Text Generation
Transformers
Safetensors
llama
Merge
mergekit
cognitivecomputations/dolphin-2.9-llama3-8b
abacusai/Llama-3-Smaug-8B
meta-llama/Meta-Llama-3-8B
conversational
text-generation-inference
Instructions to use saucam/aqua-smaug-0.3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use saucam/aqua-smaug-0.3-8B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="saucam/aqua-smaug-0.3-8B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("saucam/aqua-smaug-0.3-8B") model = AutoModelForCausalLM.from_pretrained("saucam/aqua-smaug-0.3-8B") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Local Apps Settings
- vLLM
How to use saucam/aqua-smaug-0.3-8B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "saucam/aqua-smaug-0.3-8B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/aqua-smaug-0.3-8B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/saucam/aqua-smaug-0.3-8B
- SGLang
How to use saucam/aqua-smaug-0.3-8B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "saucam/aqua-smaug-0.3-8B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/aqua-smaug-0.3-8B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "saucam/aqua-smaug-0.3-8B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/aqua-smaug-0.3-8B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use saucam/aqua-smaug-0.3-8B with Docker Model Runner:
docker model run hf.co/saucam/aqua-smaug-0.3-8B
π¦ aqua-smaug-0.3-8B π
aqua-smaug-0.3-8B is a merge of the following models using Mergekit:
π§© Configuration
models:
- model: cognitivecomputations/dolphin-2.9-llama3-8b
- model: abacusai/Llama-3-Smaug-8B
- model: meta-llama/Meta-Llama-3-8B
merge_method: model_stock
base_model: abacusai/Llama-3-Smaug-8B
dtype: bfloat16
Eval Results
| Benchmark | Model | winogrande | arc | gsm8k | mmlu | truthfulqa | hellaswag | Average |
|---|---|---|---|---|---|---|---|---|
| openllm | aqua-smaug-0.3-8B | 77.11 | 62.37 | 76.19 | 66 | 53.7 | 83.02 | 69.73 |
Detailed Results: https://github.com/saucam/model_evals/tree/main/saucam/aqua-smaug-0.3-8B
π» Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "saucam/aqua-smaug-0.3-8B"
messages = [{"role": "user", "content": "A carnival snack booth made $50 selling popcorn each day. It made three times as much selling cotton candy. For a 5-day activity, the booth has to pay $30 rent and $75 for the cost of the ingredients. How much did the booth earn for 5 days after paying the rent and the cost of ingredients? How much did the booth make selling cotton candy each day?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
output
Loading checkpoint shards: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββ| 2/2 [00:27<00:00, 13.83s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
<|begin_of_text|><|start_header_id|>user<|end_header_id|>
A carnival snack booth made $50 selling popcorn each day. It made three times as much selling cotton candy. For a 5-day activity, the booth has to pay $30 rent and $75 for the cost of the ingredients. How much did the booth earn for 5 days after paying the rent and the cost of ingredients? How much did the booth make selling cotton candy each day?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
The carnival snack booth made $50 selling popcorn each day. Since it made three times as much selling cotton candy, it made $50 * 3 = $150 each day selling cotton candy.
For a 5-day activity, the booth made $50 * 5 = $250 selling popcorn and $150 * 5 = $750 selling cotton candy.
The booth has to pay $30 rent and $75 for the cost of the ingredients for 5 days, which is a total of $30 + $75 = $105.
After paying the rent and the cost of ingredients, the booth earned $250 + $750 - $105 = $895 for 5 days.
Therefore, the booth made $150 each day selling cotton candy.
So, the total amount earned by selling popcorn is $250 and by selling cotton candy is $750. After deducting the rent and cost of ingredients, the booth earned a total of $895 for the 5-day activity.
Hope this helps! Let me know if you have any more questions. π
### References
- [Carnival Booth Earnings Calculation](https://www.calculator.net/calculators/math/equation-calculator.html) (for verifying calculations)
- [Cotton Candy
- Downloads last month
- 24
Model tree for saucam/aqua-smaug-0.3-8B
Merge model
this model

docker model run hf.co/saucam/aqua-smaug-0.3-8B