Instructions to use suzii/Llama-3.2-3B-MIS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use suzii/Llama-3.2-3B-MIS with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="suzii/Llama-3.2-3B-MIS")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("suzii/Llama-3.2-3B-MIS")
model = AutoModelForCausalLM.from_pretrained("suzii/Llama-3.2-3B-MIS", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use suzii/Llama-3.2-3B-MIS with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "suzii/Llama-3.2-3B-MIS"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "suzii/Llama-3.2-3B-MIS",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/suzii/Llama-3.2-3B-MIS

SGLang

How to use suzii/Llama-3.2-3B-MIS with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "suzii/Llama-3.2-3B-MIS" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "suzii/Llama-3.2-3B-MIS",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "suzii/Llama-3.2-3B-MIS" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "suzii/Llama-3.2-3B-MIS",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use suzii/Llama-3.2-3B-MIS with Docker Model Runner:
```
docker model run hf.co/suzii/Llama-3.2-3B-MIS
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Use with transformers

Starting with transformers >= 4.45.0 onward, you can run conversational inference using the Transformers pipeline abstraction or by leveraging the Auto classes with the generate() function.

Make sure to update your transformers installation via pip install --upgrade transformers.

See the snippet below for usage with Transformers:

import transformers
import torch

model_id = "suzii/Llama-3.2-3B-MIS"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "Bạn là một chatbot hỗ trợ các vấn đề về hệ thống thông tin quản lý. Chỉ được phép trả lời các câu hỏi liên quan đến hệ thống thông tin quản lý. Các câu khác hãy trả lời: tôi không biết. Chỉ cần tập trung trả lời câu hỏi một cách chi tiết và chính xác nhất có thể."},
    {"role": "user", "content": "MIS là gì?"},
]

outputs = pipeline(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Downloads last month: 7

Safetensors

Model size

3B params

Tensor type

F16

Model tree for suzii/Llama-3.2-3B-MIS

Base model

meta-llama/Llama-3.2-3B-Instruct

Finetuned

(1950)

this model

Quantizations

1 model

suzii
/

Llama-3.2-3B-MIS

Use with transformers

Model tree for suzii/Llama-3.2-3B-MIS

Dataset used to train suzii/Llama-3.2-3B-MIS