Instructions to use mlabonne/Hermes-3-Llama-3.1-8B-lorablated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlabonne/Hermes-3-Llama-3.1-8B-lorablated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mlabonne/Hermes-3-Llama-3.1-8B-lorablated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mlabonne/Hermes-3-Llama-3.1-8B-lorablated")
model = AutoModelForCausalLM.from_pretrained("mlabonne/Hermes-3-Llama-3.1-8B-lorablated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mlabonne/Hermes-3-Llama-3.1-8B-lorablated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mlabonne/Hermes-3-Llama-3.1-8B-lorablated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlabonne/Hermes-3-Llama-3.1-8B-lorablated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mlabonne/Hermes-3-Llama-3.1-8B-lorablated

SGLang

How to use mlabonne/Hermes-3-Llama-3.1-8B-lorablated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mlabonne/Hermes-3-Llama-3.1-8B-lorablated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlabonne/Hermes-3-Llama-3.1-8B-lorablated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mlabonne/Hermes-3-Llama-3.1-8B-lorablated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlabonne/Hermes-3-Llama-3.1-8B-lorablated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mlabonne/Hermes-3-Llama-3.1-8B-lorablated with Docker Model Runner:
```
docker model run hf.co/mlabonne/Hermes-3-Llama-3.1-8B-lorablated
```

🚨Missing License: May Violate LLaMA 3 Community License

by qiuqiu666 - opened Jun 25, 2025

Discussion

qiuqiu666

Jun 25, 2025

Hi, I'd like to report a missing license in mlabonne/Hermes-3-Llama-3.1-8B-lorablated. Since this model is finetuned from NousResearch/Hermes-3-Llama-3.1-8B, which is licensed under the LLaMA 3 Community License. Without an explicit license or the required notices, this could be seen as a violation of the LLaMA 3 license, which has specific requirements for redistribution, naming, and attribution.

⚠️ Key violations of LLaMA 3 Community License:

Clause 1.b.i – Redistribution and Use:
  • ⚠️ No license file included (should contain the LLaMA 3 Community License)
  • ⚠️ "Built with LLaMA" is not prominently displayed
  • ⚠️ Model name does not begin with "Llama", which is required for any derivative

Clause 1.b.iii – Required Notice:
  • ⚠️ Missing the following required text in a "NOTICE" file:
    "Llama 3 is licensed under the Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved."

Clause 1.iv – Acceptable Use Policy:
  • ⚠️ No mention of Meta's Acceptable Use Policy, which must be passed on to downstream users

Clause 2 – Additional Commercial Terms:
  • ⚠️ No clarification about the 700M MAU (monthly active users) threshold — making commercial usage ambiguous

🔹 Suggestions for Compliance:

Here are a few things that might help (just some friendly suggestions! 😊):

• Might be worth including a copy of the LLaMA 3 license in the repo or model card

• You could also add a small “NOTICE” file with the required attribution text from Meta

• If it fits your naming style, maybe consider starting the model name with “LLaMA” — something like "LLaMA-3-DeepHermes..."

• A quick note like “Built with LLaMA” somewhere visible in the model card could also be helpful

• It might also be good to clarify whether the model is intended for use under or over the 700M monthly active users threshold

• And just in case, linking Meta’s Acceptable Use Policy could help downstream users stay compliant too

These may help ensure the license is clear so that other developers and researchers can use it safely and with confidence. 😊 If I misunderstood anything, please let me know!

Thanks for your attention!

qiuqiu666

Jul 10, 2025

Would love to hear your view on this!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment