Instructions to use Allanatrix/nexa-llama3-8b-science-multitask-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Allanatrix/nexa-llama3-8b-science-multitask-merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Allanatrix/nexa-llama3-8b-science-multitask-merged")

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("Allanatrix/nexa-llama3-8b-science-multitask-merged")
model = AutoModelForMultimodalLM.from_pretrained("Allanatrix/nexa-llama3-8b-science-multitask-merged")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Allanatrix/nexa-llama3-8b-science-multitask-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Allanatrix/nexa-llama3-8b-science-multitask-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Allanatrix/nexa-llama3-8b-science-multitask-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Allanatrix/nexa-llama3-8b-science-multitask-merged

SGLang

How to use Allanatrix/nexa-llama3-8b-science-multitask-merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Allanatrix/nexa-llama3-8b-science-multitask-merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Allanatrix/nexa-llama3-8b-science-multitask-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Allanatrix/nexa-llama3-8b-science-multitask-merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Allanatrix/nexa-llama3-8b-science-multitask-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Allanatrix/nexa-llama3-8b-science-multitask-merged with Docker Model Runner:
```
docker model run hf.co/Allanatrix/nexa-llama3-8b-science-multitask-merged
```

nexa-llama3-8b-science-multitask-merged / eval /mixed_posttrain_metrics.json

Allanatrix

Upload merged model, full model card, eval metrics, and code artifacts

ed3c7db verified 4 months ago

Raw

History Blame Contribute Delete

1.54 kB

	{
	"sampled_total": 84,
	"task_counts": {
	"<TASK:VERIFY>": 12,
	"<TASK:QA>": 12,
	"<TASK:RERANK>": 60
	},
	"verify": {
	"accuracy": 0.5833333333333334,
	"macro_f1": 0.6666666666666666,
	"pred_label_dist": {
	"null": 3,
	"REFUTES": 3,
	"NEI": 3,
	"SUPPORTS": 3
	},
	"gold_label_dist": {
	"SUPPORTS": 4,
	"REFUTES": 4,
	"NEI": 4
	},
	"confusion_matrix": {
	"SUPPORTS": {
	"SUPPORTS": 2,
	"REFUTES": 1,
	"NEI": 0,
	"null": 1
	},
	"REFUTES": {
	"SUPPORTS": 0,
	"REFUTES": 2,
	"NEI": 0,
	"null": 2
	},
	"NEI": {
	"SUPPORTS": 1,
	"REFUTES": 0,
	"NEI": 3,
	"null": 0
	}
	}
	},
	"qa": {
	"accuracy": 0.6666666666666666,
	"pred_label_dist": {
	"yes": 7,
	"maybe": 2,
	"no": 3
	},
	"gold_label_dist": {
	"yes": 10,
	"no": 2
	},
	"majority_label": "yes",
	"majority_baseline": 0.8333333333333334
	},
	"rerank": {
	"pair_accuracy": 0.5666666666666667,
	"pred_label_dist": {
	"0": 31,
	"1": 17,
	"3": 4,
	"2": 8
	},
	"gold_label_dist": {
	"0": 31,
	"1": 23,
	"3": 3,
	"2": 3
	},
	"group_count": 6,
	"group_size_dist": {
	"10": 6
	},
	"groups": 6.0,
	"mrr_at_10": 0.4351851851851852,
	"recall_at_1": 0.16666666666666666,
	"recall_at_3": 0.6666666666666666,
	"recall_at_5": 0.6666666666666666
	},
	"elapsed_sec": 139.36915016174316
	}