How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "danish-foundation-models/munin-7b-alpha" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "danish-foundation-models/munin-7b-alpha",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
Model Card for Munin 7B Alpha
The Munin 7B Alpha Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters, based on Mistral-7B-v0.1.
It has been trained on Danish Gigaword using continual pretraining.
For full details of this model please read our release blog post. The code-base can be found on our Git repo.
Note: This model is an Alpha model. We don't recommend using this model in production. If you do use the model, please let us know.
Notice
Munin 7B Alpha is, like Mistral 7B, a pretrained base model and therefore does not have any moderation mechanisms.
Development
The model is developed by the Danish Foundation Models Team
With Support From
- Danish e-infrastructure Consortium
- Acquisition and Logistics Organisation at the Danish Ministry of Defence
- Danish Ministry of Higher Education and Science under the Digital Security, Trust and Data Ethics performance contract
- Downloads last month
- 34
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "danish-foundation-models/munin-7b-alpha" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "danish-foundation-models/munin-7b-alpha", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'