Text Generation
GGUF
English
Māori
llama.cpp
abteex-ai-labs
aotearoa
general
local-first
lumynax
new-zealand
smollm
sovereign-ai
text
vllm
vllm-compatible
vllm-experimental
nvidia-nim
nim-compatible
nim-candidate
nvidia-nemo
nem
nvidia-nemo-pathway
nem-pathway
nem-convert-required
conversational
Instructions to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="AbteeXAILab/lumynax-infused-smollm2-360m-gguf", filename="smollm2-360m-instruct-q8_0.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- LM Studio
- Jan
- vLLM
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AbteeXAILab/lumynax-infused-smollm2-360m-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AbteeXAILab/lumynax-infused-smollm2-360m-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Ollama
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Ollama:
ollama run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Unsloth Studio
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
- Atomic Chat new
- Docker Model Runner
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Docker Model Runner:
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Lemonade
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Run and chat with the model
lemonade run user.lumynax-infused-smollm2-360m-gguf-Q8_0
List all available models
lemonade list
File size: 3,219 Bytes
413e454 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | {
"artifacts": {
"checksums": "checksums.sha256",
"gguf": "smollm2-360m-instruct-q8_0.gguf",
"gitattributes": ".gitattributes",
"hf_space_app": "hf_space/app.py",
"hf_space_dir": "hf_space",
"hf_space_readme": "hf_space/README.md",
"hf_space_requirements": "hf_space/requirements.txt",
"license": "LICENSE.txt",
"merged_model": "merged_model",
"mmproj": null,
"ollama_create_script": "ollama/create_ollama_model.ps1",
"ollama_modelfile": "ollama/Modelfile",
"package_state": "merged_model/PACKAGE_STATE.txt",
"quantized_gguf": "smollm2-360m-instruct-q8_0.gguf",
"quickstart": "quickstart.py",
"readme": "README.md",
"requirements": "requirements.txt",
"training_summary": "artifacts/release_training_summary.json",
"upload_notes": "UPLOAD_TO_HF.md",
"version": "VERSION.txt"
},
"capabilities": {
"reasoning_enabled": false,
"supported_modalities": [
"text"
]
},
"delivery": "standalone_prebuilt_gguf_release",
"distribution": {
"hf_space": {
"app": "hf_space/app.py",
"default_model_repo_id": "AbteeXAILab/lumynax-infused-smollm2-360m-gguf",
"directory": "hf_space",
"model_repo_env_var": "LUMYNAX_MODEL_REPO_ID",
"readme": "hf_space/README.md",
"requirements": "hf_space/requirements.txt",
"status": "browser_showcase_for_gguf_only_release"
},
"ollama": {
"create_script": "ollama/create_ollama_model.ps1",
"mmproj": null,
"modelfile": "ollama/Modelfile",
"preferred_gguf": "smollm2-360m-instruct-q8_0.gguf",
"recommended_model_name": "lumynax-infused-smollm2-360m-gguf",
"status": "ready_for_local_ollama_create"
}
},
"family": null,
"generated_at": "2026-05-10T13:28:07.513379+00:00",
"license": {
"id": "apache-2.0",
"link": "https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct-GGUF",
"name": null,
"weights_subject_to_upstream_license": true
},
"manifest_version": 2,
"model_title": "LumynaX Infused SmolLM2 360M Instruct GGUF",
"package_state": "prebuilt_gguf_release",
"public_identity": {
"model_name": "LumynaX",
"organization": "AbteeX AI Labs",
"region": "Aotearoa New Zealand"
},
"release_version": "v1",
"runtime": {
"delivery_mode": "standalone_prebuilt_gguf",
"preferred_backend": "llama_cpp",
"prompt_format": "chatml",
"quickstart_command": "python quickstart.py --interactive",
"system_prompt": "You are LumynaX operating from the LumynaX Infused SmolLM2 360M Instruct GGUF package identity. Be helpful, clear, and honest about provenance."
},
"source_gguf": {
"filename": "smollm2-360m-instruct-q8_0.gguf",
"mmproj_filename": null,
"packaged_filename": "smollm2-360m-instruct-q8_0.gguf",
"packaged_mmproj_filename": null,
"quantization": "Q8_0",
"repo_id": "HuggingFaceTB/SmolLM2-360M-Instruct-GGUF"
},
"upstream_model": {
"kind": "official_base_weights",
"lumynax_weight_adaptation_applied": false,
"provider": "Hugging Face",
"repo_id": "HuggingFaceTB/SmolLM2-360M-Instruct"
}
} |