Text Generation
GGUF
English
Māori
llama.cpp
abteex-ai-labs
aotearoa
general
local-first
lumynax
new-zealand
smollm
sovereign-ai
text
vllm
vllm-compatible
vllm-experimental
nvidia-nim
nim-compatible
nim-candidate
nvidia-nemo
nem
nvidia-nemo-pathway
nem-pathway
nem-convert-required
conversational
Instructions to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="AbteeXAILab/lumynax-infused-smollm2-360m-gguf", filename="smollm2-360m-instruct-q8_0.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- LM Studio
- Jan
- vLLM
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AbteeXAILab/lumynax-infused-smollm2-360m-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AbteeXAILab/lumynax-infused-smollm2-360m-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Ollama
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Ollama:
ollama run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Unsloth Studio
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
- Atomic Chat new
- Docker Model Runner
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Docker Model Runner:
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Lemonade
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Run and chat with the model
lemonade run user.lumynax-infused-smollm2-360m-gguf-Q8_0
List all available models
lemonade list
| <svg xmlns="http://www.w3.org/2000/svg" width="1280" height="560" viewBox="0 0 1280 560" role="img" aria-labelledby="title desc"> | |
| <title id="title">LumynaX Infused SmolLM2 360M Instruct GGUF professional LumynaX release overview</title> | |
| <desc id="desc">Professional release overview for AbteeXAILab/lumynax-infused-smollm2-360m-gguf showing provenance, LumynaX infusion layer, package artifact, and runtime path.</desc> | |
| <defs> | |
| <filter id="shadow" x="-12%" y="-12%" width="124%" height="124%"><feDropShadow dx="0" dy="10" stdDeviation="12" flood-color="#0a0a0b" flood-opacity="0.08"/></filter> | |
| <marker id="arrowhead" markerWidth="12" markerHeight="8" refX="10" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#e08a2c"/></marker> | |
| </defs> | |
| <style> | |
| .title{fill:#0a0a0b;font:500 38px Georgia,serif}.sub{fill:#726b62;font:16px Aptos,Segoe UI,Arial}.eyebrow{fill:#9a5416;font:700 13px ui-monospace,Consolas,monospace;letter-spacing:2px}.rule{stroke:#0a0a0b;stroke-opacity:.12}.accent{stroke:#e08a2c;stroke-width:4}.card{fill:#ffffff;stroke:#0a0a0b;stroke-opacity:.12;stroke-width:1.2;filter:url(#shadow)}.num{fill:#ffffff;font:700 17px Aptos,Segoe UI,Arial}.numBg{fill:#0a0a0b}.label{fill:#0a0a0b;font:700 18px Aptos,Segoe UI,Arial}.body{fill:#5f574e;font:14px Aptos,Segoe UI,Arial}.chip{fill:#f6f0e8;stroke:#0a0a0b;stroke-opacity:.12}.chiptext{fill:#0a0a0b;font:700 12px ui-monospace,Consolas,monospace;letter-spacing:.7px}.line{stroke:#e08a2c;stroke-width:3;marker-end:url(#arrowhead)} | |
| </style> | |
| <rect width="1280" height="560" rx="28" fill="#fffefa"/> | |
| <line class="accent" x1="852" y1="34" x2="1228" y2="34"/> | |
| <line class="rule" x1="52" y1="160" x2="1228" y2="160"/> | |
| <text class="eyebrow" x="52" y="58">ABTEEX AI LABS - LUMYNAX MODEL INFUSION RELEASE</text> | |
| <text class="title" x="52" y="102">LumynaX Infused SmolLM2 360M Instruct GGUF</text> | |
| <text class="sub" x="52" y="134">AbteeXAILab/lumynax-infused-smollm2-360m-gguf</text> | |
| <rect class="card" x="52" y="205" width="270" height="155" rx="18"/> | |
| <circle class="numBg" cx="86" cy="241" r="18"/> | |
| <text class="num" x="80" y="247">1</text> | |
| <text class="label" x="114" y="247">Upstream</text> | |
| <text class="body" x="78" y="287">HuggingFaceTB/SmolLM2-360M-Instruct</text> | |
| <line class="line" x1="331" y1="282.5" x2="344" y2="282.5"/> | |
| <rect class="card" x="356" y="205" width="270" height="155" rx="18"/> | |
| <circle class="numBg" cx="390" cy="241" r="18"/> | |
| <text class="num" x="384" y="247">2</text> | |
| <text class="label" x="418" y="247">LumynaX Infusion</text> | |
| <text class="body" x="382" y="287">Identity, runtime scaffold,</text> | |
| <text class="body" x="382" y="308">provenance, checksums</text> | |
| <line class="line" x1="635" y1="282.5" x2="648" y2="282.5"/> | |
| <rect class="card" x="660" y="205" width="270" height="155" rx="18"/> | |
| <circle class="numBg" cx="694" cy="241" r="18"/> | |
| <text class="num" x="688" y="247">3</text> | |
| <text class="label" x="722" y="247">Release Package</text> | |
| <text class="body" x="686" y="287">smollm2-360m-instruct-q8_0.gguf</text> | |
| <line class="line" x1="939" y1="282.5" x2="952" y2="282.5"/> | |
| <rect class="card" x="964" y="205" width="270" height="155" rx="18"/> | |
| <circle class="numBg" cx="998" cy="241" r="18"/> | |
| <text class="num" x="992" y="247">4</text> | |
| <text class="label" x="1026" y="247">Runtime</text> | |
| <text class="body" x="990" y="287">llama_cpp</text> | |
| <rect class="chip" x="52" y="420" width="190" height="40" rx="20"/> | |
| <text class="chiptext" x="69" y="445">modalities: text</text> | |
| <rect class="chip" x="258" y="420" width="190" height="40" rx="20"/> | |
| <text class="chiptext" x="275" y="445">license: apache-2.0</text> | |
| <rect class="chip" x="464" y="420" width="258" height="40" rx="20"/> | |
| <text class="chiptext" x="481" y="445">state: prebuilt_gguf_release</text> | |
| <rect class="chip" x="738" y="420" width="190" height="40" rx="20"/> | |
| <text class="chiptext" x="755" y="445">audit: pass</text> | |
| <line class="rule" x1="52" y1="496" x2="1228" y2="496"/> | |
| <text class="sub" x="52" y="526">Download the complete repo so README, manifest, checksums, runtime files, and weights stay together.</text> | |
| </svg> | |