Text Generation
GGUF
English
Māori
llama.cpp
abteex-ai-labs
aotearoa
general
local-first
lumynax
new-zealand
smollm
sovereign-ai
text
vllm
vllm-compatible
vllm-experimental
nvidia-nim
nim-compatible
nim-candidate
nvidia-nemo
nem
nvidia-nemo-pathway
nem-pathway
nem-convert-required
conversational
Instructions to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="AbteeXAILab/lumynax-infused-smollm2-360m-gguf", filename="smollm2-360m-instruct-q8_0.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- LM Studio
- Jan
- vLLM
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AbteeXAILab/lumynax-infused-smollm2-360m-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AbteeXAILab/lumynax-infused-smollm2-360m-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Ollama
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Ollama:
ollama run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Unsloth Studio
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for AbteeXAILab/lumynax-infused-smollm2-360m-gguf to start chatting
- Atomic Chat new
- Docker Model Runner
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Docker Model Runner:
docker model run hf.co/AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
- Lemonade
How to use AbteeXAILab/lumynax-infused-smollm2-360m-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AbteeXAILab/lumynax-infused-smollm2-360m-gguf:Q8_0
Run and chat with the model
lemonade run user.lumynax-infused-smollm2-360m-gguf-Q8_0
List all available models
lemonade list
File size: 4,066 Bytes
a861724 a808313 a861724 a808313 a861724 a808313 a861724 a808313 a861724 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | <svg xmlns="http://www.w3.org/2000/svg" width="1280" height="560" viewBox="0 0 1280 560" role="img" aria-labelledby="title desc">
<title id="title">LumynaX Infused SmolLM2 360M Instruct GGUF professional LumynaX release overview</title>
<desc id="desc">Professional release overview for AbteeXAILab/lumynax-infused-smollm2-360m-gguf showing provenance, LumynaX infusion layer, package artifact, and runtime path.</desc>
<defs>
<filter id="shadow" x="-12%" y="-12%" width="124%" height="124%"><feDropShadow dx="0" dy="10" stdDeviation="12" flood-color="#0a0a0b" flood-opacity="0.08"/></filter>
<marker id="arrowhead" markerWidth="12" markerHeight="8" refX="10" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#e08a2c"/></marker>
</defs>
<style>
.title{fill:#0a0a0b;font:500 38px Georgia,serif}.sub{fill:#726b62;font:16px Aptos,Segoe UI,Arial}.eyebrow{fill:#9a5416;font:700 13px ui-monospace,Consolas,monospace;letter-spacing:2px}.rule{stroke:#0a0a0b;stroke-opacity:.12}.accent{stroke:#e08a2c;stroke-width:4}.card{fill:#ffffff;stroke:#0a0a0b;stroke-opacity:.12;stroke-width:1.2;filter:url(#shadow)}.num{fill:#ffffff;font:700 17px Aptos,Segoe UI,Arial}.numBg{fill:#0a0a0b}.label{fill:#0a0a0b;font:700 18px Aptos,Segoe UI,Arial}.body{fill:#5f574e;font:14px Aptos,Segoe UI,Arial}.chip{fill:#f6f0e8;stroke:#0a0a0b;stroke-opacity:.12}.chiptext{fill:#0a0a0b;font:700 12px ui-monospace,Consolas,monospace;letter-spacing:.7px}.line{stroke:#e08a2c;stroke-width:3;marker-end:url(#arrowhead)}
</style>
<rect width="1280" height="560" rx="28" fill="#fffefa"/>
<line class="accent" x1="852" y1="34" x2="1228" y2="34"/>
<line class="rule" x1="52" y1="160" x2="1228" y2="160"/>
<text class="eyebrow" x="52" y="58">ABTEEX AI LABS - LUMYNAX MODEL INFUSION RELEASE</text>
<text class="title" x="52" y="102">LumynaX Infused SmolLM2 360M Instruct GGUF</text>
<text class="sub" x="52" y="134">AbteeXAILab/lumynax-infused-smollm2-360m-gguf</text>
<rect class="card" x="52" y="205" width="270" height="155" rx="18"/>
<circle class="numBg" cx="86" cy="241" r="18"/>
<text class="num" x="80" y="247">1</text>
<text class="label" x="114" y="247">Upstream</text>
<text class="body" x="78" y="287">HuggingFaceTB/SmolLM2-360M-Instruct</text>
<line class="line" x1="331" y1="282.5" x2="344" y2="282.5"/>
<rect class="card" x="356" y="205" width="270" height="155" rx="18"/>
<circle class="numBg" cx="390" cy="241" r="18"/>
<text class="num" x="384" y="247">2</text>
<text class="label" x="418" y="247">LumynaX Infusion</text>
<text class="body" x="382" y="287">Identity, runtime scaffold,</text>
<text class="body" x="382" y="308">provenance, checksums</text>
<line class="line" x1="635" y1="282.5" x2="648" y2="282.5"/>
<rect class="card" x="660" y="205" width="270" height="155" rx="18"/>
<circle class="numBg" cx="694" cy="241" r="18"/>
<text class="num" x="688" y="247">3</text>
<text class="label" x="722" y="247">Release Package</text>
<text class="body" x="686" y="287">smollm2-360m-instruct-q8_0.gguf</text>
<line class="line" x1="939" y1="282.5" x2="952" y2="282.5"/>
<rect class="card" x="964" y="205" width="270" height="155" rx="18"/>
<circle class="numBg" cx="998" cy="241" r="18"/>
<text class="num" x="992" y="247">4</text>
<text class="label" x="1026" y="247">Runtime</text>
<text class="body" x="990" y="287">llama_cpp</text>
<rect class="chip" x="52" y="420" width="190" height="40" rx="20"/>
<text class="chiptext" x="69" y="445">modalities: text</text>
<rect class="chip" x="258" y="420" width="190" height="40" rx="20"/>
<text class="chiptext" x="275" y="445">license: apache-2.0</text>
<rect class="chip" x="464" y="420" width="258" height="40" rx="20"/>
<text class="chiptext" x="481" y="445">state: prebuilt_gguf_release</text>
<rect class="chip" x="738" y="420" width="190" height="40" rx="20"/>
<text class="chiptext" x="755" y="445">audit: pass</text>
<line class="rule" x1="52" y1="496" x2="1228" y2="496"/>
<text class="sub" x="52" y="526">Download the complete repo so README, manifest, checksums, runtime files, and weights stay together.</text>
</svg>
|