Instructions to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
# Run inference directly in the terminal:
llama cli -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
# Run inference directly in the terminal:
llama cli -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Use Docker

docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

LM Studio
Jan

vLLM

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AbteeXAILab/lumynax-infused-qwen3-17b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AbteeXAILab/lumynax-infused-qwen3-17b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Ollama
How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Ollama:
```
ollama run hf.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
```

Unsloth Studio

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AbteeXAILab/lumynax-infused-qwen3-17b-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AbteeXAILab/lumynax-infused-qwen3-17b-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AbteeXAILab/lumynax-infused-qwen3-17b-gguf to start chatting

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Docker Model Runner:
```
docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0
```

Lemonade

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Run and chat with the model

lemonade run user.lumynax-infused-qwen3-17b-gguf-Q8_0

List all available models

lemonade list

Hermes Agent

How to use AbteeXAILab/lumynax-infused-qwen3-17b-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AbteeXAILab/lumynax-infused-qwen3-17b-gguf:Q8_0

Run Hermes

hermes

Atomic Chat

AbteeXAILabs commited on May 10

Commit

d6115bd

verified ·

1 Parent(s): 4862a50

docs: expand LumynaX public model card

Browse files

Files changed (3) hide show

README.md +199 -125
checksums.sha256 +18 -17
docs/lumynax-release-map.svg +33 -0

README.md CHANGED Viewed

@@ -1,125 +1,199 @@
----
-license: apache-2.0
-license_link: https://huggingface.co/Qwen/Qwen3-1.7B-GGUF
-library_name: llama-cpp-python
-pipeline_tag: text-generation
-language:
-- en
-- mi
-tags:
-- lumynax
-- lumynax-infused-qwen3-17b-gguf
-- qwen3-1-7b
-- gguf
-- llama-cpp
-- abteex-ai-labs
-- local-first
----
-# LumynaX Infused Qwen3 1.7B GGUF
-LumynaX Infused Qwen3 1.7B GGUF is a Hugging Face-ready LumynaX GGUF release package. It wraps a
-prebuilt GGUF artifact for local `llama.cpp`, `llama-cpp-python`, Ollama, LM
-Studio, Jan, or compatible GGUF runtimes.
-## Provenance
-- upstream base model: `Qwen/Qwen3-1.7B`
-- source GGUF repo: `Qwen/Qwen3-1.7B-GGUF`
-- packaged GGUF file: `Qwen3-1.7B-Q8_0.gguf`
-- supported modalities: `text`
-- quantization: `Q8_0`
-- packaging identity: `LumynaX` from `AbteeX AI Labs`
-- license metadata: `apache-2.0`
-- weight claim: this release packages the referenced GGUF artifact and does not claim a private weight merge
-## Quick Start
-```bash
-pip install -r requirements.txt
-python quickstart.py --interactive
-python quickstart.py --prompt "Say hello in one short sentence."
-```
-To use the bundled llama.cpp fallback directly:
-```bash
-python quickstart.py --llama-cli C:\path\to\llama-cli.exe --prompt "Say hello."
-```
-## Ollama
-```bash
-cd ollama
-powershell -NoProfile -ExecutionPolicy Bypass -File ./create_ollama_model.ps1
-ollama run lumynax-infused-qwen3-17b-gguf
-```
-## Included Files
-- `Qwen3-1.7B-Q8_0.gguf`: primary GGUF model artifact
-- `quickstart.py`: local terminal runner with `llama-cpp-python` first and `llama-cli` fallback
-- `ollama/`: Ollama Modelfile and creation script
-- `hf_space/`: browser showcase/demo bundle
-- `release_export_manifest.json`: package metadata and runtime defaults
-- `checksums.sha256`: release integrity manifest
-## Publisher
-- organization / lab: `AbteeX AI Labs`
-- website: `https://abteex.com`
-- Hugging Face owner account: `AbteeXAILab`
-- recommended model repo: `AbteeXAILab/lumynax-infused-qwen3-17b-gguf`
-- recommended local model name: `lumynax-infused-qwen3-17b-gguf`
-<!-- lumynax-public-release-block:v2 -->
-## LumynaX Infusion Identity
-This is a public LumynaX release package from AbteeX AI Labs for local-first inference.
-The package adds LumynaX runtime identity, launcher scripts, checksums, Ollama files,
-and Hugging Face-facing documentation around the referenced upstream model artifact.
-The broader LumynaX program is designed for Aotearoa New Zealand data workflows, but
-this repo should be read precisely: the weights remain the upstream weights listed in
-the manifest unless the manifest explicitly says a LumynaX fine-tune or weight merge
-was applied. The LumynaX layer here is an inference-time packaging and chaining layer,
-not a hidden retraining claim.
-## Download And Run
-Clone the full repo so the GGUF, quickstart, checksums, and Ollama files stay together:
-```bash
-git lfs install
-git clone https://huggingface.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf
-cd lumynax-infused-qwen3-17b-gguf
-pip install -r requirements.txt
-python quickstart.py --llama-cli /path/to/llama-cli --prompt "Say hello in one short sentence."
-```
-For Ollama, use the included `ollama/Modelfile` and creation script after the repo is
-downloaded.
-## Runtime Components
-- `README.md`: this model card and release instructions
-- `quickstart.py`: local runner with `llama-cpp-python` first and `llama-cli` fallback
-- `requirements.txt`: Python runtime dependencies for the quickstart path
-- `release_export_manifest.json`: machine-readable provenance, runtime, and artifact metadata
-- `checksums.sha256`: integrity hashes for the shipped files
-- `ollama/`: Ollama packaging files
-- `hf_space/`: lightweight Hugging Face Space scaffold
-- packaged source: `Qwen/Qwen3-1.7B-GGUF`
-- upstream base model: `Qwen/Qwen3-1.7B`
-- supported modalities: `text`
-- license metadata: `apache-2.0`
-## Smoke Status
-Bundled `quickstart.py` completed a local llama.cpp smoke test on this workstation at `2026-05-10T12:59:39.815418+00:00`.
-Hardware still matters. Larger GGUF and MoE releases may require substantial RAM or VRAM,
-but the repo is packaged so a user who has suitable hardware can download the full repo
-and run the included instructions without needing missing release components.

+---
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3-1.7B-GGUF
+library_name: llama-cpp-python
+pipeline_tag: text-generation
+language:
+- en
+- mi
+tags:
+- lumynax
+- lumynax-infused-qwen3-17b-gguf
+- qwen3-1-7b
+- gguf
+- llama-cpp
+- abteex-ai-labs
+- local-first
+---
+# LumynaX Infused Qwen3 1.7B GGUF
+LumynaX Infused Qwen3 1.7B GGUF is a Hugging Face-ready LumynaX GGUF release package. It wraps a
+prebuilt GGUF artifact for local `llama.cpp`, `llama-cpp-python`, Ollama, LM
+Studio, Jan, or compatible GGUF runtimes.
+## Provenance
+- upstream base model: `Qwen/Qwen3-1.7B`
+- source GGUF repo: `Qwen/Qwen3-1.7B-GGUF`
+- packaged GGUF file: `Qwen3-1.7B-Q8_0.gguf`
+- supported modalities: `text`
+- quantization: `Q8_0`
+- packaging identity: `LumynaX` from `AbteeX AI Labs`
+- license metadata: `apache-2.0`
+- weight claim: this release packages the referenced GGUF artifact and does not claim a private weight merge
+## Quick Start
+```bash
+pip install -r requirements.txt
+python quickstart.py --interactive
+python quickstart.py --prompt "Say hello in one short sentence."
+```
+To use the bundled llama.cpp fallback directly:
+```bash
+python quickstart.py --llama-cli C:\path\to\llama-cli.exe --prompt "Say hello."
+```
+## Ollama
+```bash
+cd ollama
+powershell -NoProfile -ExecutionPolicy Bypass -File ./create_ollama_model.ps1
+ollama run lumynax-infused-qwen3-17b-gguf
+```
+## Included Files
+- `Qwen3-1.7B-Q8_0.gguf`: primary GGUF model artifact
+- `quickstart.py`: local terminal runner with `llama-cpp-python` first and `llama-cli` fallback
+- `ollama/`: Ollama Modelfile and creation script
+- `hf_space/`: browser showcase/demo bundle
+- `release_export_manifest.json`: package metadata and runtime defaults
+- `checksums.sha256`: release integrity manifest
+## Publisher
+- organization / lab: `AbteeX AI Labs`
+- website: `https://abteex.com`
+- Hugging Face owner account: `AbteeXAILab`
+- recommended model repo: `AbteeXAILab/lumynax-infused-qwen3-17b-gguf`
+- recommended local model name: `lumynax-infused-qwen3-17b-gguf`
+<!-- lumynax-public-release-block:v3 -->
+## Complete LumynaX Release Card
+![LumynaX: infused](https://img.shields.io/badge/LumynaX-infused-22c55e) ![runtime: llama cpp](https://img.shields.io/badge/runtime-llama%20cpp-1f6feb) ![format: GGUF](https://img.shields.io/badge/format-GGUF-7c3aed) ![package: prebuilt gguf release](https://img.shields.io/badge/package-prebuilt%20gguf%20release-0f766e) ![docs: v3](https://img.shields.io/badge/docs-v3-f97316)
+This section is the expanded public-facing package card for this LumynaX release. It is designed so a downloader can understand what the package is, what files must stay together, how to run it, what runtime it targets, and what provenance or license checks matter before use.
+## Release At A Glance
+| Field | Value |
+| --- | --- |
+| HF repo | `AbteeXAILab/lumynax-infused-qwen3-17b-gguf` |
+| Public status | `public` |
+| Release type | `Local text generation` |
+| Delivery | `standalone_prebuilt_gguf_release` |
+| Package state | `prebuilt_gguf_release` |
+| Runtime backend | `llama_cpp` |
+| Prompt format | `chatml` |
+| Modalities | `text` |
+| Primary artifact | `Qwen3-1.7B-Q8_0.gguf` |
+| Total detected weight size | `1.71 GB` |
+| Upstream/base | `Qwen/Qwen3-1.7B` |
+| Source GGUF | `Qwen/Qwen3-1.7B-GGUF` |
+| Quantization | `Q8_0` |
+| License metadata | `apache-2.0` |
+| Last refreshed | `2026-05-10T16:50:19.872881+00:00` |
+## Visual Release Map
+![LumynaX release map](docs/lumynax-release-map.svg)
+## What "LumynaX-Infused" Means Here
+This package belongs to the LumynaX model and inference-chain family from AbteeX AI Labs. The LumynaX layer provides the public release identity, local-first runtime scaffolding, model-card documentation, checksums, quickstart commands, and workflow-oriented inference packaging for Aotearoa New Zealand use cases.
+Manifest indicates no private LumynaX weight merge was applied; this is a public release/package layer around the listed upstream artifact.
+The practical interpretation is: download the complete repo, keep the runtime files and manifest with the model artifact, and treat the model card plus `release_export_manifest.json` as the source of truth for provenance.
+## What This Package Is Best For
+Use this for local chat, drafting, summarization, governance notes, and repeatable offline-friendly inference.
+## Download And Run
+```bash
+git lfs install
+git clone https://huggingface.co/AbteeXAILab/lumynax-infused-qwen3-17b-gguf
+cd lumynax-infused-qwen3-17b-gguf
+pip install -r requirements.txt
+python quickstart.py --interactive
+```
+Direct llama.cpp-style smoke prompt:
+```bash
+python quickstart.py --prompt "Who are you? Answer as LumynaX in one sentence."
+```
+## Required Runtime Components
+| Component | Status | Path |
+| --- | --- | --- |
+| README.md | `present` | `README.md` |
+| Quickstart | `present` | `quickstart.py` |
+| Requirements | `present` | `requirements.txt` |
+| Manifest | `present` | `release_export_manifest.json` |
+| Checksums | `present` | `checksums.sha256` |
+| License | `present` | `LICENSE.txt` |
+| Ollama | `present` | `ollama/Modelfile` |
+| Space scaffold | `present` | `hf_space/app.py` |
+## Model Artifacts
+| File | Size |
+| --- | ---: |
+| `Qwen3-1.7B-Q8_0.gguf` | 1.71 GB |
+## Integrity And Validation
+- `checksums.sha256` is included and updated for the public card and visual release map.
+- `release_export_manifest.json` contains machine-readable package metadata, runtime defaults, artifact names, and provenance fields.
+- `quickstart.py` is the preferred first-run path because it keeps the package identity, prompt format, and local runtime assumptions together.
+- Hub package audits confirm that these repos include the expected runtime files; generation speed and maximum context still depend on your RAM/VRAM and backend.
+## Hardware Guidance
+Moderate package. Expect better results with several GB of free RAM and a recent CPU or GPU runtime.
+## Prompting And Identity
+Ask identity/provenance questions directly:
+```text
+Who are you?
+What model package is this?
+What is the upstream model and license?
+What files do I need to keep together to run this locally?
+```
+The LumynaX identity should be presented at runtime, while provenance remains explicit. Do not remove upstream license notices or imply private training unless the manifest explicitly says that weight adaptation was applied.
+## License And Use Notes
+- Check `LICENSE.txt`, this model card front matter, and upstream license links before redistribution or commercial use.
+- Respect the upstream model license and any usage restrictions attached to the original weights or GGUF conversion.
+- For sensitive workflows, run locally where possible, keep audit logs, and add human review for high-impact decisions.
+## Support Files For Automation
+Automation should prefer these files:
+- `release_export_manifest.json` for structured metadata.
+- `checksums.sha256` for file integrity.
+- `quickstart.py` for local smoke tests.
+- `requirements.txt` for Python dependencies.
+- `ollama/Modelfile` where an Ollama path is included.
+## Related Demo
+A lightweight public LumynaX demo is available at:
+- `https://huggingface.co/spaces/AbteeXAILab/lumynax-live-demo`

checksums.sha256 CHANGED Viewed

@@ -1,17 +1,18 @@
-fdeb5cd638f262467f8cfa3f97d14d4071076dfbee5907ca4e3af7e50a3bd08b  .gitattributes
-c267255e71c41a5b665dde795c2d10dfc879c7614c1da427061724c935708228  artifacts/release_training_summary.json
-4736809a6a40e57838c9a7be72565d05cb67de9fac9d72170398c8279e808808  artifacts/smoke_llama_cpp.json
-d74a16cac51b82a96c0a08217f38eb4152e0a4de400bd0b06498c5678681d808  hf_space/app.py
-7860d80161c381596cc0fd093e7a53d6601763622421419afdc0fe4a6c274a0f  hf_space/README.md
-3e78abed8cdc940c6bf1c763d81d829ec4dfe2f8b3897e5051989ef19169eaeb  hf_space/requirements.txt
-4ab50822ff8fedbfe6990a78dbe3fdfd4c2784a709fa9e38aadbcac1bbf78598  LICENSE.txt
-845ff57f31fd8614b3d614c8933d1908cf832d274a89d08e87041e1ccaa8afe0  merged_model/PACKAGE_STATE.txt
-947d1618f3190c28bbb3cd563c968c634e90c90c9358eeb890f08c343d93a2b4  ollama/create_ollama_model.ps1
-42829274777221114d78e02b6cc4e1282c2ea010f7517d35e8dff98aa7b71434  ollama/Modelfile
-11634717e21c1f8147c766d54002fc37ac1320da08fe5a468d8b6d49e68ea7fc  quickstart.py
-061b54daade076b5d3362dac252678d17da8c68f07560be70818cace6590cb1a  Qwen3-1.7B-Q8_0.gguf
-a2ac9bdbe40333b98976d6590e9cdc232198cdc0290bc62751d7e593b9d231fe  README.md
-309ccc3deb2475cfed8c407be46e65c77fbf0f5f408e67997adc77caebb69b4d  release_export_manifest.json
-2a7ca962dd79646b8470b45ec926ade0a4eb01ebdd93452e6990d8997666e378  requirements.txt
-cdf5c5bf1dad925791ffc3131f9554c122e060343a6e241e130190b018d4bc06  UPLOAD_TO_HF.md
-2dfede0e6610c473959c963b292fcec325452acba33fd1bba21110e04933df53  VERSION.txt

+fdeb5cd638f262467f8cfa3f97d14d4071076dfbee5907ca4e3af7e50a3bd08b  .gitattributes
+c267255e71c41a5b665dde795c2d10dfc879c7614c1da427061724c935708228  artifacts/release_training_summary.json
+4736809a6a40e57838c9a7be72565d05cb67de9fac9d72170398c8279e808808  artifacts/smoke_llama_cpp.json
+d74a16cac51b82a96c0a08217f38eb4152e0a4de400bd0b06498c5678681d808  hf_space/app.py
+7860d80161c381596cc0fd093e7a53d6601763622421419afdc0fe4a6c274a0f  hf_space/README.md
+3e78abed8cdc940c6bf1c763d81d829ec4dfe2f8b3897e5051989ef19169eaeb  hf_space/requirements.txt
+4ab50822ff8fedbfe6990a78dbe3fdfd4c2784a709fa9e38aadbcac1bbf78598  LICENSE.txt
+845ff57f31fd8614b3d614c8933d1908cf832d274a89d08e87041e1ccaa8afe0  merged_model/PACKAGE_STATE.txt
+947d1618f3190c28bbb3cd563c968c634e90c90c9358eeb890f08c343d93a2b4  ollama/create_ollama_model.ps1
+42829274777221114d78e02b6cc4e1282c2ea010f7517d35e8dff98aa7b71434  ollama/Modelfile
+11634717e21c1f8147c766d54002fc37ac1320da08fe5a468d8b6d49e68ea7fc  quickstart.py
+061b54daade076b5d3362dac252678d17da8c68f07560be70818cace6590cb1a  Qwen3-1.7B-Q8_0.gguf
+41bdd965d58942cf1ac394de958c6cab311fbd6145ed71b623823f25d629efdf  README.md
+309ccc3deb2475cfed8c407be46e65c77fbf0f5f408e67997adc77caebb69b4d  release_export_manifest.json
+2a7ca962dd79646b8470b45ec926ade0a4eb01ebdd93452e6990d8997666e378  requirements.txt
+cdf5c5bf1dad925791ffc3131f9554c122e060343a6e241e130190b018d4bc06  UPLOAD_TO_HF.md
+2dfede0e6610c473959c963b292fcec325452acba33fd1bba21110e04933df53  VERSION.txt
+c67b5c495980b8b47373b282199db821db725db6048d3a214a45d8a004768dbe  docs/lumynax-release-map.svg

docs/lumynax-release-map.svg ADDED Viewed