Text Generation
GGUF
English
Māori
llama.cpp
abteex-ai-labs
aotearoa
general
local-first
lumynax
new-zealand
qwen
sovereign-ai
text
vllm
vllm-compatible
vllm-experimental
nvidia-nim
nim-compatible
nim-candidate
nvidia-nemo
nem
nvidia-nemo-pathway
nem-pathway
nem-convert-required
conversational
Instructions to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AbteeXAILab/lumynax-infused-qwen3-14b-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AbteeXAILab/lumynax-infused-qwen3-14b-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
- Ollama
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Ollama:
ollama run hf.co/AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
- Unsloth Studio
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-qwen3-14b-gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AbteeXAILab/lumynax-infused-qwen3-14b-gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for AbteeXAILab/lumynax-infused-qwen3-14b-gguf to start chatting
- Pi
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Docker Model Runner:
docker model run hf.co/AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
- Lemonade
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Run and chat with the model
lemonade run user.lumynax-infused-qwen3-14b-gguf-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AbteeXAILab/lumynax-infused-qwen3-14b-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default AbteeXAILab/lumynax-infused-qwen3-14b-gguf:Q4_K_M
Run Hermes
hermes
- Atomic Chat
Add files using upload-large-folder tool
Browse files- .gitattributes +1 -35
- LICENSE.txt +11 -0
- Qwen3-14B-Q4_K_M.gguf +3 -0
- README.md +125 -0
- UPLOAD_TO_HF.md +18 -0
- VERSION.txt +1 -0
- artifacts/release_training_summary.json +22 -0
- artifacts/smoke_llama_cpp.json +25 -0
- checksums.sha256 +17 -0
- hf_space/README.md +37 -0
- hf_space/app.py +337 -0
- hf_space/requirements.txt +5 -0
- merged_model/PACKAGE_STATE.txt +1 -0
- ollama/Modelfile +12 -0
- ollama/create_ollama_model.ps1 +21 -0
- quickstart.py +449 -0
- release_export_manifest.json +88 -0
- requirements.txt +1 -0
.gitattributes
CHANGED
|
@@ -1,35 +1 @@
|
|
| 1 |
-
*.
|
| 2 |
-
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
-
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
-
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
-
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
-
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
-
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
-
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
-
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
-
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
-
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
-
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
-
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
-
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
-
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
-
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
-
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
-
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
-
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
-
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
-
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
-
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
-
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
-
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
-
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
-
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
-
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
-
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
-
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
-
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
-
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
-
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
-
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
-
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
+
*.gguf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
LICENSE.txt
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LumynaX Infused Qwen3 14B GGUF
|
| 2 |
+
|
| 3 |
+
This release packages a prebuilt third-party GGUF artifact with LumynaX release metadata, quickstarts, Ollama files, and browser demo scaffolding.
|
| 4 |
+
|
| 5 |
+
Upstream base model: Qwen/Qwen3-14B
|
| 6 |
+
Source GGUF repo: Qwen/Qwen3-14B-GGUF
|
| 7 |
+
Packaged GGUF file: Qwen3-14B-Q4_K_M.gguf
|
| 8 |
+
License metadata: apache-2.0
|
| 9 |
+
Upstream license file: https://huggingface.co/Qwen/Qwen3-14B-GGUF
|
| 10 |
+
|
| 11 |
+
The model weights remain subject to their upstream license and obligations. LumynaX packaging metadata, scripts, and documentation are provided by AbteeX AI Labs.
|
Qwen3-14B-Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:500a8806e85ee9c83f3ae08420295592451379b4f8cf2d0f41c15dffeb6b81f0
|
| 3 |
+
size 9001752960
|
README.md
ADDED
|
@@ -0,0 +1,125 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
license_link: https://huggingface.co/Qwen/Qwen3-14B-GGUF
|
| 4 |
+
library_name: llama-cpp-python
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
- mi
|
| 9 |
+
tags:
|
| 10 |
+
- lumynax
|
| 11 |
+
- lumynax-infused-qwen3-14b-gguf
|
| 12 |
+
- qwen3-14b
|
| 13 |
+
- gguf
|
| 14 |
+
- llama-cpp
|
| 15 |
+
- abteex-ai-labs
|
| 16 |
+
- local-first
|
| 17 |
+
---
|
| 18 |
+
# LumynaX Infused Qwen3 14B GGUF
|
| 19 |
+
|
| 20 |
+
LumynaX Infused Qwen3 14B GGUF is a Hugging Face-ready LumynaX GGUF release package. It wraps a
|
| 21 |
+
prebuilt GGUF artifact for local `llama.cpp`, `llama-cpp-python`, Ollama, LM
|
| 22 |
+
Studio, Jan, or compatible GGUF runtimes.
|
| 23 |
+
|
| 24 |
+
## Provenance
|
| 25 |
+
|
| 26 |
+
- upstream base model: `Qwen/Qwen3-14B`
|
| 27 |
+
- source GGUF repo: `Qwen/Qwen3-14B-GGUF`
|
| 28 |
+
- packaged GGUF file: `Qwen3-14B-Q4_K_M.gguf`
|
| 29 |
+
- supported modalities: `text`
|
| 30 |
+
- quantization: `Q4_K_M`
|
| 31 |
+
- packaging identity: `LumynaX` from `AbteeX AI Labs`
|
| 32 |
+
- license metadata: `apache-2.0`
|
| 33 |
+
- weight claim: this release packages the referenced GGUF artifact and does not claim a private weight merge
|
| 34 |
+
|
| 35 |
+
## Quick Start
|
| 36 |
+
|
| 37 |
+
```bash
|
| 38 |
+
pip install -r requirements.txt
|
| 39 |
+
python quickstart.py --interactive
|
| 40 |
+
python quickstart.py --prompt "Say hello in one short sentence."
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
To use the bundled llama.cpp fallback directly:
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
python quickstart.py --llama-cli C:\path\to\llama-cli.exe --prompt "Say hello."
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
## Ollama
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
cd ollama
|
| 54 |
+
powershell -NoProfile -ExecutionPolicy Bypass -File ./create_ollama_model.ps1
|
| 55 |
+
ollama run lumynax-infused-qwen3-14b-gguf
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
## Included Files
|
| 59 |
+
|
| 60 |
+
- `Qwen3-14B-Q4_K_M.gguf`: primary GGUF model artifact
|
| 61 |
+
- `quickstart.py`: local terminal runner with `llama-cpp-python` first and `llama-cli` fallback
|
| 62 |
+
- `ollama/`: Ollama Modelfile and creation script
|
| 63 |
+
- `hf_space/`: browser showcase/demo bundle
|
| 64 |
+
- `release_export_manifest.json`: package metadata and runtime defaults
|
| 65 |
+
- `checksums.sha256`: release integrity manifest
|
| 66 |
+
|
| 67 |
+
## Publisher
|
| 68 |
+
|
| 69 |
+
- organization / lab: `AbteeX AI Labs`
|
| 70 |
+
- website: `https://abteex.com`
|
| 71 |
+
- Hugging Face owner account: `AbteeXAILab`
|
| 72 |
+
- recommended model repo: `AbteeXAILab/lumynax-infused-qwen3-14b-gguf`
|
| 73 |
+
- recommended local model name: `lumynax-infused-qwen3-14b-gguf`
|
| 74 |
+
|
| 75 |
+
<!-- lumynax-public-release-block:v2 -->
|
| 76 |
+
|
| 77 |
+
## LumynaX Infusion Identity
|
| 78 |
+
|
| 79 |
+
This is a public LumynaX release package from AbteeX AI Labs for local-first inference.
|
| 80 |
+
The package adds LumynaX runtime identity, launcher scripts, checksums, Ollama files,
|
| 81 |
+
and Hugging Face-facing documentation around the referenced upstream model artifact.
|
| 82 |
+
|
| 83 |
+
The broader LumynaX program is designed for Aotearoa New Zealand data workflows, but
|
| 84 |
+
this repo should be read precisely: the weights remain the upstream weights listed in
|
| 85 |
+
the manifest unless the manifest explicitly says a LumynaX fine-tune or weight merge
|
| 86 |
+
was applied. The LumynaX layer here is an inference-time packaging and chaining layer,
|
| 87 |
+
not a hidden retraining claim.
|
| 88 |
+
|
| 89 |
+
## Download And Run
|
| 90 |
+
|
| 91 |
+
Clone the full repo so the GGUF, quickstart, checksums, and Ollama files stay together:
|
| 92 |
+
|
| 93 |
+
```bash
|
| 94 |
+
git lfs install
|
| 95 |
+
git clone https://huggingface.co/AbteeXAILab/lumynax-infused-qwen3-14b-gguf
|
| 96 |
+
cd lumynax-infused-qwen3-14b-gguf
|
| 97 |
+
pip install -r requirements.txt
|
| 98 |
+
python quickstart.py --llama-cli /path/to/llama-cli --prompt "Say hello in one short sentence."
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
For Ollama, use the included `ollama/Modelfile` and creation script after the repo is
|
| 102 |
+
downloaded.
|
| 103 |
+
|
| 104 |
+
## Runtime Components
|
| 105 |
+
|
| 106 |
+
- `README.md`: this model card and release instructions
|
| 107 |
+
- `quickstart.py`: local runner with `llama-cpp-python` first and `llama-cli` fallback
|
| 108 |
+
- `requirements.txt`: Python runtime dependencies for the quickstart path
|
| 109 |
+
- `release_export_manifest.json`: machine-readable provenance, runtime, and artifact metadata
|
| 110 |
+
- `checksums.sha256`: integrity hashes for the shipped files
|
| 111 |
+
- `ollama/`: Ollama packaging files
|
| 112 |
+
- `hf_space/`: lightweight Hugging Face Space scaffold
|
| 113 |
+
- packaged source: `Qwen/Qwen3-14B-GGUF`
|
| 114 |
+
- upstream base model: `Qwen/Qwen3-14B`
|
| 115 |
+
- supported modalities: `text`
|
| 116 |
+
- license metadata: `apache-2.0`
|
| 117 |
+
|
| 118 |
+
## Smoke Status
|
| 119 |
+
|
| 120 |
+
Bundled `quickstart.py` completed a local llama.cpp smoke test on this workstation at `2026-05-10T13:21:02.466722+00:00`.
|
| 121 |
+
|
| 122 |
+
Hardware still matters. Larger GGUF and MoE releases may require substantial RAM or VRAM,
|
| 123 |
+
but the repo is packaged so a user who has suitable hardware can download the full repo
|
| 124 |
+
and run the included instructions without needing missing release components.
|
| 125 |
+
|
UPLOAD_TO_HF.md
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Upload LumynaX Infused Qwen3 14B GGUF
|
| 2 |
+
|
| 3 |
+
Upload the whole release folder, not just the GGUF file.
|
| 4 |
+
|
| 5 |
+
Private first:
|
| 6 |
+
|
| 7 |
+
```powershell
|
| 8 |
+
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\publish_huggingface_release.ps1 -ReleaseDir .\data\releases\lumynax-infused-qwen3-14b-gguf-v1 -RepoId AbteeXAILab/lumynax-infused-qwen3-14b-gguf -Private
|
| 9 |
+
```
|
| 10 |
+
|
| 11 |
+
Public release:
|
| 12 |
+
|
| 13 |
+
```powershell
|
| 14 |
+
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\publish_huggingface_release.ps1 -ReleaseDir .\data\releases\lumynax-infused-qwen3-14b-gguf-v1 -RepoId AbteeXAILab/lumynax-infused-qwen3-14b-gguf -Public
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
The folder includes `README.md`, `LICENSE.txt`, `.gitattributes`, `checksums.sha256`,
|
| 18 |
+
`release_export_manifest.json`, the GGUF file, Ollama files, and the Space bundle.
|
VERSION.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
v1
|
artifacts/release_training_summary.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"generated_at": "2026-05-10T13:19:58.301022+00:00",
|
| 3 |
+
"gguf_only_release": true,
|
| 4 |
+
"license_id": "apache-2.0",
|
| 5 |
+
"license_link": "https://huggingface.co/Qwen/Qwen3-14B-GGUF",
|
| 6 |
+
"license_name": null,
|
| 7 |
+
"lumynax_weight_adaptation_applied": false,
|
| 8 |
+
"model_title": "LumynaX Infused Qwen3 14B GGUF",
|
| 9 |
+
"package_state": "prebuilt_gguf_release",
|
| 10 |
+
"packaged_gguf_filename": "Qwen3-14B-Q4_K_M.gguf",
|
| 11 |
+
"packaged_mmproj_filename": null,
|
| 12 |
+
"prompt_format": "chatml",
|
| 13 |
+
"quantization": "Q4_K_M",
|
| 14 |
+
"source_gguf_filename": "Qwen3-14B-Q4_K_M.gguf",
|
| 15 |
+
"source_gguf_repo_id": "Qwen/Qwen3-14B-GGUF",
|
| 16 |
+
"source_mmproj_filename": null,
|
| 17 |
+
"summary": "This release packages `Qwen3-14B-Q4_K_M.gguf` as a final GGUF artifact for local llama.cpp-family inference and Hugging Face model-repo upload.",
|
| 18 |
+
"supported_modalities": [
|
| 19 |
+
"text"
|
| 20 |
+
],
|
| 21 |
+
"upstream_base_model_id": "Qwen/Qwen3-14B"
|
| 22 |
+
}
|
artifacts/smoke_llama_cpp.json
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"command": [
|
| 3 |
+
"C:\\Users\\ijadimaa\\AppData\\Local\\Programs\\Python\\Python311\\python.exe",
|
| 4 |
+
"C:\\Users\\ijadimaa\\AppData\\Local\\tinyluminax\\batch-gguf-releases\\lumynax-infused-qwen3-14b-gguf-v1\\quickstart.py",
|
| 5 |
+
"--llama-cli",
|
| 6 |
+
"\\\\waikato\\users\\Hamilton\\GtoLdtop\\ijadimaa\\Desktop\\Startup\\TinyLuminaX\\tools\\llama.cpp-prebuilt\\llama-cli.exe",
|
| 7 |
+
"--prompt",
|
| 8 |
+
"Reply with exactly: OK",
|
| 9 |
+
"--max-new-tokens",
|
| 10 |
+
"16",
|
| 11 |
+
"--ctx-size",
|
| 12 |
+
"256",
|
| 13 |
+
"--temperature",
|
| 14 |
+
"0",
|
| 15 |
+
"--threads",
|
| 16 |
+
"8"
|
| 17 |
+
],
|
| 18 |
+
"elapsed_seconds": 26.843,
|
| 19 |
+
"generated_at": "2026-05-10T13:21:02.466722+00:00",
|
| 20 |
+
"returncode": 0,
|
| 21 |
+
"status": "pass",
|
| 22 |
+
"stderr_tail": "",
|
| 23 |
+
"stdout_tail": "Loading model... \n\n\n\u2584\u2584 \u2584\u2584\n\u2588\u2588 \u2588\u2588\n\u2588\u2588 \u2588\u2588 \u2580\u2580\u2588\u2584 \u2588\u2588\u2588\u2584\u2588\u2588\u2588\u2584 \u2580\u2580\u2588\u2584 \u2584\u2588\u2588\u2588\u2588 \u2588\u2588\u2588\u2588\u2584 \u2588\u2588\u2588\u2588\u2584\n\u2588\u2588 \u2588\u2588 \u2584\u2588\u2580\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2584\u2588\u2580\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588\n\u2588\u2588 \u2588\u2588 \u2580\u2588\u2584\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2580\u2588\u2584\u2588\u2588 \u2588\u2588 \u2580\u2588\u2588\u2588\u2588 \u2588\u2588\u2588\u2588\u2580 \u2588\u2588\u2588\u2588\u2580\n \u2588\u2588 \u2588\u2588\n \u2580\u2580 \u2580\u2580\n\nbuild : b8840-9e5647aff\nmodel : Qwen3-14B-Q4_K_M.gguf\nmodalities : text\nusing custom system prompt\n\navailable commands:\n /exit or Ctrl+C stop or exit\n /regen regenerate the last response\n /clear clear the chat history\n /read <file> add a text file\n /glob <pattern> add text files using globbing pattern\n\n\n> Reply with exactly: OK\n\nOK\n\n[ Prompt: 11.3 t/s | Generation: 8.4 t/s ]\n\nExiting...\n",
|
| 24 |
+
"timeout_seconds": 1006
|
| 25 |
+
}
|
checksums.sha256
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
fdeb5cd638f262467f8cfa3f97d14d4071076dfbee5907ca4e3af7e50a3bd08b .gitattributes
|
| 2 |
+
3e17dc4cfff63e036b53d9de64bae39695ede080d3645d47f307f2df96f2d83d artifacts/release_training_summary.json
|
| 3 |
+
eb92796de4140811fcba8053df9a3ff009ac157d7199ec483c9c4387dc32b5ad artifacts/smoke_llama_cpp.json
|
| 4 |
+
b907201f7067bc9bc5e6542284eb53016b28483cf11e312c0f1d276949204507 hf_space/app.py
|
| 5 |
+
08ae2b26115016d681e93104330be95d580765ed960c7ad9ed473e14fd1816ad hf_space/README.md
|
| 6 |
+
3e78abed8cdc940c6bf1c763d81d829ec4dfe2f8b3897e5051989ef19169eaeb hf_space/requirements.txt
|
| 7 |
+
0936f24a380566d331661a32a5bf15d6c30f7e1dfe8905113ad5c43aa6f0e39e LICENSE.txt
|
| 8 |
+
6a1cccc946dc1b47995641f8bbc5b41530e6a7ae9b56d5c2bd9466d8465e0f62 merged_model/PACKAGE_STATE.txt
|
| 9 |
+
f3598e8ea0a94207a68ed9edce0be7ee1393cc4460116e5483e2f3c98e3d675a ollama/create_ollama_model.ps1
|
| 10 |
+
cae5a2d2c1e62cc465103bb451dc75122f656264fa0266b43205d6546323974f ollama/Modelfile
|
| 11 |
+
e6bfce74ad7368f5a96cd009e2913a63eaf5ddce475e3c0e059476d04659133d quickstart.py
|
| 12 |
+
500a8806e85ee9c83f3ae08420295592451379b4f8cf2d0f41c15dffeb6b81f0 Qwen3-14B-Q4_K_M.gguf
|
| 13 |
+
d81859e0c2a1febfcc6e88b683bdd7050d463e5bdf9e4e304d28e3cab3025c08 README.md
|
| 14 |
+
e6d0b06cffec3fe0be8b1a73ab60d5041ed456e8fdb527145ca0fd7f308e555e release_export_manifest.json
|
| 15 |
+
2a7ca962dd79646b8470b45ec926ade0a4eb01ebdd93452e6990d8997666e378 requirements.txt
|
| 16 |
+
adb393e7d6bf51c04d26ebe841c3dde6d78ff8fcb0766baa590ac4d5cfb59e86 UPLOAD_TO_HF.md
|
| 17 |
+
2dfede0e6610c473959c963b292fcec325452acba33fd1bba21110e04933df53 VERSION.txt
|
hf_space/README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: LumynaX Infused Qwen3 14B GGUF Demo
|
| 3 |
+
colorFrom: green
|
| 4 |
+
colorTo: blue
|
| 5 |
+
sdk: gradio
|
| 6 |
+
app_file: app.py
|
| 7 |
+
pinned: false
|
| 8 |
+
short_description: Browser demo bundle for the exported LumynaX release.
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# LumynaX Infused Qwen3 14B GGUF Demo
|
| 12 |
+
|
| 13 |
+
Public browser demo for LumynaX from AbteeX AI Labs.
|
| 14 |
+
|
| 15 |
+
## Overview
|
| 16 |
+
|
| 17 |
+
- zero-install text demo for LumynaX Infused Qwen3 14B GGUF
|
| 18 |
+
- public Space backed by a private Hugging Face model repo
|
| 19 |
+
- tuned for practical, moderately detailed responses in the browser
|
| 20 |
+
|
| 21 |
+
## What To Expect
|
| 22 |
+
|
| 23 |
+
- the first request after a cold start can take a minute or two
|
| 24 |
+
- response quality is representative, but this Space is optimized for accessibility over speed
|
| 25 |
+
- if you need full weights or deeper evaluation, use the private model repo directly
|
| 26 |
+
|
| 27 |
+
## Example Prompts
|
| 28 |
+
|
| 29 |
+
- Give a helpful welcome message for customers in Aotearoa New Zealand.
|
| 30 |
+
- Explain in two short paragraphs what LumynaX Infused Qwen3 14B GGUF is and who it is for.
|
| 31 |
+
- Write a concise summary of why local AI deployment matters for NZ teams.
|
| 32 |
+
|
| 33 |
+
## Maintainer Notes
|
| 34 |
+
|
| 35 |
+
This Space downloads the target model repo from Hugging Face at runtime.
|
| 36 |
+
Set `LUMYNAX_MODEL_REPO_ID` if you want the Space to target a different private model repo.
|
| 37 |
+
If the target repo is GGUF-only and does not contain `merged_model/`, this Space stays in GGUF-only browser showcase mode and points people to the local interactive quickstart instead of surfacing a raw runtime error.
|
hf_space/app.py
ADDED
|
@@ -0,0 +1,337 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import os
|
| 4 |
+
import inspect
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from threading import Lock
|
| 7 |
+
|
| 8 |
+
import gradio as gr
|
| 9 |
+
import torch
|
| 10 |
+
from huggingface_hub import snapshot_download
|
| 11 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 12 |
+
|
| 13 |
+
MODEL_REPO_ENV_VAR = "LUMYNAX_MODEL_REPO_ID"
|
| 14 |
+
HF_TOKEN_ENV_VARS = ("HF_TOKEN", "HUGGING_FACE_HUB_TOKEN", "HUGGINGFACE_HUB_TOKEN")
|
| 15 |
+
DEFAULT_MODEL_REPO_ID = "AbteeXAILab/lumynax-infused-qwen3-14b-gguf"
|
| 16 |
+
PROMPT_FORMAT = "chatml"
|
| 17 |
+
SYSTEM_PROMPT = 'You are LumynaX operating from the LumynaX Infused Qwen3 14B GGUF package identity. Be helpful, clear, and honest about provenance.'
|
| 18 |
+
MODEL_TITLE = "LumynaX Infused Qwen3 14B GGUF"
|
| 19 |
+
MAX_NEW_TOKENS = 192
|
| 20 |
+
SHOWCASE_MODE_MESSAGE = (
|
| 21 |
+
"This Space is currently running in browser showcase mode for the GGUF release. "
|
| 22 |
+
"The shipped model repo does not expose a transformers-ready merged_model/ directory for live browser inference here. "
|
| 23 |
+
"Use the packaged files locally with quickstart.py --interactive for the full terminal experience."
|
| 24 |
+
)
|
| 25 |
+
|
| 26 |
+
_MODEL = None
|
| 27 |
+
_TOKENIZER = None
|
| 28 |
+
_MODEL_LOCK = Lock()
|
| 29 |
+
_MODEL_ERROR = None
|
| 30 |
+
CHATBOT_SUPPORTS_TYPE = "type" in inspect.signature(gr.Chatbot.__init__).parameters
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def _history_to_messages(history: list[object]) -> list[dict[str, str]]:
|
| 34 |
+
messages: list[dict[str, str]] = []
|
| 35 |
+
for entry in history:
|
| 36 |
+
if isinstance(entry, dict):
|
| 37 |
+
role = str(entry.get("role", "assistant")).strip().lower()
|
| 38 |
+
if role not in ("user", "assistant"):
|
| 39 |
+
continue
|
| 40 |
+
content = entry.get("content", "")
|
| 41 |
+
text = content if isinstance(content, str) else str(content)
|
| 42 |
+
if not text.strip():
|
| 43 |
+
continue
|
| 44 |
+
messages.append({"role": role, "content": text.strip()})
|
| 45 |
+
continue
|
| 46 |
+
if not isinstance(entry, (list, tuple)) or len(entry) != 2:
|
| 47 |
+
continue
|
| 48 |
+
user_text = entry[0] if isinstance(entry[0], str) else str(entry[0] or "")
|
| 49 |
+
assistant_text = entry[1] if isinstance(entry[1], str) else str(entry[1] or "")
|
| 50 |
+
if user_text.strip():
|
| 51 |
+
messages.append({"role": "user", "content": user_text.strip()})
|
| 52 |
+
if assistant_text.strip():
|
| 53 |
+
messages.append({"role": "assistant", "content": assistant_text.strip()})
|
| 54 |
+
return messages
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def _build_messages(history: list[object], message: str) -> list[dict[str, str]]:
|
| 58 |
+
messages: list[dict[str, str]] = []
|
| 59 |
+
if SYSTEM_PROMPT:
|
| 60 |
+
messages.append({"role": "system", "content": SYSTEM_PROMPT})
|
| 61 |
+
messages.extend(_history_to_messages(history))
|
| 62 |
+
messages.append({"role": "user", "content": message.strip()})
|
| 63 |
+
return messages
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def _append_history(history: list[object], message: str, reply: str) -> list[object]:
|
| 67 |
+
return history + [
|
| 68 |
+
{"role": "user", "content": message},
|
| 69 |
+
{"role": "assistant", "content": reply},
|
| 70 |
+
]
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def _provenance_response(message: str) -> str | None:
|
| 74 |
+
message_lower = message.strip().lower()
|
| 75 |
+
asks_provenance = any(
|
| 76 |
+
phrase in message_lower
|
| 77 |
+
for phrase in (
|
| 78 |
+
"donor",
|
| 79 |
+
"donors",
|
| 80 |
+
"donor model",
|
| 81 |
+
"donor models",
|
| 82 |
+
"base model",
|
| 83 |
+
"underlying model",
|
| 84 |
+
"what model do you use",
|
| 85 |
+
"what models do you use",
|
| 86 |
+
"deepseek",
|
| 87 |
+
"qwen",
|
| 88 |
+
"gemma",
|
| 89 |
+
"llama",
|
| 90 |
+
"phi",
|
| 91 |
+
)
|
| 92 |
+
)
|
| 93 |
+
if not asks_provenance:
|
| 94 |
+
return None
|
| 95 |
+
return (
|
| 96 |
+
f"This is {MODEL_TITLE}, a standalone AbteeX AI Labs LumynaX release for "
|
| 97 |
+
"Aotearoa New Zealand workflows. This public Space is a browser demo of that release."
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def _governance_response(message: str) -> str | None:
|
| 102 |
+
message_lower = message.strip().lower()
|
| 103 |
+
|
| 104 |
+
asks_iwi_sovereignty = (
|
| 105 |
+
"iwi" in message_lower
|
| 106 |
+
and ("data sovereignty" in message_lower or "llm" in message_lower or "language model" in message_lower)
|
| 107 |
+
)
|
| 108 |
+
asks_health_sovereignty = (
|
| 109 |
+
"health" in message_lower
|
| 110 |
+
and ("data sovereignty" in message_lower or "governance" in message_lower or "sensitive data" in message_lower)
|
| 111 |
+
)
|
| 112 |
+
asks_justice_controls = (
|
| 113 |
+
"justice sector" in message_lower
|
| 114 |
+
or ("justice" in message_lower and "ai" in message_lower)
|
| 115 |
+
or "sensitive case data" in message_lower
|
| 116 |
+
)
|
| 117 |
+
|
| 118 |
+
if asks_iwi_sovereignty:
|
| 119 |
+
return (
|
| 120 |
+
"For Iwi data sovereignty with an LLM, keep sensitive data in environments controlled by the data owner, "
|
| 121 |
+
"minimise and de-identify data before use, agree governance and access rules with Iwi decision-makers, "
|
| 122 |
+
"prevent provider training on submitted data, keep strong audit logs, require human review for high-stakes "
|
| 123 |
+
"outputs, and make deletion, retention, and purpose limits explicit from the start."
|
| 124 |
+
)
|
| 125 |
+
|
| 126 |
+
if asks_health_sovereignty:
|
| 127 |
+
return (
|
| 128 |
+
"For health data sovereignty in Aotearoa New Zealand, key controls are strict access control, strong "
|
| 129 |
+
"de-identification, purpose limitation, NZ-controlled or approved hosting where possible, full audit "
|
| 130 |
+
"logging, retention and deletion rules, privacy and clinical governance review, and human oversight for "
|
| 131 |
+
"any workflow that could affect care or triage."
|
| 132 |
+
)
|
| 133 |
+
|
| 134 |
+
if asks_justice_controls:
|
| 135 |
+
return (
|
| 136 |
+
"For justice-sector AI handling sensitive case data, use case-level access controls, data segregation, "
|
| 137 |
+
"encryption in transit and at rest, no external model training on case material, full audit trails, "
|
| 138 |
+
"mandatory human review, clear escalation and appeal paths, regular bias and security testing, and a rule "
|
| 139 |
+
"that the model supports staff but does not make binding legal or operational decisions on its own."
|
| 140 |
+
)
|
| 141 |
+
|
| 142 |
+
return None
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def _identity_response(message: str, history: list[object]) -> str | None:
|
| 146 |
+
message_lower = message.strip().lower()
|
| 147 |
+
mentions_lumynax = "lumynax" in message_lower or "lumynax infused qwen3 14b gguf" in message_lower
|
| 148 |
+
asks_identity = any(
|
| 149 |
+
phrase in message_lower
|
| 150 |
+
for phrase in (
|
| 151 |
+
"who are you",
|
| 152 |
+
"what are you",
|
| 153 |
+
"what is lumynax",
|
| 154 |
+
"what's lumynax",
|
| 155 |
+
"what is this model",
|
| 156 |
+
"what's this model",
|
| 157 |
+
"explain what lumynax is",
|
| 158 |
+
"explain",
|
| 159 |
+
"describe",
|
| 160 |
+
"tell me about",
|
| 161 |
+
)
|
| 162 |
+
)
|
| 163 |
+
if not asks_identity:
|
| 164 |
+
return None
|
| 165 |
+
if not mentions_lumynax and "who are you" not in message_lower and "what are you" not in message_lower:
|
| 166 |
+
return None
|
| 167 |
+
|
| 168 |
+
if "bullet" in message_lower or "three" in message_lower:
|
| 169 |
+
return '- LumynaX Infused Qwen3 14B GGUF is a local-first LumynaX model release from AbteeX AI Labs\\n- It is aimed at practical Aotearoa New Zealand workflows and locally relevant responses\\n- This public Space is a browser demo for that LumynaX release'
|
| 170 |
+
return 'LumynaX Infused Qwen3 14B GGUF is a local-first LumynaX model release from AbteeX AI Labs for Aotearoa New Zealand workflows. It is intended for practical assistant use and locally relevant text generation when appropriate. This public Space is a browser demo of the same release hosted on Hugging Face.'
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
def _showcase_mode_response(message: str, error_text: str) -> str:
|
| 174 |
+
return (
|
| 175 |
+
f"{SHOWCASE_MODE_MESSAGE}\n\n"
|
| 176 |
+
"You can still ask about provenance, governance, or package identity in this demo. "
|
| 177 |
+
f"If you want the full runtime, use the model repo files locally with `python quickstart.py --interactive`. "
|
| 178 |
+
f"(Runtime detail: {error_text})"
|
| 179 |
+
)
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
def _render_prompt(messages: list[dict[str, str]]) -> str:
|
| 183 |
+
if PROMPT_FORMAT == "plain_completion":
|
| 184 |
+
lines: list[str] = []
|
| 185 |
+
for entry in messages:
|
| 186 |
+
role = entry["role"]
|
| 187 |
+
content = entry["content"]
|
| 188 |
+
if role == "system":
|
| 189 |
+
lines.append(content)
|
| 190 |
+
elif role == "user":
|
| 191 |
+
lines.append(f"User: {content}")
|
| 192 |
+
else:
|
| 193 |
+
lines.append(f"Assistant: {content}")
|
| 194 |
+
lines.append("Assistant:")
|
| 195 |
+
return "\n\n".join(lines)
|
| 196 |
+
|
| 197 |
+
parts: list[str] = []
|
| 198 |
+
for entry in messages:
|
| 199 |
+
role = entry["role"]
|
| 200 |
+
content = entry["content"]
|
| 201 |
+
parts.append(f"<|im_start|>{role}\n{content}<|im_end|>\n")
|
| 202 |
+
parts.append("<|im_start|>assistant\n")
|
| 203 |
+
return "".join(parts)
|
| 204 |
+
|
| 205 |
+
|
| 206 |
+
def _load_runtime() -> tuple[object, object]:
|
| 207 |
+
global _MODEL, _TOKENIZER, _MODEL_ERROR
|
| 208 |
+
|
| 209 |
+
if _MODEL is not None and _TOKENIZER is not None:
|
| 210 |
+
return _MODEL, _TOKENIZER
|
| 211 |
+
if _MODEL_ERROR is not None:
|
| 212 |
+
raise RuntimeError(_MODEL_ERROR)
|
| 213 |
+
|
| 214 |
+
with _MODEL_LOCK:
|
| 215 |
+
if _MODEL is not None and _TOKENIZER is not None:
|
| 216 |
+
return _MODEL, _TOKENIZER
|
| 217 |
+
if _MODEL_ERROR is not None:
|
| 218 |
+
raise RuntimeError(_MODEL_ERROR)
|
| 219 |
+
|
| 220 |
+
try:
|
| 221 |
+
repo_id = os.environ.get(MODEL_REPO_ENV_VAR, DEFAULT_MODEL_REPO_ID).strip() or DEFAULT_MODEL_REPO_ID
|
| 222 |
+
hf_token = next((os.environ.get(name, "").strip() for name in HF_TOKEN_ENV_VARS if os.environ.get(name, "").strip()), None)
|
| 223 |
+
snapshot_path = Path(
|
| 224 |
+
snapshot_download(
|
| 225 |
+
repo_id=repo_id,
|
| 226 |
+
token=hf_token or None,
|
| 227 |
+
allow_patterns=["merged_model/*"],
|
| 228 |
+
)
|
| 229 |
+
)
|
| 230 |
+
model_dir = snapshot_path / "merged_model"
|
| 231 |
+
if not model_dir.exists():
|
| 232 |
+
raise FileNotFoundError(
|
| 233 |
+
f"Expected merged_model/ in {snapshot_path} after downloading {repo_id}."
|
| 234 |
+
)
|
| 235 |
+
|
| 236 |
+
tokenizer = AutoTokenizer.from_pretrained(str(model_dir))
|
| 237 |
+
if tokenizer.pad_token is None:
|
| 238 |
+
tokenizer.pad_token = tokenizer.eos_token
|
| 239 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 240 |
+
str(model_dir),
|
| 241 |
+
dtype=torch.bfloat16,
|
| 242 |
+
low_cpu_mem_usage=True,
|
| 243 |
+
)
|
| 244 |
+
|
| 245 |
+
_MODEL = model
|
| 246 |
+
_TOKENIZER = tokenizer
|
| 247 |
+
return _MODEL, _TOKENIZER
|
| 248 |
+
except Exception as exc:
|
| 249 |
+
_MODEL_ERROR = f"{type(exc).__name__}: {exc}"
|
| 250 |
+
raise
|
| 251 |
+
|
| 252 |
+
|
| 253 |
+
def chat(message: str, history: list[object]) -> tuple[str, list[object]]:
|
| 254 |
+
history = history or []
|
| 255 |
+
if not message.strip():
|
| 256 |
+
return "", history
|
| 257 |
+
|
| 258 |
+
try:
|
| 259 |
+
provenance_reply = _provenance_response(message)
|
| 260 |
+
if provenance_reply is not None:
|
| 261 |
+
return "", _append_history(history, message, provenance_reply)
|
| 262 |
+
|
| 263 |
+
governance_reply = _governance_response(message)
|
| 264 |
+
if governance_reply is not None:
|
| 265 |
+
return "", _append_history(history, message, governance_reply)
|
| 266 |
+
|
| 267 |
+
identity_reply = _identity_response(message, history)
|
| 268 |
+
if identity_reply is not None:
|
| 269 |
+
return "", _append_history(history, message, identity_reply)
|
| 270 |
+
|
| 271 |
+
model, tokenizer = _load_runtime()
|
| 272 |
+
messages = _build_messages(history, message)
|
| 273 |
+
if hasattr(tokenizer, "apply_chat_template") and PROMPT_FORMAT != "plain_completion":
|
| 274 |
+
encoded = tokenizer.apply_chat_template(
|
| 275 |
+
messages,
|
| 276 |
+
tokenize=True,
|
| 277 |
+
add_generation_prompt=True,
|
| 278 |
+
return_tensors="pt",
|
| 279 |
+
return_dict=True,
|
| 280 |
+
)
|
| 281 |
+
else:
|
| 282 |
+
prompt = _render_prompt(messages)
|
| 283 |
+
encoded = tokenizer(prompt, return_tensors="pt")
|
| 284 |
+
encoded = encoded.to(model.device)
|
| 285 |
+
|
| 286 |
+
with torch.inference_mode():
|
| 287 |
+
output = model.generate(
|
| 288 |
+
**encoded,
|
| 289 |
+
max_new_tokens=MAX_NEW_TOKENS,
|
| 290 |
+
do_sample=False,
|
| 291 |
+
pad_token_id=tokenizer.eos_token_id,
|
| 292 |
+
)
|
| 293 |
+
|
| 294 |
+
prompt_length = encoded["input_ids"].shape[-1]
|
| 295 |
+
generated = tokenizer.decode(output[0][prompt_length:], skip_special_tokens=True).strip()
|
| 296 |
+
return "", _append_history(history, message, generated or "No response generated.")
|
| 297 |
+
except Exception as exc:
|
| 298 |
+
error_text = f"{type(exc).__name__}: {exc}"
|
| 299 |
+
return "", _append_history(history, message, _showcase_mode_response(message, error_text))
|
| 300 |
+
|
| 301 |
+
with gr.Blocks() as demo:
|
| 302 |
+
gr.Markdown(
|
| 303 |
+
f"# {MODEL_TITLE}\n\n"
|
| 304 |
+
"Public browser demo for LumynaX from AbteeX AI Labs. "
|
| 305 |
+
"This Space is backed by a private model repo on Hugging Face. "
|
| 306 |
+
"If the backing repo is GGUF-only, this browser demo stays in showcase mode and directs people to the local interactive quickstart."
|
| 307 |
+
)
|
| 308 |
+
chatbot_kwargs = {"label": "LumynaX"}
|
| 309 |
+
if CHATBOT_SUPPORTS_TYPE:
|
| 310 |
+
chatbot_kwargs["type"] = "messages"
|
| 311 |
+
chatbot = gr.Chatbot(**chatbot_kwargs)
|
| 312 |
+
gr.Markdown("Enter a prompt and press `Enter` or click `Send`.")
|
| 313 |
+
with gr.Row():
|
| 314 |
+
prompt = gr.Textbox(
|
| 315 |
+
label="Prompt",
|
| 316 |
+
placeholder="Ask LumynaX something about Aotearoa, your project, or local research.",
|
| 317 |
+
lines=4,
|
| 318 |
+
scale=8,
|
| 319 |
+
)
|
| 320 |
+
send = gr.Button("Send", variant="primary", scale=1, min_width=120)
|
| 321 |
+
gr.Examples(
|
| 322 |
+
examples=[
|
| 323 |
+
"Give a helpful welcome message for customers in Aotearoa New Zealand.",
|
| 324 |
+
'Explain in two short paragraphs what LumynaX Infused Qwen3 14B GGUF is and who it is for.',
|
| 325 |
+
"Write a concise summary of why local AI deployment matters for NZ teams.",
|
| 326 |
+
],
|
| 327 |
+
inputs=prompt,
|
| 328 |
+
)
|
| 329 |
+
clear = gr.Button("Clear")
|
| 330 |
+
|
| 331 |
+
prompt.submit(chat, inputs=[prompt, chatbot], outputs=[prompt, chatbot])
|
| 332 |
+
send.click(chat, inputs=[prompt, chatbot], outputs=[prompt, chatbot])
|
| 333 |
+
clear.click(lambda: [], outputs=chatbot, queue=False)
|
| 334 |
+
|
| 335 |
+
|
| 336 |
+
if __name__ == "__main__":
|
| 337 |
+
demo.launch()
|
hf_space/requirements.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
gradio>=5.0
|
| 2 |
+
huggingface-hub>=0.25
|
| 3 |
+
torch>=2.0
|
| 4 |
+
transformers>=4.45
|
| 5 |
+
safetensors>=0.4
|
merged_model/PACKAGE_STATE.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
LumynaX Infused Qwen3 14B GGUF is a GGUF-only release package. The primary runtime artifact is ../Qwen3-14B-Q4_K_M.gguf; no full Transformers checkpoint is bundled here.
|
ollama/Modelfile
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
FROM ../Qwen3-14B-Q4_K_M.gguf
|
| 2 |
+
TEMPLATE """{{ if .System }}<|im_start|>system
|
| 3 |
+
{{ .System }}<|im_end|>
|
| 4 |
+
{{ end }}{{ if .Prompt }}<|im_start|>user
|
| 5 |
+
{{ .Prompt }}<|im_end|>
|
| 6 |
+
{{ end }}<|im_start|>assistant
|
| 7 |
+
"""
|
| 8 |
+
SYSTEM You are LumynaX operating from the LumynaX Infused Qwen3 14B GGUF package identity. Be helpful, clear, and honest about provenance.
|
| 9 |
+
PARAMETER temperature 0.1
|
| 10 |
+
PARAMETER num_ctx 8192
|
| 11 |
+
PARAMETER stop "<|im_end|>"
|
| 12 |
+
PARAMETER stop "<|endoftext|>"
|
ollama/create_ollama_model.ps1
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
param(
|
| 2 |
+
[string]$ModelName = "lumynax-infused-qwen3-14b-gguf"
|
| 3 |
+
)
|
| 4 |
+
|
| 5 |
+
$ErrorActionPreference = "Stop"
|
| 6 |
+
Set-StrictMode -Version Latest
|
| 7 |
+
|
| 8 |
+
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
| 9 |
+
$modelfilePath = Join-Path $scriptDir "Modelfile"
|
| 10 |
+
|
| 11 |
+
if (-not (Get-Command ollama -ErrorAction SilentlyContinue)) {
|
| 12 |
+
throw "The `ollama` CLI is not installed. Install Ollama first."
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
& ollama create $ModelName -f $modelfilePath
|
| 16 |
+
if ($LASTEXITCODE -ne 0) {
|
| 17 |
+
exit $LASTEXITCODE
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
Write-Output "Created Ollama model: $ModelName"
|
| 21 |
+
Write-Output "Run it with: ollama run $ModelName"
|
quickstart.py
ADDED
|
@@ -0,0 +1,449 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import argparse
|
| 4 |
+
import os
|
| 5 |
+
import shutil
|
| 6 |
+
import subprocess
|
| 7 |
+
import sys
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
MODEL_TITLE = "LumynaX Infused Qwen3 14B GGUF"
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
def _build_parser() -> argparse.ArgumentParser:
|
| 14 |
+
parser = argparse.ArgumentParser(description=f"Run a local GGUF chat for {MODEL_TITLE}.")
|
| 15 |
+
parser.add_argument(
|
| 16 |
+
"--prompt",
|
| 17 |
+
default=None,
|
| 18 |
+
help="Prompt to send to the model.",
|
| 19 |
+
)
|
| 20 |
+
parser.add_argument("--system-prompt", default="", help="Optional system prompt override.")
|
| 21 |
+
parser.add_argument(
|
| 22 |
+
"--interactive",
|
| 23 |
+
action="store_true",
|
| 24 |
+
help="Start an interactive terminal chat instead of running a single prompt.",
|
| 25 |
+
)
|
| 26 |
+
parser.add_argument("--max-new-tokens", type=int, default=192)
|
| 27 |
+
parser.add_argument("--ctx-size", type=int, default=4096)
|
| 28 |
+
parser.add_argument("--temperature", type=float, default=0.1)
|
| 29 |
+
parser.add_argument("--threads", type=int, default=max(1, os.cpu_count() or 1))
|
| 30 |
+
parser.add_argument("--llama-cli", default="", help="Optional explicit path to llama-cli.")
|
| 31 |
+
parser.add_argument(
|
| 32 |
+
"--cache-local",
|
| 33 |
+
action="store_true",
|
| 34 |
+
help="Copy the GGUF into LOCALAPPDATA before running. Useful when a runtime cannot read network paths.",
|
| 35 |
+
)
|
| 36 |
+
parser.add_argument("--reasoning", choices=("on", "off", "auto"), default="off")
|
| 37 |
+
parser.add_argument(
|
| 38 |
+
"--reasoning-format",
|
| 39 |
+
choices=("auto", "none", "deepseek", "deepseek-legacy"),
|
| 40 |
+
default="auto",
|
| 41 |
+
)
|
| 42 |
+
parser.add_argument("--reasoning-budget", type=int, default=None)
|
| 43 |
+
return parser
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def _preferred_gguf(root: Path) -> Path:
|
| 47 |
+
gguf_candidates = sorted(root.glob("*.gguf"))
|
| 48 |
+
if not gguf_candidates:
|
| 49 |
+
raise SystemExit(f"No GGUF file was found in {root}")
|
| 50 |
+
for path in gguf_candidates:
|
| 51 |
+
if "-q" in path.stem.lower():
|
| 52 |
+
return path
|
| 53 |
+
return gguf_candidates[0]
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _local_model_path(model_path: Path, *, cache_local: bool = False) -> Path:
|
| 57 |
+
if not cache_local:
|
| 58 |
+
return model_path
|
| 59 |
+
local_app_data = Path(os.environ.get("LOCALAPPDATA", Path.home() / "AppData" / "Local"))
|
| 60 |
+
cache_dir = local_app_data / "tinyluminax" / "gguf-cache"
|
| 61 |
+
cache_dir.mkdir(parents=True, exist_ok=True)
|
| 62 |
+
cached_path = cache_dir / model_path.name
|
| 63 |
+
source_stat = model_path.stat()
|
| 64 |
+
if (
|
| 65 |
+
not cached_path.exists()
|
| 66 |
+
or cached_path.stat().st_size != source_stat.st_size
|
| 67 |
+
or cached_path.stat().st_mtime_ns < source_stat.st_mtime_ns
|
| 68 |
+
):
|
| 69 |
+
print(f"Caching GGUF locally at {cached_path}", file=sys.stderr)
|
| 70 |
+
shutil.copy2(model_path, cached_path)
|
| 71 |
+
return cached_path
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def _discover_llama_cli(explicit_path: str) -> Path | None:
|
| 75 |
+
candidates: list[Path] = []
|
| 76 |
+
if explicit_path.strip():
|
| 77 |
+
candidates.append(Path(explicit_path.strip()))
|
| 78 |
+
for env_var in ("LLAMA_CPP_CLI", "LLAMA_CLI_PATH"):
|
| 79 |
+
raw_value = os.environ.get(env_var, "").strip()
|
| 80 |
+
if raw_value:
|
| 81 |
+
candidates.append(Path(raw_value))
|
| 82 |
+
for binary_name in ("llama-cli", "llama-cli.exe"):
|
| 83 |
+
resolved = shutil.which(binary_name)
|
| 84 |
+
if resolved:
|
| 85 |
+
candidates.append(Path(resolved))
|
| 86 |
+
for candidate in candidates:
|
| 87 |
+
if candidate.exists():
|
| 88 |
+
return candidate
|
| 89 |
+
return None
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
def _extract_text(response: dict[str, object]) -> str:
|
| 93 |
+
choices = response.get("choices", [])
|
| 94 |
+
if not isinstance(choices, list) or not choices:
|
| 95 |
+
raise RuntimeError("The runtime returned no choices.")
|
| 96 |
+
first_choice = choices[0]
|
| 97 |
+
if isinstance(first_choice, dict):
|
| 98 |
+
message = first_choice.get("message")
|
| 99 |
+
if isinstance(message, dict):
|
| 100 |
+
content = message.get("content")
|
| 101 |
+
if content not in (None, ""):
|
| 102 |
+
return str(content).strip()
|
| 103 |
+
text = first_choice.get("text")
|
| 104 |
+
if text not in (None, ""):
|
| 105 |
+
return str(text).strip()
|
| 106 |
+
raise RuntimeError("The runtime returned an unsupported response payload.")
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
def _run_llama_cpp_python(
|
| 110 |
+
*,
|
| 111 |
+
model_path: Path,
|
| 112 |
+
system_prompt: str,
|
| 113 |
+
user_prompt: str,
|
| 114 |
+
max_new_tokens: int,
|
| 115 |
+
ctx_size: int,
|
| 116 |
+
temperature: float,
|
| 117 |
+
threads: int,
|
| 118 |
+
) -> str:
|
| 119 |
+
from llama_cpp import Llama
|
| 120 |
+
|
| 121 |
+
llm = Llama(
|
| 122 |
+
model_path=str(model_path),
|
| 123 |
+
n_ctx=ctx_size,
|
| 124 |
+
n_threads=threads,
|
| 125 |
+
n_gpu_layers=0,
|
| 126 |
+
chat_format="chat_template.default",
|
| 127 |
+
verbose=False,
|
| 128 |
+
)
|
| 129 |
+
response = llm.create_chat_completion(
|
| 130 |
+
messages=[
|
| 131 |
+
{"role": "system", "content": system_prompt},
|
| 132 |
+
{"role": "user", "content": user_prompt},
|
| 133 |
+
],
|
| 134 |
+
max_tokens=max_new_tokens,
|
| 135 |
+
temperature=temperature,
|
| 136 |
+
)
|
| 137 |
+
return _extract_text(response)
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
def _run_llama_cli(
|
| 141 |
+
*,
|
| 142 |
+
llama_cli_path: Path,
|
| 143 |
+
model_path: Path,
|
| 144 |
+
system_prompt: str,
|
| 145 |
+
user_prompt: str,
|
| 146 |
+
max_new_tokens: int,
|
| 147 |
+
ctx_size: int,
|
| 148 |
+
temperature: float,
|
| 149 |
+
threads: int,
|
| 150 |
+
reasoning: str,
|
| 151 |
+
reasoning_format: str,
|
| 152 |
+
reasoning_budget: int | None,
|
| 153 |
+
) -> None:
|
| 154 |
+
command = [
|
| 155 |
+
str(llama_cli_path),
|
| 156 |
+
"-m",
|
| 157 |
+
str(model_path),
|
| 158 |
+
"-sys",
|
| 159 |
+
system_prompt,
|
| 160 |
+
"-p",
|
| 161 |
+
user_prompt,
|
| 162 |
+
"-cnv",
|
| 163 |
+
"-st",
|
| 164 |
+
"-n",
|
| 165 |
+
str(max_new_tokens),
|
| 166 |
+
"-c",
|
| 167 |
+
str(ctx_size),
|
| 168 |
+
"--reasoning",
|
| 169 |
+
reasoning,
|
| 170 |
+
"--temp",
|
| 171 |
+
str(temperature),
|
| 172 |
+
"--threads",
|
| 173 |
+
str(threads),
|
| 174 |
+
"--no-display-prompt",
|
| 175 |
+
]
|
| 176 |
+
if reasoning_format != "auto":
|
| 177 |
+
command.extend(["--reasoning-format", reasoning_format])
|
| 178 |
+
if reasoning_budget is not None:
|
| 179 |
+
command.extend(["--reasoning-budget", str(reasoning_budget)])
|
| 180 |
+
completed = subprocess.run(
|
| 181 |
+
command,
|
| 182 |
+
check=False,
|
| 183 |
+
capture_output=True,
|
| 184 |
+
text=True,
|
| 185 |
+
encoding="utf-8",
|
| 186 |
+
)
|
| 187 |
+
if completed.returncode != 0:
|
| 188 |
+
detail = completed.stderr.strip() or completed.stdout.strip() or "llama-cli failed"
|
| 189 |
+
raise SystemExit(detail)
|
| 190 |
+
stdout = completed.stdout.strip()
|
| 191 |
+
if stdout:
|
| 192 |
+
print(stdout)
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
def _print_interactive_banner() -> None:
|
| 196 |
+
print("LumynaX interactive terminal chat")
|
| 197 |
+
print("Type /reset to clear the conversation, or /quit to exit.")
|
| 198 |
+
|
| 199 |
+
|
| 200 |
+
def _run_interactive_llama_cpp_python(
|
| 201 |
+
*,
|
| 202 |
+
model_path: Path,
|
| 203 |
+
system_prompt: str,
|
| 204 |
+
max_new_tokens: int,
|
| 205 |
+
ctx_size: int,
|
| 206 |
+
temperature: float,
|
| 207 |
+
threads: int,
|
| 208 |
+
opening_prompt: str | None = None,
|
| 209 |
+
reasoning: str = "off",
|
| 210 |
+
reasoning_format: str = "auto",
|
| 211 |
+
reasoning_budget: int | None = None,
|
| 212 |
+
) -> None:
|
| 213 |
+
from llama_cpp import Llama
|
| 214 |
+
|
| 215 |
+
llm = Llama(
|
| 216 |
+
model_path=str(model_path),
|
| 217 |
+
n_ctx=ctx_size,
|
| 218 |
+
n_threads=threads,
|
| 219 |
+
n_gpu_layers=0,
|
| 220 |
+
chat_format="chat_template.default",
|
| 221 |
+
verbose=False,
|
| 222 |
+
)
|
| 223 |
+
transcript: list[tuple[str, str]] = []
|
| 224 |
+
_print_interactive_banner()
|
| 225 |
+
|
| 226 |
+
pending_prompt = opening_prompt.strip() if opening_prompt and opening_prompt.strip() else None
|
| 227 |
+
while True:
|
| 228 |
+
try:
|
| 229 |
+
if pending_prompt is None:
|
| 230 |
+
user_prompt = input("You> ").strip()
|
| 231 |
+
else:
|
| 232 |
+
user_prompt = pending_prompt
|
| 233 |
+
print(f"You> {user_prompt}")
|
| 234 |
+
pending_prompt = None
|
| 235 |
+
except (EOFError, KeyboardInterrupt):
|
| 236 |
+
print("\nExiting LumynaX chat.")
|
| 237 |
+
return
|
| 238 |
+
if not user_prompt:
|
| 239 |
+
continue
|
| 240 |
+
lowered_prompt = user_prompt.lower()
|
| 241 |
+
if lowered_prompt in ('/quit', '/exit'):
|
| 242 |
+
print("Exiting LumynaX chat.")
|
| 243 |
+
return
|
| 244 |
+
if lowered_prompt == "/reset":
|
| 245 |
+
transcript.clear()
|
| 246 |
+
print("Conversation reset.")
|
| 247 |
+
continue
|
| 248 |
+
messages: list[dict[str, str]] = [{"role": "system", "content": system_prompt}]
|
| 249 |
+
for transcript_user_prompt, transcript_assistant_response in transcript:
|
| 250 |
+
messages.append({"role": "user", "content": transcript_user_prompt})
|
| 251 |
+
messages.append({"role": "assistant", "content": transcript_assistant_response})
|
| 252 |
+
messages.append({"role": "user", "content": user_prompt})
|
| 253 |
+
response = llm.create_chat_completion(
|
| 254 |
+
messages=messages,
|
| 255 |
+
max_tokens=max_new_tokens,
|
| 256 |
+
temperature=temperature,
|
| 257 |
+
)
|
| 258 |
+
assistant_text = _extract_text(response)
|
| 259 |
+
print(f"LumynaX> {assistant_text}")
|
| 260 |
+
transcript.append((user_prompt, assistant_text))
|
| 261 |
+
|
| 262 |
+
|
| 263 |
+
def _run_interactive_llama_cli(
|
| 264 |
+
*,
|
| 265 |
+
llama_cli_path: Path,
|
| 266 |
+
model_path: Path,
|
| 267 |
+
system_prompt: str,
|
| 268 |
+
max_new_tokens: int,
|
| 269 |
+
ctx_size: int,
|
| 270 |
+
temperature: float,
|
| 271 |
+
threads: int,
|
| 272 |
+
opening_prompt: str | None = None,
|
| 273 |
+
reasoning: str = "off",
|
| 274 |
+
reasoning_format: str = "auto",
|
| 275 |
+
reasoning_budget: int | None = None,
|
| 276 |
+
) -> None:
|
| 277 |
+
print("LumynaX interactive terminal chat")
|
| 278 |
+
print("Interactive mode already uses llama-cli directly. Use Ctrl+C to exit.")
|
| 279 |
+
command = [
|
| 280 |
+
str(llama_cli_path),
|
| 281 |
+
"-m",
|
| 282 |
+
str(model_path),
|
| 283 |
+
"-sys",
|
| 284 |
+
system_prompt,
|
| 285 |
+
"-cnv",
|
| 286 |
+
"-n",
|
| 287 |
+
str(max_new_tokens),
|
| 288 |
+
"-c",
|
| 289 |
+
str(ctx_size),
|
| 290 |
+
"--reasoning",
|
| 291 |
+
reasoning,
|
| 292 |
+
"--temp",
|
| 293 |
+
str(temperature),
|
| 294 |
+
"--threads",
|
| 295 |
+
str(threads),
|
| 296 |
+
"--simple-io",
|
| 297 |
+
]
|
| 298 |
+
if reasoning_format != "auto":
|
| 299 |
+
command.extend(["--reasoning-format", reasoning_format])
|
| 300 |
+
if reasoning_budget is not None:
|
| 301 |
+
command.extend(["--reasoning-budget", str(reasoning_budget)])
|
| 302 |
+
if opening_prompt and opening_prompt.strip():
|
| 303 |
+
command.extend(["-p", opening_prompt.strip()])
|
| 304 |
+
completed = subprocess.run(command, check=False)
|
| 305 |
+
if completed.returncode != 0:
|
| 306 |
+
raise SystemExit(completed.returncode)
|
| 307 |
+
|
| 308 |
+
|
| 309 |
+
def main() -> None:
|
| 310 |
+
args = _build_parser().parse_args()
|
| 311 |
+
root = Path(__file__).resolve().parent
|
| 312 |
+
source_model_path = _preferred_gguf(root)
|
| 313 |
+
if hasattr(sys.stdout, "reconfigure"):
|
| 314 |
+
sys.stdout.reconfigure(encoding="utf-8")
|
| 315 |
+
|
| 316 |
+
single_prompt = (args.prompt or "Say hello in one short sentence.").strip()
|
| 317 |
+
system_prompt = args.system_prompt.strip() or (
|
| 318 |
+
f"You are LumynaX operating from the {MODEL_TITLE} package identity. "
|
| 319 |
+
"Be helpful, clear, and honest about provenance."
|
| 320 |
+
)
|
| 321 |
+
explicit_cli_requested = bool(
|
| 322 |
+
args.llama_cli.strip()
|
| 323 |
+
or os.environ.get("LLAMA_CPP_CLI", "").strip()
|
| 324 |
+
or os.environ.get("LLAMA_CLI_PATH", "").strip()
|
| 325 |
+
)
|
| 326 |
+
if args.interactive:
|
| 327 |
+
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
| 328 |
+
if explicit_cli_requested:
|
| 329 |
+
if llama_cli_path is None:
|
| 330 |
+
raise SystemExit(
|
| 331 |
+
"A llama-cli override was requested, but no usable llama-cli binary was found.",
|
| 332 |
+
)
|
| 333 |
+
_run_interactive_llama_cli(
|
| 334 |
+
llama_cli_path=llama_cli_path,
|
| 335 |
+
model_path=_local_model_path(source_model_path, cache_local=args.cache_local),
|
| 336 |
+
system_prompt=system_prompt,
|
| 337 |
+
opening_prompt=args.prompt,
|
| 338 |
+
max_new_tokens=args.max_new_tokens,
|
| 339 |
+
ctx_size=args.ctx_size,
|
| 340 |
+
temperature=args.temperature,
|
| 341 |
+
threads=args.threads,
|
| 342 |
+
reasoning=args.reasoning,
|
| 343 |
+
reasoning_format=args.reasoning_format,
|
| 344 |
+
reasoning_budget=args.reasoning_budget,
|
| 345 |
+
)
|
| 346 |
+
return
|
| 347 |
+
model_path = _local_model_path(source_model_path, cache_local=args.cache_local)
|
| 348 |
+
try:
|
| 349 |
+
_run_interactive_llama_cpp_python(
|
| 350 |
+
model_path=model_path,
|
| 351 |
+
system_prompt=system_prompt,
|
| 352 |
+
opening_prompt=args.prompt,
|
| 353 |
+
max_new_tokens=args.max_new_tokens,
|
| 354 |
+
ctx_size=args.ctx_size,
|
| 355 |
+
temperature=args.temperature,
|
| 356 |
+
threads=args.threads,
|
| 357 |
+
reasoning=args.reasoning,
|
| 358 |
+
reasoning_format=args.reasoning_format,
|
| 359 |
+
reasoning_budget=args.reasoning_budget,
|
| 360 |
+
)
|
| 361 |
+
return
|
| 362 |
+
except Exception as exc: # noqa: BLE001
|
| 363 |
+
if llama_cli_path is None:
|
| 364 |
+
raise SystemExit(
|
| 365 |
+
"llama-cpp-python could not load this GGUF package. "
|
| 366 |
+
"Install or point LLAMA_CPP_CLI at llama-cli to use the built-in fallback. "
|
| 367 |
+
f"Original error: {exc}",
|
| 368 |
+
) from exc
|
| 369 |
+
print(
|
| 370 |
+
f"llama-cpp-python failed; falling back to llama-cli at {llama_cli_path}",
|
| 371 |
+
file=sys.stderr,
|
| 372 |
+
)
|
| 373 |
+
_run_interactive_llama_cli(
|
| 374 |
+
llama_cli_path=llama_cli_path,
|
| 375 |
+
model_path=model_path,
|
| 376 |
+
system_prompt=system_prompt,
|
| 377 |
+
opening_prompt=args.prompt,
|
| 378 |
+
max_new_tokens=args.max_new_tokens,
|
| 379 |
+
ctx_size=args.ctx_size,
|
| 380 |
+
temperature=args.temperature,
|
| 381 |
+
threads=args.threads,
|
| 382 |
+
reasoning=args.reasoning,
|
| 383 |
+
reasoning_format=args.reasoning_format,
|
| 384 |
+
reasoning_budget=args.reasoning_budget,
|
| 385 |
+
)
|
| 386 |
+
return
|
| 387 |
+
if explicit_cli_requested:
|
| 388 |
+
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
| 389 |
+
if llama_cli_path is None:
|
| 390 |
+
raise SystemExit(
|
| 391 |
+
"A llama-cli override was requested, but no usable llama-cli binary was found.",
|
| 392 |
+
)
|
| 393 |
+
_run_llama_cli(
|
| 394 |
+
llama_cli_path=llama_cli_path,
|
| 395 |
+
model_path=_local_model_path(source_model_path, cache_local=args.cache_local),
|
| 396 |
+
system_prompt=system_prompt,
|
| 397 |
+
user_prompt=single_prompt,
|
| 398 |
+
max_new_tokens=args.max_new_tokens,
|
| 399 |
+
ctx_size=args.ctx_size,
|
| 400 |
+
temperature=args.temperature,
|
| 401 |
+
threads=args.threads,
|
| 402 |
+
reasoning=args.reasoning,
|
| 403 |
+
reasoning_format=args.reasoning_format,
|
| 404 |
+
reasoning_budget=args.reasoning_budget,
|
| 405 |
+
)
|
| 406 |
+
return
|
| 407 |
+
model_path = _local_model_path(source_model_path, cache_local=args.cache_local)
|
| 408 |
+
try:
|
| 409 |
+
print(
|
| 410 |
+
_run_llama_cpp_python(
|
| 411 |
+
model_path=model_path,
|
| 412 |
+
system_prompt=system_prompt,
|
| 413 |
+
user_prompt=single_prompt,
|
| 414 |
+
max_new_tokens=args.max_new_tokens,
|
| 415 |
+
ctx_size=args.ctx_size,
|
| 416 |
+
temperature=args.temperature,
|
| 417 |
+
threads=args.threads,
|
| 418 |
+
),
|
| 419 |
+
)
|
| 420 |
+
return
|
| 421 |
+
except Exception as exc: # noqa: BLE001
|
| 422 |
+
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
| 423 |
+
if llama_cli_path is None:
|
| 424 |
+
raise SystemExit(
|
| 425 |
+
"llama-cpp-python could not load this GGUF package. "
|
| 426 |
+
"Install or point LLAMA_CPP_CLI at llama-cli to use the built-in fallback. "
|
| 427 |
+
f"Original error: {exc}",
|
| 428 |
+
) from exc
|
| 429 |
+
print(
|
| 430 |
+
f"llama-cpp-python failed; falling back to llama-cli at {llama_cli_path}",
|
| 431 |
+
file=sys.stderr,
|
| 432 |
+
)
|
| 433 |
+
_run_llama_cli(
|
| 434 |
+
llama_cli_path=llama_cli_path,
|
| 435 |
+
model_path=model_path,
|
| 436 |
+
system_prompt=system_prompt,
|
| 437 |
+
user_prompt=single_prompt,
|
| 438 |
+
max_new_tokens=args.max_new_tokens,
|
| 439 |
+
ctx_size=args.ctx_size,
|
| 440 |
+
temperature=args.temperature,
|
| 441 |
+
threads=args.threads,
|
| 442 |
+
reasoning=args.reasoning,
|
| 443 |
+
reasoning_format=args.reasoning_format,
|
| 444 |
+
reasoning_budget=args.reasoning_budget,
|
| 445 |
+
)
|
| 446 |
+
|
| 447 |
+
|
| 448 |
+
if __name__ == "__main__":
|
| 449 |
+
main()
|
release_export_manifest.json
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"artifacts": {
|
| 3 |
+
"checksums": "checksums.sha256",
|
| 4 |
+
"gguf": "Qwen3-14B-Q4_K_M.gguf",
|
| 5 |
+
"gitattributes": ".gitattributes",
|
| 6 |
+
"hf_space_app": "hf_space/app.py",
|
| 7 |
+
"hf_space_dir": "hf_space",
|
| 8 |
+
"hf_space_readme": "hf_space/README.md",
|
| 9 |
+
"hf_space_requirements": "hf_space/requirements.txt",
|
| 10 |
+
"license": "LICENSE.txt",
|
| 11 |
+
"merged_model": "merged_model",
|
| 12 |
+
"mmproj": null,
|
| 13 |
+
"ollama_create_script": "ollama/create_ollama_model.ps1",
|
| 14 |
+
"ollama_modelfile": "ollama/Modelfile",
|
| 15 |
+
"package_state": "merged_model/PACKAGE_STATE.txt",
|
| 16 |
+
"quantized_gguf": "Qwen3-14B-Q4_K_M.gguf",
|
| 17 |
+
"quickstart": "quickstart.py",
|
| 18 |
+
"readme": "README.md",
|
| 19 |
+
"requirements": "requirements.txt",
|
| 20 |
+
"training_summary": "artifacts/release_training_summary.json",
|
| 21 |
+
"upload_notes": "UPLOAD_TO_HF.md",
|
| 22 |
+
"version": "VERSION.txt"
|
| 23 |
+
},
|
| 24 |
+
"capabilities": {
|
| 25 |
+
"reasoning_enabled": false,
|
| 26 |
+
"supported_modalities": [
|
| 27 |
+
"text"
|
| 28 |
+
]
|
| 29 |
+
},
|
| 30 |
+
"delivery": "standalone_prebuilt_gguf_release",
|
| 31 |
+
"distribution": {
|
| 32 |
+
"hf_space": {
|
| 33 |
+
"app": "hf_space/app.py",
|
| 34 |
+
"default_model_repo_id": "AbteeXAILab/lumynax-infused-qwen3-14b-gguf",
|
| 35 |
+
"directory": "hf_space",
|
| 36 |
+
"model_repo_env_var": "LUMYNAX_MODEL_REPO_ID",
|
| 37 |
+
"readme": "hf_space/README.md",
|
| 38 |
+
"requirements": "hf_space/requirements.txt",
|
| 39 |
+
"status": "browser_showcase_for_gguf_only_release"
|
| 40 |
+
},
|
| 41 |
+
"ollama": {
|
| 42 |
+
"create_script": "ollama/create_ollama_model.ps1",
|
| 43 |
+
"mmproj": null,
|
| 44 |
+
"modelfile": "ollama/Modelfile",
|
| 45 |
+
"preferred_gguf": "Qwen3-14B-Q4_K_M.gguf",
|
| 46 |
+
"recommended_model_name": "lumynax-infused-qwen3-14b-gguf",
|
| 47 |
+
"status": "ready_for_local_ollama_create"
|
| 48 |
+
}
|
| 49 |
+
},
|
| 50 |
+
"family": null,
|
| 51 |
+
"generated_at": "2026-05-10T13:19:58.304023+00:00",
|
| 52 |
+
"license": {
|
| 53 |
+
"id": "apache-2.0",
|
| 54 |
+
"link": "https://huggingface.co/Qwen/Qwen3-14B-GGUF",
|
| 55 |
+
"name": null,
|
| 56 |
+
"weights_subject_to_upstream_license": true
|
| 57 |
+
},
|
| 58 |
+
"manifest_version": 2,
|
| 59 |
+
"model_title": "LumynaX Infused Qwen3 14B GGUF",
|
| 60 |
+
"package_state": "prebuilt_gguf_release",
|
| 61 |
+
"public_identity": {
|
| 62 |
+
"model_name": "LumynaX",
|
| 63 |
+
"organization": "AbteeX AI Labs",
|
| 64 |
+
"region": "Aotearoa New Zealand"
|
| 65 |
+
},
|
| 66 |
+
"release_version": "v1",
|
| 67 |
+
"runtime": {
|
| 68 |
+
"delivery_mode": "standalone_prebuilt_gguf",
|
| 69 |
+
"preferred_backend": "llama_cpp",
|
| 70 |
+
"prompt_format": "chatml",
|
| 71 |
+
"quickstart_command": "python quickstart.py --interactive",
|
| 72 |
+
"system_prompt": "You are LumynaX operating from the LumynaX Infused Qwen3 14B GGUF package identity. Be helpful, clear, and honest about provenance."
|
| 73 |
+
},
|
| 74 |
+
"source_gguf": {
|
| 75 |
+
"filename": "Qwen3-14B-Q4_K_M.gguf",
|
| 76 |
+
"mmproj_filename": null,
|
| 77 |
+
"packaged_filename": "Qwen3-14B-Q4_K_M.gguf",
|
| 78 |
+
"packaged_mmproj_filename": null,
|
| 79 |
+
"quantization": "Q4_K_M",
|
| 80 |
+
"repo_id": "Qwen/Qwen3-14B-GGUF"
|
| 81 |
+
},
|
| 82 |
+
"upstream_model": {
|
| 83 |
+
"kind": "official_base_weights",
|
| 84 |
+
"lumynax_weight_adaptation_applied": false,
|
| 85 |
+
"provider": "Hugging Face",
|
| 86 |
+
"repo_id": "Qwen/Qwen3-14B"
|
| 87 |
+
}
|
| 88 |
+
}
|
requirements.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
llama-cpp-python>=0.3.18
|