Instructions to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SC117/Ornith-1.0-35B-MTP-APEX-GGUF")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("SC117/Ornith-1.0-35B-MTP-APEX-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SC117/Ornith-1.0-35B-MTP-APEX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF

SGLang

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SC117/Ornith-1.0-35B-MTP-APEX-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SC117/Ornith-1.0-35B-MTP-APEX-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with Docker Model Runner:
```
docker model run hf.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF
```

Loving this version

by SaturnsVoid - opened Jul 11

Discussion

SaturnsVoid

Jul 11

Running the Quality version at 128k on my 9070XT and 48GB ram and getting 45/ts its able to one shot a lot of GoLang coding prompts in Pi.dev, Just wish it was uncensored.

SC117

Owner Jul 11

Running the Quality version at 128k on my 9070XT and 48GB ram and getting 45/ts its able to one shot a lot of GoLang coding prompts in Pi.dev, Just wish it was uncensored.

Thank you, I’m glad the Quality version is working well for you, especially at 128K context.

I’m currently working on an uncensored version of this model. It has been more difficult than expected, so I can’t promise the final result yet. My preference is still to complete the uncensoring myself, so I can fully control the processing and validation pipeline.

If I cannot reach an acceptable result, the fallback would be to start from a high-quality uncensored base released by another researcher, then add native MTP preservation and APEX quantization on top of it.

For now, I’m still trying to finish the uncensored version independently and verify that it does not introduce significant quality regressions.

SC117

Owner Jul 13

Running the Quality version at 128k on my 9070XT and 48GB ram and getting 45/ts its able to one shot a lot of GoLang coding prompts in Pi.dev, Just wish it was uncensored.

Quick update: I actually finished the uncensored version much sooner than expected 😅

After a lot of testing and some custom Heretic modifications for the 35B Qwen3.5 MoE architecture, I was able to get a stable result, preserve native MTP, and complete the APEX GGUF release.

BF16 + MTP:
https://huggingface.co/SC117/Ornith-1.0-35B-Heretic-MTP

APEX GGUF:
https://huggingface.co/SC117/Ornith-1.0-35B-Heretic-MTP-APEX-GGUF

Since you mentioned using the Quality version at 128K, you may want to try the Heretic Quality build directly.

TEPTOMTEPTOM

23 days ago

Dear Author,
I love the SC117/Ornith-1.0-35B-MTP-APEX-GGUF compact version (after test many model)
Does the Heretic version is better and faster,
Thanks you!

SC117

Owner 23 days ago

Dear Author,
I love the SC117/Ornith-1.0-35B-MTP-APEX-GGUF compact version (after test many model)
Does the Heretic version is better and faster,
Thanks you!

Thanks for your support!

The Heretic version is mainly focused on reducing refusals while keeping the original capabilities as much as possible. It is not necessarily faster than the normal version, because the speed mainly depends on the quantization profile and your hardware.

If you are already happy with the Compact version, the Heretic Compact build would be the closest comparison. The Quality/Balanced versions can provide better accuracy but require more memory.

Thanks again for testing!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment