Instructions to use EchoLabs33/olmoe-1b-7b-instruct-hxq with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EchoLabs33/olmoe-1b-7b-instruct-hxq with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="EchoLabs33/olmoe-1b-7b-instruct-hxq")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("EchoLabs33/olmoe-1b-7b-instruct-hxq")
model = AutoModelForCausalLM.from_pretrained("EchoLabs33/olmoe-1b-7b-instruct-hxq", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use EchoLabs33/olmoe-1b-7b-instruct-hxq with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "EchoLabs33/olmoe-1b-7b-instruct-hxq"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EchoLabs33/olmoe-1b-7b-instruct-hxq",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/EchoLabs33/olmoe-1b-7b-instruct-hxq

SGLang

How to use EchoLabs33/olmoe-1b-7b-instruct-hxq with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "EchoLabs33/olmoe-1b-7b-instruct-hxq" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EchoLabs33/olmoe-1b-7b-instruct-hxq",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "EchoLabs33/olmoe-1b-7b-instruct-hxq" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EchoLabs33/olmoe-1b-7b-instruct-hxq",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use EchoLabs33/olmoe-1b-7b-instruct-hxq with Docker Model Runner:
```
docker model run hf.co/EchoLabs33/olmoe-1b-7b-instruct-hxq
```

voidstream commited on Apr 3

Commit

e237124

verified ·

1 Parent(s): 5ee8bd4

Initial upload: OLMoE-1B-7B-Instruct HXQ (1.9x compression, paired eval receipts)

Browse files

Files changed (8) hide show

README.md +186 -0
completeness_gate_receipt.json +61 -0
config.json +0 -0
conversion_receipt.json +17 -0
model.safetensors +3 -0
special_tokens_map.json +23 -0
tokenizer.json +0 -0
tokenizer_config.json +247 -0

README.md ADDED Viewed

	@@ -0,0 +1,186 @@

+---
+license: apache-2.0
+base_model: allenai/OLMoE-1B-7B-0924-Instruct
+tags:
+  - olmoe
+  - moe
+  - mixture-of-experts
+  - compressed
+  - hxq
+  - helix-substrate
+  - vector-quantization
+  - helixcode
+library_name: transformers
+pipeline_tag: text-generation
+model-index:
+  - name: olmoe-1b-7b-instruct-helix
+    results:
+      - task:
+          type: text-generation
+          name: Text Generation
+        dataset:
+          name: HellaSwag
+          type: hellaswag
+        metrics:
+          - type: acc_norm
+            value: 0.7876
+            name: Accuracy (norm)
+      - task:
+          type: text-generation
+          name: Text Generation
+        dataset:
+          name: ARC-Easy
+          type: ai2_arc
+          config: ARC-Easy
+        metrics:
+          - type: acc_norm
+            value: 0.7685
+            name: Accuracy (norm)
+      - task:
+          type: text-generation
+          name: Text Generation
+        dataset:
+          name: ARC-Challenge
+          type: ai2_arc
+          config: ARC-Challenge
+        metrics:
+          - type: acc_norm
+            value: 0.5205
+            name: Accuracy (norm)
+---
+# OLMoE-1B-7B-Instruct-HXQ
+> **1.9x smaller from BF16. HellaSwag 78.8%. First MoE compressed with HXQ.**
+>
+> OLMoE-1B-7B-Instruct (64-expert Mixture-of-Experts, 1B active / 6.9B total) compressed from 13 GB (BF16) to 6.7 GB. All three downstream benchmarks within noise of the dense baseline. No calibration data. No architecture-specific tuning. Just `pip install` and `from_pretrained()`.
+## Install and Run
+```bash
+pip install "helix-substrate[hf]"
+```
+```python
+import helix_substrate  # registers the HXQ quantizer with HuggingFace
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained(
+    "EchoLabs33/olmoe-1b-7b-instruct-helix",
+    trust_remote_code=True,
+    torch_dtype="bfloat16",
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("EchoLabs33/olmoe-1b-7b-instruct-helix")
+inputs = tokenizer("The capital of France is", return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=50)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+That's it. `import helix_substrate` registers the quantizer. `from_pretrained()` handles the rest automatically.
+## Downstream Benchmarks
+Evaluated with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) v0.4.11 on an NVIDIA RTX 3090 (batch=4, dtype=bfloat16):
+| Benchmark | Dense (acc_norm) | HXQ (acc_norm) | Delta |
+|-----------|-----------------|----------------|-------|
+| **HellaSwag** | 78.92% | **78.76%** | **-0.16%** |
+| **ARC-Challenge** | 52.13% | **52.05%** | **-0.08%** |
+| **ARC-Easy** | 75.72% | **76.85%** | **+1.14%** |
+All deltas within standard error. Task performance is preserved after 1.9x compression. These are real downstream scores from paired dense/HXQ evaluations, not PPL proxies.
+## Compression Benchmark
+| | Dense (BF16) | HXQ |
+|---|---|---|
+| **Size** | 13 GB | **6.7 GB** |
+| **Compression ratio** | — | **1.9x** |
+| **VRAM (eval)** | 13,886 MB | **7,540 MB** |
+| **Compressed modules** | — | 3,152 HelixLinear layers |
+| **Architecture** | OLMoE (64-expert MoE) | unchanged |
+## Verification Status
+- **Compression receipt:** PASS — 3,152 compressed, 67 exact, 12,675 total keys
+- **Conversion receipt:** PASS — SHA256 `a9f74982b746853077d13dc11c8bc863dc91219c81e22577de1de2b195c7b836`
+- **Downstream eval:** PASS — paired dense/HXQ on HellaSwag, ARC-Easy, ARC-Challenge
+## Good to Know
+- **GPU and CPU supported** — runs on any CUDA GPU or CPU via standard PyTorch.
+- **`trust_remote_code=True` required** — OLMoE uses custom modeling code.
+- **Not fine-tunable** — compressed weights are read-only (`is_trainable = False`).
+- **Requires `helix-substrate`** — the quantizer is not built into transformers. You need `pip install "helix-substrate[hf]"`.
+- **64 experts = slow eval** — lm-eval-harness takes ~5.5 hours on a 3090 due to MoE routing overhead. Inference speed is normal for interactive use.
+## What is HelixCode?
+HelixCode is a universal weight compression codec based on vector quantization:
+- Each weight matrix is replaced by a **256-entry codebook** (float32) + **uint8 index matrix** + optional **sidecar corrections** for outlier values
+- The compressed form *is* the executable — `HelixLinear` performs `codebook[indices] @ x` directly, no decompression step
+- Works on any `nn.Linear` regardless of architecture (Transformer, Mamba, MoE, CNN)
+- **No calibration data required** — unlike GPTQ/AWQ, codebooks are fit from the weights alone
+## How It Works
+1. `import helix_substrate` registers the `hxq` quantizer with HuggingFace
+2. `from_pretrained()` reads `quantization_config.quant_method = "hxq"` from `config.json`
+3. The quantizer replaces 3,152 `nn.Linear` modules with `HelixLinear` shells before weight loading
+4. Safetensors populates the codebook, indices, and sidecar buffers directly
+5. The model runs in compressed form — no decompression needed
+## Architecture Details
+OLMoE-1B-7B-Instruct is a Mixture-of-Experts architecture with:
+- **16 transformer layers**, each with attention + MoE MLP
+- **64 experts per layer**, top-8 routing (1B active / 6.9B total parameters)
+- **hidden_size=2048**, intermediate_size=1024 per expert
+- **16 attention heads**, no GQA (num_kv_heads=16)
+All 3,152 linear layers are compressed:
+- **3,072 expert projections** (64 experts x 3 projections x 16 layers)
+- **64 attention projections** (Q/K/V/O across 16 layers)
+- **16 router gates** (expert routing per layer)
+Normalization layers (33), embeddings (1), and lm_head (1) are stored at full precision.
+## Why This Matters
+OLMoE is the first **Mixture-of-Experts** model compressed with HXQ. Combined with existing Transformer, SSM, and Hybrid results, this demonstrates that the same codec — same codebook size, same algorithm, same `pip install` — works across four distinct architecture families without modification.
+## Companion Models
+Same codec, same `pip install`, multiple architectures:
+| Model | Architecture | Ratio | Eval Delta |
+|-------|-------------|-------|------------|
+| **olmoe-1b-7b-instruct-helix** | **MoE (64 experts)** | **1.9x** | **-0.16% HellaSwag** |
+| [zamba2-2.7b-instruct-helix](https://huggingface.co/EchoLabs33/zamba2-2.7b-instruct-helix) | Hybrid (Mamba2+Transformer) | 1.8x | +6.59% PPL |
+| [zamba2-1.2b-helix](https://huggingface.co/EchoLabs33/zamba2-1.2b-helix) | Hybrid (Mamba2+Transformer) | 1.7x | +2.90% PPL |
+| [qwen2.5-14b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-14b-instruct-helix) | Transformer | 3.4x | pending |
+| [qwen2.5-7b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-7b-instruct-helix) | Transformer | 2.2x | +6.34% PPL |
+| [qwen2.5-3b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-3b-instruct-helix) | Transformer | 1.6x | +0.69% PPL |
+| [qwen2.5-coder-3b-helix](https://huggingface.co/EchoLabs33/qwen2.5-coder-3b-helix) | Transformer (code) | 1.6x | +1.92% PPL |
+| [qwen2.5-coder-1.5b-helix](https://huggingface.co/EchoLabs33/qwen2.5-coder-1.5b-helix) | Transformer (code) | 1.5x | +1.73% PPL |
+| [tinyllama-1.1b-helix](https://huggingface.co/EchoLabs33/tinyllama-1.1b-helix) | Transformer | 4.0x | +0.78% PPL |
+| [mamba2-1.3b-helix](https://huggingface.co/EchoLabs33/mamba2-1.3b-helix) | Pure SSM (Mamba2) | 2.1x | +8.0% PPL |
+| [mamba-130m-helix](https://huggingface.co/EchoLabs33/mamba-130m-helix) | Pure SSM | 3.8x | +18.4% PPL |
+## Citation
+```bibtex
+@software{helix_substrate_2026,
+  title={Helix Substrate: Universal Weight Compression via HelixCode},
+  author={EchoLabs},
+  year={2026},
+  url={https://github.com/echo313unfolding/helix-substrate}
+}
+```
+## License
+Apache 2.0 (inherited from [allenai/OLMoE-1B-7B-0924-Instruct](https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct)).

completeness_gate_receipt.json ADDED Viewed

	@@ -0,0 +1,61 @@

+{
+  "verdict": "PASS",
+  "summary": {
+    "dense_tensors": 3219,
+    "output_tensors": 12675,
+    "compressed_weights": 3152,
+    "accounted": 3219,
+    "missing": 0,
+    "skip_tensors_found": 35,
+    "skip_categories": {
+      "embedding": "OK (1/1)",
+      "layernorm": "OK (33/33)",
+      "output_head": "OK (1/1)"
+    }
+  },
+  "failures": [],
+  "missing_tensors": [],
+  "skip_found": {
+    "output_head": [
+      "lm_head.weight"
+    ],
+    "embedding": [
+      "model.embed_tokens.weight"
+    ],
+    "layernorm": [
+      "model.layers.0.input_layernorm.weight",
+      "model.layers.0.post_attention_layernorm.weight",
+      "model.layers.1.input_layernorm.weight",
+      "model.layers.1.post_attention_layernorm.weight",
+      "model.layers.10.input_layernorm.weight",
+      "model.layers.10.post_attention_layernorm.weight",
+      "model.layers.11.input_layernorm.weight",
+      "model.layers.11.post_attention_layernorm.weight",
+      "model.layers.12.input_layernorm.weight",
+      "model.layers.12.post_attention_layernorm.weight",
+      "model.layers.13.input_layernorm.weight",
+      "model.layers.13.post_attention_layernorm.weight",
+      "model.layers.14.input_layernorm.weight",
+      "model.layers.14.post_attention_layernorm.weight",
+      "model.layers.15.input_layernorm.weight",
+      "model.layers.15.post_attention_layernorm.weight",
+      "model.layers.2.input_layernorm.weight",
+      "model.layers.2.post_attention_layernorm.weight",
+      "model.layers.3.input_layernorm.weight",
+      "model.layers.3.post_attention_layernorm.weight",
+      "model.layers.4.input_layernorm.weight",
+      "model.layers.4.post_attention_layernorm.weight",
+      "model.layers.5.input_layernorm.weight",
+      "model.layers.5.post_attention_layernorm.weight",
+      "model.layers.6.input_layernorm.weight",
+      "model.layers.6.post_attention_layernorm.weight",
+      "model.layers.7.input_layernorm.weight",
+      "model.layers.7.post_attention_layernorm.weight",
+      "model.layers.8.input_layernorm.weight",
+      "model.layers.8.post_attention_layernorm.weight",
+      "model.layers.9.input_layernorm.weight",
+      "model.layers.9.post_attention_layernorm.weight",
+      "model.norm.weight"
+    ]
+  }
+}

config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

conversion_receipt.json ADDED Viewed

	@@ -0,0 +1,17 @@

+{
+  "verdict": "PASS",
+  "validation": {
+    "checks": {
+      "readable": true,
+      "compressed_complete": true,
+      "indices_in_range": true,
+      "exact_no_nan": true,
+      "key_count_match": true
+    },
+    "details": [],
+    "total_keys": 12675,
+    "compressed_modules": 3152,
+    "exact_tensors": 67,
+    "sha256": "a9f74982b746853077d13dc11c8bc863dc91219c81e22577de1de2b195c7b836"
+  }
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a9f74982b746853077d13dc11c8bc863dc91219c81e22577de1de2b195c7b836
+size 7139966784

special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,23 @@

+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,247 @@

+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{{ bos_token }}{% for message in messages %}\n{% if message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] }}\n{% elif message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": null
+}