--- license: apache-2.0 base_model: Zyphra/Zamba2-7B-instruct tags: - zamba2 - mamba - hybrid - compressed - hxq - helix-substrate - vector-quantization - 2d-vq - 12bit-packing library_name: transformers pipeline_tag: text-generation model-index: - name: zamba2-7b-instruct-hxq results: - task: type: text-generation dataset: name: WikiText-2 type: wikitext metrics: - name: Perplexity type: perplexity value: 3.8454 - task: type: text-generation dataset: name: HellaSwag type: hellaswag metrics: - name: acc_norm type: acc_norm value: 0.8106 - task: type: text-generation dataset: name: ARC-Challenge type: ai2_arc metrics: - name: acc_norm type: acc_norm value: 0.5811 - task: type: text-generation dataset: name: ARC-Easy type: ai2_arc metrics: - name: acc_norm type: acc_norm value: 0.8190 --- # Zamba2-7B-Instruct-HXQ > **2D Vector Quantization (k=4096) with 12-bit packed indices. 81-layer hybrid Mamba2+Transformer. Faster than dense at 60% less VRAM.** Zamba2-7B-Instruct compressed with HXQ 2D VQ and 12-bit index packing (6 bits/weight). Beats bnb 4-bit NF4 on quality, speed, and requires no calibration data. ## Benchmark: Native HelixLinear Inference on RTX 3090 All numbers from a single session, same GPU, same WikiText-2 test set (50 chunks x 512 tokens). | Method | PPL | Throughput | VRAM (load) | VRAM (peak) | Bits/weight | |--------|-----|------------|-------------|-------------|-------------| | Dense BF16 | 4.82 | 1,446 tok/s | 14,032 MB | 14,686 MB | 16 | | bnb 8-bit | 4.85 | 515 tok/s | 7,831 MB | 8,635 MB | 8 | | bnb 4-bit NF4 | 5.07 | 1,579 tok/s | 5,129 MB | 5,904 MB | 4 | | **HXQ 12-bit packed** | **5.02** | **1,764 tok/s** | **5,657 MB** | **6,511 MB** | **6** | ### Why HXQ wins - **Faster than dense** (1,764 vs 1,446 tok/s) -- fused Triton gather kernel eliminates memory bottleneck - **60% less VRAM** than dense (5.7 GB vs 14.0 GB) - **Better quality than bnb 4-bit** (5.02 vs 5.07 PPL) at comparable VRAM - **No calibration data required** -- unlike GPTQ, AWQ, or bnb, HXQ compresses from weights alone ## Downstream Task Evaluation (lm-eval-harness v0.4.11) Same GPU (RTX 3090), same harness, same settings. All metrics are `acc_norm`. | Task | Dense BF16 | HXQ 2D VQ | Delta | |------|-----------|-----------|-------| | **HellaSwag** | 80.79% | 81.06% | +0.27% | | **ARC-Challenge** | 59.39% | 58.11% | -1.28% | | **ARC-Easy** | 83.21% | 81.90% | -1.31% | Compression preserves task performance within noise. HellaSwag (commonsense reasoning) is slightly *better* under compression. ARC drops are within 1.3%. ## Install and Run ```bash pip install "helix-substrate>=0.3.3" ``` ```python import helix_substrate # registers the HXQ quantizer from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "EchoLabs33/zamba2-7b-instruct-hxq", trust_remote_code=True, ) tokenizer = AutoTokenizer.from_pretrained( "EchoLabs33/zamba2-7b-instruct-hxq", trust_remote_code=True, ) inputs = tokenizer("The capital of France is", return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=32) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` **Note:** Tested with `transformers>=4.49`. `mamba-ssm` and `causal-conv1d` recommended for fast Mamba2 inference. ## Codec Details | Property | Value | |----------|-------| | **Quantization** | 2D Vector Quantization | | **Codebook size (k)** | 4,096 | | **Vector dimension** | 2 (pairs of adjacent weights) | | **Bits per weight** | 6 effective (12-bit packed index / 2 weights) | | **Index packing** | 12-bit (3 bytes per 2 indices, lossless) | | **Compressed modules** | 213 HelixLinear layers | | **Exact tensors** | 573 (norms, embeddings, conv1d, A_log, D, dt_bias) | | **Sidecar corrections** | Yes (sparse outlier compensation) | | **Calibration data** | None required | | **HXQ storage** | 5.7 GB (12-bit packed) | | **Dense BF16** | 14.0 GB | ### Why 2D VQ Standard scalar VQ assigns one codebook entry per weight. 2D VQ groups pairs of adjacent weights and clusters in R^2 -- the codebook captures the joint distribution. With k=4096 and 12-bit index packing, each pair of weights costs 12 bits = 6 bits per weight. This beats scalar k=256 (8 bits/weight) on both quality and compression. On Zamba2-7B, 2D VQ k=4096 achieves PPL 3.8454 vs the scalar baseline's higher PPL on this architecture. SSM and hybrid architectures benefit from 2D VQ more than pure Transformers. ## Architecture Zamba2-7B-Instruct is a hybrid architecture: - **81 total layers** (Mamba2 + shared Transformer) - **hidden_size=3584**, **attention_hidden_size=7168**, **32 attention heads** - **mamba_d_state=64**, **mamba_d_conv=4** - **vocab_size=32000** 213 linear layers compressed (Mamba projections, attention/MLP, LoRA adapters). Normalization layers, embeddings, conv1d, and Mamba-specific parameters stored at full precision. ## Verification - **Benchmark receipt:** [`zamba2_7b_comparison.json`](https://huggingface.co/EchoLabs33/zamba2-7b-instruct-hxq/blob/main/zamba2_7b_comparison.json) -- RTX 3090, 2026-04-02 - **Conversion receipt:** [`conversion_receipt.json`](https://huggingface.co/EchoLabs33/zamba2-7b-instruct-hxq/blob/main/conversion_receipt.json) -- Gate 1 PASS - **helix-substrate version:** 0.3.3 (12-bit packing, fused Triton gather) ## Companion Models | Model | Architecture | PPL Delta | |-------|-------------|-----------| | [qwen2.5-14b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-14b-instruct-helix) | Transformer | pending | | [qwen2.5-7b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-7b-instruct-helix) | Transformer | +6.34% | | [qwen2.5-3b-instruct-helix](https://huggingface.co/EchoLabs33/qwen2.5-3b-instruct-helix) | Transformer | +0.69% | | [zamba2-2.7b-instruct-helix](https://huggingface.co/EchoLabs33/zamba2-2.7b-instruct-helix) | Hybrid | +6.59% | | [zamba2-1.2b-helix](https://huggingface.co/EchoLabs33/zamba2-1.2b-helix) | Hybrid | +2.90% | | [mamba2-1.3b-helix](https://huggingface.co/EchoLabs33/mamba2-1.3b-helix) | Pure SSM | +8.0% | | [tinyllama-1.1b-helix](https://huggingface.co/EchoLabs33/tinyllama-1.1b-helix) | Transformer | +0.78% | ## Citation ```bibtex @software{helix_substrate_2026, title={Helix Substrate: Universal Weight Compression via HXQ}, author={EchoLabs}, year={2026}, url={https://pypi.org/project/helix-substrate/} } ``` ## License Apache 2.0 (inherited from [Zyphra/Zamba2-7B-instruct](https://huggingface.co/Zyphra/Zamba2-7B-instruct)).