abgunaydin/postnet-qwen05b-with-gates

Qwen2.5-0.5B-Instruct + NTK-Mirror gates (int8 ONNX)

Single-file int8 ONNX (~906 MB) used by the browser workers in postnet-cf — federated learning where the workers are browser tabs and the coordinator is a Cloudflare Durable Object. The /ntk.html demo runs this model forward in the tab via onnxruntime-web and federated-SPSA-trains a sparse NTK-Mirror gate controller (K = 5000 signed log-gates) across everyone who opens the URL.

▶ Live demo: https://postnet-cf.abgunaydin94.workers.dev/ntk — open it, click Join, and your tab starts federated-SPSA-training against everyone else who's connected. No install, no clone; this model streams in from the Hub.

What this is

• Base: Qwen/Qwen2.5-0.5B-Instruct (Apache-2.0), exported with optimum-cli (ORT-web compatible), then int8-dynamic-quantized to a single file (no external-data sidecar) so it fits in one ArrayBuffer and loads in Chrome's onnxruntime-web 1.22.
• Gate surgery: 24 forward-only Mul nodes injected on each decoder layer's post-MLP residual, driven by an extra graph input gate_mults: [24, 896] float32. With gate_mults = 1 the graph is byte-identical to the base export. Parameterization from NTK-Mirror (MIT).

Inputs / outputs

input	dtype	shape
input_ids	int64	[batch, seq]
attention_mask	int64	[batch, seq]
position_ids	int64	[batch, seq]
gate_mults	float32	[24, 896]

Output: logits [batch, seq, vocab].

License / attribution

Apache-2.0 (inherits the base model's license). This is a quantized derivative of Qwen2.5-0.5B-Instruct © the Alibaba Qwen team. The gate parameterization is from Leon Chlon's NTK-Mirror (Hassana Labs), MIT.