Brooooooklyn/Qwen-AgentWorld-35B-A3B-mxfp4-mlx

Qwen-AgentWorld-35B-A3B — MXFP4 (mlx-node)

MXFP4 microscaling floating-point quantization of Qwen/Qwen-AgentWorld-35B-A3B for Apple Silicon (with imatrix-AWQ pre-scaling), via mlx-node.

Qwen-AgentWorld-35B-A3B is the first native language world model for agentic environment simulation — a Qwen3.5-VL-MoE (hybrid Gated-DeltaNet + full attention, 256 experts, vision-language) that simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state from an agent's action and interaction history. A single model spans seven interaction domains: MCP (tool calling), Search, Terminal, SWE, Android, Web, and OS. Trained CPT → SFT → RL on Qwen3.5-35B-A3B-Base. (technical report)

	Original (BF16)	This Model
Size	~65 GB	21 GB
Format	SafeTensors (sharded)	SafeTensors (sharded)
Precision	BF16 uniform	MXFP4 (FP4 E2M1, gs32) body + 8-bit affine routers + BF16 (imatrix-AWQ)

All Variants

Repo	Format	Size	Decode (tok/s)
Brooooooklyn/Qwen-AgentWorld-35B-A3B-UD-Q3_K_XL-mlx	UD-Q3_K_XL	17 GB	112.3
Brooooooklyn/Qwen-AgentWorld-35B-A3B-mxfp4-mlx (this model)	MXFP4	21 GB	102.6
Brooooooklyn/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-mlx	UD-Q4_K_XL	22 GB	99.8
Brooooooklyn/Qwen-AgentWorld-35B-A3B-nvfp4-mlx	NVFP4	23 GB	94.1
Brooooooklyn/Qwen-AgentWorld-35B-A3B-UD-Q5_K_XL-mlx	UD-Q5_K_XL	26 GB	95.4
Brooooooklyn/Qwen-AgentWorld-35B-A3B-UD-Q6_K_XL-mlx	UD-Q6_K_XL	31 GB	92.7
Brooooooklyn/Qwen-AgentWorld-35B-A3B-mxfp8-mlx	MXFP8	36 GB	91.0
Brooooooklyn/Qwen-AgentWorld-35B-A3B-UD-Q8_K_XL-mlx	UD-Q8_K_XL	37 GB	91.1

Benchmarked on a cool Apple M5 Max: median decode throughput over three 512-token generations, with a 60-second idle GPU cooldown after every generation. (Sustained decode on Apple Silicon is thermally sensitive — back-to-back benchmarking on a hot chip can understate throughput by 20–30%, so every model here was measured from a comparable cool start.)

Performance

Steady-state decode: 102.6 tok/s (1.7x vs BF16) on Apple M5 Max. Decode is memory-bandwidth bound on Apple Silicon — fewer bytes per token directly translates to higher throughput. The MoE architecture activates only 8 of 256 experts per token (~3B active out of ~34.7B total), so the active-weight footprint streamed per token is what matters.

Output Quality

Decoded-text quality was verified against the BF16 reference with a multi-judge review of the actual generated output (not a heuristic): a multi-turn factual chat plus a structured reasoning/code task. This MXFP4 build produced coherent prose, correct facts, and a correct implementation — no runaway generation, repetition loops, or stray tokens — on par with full precision.

Per-Tensor Quantization

Weight	Format	Rationale
switch_mlp.gate_proj/up_proj/down_proj	MXFP4 (FP E2M1, gs32)	MoE expert bulk — microscaled FP
self_attn.q/k/v/o_proj	MXFP4 (gs32)	attention projections
linear_attn.in_proj_qkv/z, out_proj, in_proj_a/b	MXFP4 (gs32)	GatedDeltaNet projections
Router gates (mlp.gate, shared_expert_gate)	8-bit affine	MoE routing accuracy (never FP)
embed_tokens, lm_head	bf16	uniform-FP path keeps embeddings/head full precision
GDN params (A_log, dt_bias)	bf16	state-space dynamics
visual.* (vision tower)	bf16	vision encoder kept full precision

Quantization Strategy

MXFP4 is a microscaling floating-point format (4-bit elements with a shared block exponent per group of 32). Unlike integer-affine quantization, the per-group exponent adapts the dynamic range locally, which suits weight distributions with outliers. MLX runs it natively on Metal via the fp_gather_qmm kernels for MoE experts — no dequantize-to-bf16 fallback. These builds add imatrix-AWQ pre-scaling on top. Router gates remain 8-bit affine and the embeddings, head, GatedDeltaNet state params and vision tower stay bf16.

Architecture

Parameter	Value
Total parameters	34.7B (3B active per token)
Hidden size	2,048
Layers	40 (30 linear GatedDeltaNet + 10 full attention, interval 4)
Attention heads	16 (2 KV heads, GQA 8:1)
Head dimension	256
Experts	256 per MoE layer, top-8 routing
Vocab size	248,320
Vision	yes (Qwen3.5-VL vision tower, kept bf16)
Max context	262,144 tokens

Usage

import { loadSession } from '@mlx-node/lm';

const session = await loadSession('./Qwen-AgentWorld-35B-A3B-mxfp4-mlx');

for await (const event of session.sendStream('An agent runs `ls -la` in /home/user. Predict the terminal output and the resulting environment state.', {
  config: { maxNewTokens: 2048, temperature: 0.6, reasoningEffort: 'low' },
})) {
  if (!event.done) process.stdout.write(event.text);
}

How It Was Made

mlx convert \
  -i Qwen-AgentWorld-35B-A3B \
  -o Qwen-AgentWorld-35B-A3B-mxfp4-mlx \
  -q --q-recipe unsloth --q-mxfp --q-bits 4\
  --imatrix-path imatrix_unsloth.gguf_file

The --q-recipe unsloth --q-mxfp path upgrades the quantizable linears to MXFP4 microscaled floating-point (shared block exponents, group_size 32) with imatrix-AWQ pre-scaling (unsloth/Qwen-AgentWorld-35B-A3B-GGUF). Router gates stay 8-bit affine for routing accuracy; embeddings, head, GatedDeltaNet state params and the vision tower stay bf16.

Acknowledgments

• Qwen Team — For the Qwen-AgentWorld model and the Qwen3.5 base architecture
• Unsloth — Per-layer KLD bit-allocation (Dynamic 2.0) and the imatrix used for AWQ pre-scaling
• Apple MLX — For the Metal-accelerated ML framework

License

Apache-2.0 (inherited from base model).