This bundle requires vMLX Python 0.21. That runtime update is currently being built and will be published within 24 hours. Until vMLX Python 0.21 is available, this bundle is not compatible with the current public vMLX Python release.

DeepSeek-V4-Flash-0731-JANG

Dynamic affine JANG quantization of the official deepseek-ai/DeepSeek-V4-Flash-0731 release for DSV4-aware Apple Silicon MLX runtimes.

This is the 0731 release, not the earlier DeepSeek-V4-Flash preview. The source is pinned to immutable commit 9e165c30e2704aec5d9d593cce3eebd58bbef1cb.

Source	deepseek-ai/DeepSeek-V4-Flash-0731
Source revision	9e165c30e2704aec5d9d593cce3eebd58bbef1cb
License	MIT, inherited from upstream
Format	JANG mixed affine
Bundle size	104.60 GB / 97.41 GiB
Weight shards	105
Indexed tensor keys	101,295
Context configuration	1,048,576 tokens with YaRN
Runtime cache schema	deepseek_v4_v9
MTP / DSpark	Dropped from this runtime artifact

What this quant is

The 97.41 GiB policy recovers the projection-sensitive affine layout that was needed for coherent DSV4 generation instead of forcing every routed expert to the same 2-bit/group-size setting. It keeps small groups on gate and down projections, uses a larger group only for up projections, and lifts the gate projection in six sensitive layers.

Tensor role	Bits	Group size	Policy
Routed expert gate / w1	2	32	Default
Routed expert gate / w1 in layers 5, 14, 30, 34, 37, 42	3	32	Quality lifts
Routed expert down / w2	2	32	All routed layers
Routed expert up / w3	2	64	All routed layers
Attention, Compressor, Indexer, shared expert	4	64	Non-routed path
Token embedding and output head	8	64	Bookends
Norms, router, mHC, sinks and controls	Source dtype	—	Critical F32 retained

The index records 20,480 tensors at 2b/G32, 11,008 at 2b/G64, 1,536 at 3b/G32, 510 at 4b/G64, and two token bookends at 8b/G64. This is affine JANG, not JANGTQ.

JANG versus uniform MLX quantization

	This repository	Uniform MLX quant
Weight policy	Per tensor role, projection, and selected layer	One global/default policy
Routed gate/down/up groups	G32 / G32 / G64	Not independently represented
Sensitive gate lifts	Six layers at 3-bit	Not represented by a global bit width
DSV4 controls	Source dtype; critical F32 retained	Loader-dependent
Required runtime	DSV4-aware vMLX mixed-affine path	Stock uniform loader

No like-for-like accuracy benchmark against a uniform MLX artifact is claimed. A stock loader that applies one bit width to every quantized tensor cannot faithfully interpret this bundle's plan.

Runtime requirement

Use a current DSV4-aware vMLX runtime that supports:

• per-tensor JANG affine bits and group sizes;
• DSV4 SWA + CSA + HCA composite cache state;
• Compressor and Sparse Indexer state;
• the bundled official 0731 Python encoder and DSML output parser;
• native low, high, and max reasoning effort, with Low as the reasoning default.

Preview-era adapters that expose only High/Max or map omitted reasoning to Instruct are incompatible with the 0731 contract.

Native 0731 chat contract

The official release has no Jinja chat template. This repository does not synthesize chat_template.jinja; tokenizer_config.json.chat_template is unset. Use encoding/encoding_dsv4.py:

from encoding.encoding_dsv4 import (
    encode_messages,
    parse_message_from_completion_text,
)

messages = [{"role": "user", "content": "Explain why 17 is prime."}]

# Native default reasoning: thinking mode, Low effort.
prompt = encode_messages(messages, thinking_mode="thinking")

# Non-reasoning / Instruct behavior.
chat_prompt = encode_messages(messages, thinking_mode="chat")

# Explicit 0731 reasoning rails.
high_prompt = encode_messages(
    messages, thinking_mode="thinking", reasoning_effort="high"
)
max_prompt = encode_messages(
    messages, thinking_mode="thinking", reasoning_effort="max"
)

The four intended surfaces are Instruct, Reasoning Low, Reasoning High, and Reasoning Max. Low is the default when reasoning_effort is omitted in thinking mode. Tool calls use native DSML, and tool results are merged into user messages as <tool_result>...</tool_result> blocks. See encoding/README.md for the full contract.

Generation and stop contract

The unchanged upstream generation_config.json uses do_sample=true, temperature=1.0, top_p=1.0, BOS 0, and EOS 1. It has no top_k or repetition-penalty override. DeepSeek separately recommends top_p=0.95 for agentic scenarios.

DSV4-aware servers should recognize EOS 1 and the 0731 role-boundary tokens User 128803, Assistant 128804, and latest-reminder 128828 where the API surface uses boundary stopping.

Cache, context, and speculative decoding

The bundle preserves the 1M-token YaRN configuration, sliding window 128, and the layerwise compression schedule. Its deepseek_v4_v9 metadata names SWA, CSA, HCA, Compressor, and Indexer cache state. Generic TurboQuant KV is disabled and pool-cache quantization defaults off.

MTP / DSpark weights are intentionally absent from this runtime artifact. Speculative decoding would require a separate drafter plus atomic rollback of the complete DSV4 composite cache; this repository does not claim that path.

Validation boundary

Source identity, complete shard/index structure, the per-tensor affine plan, official encoder fixtures, tokenizer, generation config, and sampled source-versus-dequantized reconstruction checks were validated on the exact published bundle.

Live vMLX Python checks produced the exact non-thinking marker JANG-0731-OK, correctly answered seven times eight in natural language, and generated a coherent 48-token greedy explanation of why 17 is prime. Cached and full-refeed greedy top-1 tokens agreed at all 48 decode steps.

This is short-context generation/cache evidence. A complete final-bundle Low/High/Max, multi-turn DSML, 30K+ long-context recall, cache trim/restart, and pool-quantization matrix has not been completed, so those production claims are intentionally not made here.

Download

hf download JANGQ-AI/DeepSeek-V4-Flash-0731-JANG \
  --local-dir ~/models/DeepSeek-V4-Flash-0731-JANG

Korean summary

이 모델은 공식 deepseek-ai/DeepSeek-V4-Flash-0731 릴리스를 Apple Silicon 용으로 동적 affine JANG 양자화한 97.41 GiB 번들입니다. 이전 preview 모델이 아니며, 공식 0731 Python 인코더를 그대로 사용합니다. 기본 추론 모드는 Reasoning Low이고, Low/High/Max와 비추론 Instruct 모드를 지원합니다. 라우티드 전문가의 gate/down/up projection마다 비트와 그룹 크기를 다르게 적용했으며, 중요한 F32 제어 텐서는 보존했습니다. 짧은 생성과 캐시/리피드 토큰 일치는 확인했지만 최종 번들의 전체 장문 컨텍스트 및 DSML 매트릭스는 아직 완료되지 않았습니다.

Contact

eric@jangq.ai