Instructions to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Use Docker

docker model run hf.co/fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Ollama
How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Ollama:
```
ollama run hf.co/fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
```

Unsloth Studio

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF to start chatting

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Docker Model Runner:
```
docker model run hf.co/fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M
```

Lemonade

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Gemma-4-E2B-it-Hi-Fi-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

Gemma-4-E2B Q4_K_M GGUF — fraQtl Hi-Fi (calibration-aware quantization)

36% closer to the original model than the community Q4_K_M — at the same file size. The most faithful Gemma-4-E2B quant we've measured, on every arm we tested. Every number reproducible. Receipts inside.

Gemma-4-E2B Q4_K_M GGUF built with calibration-aware quantization: −36.0% code/math KLD and −27.2% general KLD to the original bf16 model versus the pinned community Q4_K_M baseline, at the same file size (−0.03%). A second 2.86 GB phone artifact is 17.5% smaller and still −31.8%/−26.7% versus that same-size baseline. Every number on this page is measured, pinned, and reproducible from the provenance table below.

Model Overview

Description

This repository contains two GGUF quantizations of google/gemma-4-E2B-it produced with fraQtl's calibration-aware per-tensor quantization: the same llama.cpp Q4_K machinery everyone uses, with per-tensor precision decisions driven by a calibration analysis instead of fixed defaults. The claim is smarter bits, not more bits.

Artifact	Bytes	True bpw (all tensors)	KLD code/math (vs original bf16)	KLD general	Verdict vs baseline
`Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf` (iso-size)	3,461,675,328	5.93	0.05846 (3-run, drift 0.0)	0.09646 (3-run, drift 0.0)	−36.0% / −27.2%
`Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf`	2,856,122,688	4.89	0.06231	0.09716	−31.8% / −26.7% (and 17.5% smaller)
bartowski Q4_K_M (pinned baseline)	3,462,678,272	5.93	0.09139 (3-run)	0.13249 (3-run)	—

Fidelity metric: symmetric top-20 KL divergence on Q8 teacher support, mean over positions; teacher is a Q8_0 quantization of the ORIGINAL bf16 checkpoint. Lower is better; this measures how faithfully the quantized model reproduces the original model's token distributions.

On-device: what it does on an iPhone (airplane mode)

These are unedited screenshots from our demo assistant running this model family entirely on an iPhone — on-device speech recognition, this model for understanding, deterministic Swift code executing the actions. The airplane icon in every status bar is the point: zero network at runtime.

The proof pair: the app says it booked — the Calendar shows it did



Spoken request (note the waveform icons): "Find some time on Thursday for VC meeting." The model finds the free slot, the code layer writes the event, a receipt chip states exactly what was written — and the reply is read aloud by on-device TTS ("Speaking — tap to stop").	Apple Calendar, same phone, still in airplane mode: "VC meeting", Thursday Jul 30, 11:30–12:30 — the exact event the chip claimed. Note the "◀ fraQtl" breadcrumb: navigated straight from the app. The app cannot claim a write that did not happen.

Scope, receipts, and speed



Asked about email, it answers "I do not have access to your emails" with the context tag `no connectors`. Then a two-step instruction (move + alert) executes as two real calendar writes.	Typed request, same behavior: finds a free 90-minute block and writes the event, receipt chip naming calendar and time.

The receipts panel: model sha256-verified on load, live decode rate, resident memory, thermal state, network status.	In-app benchmark, 256 tokens greedy: 21.6 tok/s sustained* against a stated gate of ≥15 tok/s — the number ships with its pass condition.*

Dijkstra's algorithm in C++, generated offline at ~11 tok/s interactive.

It also has opinions. (Every answer still carries its context tag.)

Measured on an iPhone-16-class device: ≈516 MB resident · ~12 tok/s interactive (last turn, includes prefill) · 21.6 tok/s sustained on the 256-token greedy benchmark (gate ≥15: PASS) · sha256-verified artifact · llama.cpp b10075 · Metal · mmap · airplane mode throughout.

What it can do (all offline): voice input and spoken replies, calendar questions, find-a-slot-and-book, move/retitle events, chained reminders, contacts-aware event titles, reading documents and photos, general knowledge, code generation.

What it cannot do — by design, and labeled as such in the app: no email, no messages, no network calls of any kind (there are no connectors, and it says so when asked); no action without a receipt chip — if a write fails, no chip appears; and the model never announces an action itself — a deterministic code layer executes and reports.

Artifact note: the demo runs Gemma-4-E2B-it-fraQtl-SmartEdge-IQ3XXS.gguf — in this repository now, 2.45 GB, sha256 761c7002…816776, byte-identical to the sha shown in the receipts panel screenshot above. What you download is exactly what the phone is running. Fidelity receipts (KLD ladder) for this edge variant are the next lane item; until they land, its published evidence is the on-device behavior and benchmark shown here. The Q4_K_M and Phone artifacts carry full fidelity receipts today.

License/Terms of Use

Apache 2.0 for the quantization artifacts and receipts in this repository. The model weights derive from google/gemma-4-E2B-it and remain subject to Google's Gemma terms (license link). Nothing here modifies the base model's own usage terms.

Use Case

On-device and edge inference of Gemma-4-E2B-it via llama.cpp and llama-cpp-python where fidelity to the original model matters at 4-bit-class sizes: assistants, code/math workloads, and phone/laptop deployments (the Phone artifact targets memory-constrained devices at 2.86 GB).

Release Date

Hugging Face 07/2026 via this repository.

References

Serving lane (KV-cache compression for vLLM, same receipt discipline): Qwen3-4B kit · Mistral-7B kit
All fraQtl artifacts: huggingface.co/fraQtl

Model Architecture

Base model: google/gemma-4-E2B-it (instruction-tuned E2B). The GGUFs were converted from the pinned bf16 checkpoint with llama.cpp's convert_hf_to_gguf.py --outtype bf16, then quantized with llama-quantize using per-tensor --tensor-type overrides selected by fraQtl's calibration-aware analysis on a code+math calibration packet. How the per-tensor decisions are made is not published; the recipe hash, the imatrix, and every artifact hash are, so the result is verifiable end to end.

Phone artifact: identical transformer quantization; the per-layer-embedding (PLE) table is additionally demoted to Q4_K. Speed-neutral by construction: bandwidth-relevant bytes (everything except the two lookup tables) are byte-identical between the two artifacts (1,188,795,532 B).

True bits-per-weight: "Q4_K_M" for E2B is really 5.3–5.9 true bits/weight because the 2.35B-parameter per-layer-embedding table dominates bytes. We publish true average bits/weight over all tensors AND transformer-resident ("bandwidth-relevant") bytes — E2B decode reads ≈1.15 GB of transformer weights per token; the PLE table is a ≈7 KB/token lookup. Size comparisons on this page are total bytes; speed-relevant comparisons are bandwidth-relevant bytes.

Input / Output

Input: text (Gemma chat format)
Output: text
Context length: as the base model; evaluation on this page used ctx 2048 (calibration) and ctx 512 (perplexity sanity metric).

Calibration Dataset

Dataset: fraQtl Apex code+math packet (sha256 9bec84a28dcb0c940047e6084561a00857fd610b8a1e148cc38e27929e0a7e02), decontaminated against the eval slices.
Data collection method: Automated (curated public code/math text).
Properties: full packet, 212 chunks / 434,176 tokens; importance matrix computed with llama-imatrix at ctx 2048 on the BF16 substrate. The exact imatrix.dat ships in this repository (sha256 892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c) so the quantization is reproducible.
Recorded deviation from the 26B build pattern: single full-packet imatrix; no imatrix-budget sweep (E1 lean scope).

Usage

Receipt-validated stack: llama.cpp @ 78d2f524682d9fee790a6460c93d018dafeb5229 (2026-07-05) / llama-cpp-python==0.3.33 (vendors that commit). Newer releases should work; the pin is what every number on this page was measured with.

pip install llama-cpp-python==0.3.33
hf download fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf --local-dir .

from llama_cpp import Llama

llm = Llama(model_path="Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf", n_ctx=4096, seed=0)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain in two sentences why the sky is blue."}],
    temperature=0.0,
)
print(out["choices"][0]["message"]["content"])

Or with llama.cpp directly:

llama-cli -m Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf -cnv -p "You are a helpful assistant."

Chat template / turn-token note: the Gemma chat template is embedded in the GGUF — use create_chat_completion (or llama-cli -cnv) and let the runtime apply it. Gemma turns are delimited by <start_of_turn>user / <start_of_turn>model and terminated by <end_of_turn>, and the assistant role is named model, not assistant. If you hand-roll prompts, make sure <end_of_turn> is in your stop set: stacks that only stop on <eos> will run past the turn boundary and generate spurious extra turns.

Evaluation

All evaluation is fidelity to the ORIGINAL gemma-4-E2B-it (the axis a user migrating from the bf16 model experiences), on an A100-80GB with seed 0, deterministic settings, 3 independent runs where stated (run-to-run drift 0.0). Eval slices: code/math 400-record held-out (sha256 cce6860274ab715e3ab8493c9f817b1c29ff99c148f83a25de8944e027db1665); general chat/tool/wikitext 400-record (sha256 b10a79caf2c17cc10cd1edcae44d4655278baf5b09b6a867b4d3ade2f996b276).

Fidelity to the original model (KLD, lower is better)

Arm	bytes	true bpw	KLD code/math ↓	KLD general ↓	top-1 agree (gen)
bartowski Q4_K_M (community)	3,462,678,272	5.93	0.09139 (3-run)	0.13249 (3-run)	0.888
unsloth Q4_K_M (community)	3,106,738,272	5.32	0.10348	0.14767	0.882
google QAT q4_0	3,349,516,256	5.76	0.24730	0.35353	0.831
unsloth qat UD-Q4_K_XL	2,620,370,976	4.50	0.20666 (3-run)	0.30275 (3-run)	0.849
fraQtl Hi-Fi (iso-size)	3,461,675,328 (−0.03% vs baseline)	5.93	0.05846 (3-run, −36.0%)	0.09646 (3-run, −27.2%)	0.905
fraQtl Hi-Fi Phone (PLE-Q4_K)	2,856,122,688 (−17.5%)	4.89	0.06231 (−31.8%)	0.09716 (−26.7%)	—

QAT context (read before comparing QAT rows): the QAT GGUFs descend from google/gemma-4-E2B-it-qat-q4_0-unquantized — a retrained checkpoint. Unsloth's published E2B QAT mean KLD 0.00173 is measured against that retrained checkpoint, not against the original gemma-4-E2B-it. All rows above measure fidelity to the ORIGINAL model, which is the axis a user migrating from the bf16 model experiences. Both statements are true simultaneously; they answer different questions.

GSM8K

Metric	fraQtl Hi-Fi (iso-size)	bartowski Q4_K_M	Verdict
GSM8K, 0-shot instructed chain-of-thought, n=200, seed 0, greedy	86.0% (172/200)	79.0% (158/200)	diff +7.0pp, 95% tie band ±7.42pp → statistical tie

The +7.0pp difference is inside the ±7.42pp tie band (1.96·sqrt((p₁(1−p₁)+p₂(1−p₂))/n), recomputed for this model at n=200). We make no win claim on GSM8K. The protocol label is exact: 0-shot with a chain-of-thought instruction; no few-shot exemplars; not the full 1319-item set. Phone artifact: not run (lean scope).

Wikitext-2 perplexity (sanity flag — disclosed loss)

Metric	fraQtl Hi-Fi (iso-size)	bartowski Q4_K_M	Verdict
Wikitext-2 PPL, ctx 512	20009.7	18005.3	LOSS (flag)

One result runs against this artifact: it is worse on raw-web perplexity. Two caveats, neither of which removes the flag: (1) both absolute values are harness-inflated — ctx-512 chunked raw text on an instruction-tuned model is a sanity metric, not a capability measure; (2) the code/math-domain imatrix likely explains the relative gap (domain shift). Note the general-slice KLD — which includes a wikitext-anchor domain — still favors this artifact. PPL here is a sanity flag, not a gate.

Reproducibility

Deterministic eval rig: 3 independent KLD runs are byte-identical (drift 0.0). Verdicts are by mean; no confidence-interval claim is made on KLD. GSM8K uses the tie-band discipline above.

Provenance

Field	Value
Base model	`google/gemma-4-E2B-it` @ `179516f0c449474fdc46f08f30ead5b11e178497`
Base checkpoint sha256	`model.safetensors` = `2db5482b20d746879bb3ef79b5203e9075a2e2b98f54ec7c2f281c1477ddc550` (10,246,621,918 B)
Conversion + quantization	llama.cpp @ `78d2f524682d9fee790a6460c93d018dafeb5229` (2026-07-05); `convert_hf_to_gguf.py --outtype bf16` then `llama-quantize` with per-tensor `--tensor-type` overrides
Calibration	fraQtl Apex code+math packet (sha256 `9bec84a28dcb0c940047e6084561a00857fd610b8a1e148cc38e27929e0a7e02`), `llama-imatrix` ctx 2048, BF16 substrate, full packet (212 chunks / 434,176 tokens)
imatrix (ships in this repo)	`imatrix.dat` sha256 `892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c`
Recipe sha256	`69f2151874412bcda4df1028c5b76743b1a60afa9f0a0abb2ff87c26a982bd37` (phone artifact differs only in the PLE embedding override)
Iso-size GGUF	`Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf` — 3,461,675,328 B, sha256 `d008ba9d736b1d51e55d1d1944196441c1dbfe4fe9eb4ebd9bfbda4c2f36e013`, true bpw 5.93, bandwidth-relevant bytes 1,188,795,532
Phone GGUF	`Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf` — 2,856,122,688 B, sha256 `d535787b323f79d8faf3867501c028dc05be4e0a3413a6ccdf54cb88bb737866`, true bpw 4.89, bandwidth-relevant bytes 1,188,795,532 (identical by construction — speed-neutral)
Eval runtime	`llama-cpp-python==0.3.33` (vendored llama.cpp `78d2f5246`), A100-80GB, seed 0, deterministic (3-run drift 0.0)
Fidelity metric	symmetric top-20 KLD on Q8 teacher support, mean over positions; teacher = `unsloth/gemma-4-E2B-it-GGUF` Q8_0 @ `0314792d7f1f7e229411f620751375812bb9faf2` (sha256 `605d3c2647d7c58c1e4b5375ccb5702acf94c2611b4c8d4877812f8fdd32d053`), quantized from the ORIGINAL bf16
Eval slices	code/math 400-record held-out (sha256 `cce6860274ab715e3ab8493c9f817b1c29ff99c148f83a25de8944e027db1665`); general chat/tool/wikitext 400-record (sha256 `b10a79caf2c17cc10cd1edcae44d4655278baf5b09b6a867b4d3ade2f996b276`)
Baseline (iso-size)	`bartowski/google_gemma-4-E2B-it-GGUF` Q4_K_M @ `b5e99bd964eaacc27ba484bb2eb3e9f6160b9143` — 3,462,678,272 B, sha256 `b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705`, true 5.93 bpw (all tensors)
Probe receipt	`gemma4_e2b_probe_2026-07-19.partial.json` sha256 `8511c8d6f1530a94e66f1f344e1e0bed57cc692819e36ca801946a71452f036a`
Drift	3-run KLD drift 0.0 (deterministic rig)

Model Limitations

Raw-web perplexity flag: wikitext-2 PPL (ctx 512) is worse than the community baseline (20009.7 vs 18005.3) — see the Evaluation section for the domain-shift note. If your workload is raw web-text continuation rather than chat/code/math, weigh this flag accordingly.
Calibration domain: the imatrix is code+math. Both eval slices (including the general chat/tool/wikitext slice) favor this artifact on KLD, but calibration is not domain-neutral by construction.
Lean eval scope (E1): GSM8K n=200 only; no MMLU, MATH-500, or long-context retrieval benchmarks were run for this model. The GSM8K result is a statistical tie, not a win.
Phone artifact: KLD-evaluated (single run) and smoke-tested; GSM8K/PPL were not run on it.
Quantization is lossy by construction; the evidence standard here is measured distributional fidelity (KLD) with pinned receipts, not bit-identity.
The base model's own limitations and biases are inherited unchanged.

Files

File	What	sha256
`Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf`	Iso-size artifact (3.46 GB, true bpw 5.93)	`d008ba9d736b1d51e55d1d1944196441c1dbfe4fe9eb4ebd9bfbda4c2f36e013`
`Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf`	Phone artifact (2.86 GB, true bpw 4.89, PLE at Q4_K)	`d535787b323f79d8faf3867501c028dc05be4e0a3413a6ccdf54cb88bb737866`
`imatrix.dat`	Calibration importance matrix (reproducibility)	`892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c`

More from fraQtl

Weight quantization is the on-device lane. The serving lane — KV-cache compression sidecars for vLLM with per-user retrieval verification — lives at fraQtl/qwen3-4b-instruct-2507-kv-sidecars and fraQtl/mistral-7b-instruct-v0.3-kv-sidecars. Same standard per artifact: pinned provenance, measured numbers, results reported in both directions. Org page: huggingface.co/fraQtl.