fraQtl/qwen3-4b-instruct-2507-kv-sidecars
Updated
KV-Cache Receipts — 3× the 128K users per A100 (vLLM)
Three-arm, needle-gated serving receipts (fp16 / fp8-KV / compressed) — reproducible for ≈$12 with one command.
Note Start here — 128K receipts: 140.9 tok/s @ 6 preemption-free users vs fp16's 66.6 @ 2, all needles verified
fraQtl/mistral-7b-instruct-v0.3-kv-sidecars
UpdatedNote 2.4× the fp16 KV pool at ≈95-100% decode speed, 8K/32K, byte-exact independent re-run
fraQtl/fraqtl-sm80-runtime
UpdatedNote The prebuilt A100 wheel + one-command repro script — verify every number yourself, ≈$12