Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -18,10 +18,13 @@ packed signed INT4 weights with group-256 regular-Hadamard **ConvRot** rotation
|
|
| 18 |
(arXiv [2512.03673](https://arxiv.org/abs/2512.03673)), activations rotated online
|
| 19 |
and dynamically quantized inside the kernel. Calibration-free.
|
| 20 |
|
| 21 |
-
Mixed recipe: 224 block Linears = **
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Samples
|
| 27 |
|
|
@@ -48,7 +51,7 @@ fails with a clear "format not available" error.
|
|
| 48 |
| bf16 | 252 s |
|
| 49 |
|
| 50 |
Weights: **9.25 GB** vs 28 GB bf16 (−67%) — fits a 24 GB card with headroom where
|
| 51 |
-
bf16 needs weight streaming. PSNR vs bf16 (same seed):
|
| 52 |
|
| 53 |
## Reproduce
|
| 54 |
|
|
|
|
| 18 |
(arXiv [2512.03673](https://arxiv.org/abs/2512.03673)), activations rotated online
|
| 19 |
and dynamically quantized inside the kernel. Calibration-free.
|
| 20 |
|
| 21 |
+
Mixed recipe: 224 block Linears = **128 INT4 + 96 int8_tensorwise fallback**,
|
| 22 |
+
selected by measured per-layer sensitivity: each layer was swapped alone to
|
| 23 |
+
W4A4 on real sampling inputs, ranked by final-output drift, then chosen by
|
| 24 |
+
impact-per-byte at a fixed size budget (all `wv`/`wk`, 27/28 `wo`, the most
|
| 25 |
+
sensitive `mlp.down`/`gate`/`wq` layers). +2.2 dB over the class-heuristic
|
| 26 |
+
recipe at identical size and speed. The shared modulation projector `tproj`
|
| 27 |
+
and the text-fusion transformer stay bf16.
|
| 28 |
|
| 29 |
## Samples
|
| 30 |
|
|
|
|
| 51 |
| bf16 | 252 s |
|
| 52 |
|
| 53 |
Weights: **9.25 GB** vs 28 GB bf16 (−67%) — fits a 24 GB card with headroom where
|
| 54 |
+
bf16 needs weight streaming. PSNR vs bf16 (same seed): 24.3 dB (min 18.6, max 31.0).
|
| 55 |
|
| 56 |
## Reproduce
|
| 57 |
|