Update README.md
Browse files
README.md
CHANGED
|
@@ -85,41 +85,7 @@ loop on the teacher too** (intrinsic); the other ~3.6 points are the price of pr
|
|
| 85 |
|
| 86 |
> A **knowledge-recovery LoRA** (separate, optional adapter) is being trained to add capacity back and
|
| 87 |
> close this gap — see the model index. The pruned model remains a strong ~34%-smaller option; just
|
| 88 |
-
> know it trades ~3.6 points of termination robustness for the
|
| 89 |
-
|
| 90 |
-
## Coding benchmark results
|
| 91 |
-
|
| 92 |
-
These scores are for `0xSero/GLM-5.2-504B` (REAP keep-168 + Router-KD, NVFP4) served through an OpenAI-compatible endpoint.
|
| 93 |
-
|
| 94 |
-
| Benchmark | Score | Notes |
|
| 95 |
-
|---|---:|---|
|
| 96 |
-
| Terminal-Bench 2.1 full-89 | **70.5%** | Locked 89-task board-faithful config, 5 attempts, `terminus-2`, `reasoning_effort=max`, temperature 1.0. |
|
| 97 |
-
| Aider Polyglot | **90.1%** | Aggregate language score across Python, JavaScript, Java, C++, Go, and Rust. |
|
| 98 |
-
|
| 99 |
-
### Aider Polyglot language breakdown
|
| 100 |
-
|
| 101 |
-
`FULL` is the unpruned GLM-5.2 baseline. `REAP` is this 504B REAP + Router-KD checkpoint. Scores are percentages.
|
| 102 |
-
|
| 103 |
-
| Language | FULL | REAP | Change |
|
| 104 |
-
|---|---:|---:|---:|
|
| 105 |
-
| Python | 97.1 | 94.1 | -3.0 |
|
| 106 |
-
| JavaScript | 93.9 | 93.9 | 0.0 |
|
| 107 |
-
| Java | 85.1 | 85.1 | 0.0 |
|
| 108 |
-
| C++ | 92.3 | 96.2 | +3.9 |
|
| 109 |
-
| Go | 89.7 | 87.2 | -2.5 |
|
| 110 |
-
| Rust | 86.7 | 83.3 | -3.4 |
|
| 111 |
-
|
| 112 |
-
## Serving (vLLM)
|
| 113 |
-
|
| 114 |
-
```bash
|
| 115 |
-
vllm serve 0xSero/GLM-5.2-504B \
|
| 116 |
-
--tensor-parallel-size 8 \
|
| 117 |
-
--quantization modelopt_fp4 \
|
| 118 |
-
--kv-cache-dtype fp8 \
|
| 119 |
-
--trust-remote-code \
|
| 120 |
-
--max-model-len 262144
|
| 121 |
-
```
|
| 122 |
-
|
| 123 |
### Recommended serving — recover most of the loop gap for free
|
| 124 |
|
| 125 |
Two no-retrain knobs, measured at n=2000:
|
|
|
|
| 85 |
|
| 86 |
> A **knowledge-recovery LoRA** (separate, optional adapter) is being trained to add capacity back and
|
| 87 |
> close this gap — see the model index. The pruned model remains a strong ~34%-smaller option; just
|
| 88 |
+
> know it trades ~3.6 points of termination robustness for the
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
### Recommended serving — recover most of the loop gap for free
|
| 90 |
|
| 91 |
Two no-retrain knobs, measured at n=2000:
|