docs: C engine exists now — retire the no-engine limitation, add torch-free inference section
Browse files
README.md
CHANGED
|
@@ -351,14 +351,23 @@ Two of these reversed conclusions that would otherwise have shipped on intuition
|
|
| 351 |
`live_parallel` 0.0 on BFCL — repeated instantiations of one schema, as
|
| 352 |
opposed to calls the query motivates in sequence.
|
| 353 |
- **768-token context.** 151 of 3,641 BFCL rows (4.1%) do not fit and score as misses.
|
| 354 |
-
- **
|
| 355 |
-
|
| 356 |
-
|
| 357 |
-
|
| 358 |
|
| 359 |
Scale is the honest explanation for most of this: ~1B unique tokens, no
|
| 360 |
pretraining phase, a corpus deliberately spent on depth instead of breadth.
|
| 361 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 362 |
## Model details
|
| 363 |
|
| 364 |
| | |
|
|
|
|
| 351 |
`live_parallel` 0.0 on BFCL — repeated instantiations of one schema, as
|
| 352 |
opposed to calls the query motivates in sequence.
|
| 353 |
- **768-token context.** 151 of 3,641 BFCL rows (4.1%) do not fit and score as misses.
|
| 354 |
+
- **Microcontrollers.** ~11.5MB at 2-bit is a property of the parameter count,
|
| 355 |
+
not a shippable artifact: 2-bit would need quantization-aware retraining this
|
| 356 |
+
model never had. The smallest thing that actually runs is the 48MB int8
|
| 357 |
+
build of the C engine below — Pi-class and up, not Cortex-M.
|
| 358 |
|
| 359 |
Scale is the honest explanation for most of this: ~1B unique tokens, no
|
| 360 |
pretraining phase, a corpus deliberately spent on depth instead of breadth.
|
| 361 |
|
| 362 |
+
## Torch-free inference (C engine)
|
| 363 |
+
|
| 364 |
+
The repo ships [cengine/](https://github.com/nikshepsvn/thimble/tree/master/cengine),
|
| 365 |
+
the full decoder — tokenizer, trunk, grammar walk, name head — in one
|
| 366 |
+
dependency-free C file. 48MB int8 weights, ~20ms load, ~350ms per full request
|
| 367 |
+
on an Apple M3, and fp32 output is byte-identical to this checkpoint run
|
| 368 |
+
through the Python stack on 100/100 checked eval rows. Export the weights from
|
| 369 |
+
this repo's checkpoint with `cengine/export.py`.
|
| 370 |
+
|
| 371 |
## Model details
|
| 372 |
|
| 373 |
| | |
|