Upload folder using huggingface_hub

.gitattributes

@@ -84,3 +84,5 @@ function-calls/functiongemma-270m-cellm/functiongemma-270m-int8-e4g32.cellm filt
 function-calls/functiongemma-270m-cellm/functiongemma-270m-f16.cellm filter=lfs diff=lfs merge=lfs -text
 privacy/privacy-filter-int4-g32-f16s.cellm filter=lfs diff=lfs merge=lfs -text
 privacy/privacy-filter-int3-g128.cellm filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-int4-g64/LFM2-2.6B-int4-g64.cellm filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-int4-g64/tokenizer.json filter=lfs diff=lfs merge=lfs -text

LFM2-2.6B-int4-g64/LFM2-2.6B-int4-g64.cellm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74e7429152b0990d1f7a26a58bfcfafc853d7b1d210ede1439589c7118d895fb
+size 1686190208

LFM2-2.6B-int4-g64/README.md

@@ -0,0 +1,159 @@
+---
+library_name: cellm
+license: apache-2.0
+tags:
+- cellm
+- rust
+- quantized
+- int4
+- cpu
+- lfm2
+- reasoning
+---
+
+# LFM2-2.6B INT4 g64 (cellm)
+
+Liquid Foundation Model 2 (2.6B) converted to cellm format, MLX-style affine
+INT4 at group size 64. Runs on CPU with an ARM `SDOT` W4A8 prefill kernel.
+
+This build emits `<think>` reasoning blocks before its answer.
+
+## Files
+
+| file | size |
+| ---- | ---- |
+| `LFM2-2.6B-int4-g64.cellm` | 1,686,190,208 bytes (1.57 GiB) |
+| `tokenizer.json` | 17 MB |
+
+## Model details
+
+- **Format**: cellm (`.cellm`)
+- **Precision**: affine INT4, nibbles packed 8-per-`u32`, f32 `.scales` /
+  `.biases` sidecars at group size 64
+- **Parameters**: 2,697,198,592
+- **Architecture**: LFM2 hybrid, 30 layers (LIV convolution + grouped-query
+  attention)
+- **Hidden size**: 2048
+- **Heads**: 32 query / 8 key-value
+- **Vocab**: 128,000, tied embeddings
+- **RoPE theta**: 10,000,000
+- **Chat format**: ChatML, EOS 124900, `<think>` = 124901
+
+## Usage
+
+```sh
+./target/release/infer \
+  --model LFM2-2.6B-int4-g64.cellm \
+  --tokenizer tokenizer.json \
+  --prompt "What is the capital of France?" \
+  --chat --gen 120 --temperature 0 --stop-eos
+```
+
+Output:
+
+```
+Prefill: 16 tokens in 4.31s (next=124901)
+<think>The user is asking a straightforward factual question: "What is the
+capital of France?"
+
+1.  **Identify the core entity and attribute:** The entity is "France", the
+    attribute is "capital".
+2.  **Retrieve knowledge:** My internal knowledge base contains information
+    about world capitals. The capital of France is Paris.
+3.  **Formulate the answer:** The direct answer is "Paris".
+4.  **Check for nuances or additional context:** Sometimes people mention
+    "Île-de-France" as the region, but the capital city is definitely Paris.
+Decode: 120 tokens in 18.23s
+```
+
+Code generation:
+
+```sh
+./target/release/infer \
+  --model LFM2-2.6B-int4-g64.cellm \
+  --tokenizer tokenizer.json \
+  --prompt "Write a Python function that reverses a string. Answer with code only." \
+  --chat --gen 200 --temperature 0 --stop-eos
+```
+
+~~~
+Prefill: 23 tokens in 0.84s (next=124901)
+<think>The user wants a Python function that reverses a string.
+The constraint is "Answer with code only." This means I should not include
+explanations, comments outside the function, or any other text. Just the
+function definition.
+
+There are several ways to do this in Python:
+1. Slicing: `return s[::-1]` - This is the most Pythonic and efficient way.
+2. `reversed()` function combined with `join`: `return ''.join(reversed(s))`
+
+The slicing method `[::-1]` is concise, readable, and standard for this task.
+
+So the code should look like:
+```python
+def reverse_string(s):
+    return s[::-1]
+```
+Decode: 200 tokens in 29.19s
+~~~
+
+Both runs are greedy (`--temperature 0`) on an Apple M-series CPU with the
+model already in the page cache. Raise `--gen` to let the model finish past the
+reasoning block.
+
+## Performance
+
+Prefill uses a W4A8 kernel: activations are quantized to INT8 and dotted
+against the packed nibbles with ARM `SDOT`. Affine nibbles are unsigned 0..15,
+so they reinterpret as non-negative `i8` exactly, and the `bias * sum(x)` term
+stays in f32 against unquantized activations. Greedy output is byte-identical
+to the f32 path.
+
+| prompt tokens | f32 NEON | W4A8 | speedup |
+| ------------- | -------- | ------ | ------- |
+| 70 | 11.30s | 2.28s | 4.96x |
+| 210 | 27.74s | 6.63s | 4.18x |
+| 460 | 61.99s | 19.68s | 3.15x |
+
+The speedup shrinks as prompts grow because attention is quadratic and not
+covered by this kernel.
+
+Notes on measuring:
+
+- Run-to-run variance is about ±7%, so treat any single timing as a range.
+- A cold page cache on the 1.6 GiB file moves prefill by more than 2x. Warm it
+  with a throwaway `--gen 1` run first.
+- Peak resident size is ~1.7 GB. Measure with `vmmap --summary` "Physical
+  footprint"; `ps` RSS counts clean file-backed mmap pages and overstates this
+  by roughly 3x.
+- Override the weight cache bound with `CELLM_WEIGHT_CACHE_MB` if needed.
+
+## Known issues
+
+- Metal is not implemented for the LFM2 runner. `--backend metal` silently
+  falls back to CPU.
+- The tokenizer carries vision and audio tokens this text-only build does not
+  define, so startup prints hundreds of benign lines like
+  `Token '<|img_row_9_col_9|>' was expected to have ID '124996' but was given ID 'None'`.
+  Filter with `2>/dev/null` or `2>&1 | grep -v 'tokenizers::'`.
+
+## Quantization
+
+INT4 is the floor for this model. INT2 was tested end-to-end by writing
+INT2-derived `(q, scale, bias)` triples into the INT4 container, which
+reproduces INT2 numerics exactly. Even the gentlest variant measured (FFN-only,
+group 64, MSE-optimal clipping, 34.2% mean weight error) produced grammatical
+but broken output: it lost the `<think>` behavior, hallucinated a phantom input
+token, inverted basic physics, and would not write code when asked.
+
+| scheme | rel. error | cosine |
+| ------ | ---------- | ------ |
+| int4 g64 | 0.00% | 1.0000 |
+| int3 g64 | 20.4% | 0.9872 |
+| int2 g64 mse | 33.8% | 0.9437 |
+| int2 g64 | 46.0% | 0.9161 |
+| int2 g128 | 53.5% | 0.8946 |
+
+## License
+
+Apache 2.0, same as upstream LFM2.

LFM2-2.6B-int4-g64/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
+size 17905598