# NanoWhale-100M (cellm)

NanoWhale-100M is a tiny 100M parameter model based on the DeepSeek-V4 architecture (MLA + MoE), converted to the cellm format for efficient on-device inference.

## Model Details

- **Architecture**: DeepSeek-V4 (MLA + MoE)
- **Parameters**: ~100M
- **Layers**: 8
- **Hidden Size**: 320
- **MLA Config**: 8 heads, 96 head_dim, 32 qk_rope_head_dim
- **MoE Config**: 4 routed experts, 1 shared expert, 2 experts per token
- **Vocab Size**: 129,280

## Files

| File | Format | Size |
|------|--------|------|
| nanowhale-100m-v1.cellm | f16/int8 | 210 MB |

## Usage

```sh
./target/release/infer \
  --model nanowhale-100m-v1.cellm \
  --tokenizer tokenizer.json \
  --prompt "<｜begin of sentence｜><｜User｜>what's sycophancy?<｜Assistant｜>" \
  --gen 100 --temperature 0 --backend cpu
```

## Notes

- This model uses **Multi-head Latent Attention (MLA)** for efficient KV cache management.
- It uses **DeepSeekMoE** with routed and shared experts.
- Designed for extremely lightweight inference on mobile and edge devices.