# Reproducing these APEX quants

No-imatrix, CPU-only path. All MIT (see NOTICE).

## Pinned
- **llama.cpp** — a build supporting `kimi_linear` + `kimi-k2` pre-tokenizer +
  `--tensor-type-file` (e.g. commit `bbf4a8a`/b8833 or newer).
- **apex-quant** — commit `a445a12` (https://github.com/localai-org/apex-quant),
  for `generate_config.sh` (bundled here).

## Baseline
```bash
hf download bartowski/moonshotai_Kimi-Linear-48B-A3B-Instruct-GGUF \
  --include "moonshotai_Kimi-Linear-48B-A3B-Instruct-bf16/*" --local-dir .
```
(A correctly-converted bf16 GGUF: kimi-k2 tokenizer with BPE merges present.)

## Config (imatrix-free "balanced")
The final configs are included (`configs/kimi_balanced.txt`, `configs/kimi_handroll.txt`).
To regenerate:
```bash
# 43 layers, layer 0 dense
bash generate_config.sh --profile balanced --layers 43 --dense-layers 1 -o kimi_bal.base.txt
# add the MLA + KDA(ssm) tensors the stock generator misses
python patch_kimi_config.py kimi_bal.base.txt configs/kimi_balanced.txt
# hand-roll variant: KDA recurrence pinned to Q8_0 (found to make no PPL difference)
python patch_kimi_config.py kimi_bal.base.txt configs/kimi_handroll.txt --ssm-type Q8_0
```

## Quantize (base type Q6_K; NO imatrix)
```bash
SHARD=moonshotai_Kimi-Linear-48B-A3B-Instruct-bf16/moonshotai_Kimi-Linear-48B-A3B-Instruct-bf16-00001-of-00003.gguf
llama-quantize --tensor-type-file configs/kimi_balanced.txt \
  "$SHARD" Kimi-Linear-48B-A3B-Instruct-APEX-balanced.gguf Q6_K
```

## Evaluate
```bash
# NOTE: PPL prints to STDERR — capture 2>&1.
llama-perplexity -m Kimi-Linear-48B-A3B-Instruct-APEX-balanced.gguf \
  -f wiki.test.raw -ngl 999 --chunks 200 2>&1 | grep -oP 'Final estimate: PPL = \K[0-9.]+'
```
Expected: ~7.38 (wikitext-2, 200×512). bf16 reference pending.