# EchoCoder — FSI Coding Specialist

> A small, **from-scratch** code-generation model. No pretrained weights, no base model,
> no tokenizer dependency. Trained on a corpus we generate. Runs locally on CPU.
> Part of the **FSI (Ferrell Synthetic Intelligence) sovereign, off-grid, private-first**
> model stack.

## Why "our own model"
- Architecture (RMSNorm + RoPE + SwiGLU + causal MHA) is written from scratch in
  `echocoder.py` — not a fine-tune of someone else's checkpoint.
- Trained from zero on `tinycode.txt`, a clean Python corpus we synthesize, so the whole
  pipeline is reproducible and auditable.
- Exports to **TorchScript** (portable CPU runtime) **and GGUF** (llama.cpp compatible —
  tensor names match llama.cpp so it loads in `llama.cpp` / Ollama / LM Studio).

## Efficiency (the contest angle)
- ~**720K parameters**, char-level vocab → tiny, fast, cheap to ship.
- Runs on CPU with no GPU, no API key, no cloud. Your code never leaves the machine.

## Train
```bash
python3 echocoder.py            # builds corpus, trains, exports .pt + GGUF + demo
```
Config lives in `Cfg` (d_model, n_layers, ctx, ffn_mult). Retrain on GPU by pointing
`device` at CUDA — the code is device-agnostic.

## Use (local, private)
```python
import echocoder as e, torch
m = e.EchoCoder(e.Cfg()); m.load_state_dict(torch.load("echocoder.pt")); m.eval()
print(e.generate(m, e.Cfg(), prompt="def fib", length=160))
```
Or load `echocoder.f32.gguf` in llama.cpp / Ollama for a fully offline runtime.

## Files
- `echocoder.py` — model + corpus generator + trainer + generator + GGUF exporter
- `tinycode.txt` — generated training corpus
- `echocoder.pt` / `echocoder_ts.pt` — weights / TorchScript
- `echocoder.f32.gguf` — llama.cpp-compatible export

## The FSI stack
| Model | Role |
|-------|------|
| **NanoMind** | Efficiency proof: ~7K-param vision model, sub-10ms CPU, 95%+ MNIST |
| **EchoCoder** (this repo) | Coding specialist — small, from-scratch, local-first |
| **Sovereign Research Analyst** | Local-first, *legal* public-OSINT collaborator |

See `STRATEGY.md` for the full plan (how Mistral / llama.cpp / Karpathy shaped it, and
the sovereign-AI niche we are claiming).