EchoCoder — FSI Coding Specialist

A small, from-scratch code-generation model. No pretrained weights, no base model, no tokenizer dependency. Trained on a corpus we generate. Runs locally on CPU. Part of the FSI (Ferrell Synthetic Intelligence) sovereign, off-grid, private-first model stack.

Why "our own model"

• Architecture (RMSNorm + RoPE + SwiGLU + causal MHA) is written from scratch in echocoder.py — not a fine-tune of someone else's checkpoint.
• Trained from zero on tinycode.txt, a clean Python corpus we synthesize, so the whole pipeline is reproducible and auditable.
• Exports to TorchScript (portable CPU runtime) and GGUF (llama.cpp compatible — tensor names match llama.cpp so it loads in llama.cpp / Ollama / LM Studio).

Efficiency (the contest angle)

• ~720K parameters, char-level vocab → tiny, fast, cheap to ship.
• Runs on CPU with no GPU, no API key, no cloud. Your code never leaves the machine.

Train

python3 echocoder.py            # builds corpus, trains, exports .pt + GGUF + demo

Config lives in Cfg (d_model, n_layers, ctx, ffn_mult). Retrain on GPU by pointing device at CUDA — the code is device-agnostic.

Use (local, private)

import echocoder as e, torch
m = e.EchoCoder(e.Cfg()); m.load_state_dict(torch.load("echocoder.pt")); m.eval()
print(e.generate(m, e.Cfg(), prompt="def fib", length=160))

Or load echocoder.f32.gguf in llama.cpp / Ollama for a fully offline runtime.

Files

• echocoder.py — model + corpus generator + trainer + generator + GGUF exporter
• tinycode.txt — generated training corpus
• echocoder.pt / echocoder_ts.pt — weights / TorchScript
• echocoder.f32.gguf — llama.cpp-compatible export

The FSI stack

Model	Role
NanoMind	Efficiency proof: ~7K-param vision model, sub-10ms CPU, 95%+ MNIST
EchoCoder (this repo)	Coding specialist — small, from-scratch, local-first
Sovereign Research Analyst	Local-first, legal public-OSINT collaborator

See STRATEGY.md for the full plan (how Mistral / llama.cpp / Karpathy shaped it, and the sovereign-AI niche we are claiming).