staccs/Qwen3.5-0.8B-lecore-assimilated

Qwen3.5-0.8B — leCore/Unicron assimilated

This is Qwen3.5-0.8B after one pass of leCore's Unicron assimilation: every attention/MLP projection matrix spectrally filtered by a Marchenko–Pastur criterion — keep the learned spectral outliers, drop the still-random bulk — then rebuilt into the original architecture. Same tensor names, same shapes: it loads exactly like the original with transformers, llama.cpp, or anything else that loads Qwen3.5-0.8B.

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("staccs/Qwen3.5-0.8B-lecore-assimilated")
t = AutoTokenizer.from_pretrained("staccs/Qwen3.5-0.8B-lecore-assimilated")

What was measured (the honesty contract)

The assimilation kit refuses to call a model anything but an UNVERIFIED claim until perplexity is measured before vs after. Our run (RTX 3060, 2026-08-16, fp16):

metric	original	assimilated	delta
perplexity, kit's eval sample	2.506	2.528	+0.86%
perplexity, full wikitext-2 test (sliding window, 2048/1024)	15.2443	15.2389	−0.04%

The kit's own standard, quoted: "RETENTION MEASURED: within 2% on this sample. Run a full corpus (wikitext) before shipping." Done: on the full wikitext-2 test set the assimilated model is marginally better than the original. Same pass, independently reproduced on an RTX 4090 with digit-identical sample deltas (+0.86% on both boxes) — the pipeline is deterministic end to end.

What's in the repo

• model.safetensors-* — the dense assimilated weights (full original shape, because transformers/llama.cpp demand the original architecture).
• *.unicron_report.json — the receipt: per-shard, per-layer rank reports of what the Marchenko–Pastur filter kept vs dropped.
• *.lecore.safetensors — the factored form: each filtered layer as its thin (U, V) pair. This is the model's true information size (~2× smaller on the rehearsal subject). Loads via leCore's unicron_reconstruct; a transformers shim that runs the factored form directly is upstream's planned next step.

How it works

Marchenko–Pastur filtering treats each weight matrix's singular value spectrum as signal + noise: the MP distribution predicts exactly what the spectrum of a random matrix of that shape looks like, so singular values above the MP edge are learned structure and the bulk below it is indistinguishable from initialization noise. Unicron keeps the outliers, drops the bulk, and rebuilds. Embeddings and norms are policy-skipped; layers whose outliers carry <1% of energy are guarded rather than filtered (random ≠ useless — measured, not assumed). The engine that rewrites the weights is NumPy + stdlib only; torch lives only in the measurement harness.

Reproduce it

Three commands, no HF account needed (the kit downloads anonymously):

git clone https://github.com/AnOversizedMooseWithSocks/leCore.git
cd leCore
./assimilation/assimilate.sh --eval   # download + assimilate + measure
./assimilation/chat.sh --both         # same prompt to both models, side by side

A private venv appears at assimilation/.venv; your system Python is never touched.

Provenance & license

• Base model: Qwen/Qwen3.5-0.8B (Apache-2.0) — this repo redistributes a derivative of those weights under the same license.
• Assimilation engine: leCore by AnOversizedMooseWithSocks, MIT. This upload is by the kit's first external users (the openzoo crew), not by leCore's author; the assessment bundle from this exact run went back upstream per the kit's README.
• Run: RTX 3060 12GB, 2026-08-16, assimilate.sh --eval, stock settings.