Add model card

README.md

@@ -0,0 +1,153 @@
+---
+license: other
+license_name: set-before-distribution
+pipeline_tag: image-text-to-text
+tags:
+  - multimodal
+  - vision-language
+  - knot
+  - gnosis
+  - fat-station
+  - sovereign-inference
+  - cloudflare-containers
+library_name: gnosis-distributed-inference
+---
+
+# Muse-Glimmer-30B
+
+**Muse-Glimmer-30B** is a ~30B-parameter multimodal (vision-language) model distributed in
+the sovereign **`.knot`** format and served by the [Gnosis](https://forkjoin.ai)
+`fat-station` inference runtime. It pairs a Qwen2.5-VL-class perception encoder with a
+Qwen3-class dense text decoder, and is designed to run **CPU-only, scale-to-zero** on
+commodity edge compute (e.g. Cloudflare Containers) with no GPU and no fixed cost.
+
+> **License note:** `license: other` is a placeholder. Set the correct license and
+> base-model attribution before wide distribution — this card does not assert one.
+
+## Files
+
+| File | Role | Notes |
+|---|---|---|
+| `muse-glimmer-30b.knot` | Text decoder | 52-layer dense transformer |
+| `muse-glimmer-30b-mmproj.knot` | Vision encoder + projector | CLIP ViT + 3-layer GELU projector → text hidden space |
+
+The `.knot` format is the Gnosis distributed-inference container format: a
+streamable, sharded, tensor-cache-friendly package (quantized weights are demand-streamed
+from object storage and held resident per-shard). It is **not** GGUF or safetensors.
+
+## Architecture
+
+### Text decoder (`muse-glimmer-30b.knot`)
+Qwen3-class dense transformer:
+
+| Field | Value |
+|---|---|
+| Layers | 52 |
+| Hidden size | 6656 |
+| Attention | GQA — 32 query / 2 KV heads, head_dim 128 |
+| Normalization | RMSNorm + **QK-norm** (per-head query/key norm) |
+| Attention gate | **output gate** — `attn_out *= sigmoid(x @ attn_gate)` before `o_proj` |
+| FFN | SwiGLU |
+| Vocabulary | 202048 |
+| RoPE θ | 500000 |
+| Image token id | 200092 (placeholder positions spliced with vision embeddings) |
+
+### Vision encoder (`muse-glimmer-30b-mmproj.knot`)
+Qwen2.5-VL-class CLIP ViT + projector:
+
+| Field | Value |
+|---|---|
+| Layers | 50 |
+| Hidden size | 1536 |
+| Heads | 16 (head_dim 96) |
+| Input | 896×896, patch 14 → 4096 patches |
+| Position | 32×32 learned embeddings, bicubic-interpolated |
+| Attention | 2D-RoPE (θ 10000), window/full pattern `[w, w, w, f]` |
+| Spatial merge | 2×2 |
+| Projector | 3-layer GELU MLP: 6144 → 4096 → 4096 → **6656** (into the text hidden space) |
+
+### Quantization
+- **Q4_K / Q6_K** — attention & FFN weights
+- **F32** — token embeddings, norms, biases, and the full vision projector
+
+## Intended use
+
+Multimodal understanding — describe / reason about images, and vision-grounded text
+generation — on sovereign, GPU-free infrastructure. The perception encoder emits dense,
+image-sensitive `[n_tokens × 6656]` embeddings that splice directly into the text decoder's
+hidden space at `image_token_id` positions.
+
+## Serving
+
+Served natively by `fat-station` (Rust HTTP inference). Two deployment shapes:
+
+- **Single container** (`role=both`) — one box serves `/embed`, `/forward`, `/lm-head`,
+  `/tokenize`, `/generate`, `/detokenize`.
+- **Sharded mesh** — the 52 decoder layers split across N containers (entry / mid / exit),
+  coordinated over a lightweight `/embed → /forward → /sample-next` protocol.
+
+Vision:
+```
+POST /glimmer-vision-embed
+{ "pixels": [<CHW f32, normalized>], "grid_h": H, "grid_w": W }
+→ octet-stream [n_tokens × 6656] f32 embeddings
+```
+
+Text (OpenAI-compatible, via the `aeon-openai-adapter` front):
+```
+POST /v1/chat/completions
+{ "model": "muse-glimmer-30b", "messages": [...], "stream": true|false }
+```
+
+### Operational notes
+- **`KNOT_HTTP_PIN_ALL=0`** for the vision encoder and for text shards — pinning the full
+  vision weights resident dequantizes to ~10 GB f32 and OOMs; stream instead.
+- Text shards benefit from a resident **quantized** tensor cache
+  (`KNOT_HTTP_TENSOR_CACHE_CAPACITY`) so warm tokens skip re-streaming.
+
+## Performance (measured, CPU)
+
+On a 4-shard mesh of scale-to-zero Cloudflare Containers (4 vCPU / 12 GiB each):
+
+| Path | Warm | Notes |
+|---|---|---|
+| Text generation | ~3.0 s/token | down from ~42 s/token via resident tensor cache + on-device sampling (~14×) |
+| Vision embed | ~1.5–2.2 s | with a dequantized-weight cache (~25× over cold) |
+| Cold start | ~160 s | full weight stream on wake (accepted tradeoff for scale-to-zero) |
+
+Numbers are latency on CPU edge compute, not throughput benchmarks; there are no GPU or
+accuracy benchmarks in this card.
+
+## Limitations & status
+
+- **Vision encoder**: admission-gated — proven to produce finite, dense, non-degenerate,
+  **image-sensitive** embeddings (a gradient vs a checkerboard yield different outputs).
+- **Text decoder**: functional after the attention-output-gate fix (distinct, non-degenerate
+  token sequences); a formal generation-quality admission gate is still pending. Treat text
+  generation as a **research artifact**, not a production-graded chat model.
+- CPU-first design: high-quality but latency-bound (seconds/token), with multi-minute cold
+  starts under scale-to-zero.
+- No safety tuning, RLHF, or content filtering is claimed. No training-data or provenance
+  disclosure is made here — set attribution and license before distribution.
+
+## Download
+
+```python
+from huggingface_hub import hf_hub_download
+
+text   = hf_hub_download("forkjoin-ai/muse-glimmer-30b", "muse-glimmer-30b.knot")
+vision = hf_hub_download("forkjoin-ai/muse-glimmer-30b", "muse-glimmer-30b-mmproj.knot")
+```
+
+The knots are also mirrored on the Forkjoin object-storage plane for edge serving.
+
+## Citation
+
+```bibtex
+@misc{muse_glimmer_30b,
+  title  = {Muse-Glimmer-30B: a sovereign CPU-first vision-language model in the knot format},
+  author = {Forkjoin.ai},
+  year   = {2026},
+  url    = {https://huggingface.co/forkjoin-ai/muse-glimmer-30b}
+}
+```