---
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
  - multimodal
  - vision-language
  - knot
  - gnosis
  - fat-station
  - sovereign-inference
  - cloudflare-containers
library_name: gnosis-distributed-inference
---

# Muse-Glimmer-30B

**Muse-Glimmer-30B** is a ~30B-parameter multimodal (vision-language) model distributed in
the sovereign **`.knot`** format and served by the [Gnosis](https://forkjoin.ai)
`fat-station` inference runtime. It pairs a Qwen2.5-VL-class perception encoder with a
Qwen3-class dense text decoder, and is designed to run **CPU-only, scale-to-zero** on
commodity edge compute (e.g. Cloudflare Containers) with no GPU and no fixed cost.

Released under the **Apache-2.0** license.

## Files

| File | Role | Notes |
|---|---|---|
| `muse-glimmer-30b.knot` | Text decoder | 52-layer dense transformer |
| `muse-glimmer-30b-mmproj.knot` | Vision encoder + projector | CLIP ViT + 3-layer GELU projector → text hidden space |

The `.knot` format is the Gnosis distributed-inference container format: a
streamable, sharded, tensor-cache-friendly package (quantized weights are demand-streamed
from object storage and held resident per-shard). It is **not** GGUF or safetensors.

## Architecture

### Text decoder (`muse-glimmer-30b.knot`)
Qwen3-class dense transformer:

| Field | Value |
|---|---|
| Layers | 52 |
| Hidden size | 6656 |
| Attention | GQA — 32 query / 2 KV heads, head_dim 128 |
| Normalization | RMSNorm + **QK-norm** (per-head query/key norm) |
| Attention gate | **output gate** — `attn_out *= sigmoid(x @ attn_gate)` before `o_proj` |
| FFN | SwiGLU |
| Vocabulary | 202048 |
| RoPE θ | 500000 |
| Image token id | 200092 (placeholder positions spliced with vision embeddings) |

### Vision encoder (`muse-glimmer-30b-mmproj.knot`)
Qwen2.5-VL-class CLIP ViT + projector:

| Field | Value |
|---|---|
| Layers | 50 |
| Hidden size | 1536 |
| Heads | 16 (head_dim 96) |
| Input | 896×896, patch 14 → 4096 patches |
| Position | 32×32 learned embeddings, bicubic-interpolated |
| Attention | 2D-RoPE (θ 10000), window/full pattern `[w, w, w, f]` |
| Spatial merge | 2×2 |
| Projector | 3-layer GELU MLP: 6144 → 4096 → 4096 → **6656** (into the text hidden space) |

### Quantization
- **Q4_K / Q6_K** — attention & FFN weights
- **F32** — token embeddings, norms, biases, and the full vision projector

## Intended use

Multimodal understanding — describe / reason about images, and vision-grounded text
generation — on sovereign, GPU-free infrastructure. The perception encoder emits dense,
image-sensitive `[n_tokens × 6656]` embeddings that splice directly into the text decoder's
hidden space at `image_token_id` positions.

## Serving

Served natively by `fat-station` (Rust HTTP inference). Two deployment shapes:

- **Single container** (`role=both`) — one box serves `/embed`, `/forward`, `/lm-head`,
  `/tokenize`, `/generate`, `/detokenize`.
- **Sharded mesh** — the 52 decoder layers split across N containers (entry / mid / exit),
  coordinated over a lightweight `/embed → /forward → /sample-next` protocol.

Vision:
```
POST /glimmer-vision-embed
{ "pixels": [<CHW f32, normalized>], "grid_h": H, "grid_w": W }
→ octet-stream [n_tokens × 6656] f32 embeddings
```

Text (OpenAI-compatible, via the `aeon-openai-adapter` front):
```
POST /v1/chat/completions
{ "model": "muse-glimmer-30b", "messages": [...], "stream": true|false }
```

### Operational notes
- **`KNOT_HTTP_PIN_ALL=0`** for the vision encoder and for text shards — pinning the full
  vision weights resident dequantizes to ~10 GB f32 and OOMs; stream instead.
- Text shards benefit from a resident **quantized** tensor cache
  (`KNOT_HTTP_TENSOR_CACHE_CAPACITY`) so warm tokens skip re-streaming.

## Performance (measured, CPU)

On a 4-shard mesh of scale-to-zero Cloudflare Containers (4 vCPU / 12 GiB each):

| Path | Warm | Notes |
|---|---|---|
| Text generation | ~3.0 s/token | down from ~42 s/token via resident tensor cache + on-device sampling (~14×) |
| Vision embed | ~1.5–2.2 s | with a dequantized-weight cache (~25× over cold) |
| Cold start | ~160 s | full weight stream on wake (accepted tradeoff for scale-to-zero) |

Numbers are latency on CPU edge compute, not throughput benchmarks; there are no GPU or
accuracy benchmarks in this card.

## Limitations & status

- **Vision encoder**: admission-gated — proven to produce finite, dense, non-degenerate,
  **image-sensitive** embeddings (a gradient vs a checkerboard yield different outputs).
- **Text decoder**: functional after the attention-output-gate fix (distinct, non-degenerate
  token sequences); a formal generation-quality admission gate is still pending. Treat text
  generation as a **research artifact**, not a production-graded chat model.
- CPU-first design: high-quality but latency-bound (seconds/token), with multi-minute cold
  starts under scale-to-zero.
- No safety tuning, RLHF, or content filtering is claimed. No training-data or provenance
  disclosure is made in this card beyond the architecture above.

## Download

```python
from huggingface_hub import hf_hub_download

text   = hf_hub_download("forkjoin-ai/muse-glimmer-30b", "muse-glimmer-30b.knot")
vision = hf_hub_download("forkjoin-ai/muse-glimmer-30b", "muse-glimmer-30b-mmproj.knot")
```

The knots are also mirrored on the Forkjoin object-storage plane for edge serving.

## Citation

```bibtex
@misc{muse_glimmer_30b,
  title  = {Muse-Glimmer-30B: a sovereign CPU-first vision-language model in the knot format},
  author = {Forkjoin.ai},
  year   = {2026},
  url    = {https://huggingface.co/forkjoin-ai/muse-glimmer-30b}
}
```