---
license: gemma
base_model: google/gemma-3-270m-it
tags:
- gemma-3
- open-mythos
- recursive-transformer
- cognitive-routing
- experimental
---

# Gemma-3 270M-IT "Open-Mythos" (Phase 2.8)

This is an experimental architectural modification of the **Google Gemma-3 270M-IT** base model. It implements the "Open-Mythos" (PX) architecture, introducing **Recursive Computational Headroom** and **Fluid Gaussian Cognitive Routing**.

## ⚠️ Transparency Notice
**This is not a standard fine-tune.** It is a structural mod that changes how the transformer processes tokens at inference time.
- **Base Model:** [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it)
- **Modifications:** Runtime patching of the forward pass to allow for recursive layer execution and dynamic cognitive routing.

## 🚀 Key Innovations

### 1. Recursive Computational Headroom (PX)
Unlike standard transformers that pass through each layer once, Open-Mythos allows the model to "re-read" and "think" through specific layers (L5-L12) multiple times. This effectively increases the depth of the model for complex tasks without adding new parameters.

### 2. Fluid Gaussian Cognitive Routing
The model dynamically analyzes the "cognitive signature" (Kurtosis) of each prompt during the prefill phase. Based on this signature, it automatically routes the task through a specific "Cognitive Envelope":
- **Math Mode:** Optimized for numerical precision (L5-L11).
- **Logic Mode:** Optimized for multi-step reasoning (L8-L12).
- **Creative Mode:** Optimized for semantic drift and metaphor (L10-L16).
- **Synthesis Mode:** Optimized for extraction and summarization (L6-L14).

Transitions between these modes are **continuous and fluid** using Gaussian blending, allowing the model to self-determine its reasoning path.

### 3. Numerical Stability (RMSNorm Fix)
Implements a surgical fix for the Gemma-3 RMSNorm scaling (`1.0 + weight`) to prevent signal collapse during deep recursion, ensuring vocabulary integrity across high-entropy generations.

## 💻 Usage

To use this model, you **must** set `trust_remote_code=True` because it uses custom modeling code to implement the recursive logic.

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "neuralworm/gemma-3-270m-it-p2.8"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id, 
    torch_dtype=torch.bfloat16, 
    device_map="auto", 
    trust_remote_code=True
)

prompt = "Sally has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?"
chat = [{"role": "user", "content": prompt}]
inputs = tokenizer(tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True), return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

## 📊 Performance
Open-Mythos (Phase 2.8) significantly improves zero-shot performance on "Logical Traps" and multi-step reasoning compared to the pure 270M baseline, while remaining multimodal-ready and regression-free for standard NLP tasks.

---
*Created as part of the Open-Mythos Research Project (2026).*