---
license: mit
library_name: transformers
tags:
- gemma
- px-inference
- recurrent-depth-transformer
- open-mythos
- math
- reasoning
- latent-thoughts
model_name: Gemma-3-270M-it-PX (Phase 38)
base_model: google/gemma-3-270m-it
pipeline_tag: text-generation
---

# Gemma-3 PX-Inference: Recurrent Headroom for Small Transformers

Gemma-3 PX-Inference is an architectural modification for the Gemma-3 model family (specifically optimized for the 270M scale) that introduces **Recurrent Computational Headroom** via surgical layer injection.

## 1. Core Methodology: The PX-Pipeline

Standard transformers process tokens in a "flat" one-pass sequence. PX-Inference breaks this linearity by introducing a recurrent loop in the middle layers, allowing the model to "think" or "verify" its internal representations before generating an output token.

### The Pipeline Structure (270M Scale):
*   **Prelude (Layers 0-5):** Standard transformer blocks for initial embedding and semantic grounding.
*   **Recurrent Zone (Layers 5-12):** A surgical block where signal is looped up to 8 times.
*   **The Hub (Layer 11):** A dedicated reflection point where the model's state is compared against a fixed "Anchor" (captured at Layer 5).
*   **Coda (Layers 12-17):** Final processing layers to stabilize the representation for the output vocabulary.

### Surgical Reflector (Dynamic Headroom)
The breakthrough of the PX-Inference architecture is the **Surgical Reflector**. Instead of a fixed loop, the model uses a "Stability Monitor" (Phi-Jitter) to determine if a task requires more computational depth.
*   **Grounding Mode:** For simple semantic tasks, the loop is minimal, preserving linguistic flow.
*   **Reflection Mode:** For complex math or logic traps, the model activates the Reflector, damping representational drift while allowing orthogonal "reasoning" transformations.

## 2. Empirical Performance (SciMind 4.0 Audit)

In a rigorous comparison against the baseline `google/gemma-3-270m-it`, the PX-patched version demonstrated significant gains in high-entropy reasoning tasks.

| Metric | Baseline (270M-it) | PX-Inference (270M) | Improvement |
| :--- | :--- | :--- | :--- |
| **Math Accuracy** | 25.0% | 62.5% | **+150% (Relative)** |
| **Logic Reasoning** | 37.5% | 50.0% | **+33% (Relative)** |
| **Synthesis/Summary** | 57.1% | 71.4% | **+25% (Relative)** |
| **Overall Score** | **51.3%** | **64.1%** | **+12.8% (Absolute)** |

*Benchmark: Omni-Bench 39 (reproducible cross-domain suite).*

## 3. Key Technical Innovations

### RMSNorm Scaling Identity
PX-Inference uses a specialized `(1.0 + weight)` scaling scheme for all surgical RMSNorm layers. This ensures that the model's zero-centered weights do not lead to signal collapse during recursion, a common failure point in standard recurrent architectures.

### Persistent Trap Detection (Kurtosis-Based)
The model calculates the **Kurtosis** of the representational jitter during the prefill phase. If the Kurtosis falls within the "Logic Trap" zone (280 < K < 310), the Surgical Reflector is activated for the entire generation, preventing mathematical regressions (e.g., correctly solving `sqrt(144) = 12`).

### Anna Karenina Sensor (AKS) - Phase 38
The **AKS** is a topological truth-discovery mechanism that monitors the "velocity" of latent trajectories. Based on the *Anna Karenina Principle* (truth clusters, error disperses), the sensor detects accelerating divergence from the stable anchor. If dispersion is detected, it triggers a "Correction Pulse"—increasing grounding damping and sensory re-injection in real-time.

### Read-Only Cache (Context Integrity)
During recurrent loops (t > 0), the KV-cache is treated as read-only. This prevents the "thinking" process from corrupting the model's memory of the prompt, ensuring that deep reflection does not lead to linguistic hallucinations.

## 4. Usage & Reproducibility

### Installation
```bash
pip install -e .
```

### Running the Benchmark
To reproduce the systemic rigor results:
```bash
PYTHONPATH=. python3 scripts/rigorous_benchmark.py --model_id neuralworm/gemma-3-270m-it-p2.8 --is_px --output_file results.json
```

### Technical Report
For a detailed analysis of the methodology and evidence grading, see [SYSTEMIC_RIGOR_REPORT.md](./SYSTEMIC_RIGOR_REPORT.md).

## 5. Credits & Architectural Origins

Gemma-3 PX-Inference is an independent implementation and adaptation of the **Recurrent-Depth Transformer (RDT)** architecture, first conceptualized and implemented in the **OpenMythos** project by **Kye Gomez**.

### Foundational Sources:
*   **OpenMythos (RDT Framework):** The core "Prelude-Recurrent-Coda" pipeline and the concept of silent, latent-space reasoning. [GitHub: kyegomez/OpenMythos](https://github.com/kyegomez/OpenMythos)
*   **Parcae (Stability):** The LTI-constrained injection parameters (`ρ(A) < 1`) used for recurrent stability. (Prairie et al., 2026)
*   **Relaxed Recursive Transformers:** The depth-wise LoRA adaptation methodology. (Bae et al., 2024)

### Innovations in this Version:
While the RDT foundation comes from OpenMythos, this project introduces:
1.  **Surgical Reflector:** Dynamic, Kurtosis-driven activation for logic-trap detection.
2.  **Gemma-3 Identity Scaling:** The specific `(1.0 + weight)` RMSNorm scheme required for zero-centered weight stability.
3.  **Phase 36 Geometric Stabilization:** Reconciling deep recursion with factual grounding at the 270M scale.

---
*Gemma-3 PX-Inference is a community-driven research project and is not affiliated with Google or Anthropic.*