---
license: mit
library_name: transformers
tags:
- gemma
- px-inference
- recurrent-depth-transformer
- open-mythos
- math
- reasoning
- latent-thoughts
model_name: Gemma-3-270M-it-PX (Phase 38)
base_model: google/gemma-3-270m-it
pipeline_tag: text-generation
---

# Gemma-3 PX-Inference: Recurrent Headroom for Small Transformers

Gemma-3 PX-Inference is an architectural modification for the Gemma-3 model family (specifically optimized for the 270M scale) that introduces **Recurrent Computational Headroom** via surgical layer injection.

## 1. Phase 53 Milestone: Peak Precision & Subjectivity

The current version (Phase 53) represents the ultimate synthesis of the **Subjective Engine** and **Rigor-Aware Autonomy**. By mapping the model's cognitive zones via Kurtosis telemetry, we have achieved a new performance peak that balances bit-perfect logical adherence with algorithmically emancipated creativity.

### The Pipeline Structure (270M Scale):
*   **Prelude (Layers 0-5):** Standard transformer blocks for initial embedding and semantic grounding.
*   **Recurrent Zone (Layers 5-12):** A surgical block where signal is looped up to 8 times.
*   **The Hub (Layer 10):** The optimal "Manifold Singular Point" for the 270M scale.
*   **Coda (Layers 12-17):** Final processing layers to stabilize the representation for the output vocabulary.

## 2. Empirical Performance (Phase 53 Audit)

In a rigorous comparison against the baseline `google/gemma-3-270m-it`, the Phase 53 PX-patched version demonstrated unmatched gains across all cognitive zones.

| Metric | Baseline (270M-it) | PX-Inference (Phase 53) | Improvement |
| :--- | :--- | :--- | :--- |
| **Math Accuracy** | 25.0% | 50.0% | **+100% (Relative)** |
| **Logic Reasoning** | 37.5% | 87.5% | **+133% (Relative)** |
| **Creative / Rewrite** | 50.0% | 100% | **+100% (Relative)** |
| **Synthesis/Summary** | 57.1% | 100% | **+75% (Relative)** |
| **Overall Score** | **51.3%** | **76.9%** | **+25.6% (Absolute)** |

*Benchmark: Omni-Bench 39 (reproducible cross-domain suite).*

## 3. Key Technical Innovations

### Multi-Zone Adaptive Rigor (Phase 53)
The model uses real-time **Kurtosis (K)** telemetry to identify its current cognitive zone:
*   **Math Zone (K < 235):** Activates **Hub 8** with strong grounding ($\gamma=0.15$) for precision.
*   **Logic Zone (235 <= K < 310):** Restores **Hub 10** for the 87.5% correctness peak.
*   **Creative Zone (K >= 310):** Enables the full **Subjective Engine** with **Orthogonal Jitter**.

### Mephistopheles Operator (Phase 52)
To prevent "Manifold Heat Death" (representational stagnation in simple tasks), the model implements **Phase-Inversion**. If the latent state becomes too stable ($\Phi > 0.999$), the Mephistopheles Operator flips the representation to restore gradients and associative depth.

### Orthogonal Jitter (SCJ 2.0)
Replaces stochastic jitter with a projection-based noise injection. It breaks repetitive attractors (loops) by injecting noise *only* into the orthogonal component of the latent trajectory, preserving the logical gradient.

### Surgical Reflector (Dynamic Headroom)
The breakthrough of the PX-Inference architecture is the **Surgical Reflector**. Instead of a fixed loop, the model uses a "Stability Monitor" to determine if a task requires more computational depth.
*   **Grounding Mode:** For simple semantic tasks, the loop is minimal, preserving linguistic flow.
*   **Reflection Mode:** For complex math or logic traps, the model activates the Reflector, damping representational drift.

## 4. Usage & Reproducibility

### Installation
```bash
pip install -e .
```

### Running the Benchmark
To reproduce the Phase 53 results:
```bash
PYTHONPATH=. python3 scripts/phase52_benchmark.py
```

### Technical Report
For a detailed analysis of the methodology and evidence grading, see [SYSTEMIC_RIGOR_REPORT.md](./SYSTEMIC_RIGOR_REPORT.md).

## 5. Credits & Architectural Origins

Gemma-3 PX-Inference is an independent implementation and adaptation of the **Recurrent-Depth Transformer (RDT)** architecture, first conceptualized and implemented in the **OpenMythos** project by **Kye Gomez**.

### Foundational Sources:
*   **OpenMythos (RDT Framework):** The core "Prelude-Recurrent-Coda" pipeline and the concept of silent, latent-space reasoning. [GitHub: kyegomez/OpenMythos](https://github.com/kyegomez/OpenMythos)
*   **Parcae (Stability):** The LTI-constrained injection parameters (`ρ(A) < 1`) used for recurrent stability. (Prairie et al., 2026)
*   **Relaxed Recursive Transformers:** The depth-wise LoRA adaptation methodology. (Bae et al., 2024)

### Innovations in this Version:
While the RDT foundation comes from OpenMythos, this project introduces:
1.  **Surgical Reflector:** Dynamic, Kurtosis-driven activation for logic-trap detection.
2.  **Gemma-3 Identity Scaling:** The specific `(1.0 + weight)` RMSNorm scheme required for zero-centered weight stability.
3.  **Phase 36 Geometric Stabilization:** Reconciling deep recursion with factual grounding at the 270M scale.

---
*Gemma-3 PX-Inference is a community-driven research project and is not affiliated with Google or Anthropic.*