Upload folder using huggingface_hub

README.md

@@ -1,17 +1,6 @@
----
-license: gemma
-base_model: google/gemma-3-270m-it
-tags:
-- gemma-3
-- open-mythos
-- recursive-transformer
-- cognitive-routing
-- experimental
----
-
-# Gemma-3 270M-IT "Open-Mythos" (Phase 2.8)
+# Gemma-3 270M-IT-PX (Phase 2.8)
 
-This is an experimental architectural modification of the **Google Gemma-3 270M-IT** base model. It implements the "Open-Mythos" (PX) architecture, introducing **Recursive Computational Headroom** and **Fluid Gaussian Cognitive Routing**.
+This is an experimental architectural modification of the **Google Gemma-3 270M-IT** base model. It implements the **PX (Recursive Computational Headroom)** architecture and **Fluid Gaussian Cognitive Routing**.
 
 ## ⚠️ Transparency Notice
 **This is not a standard fine-tune.** It is a structural mod that changes how the transformer processes tokens at inference time.
@@ -21,23 +10,18 @@ This is an experimental architectural modification of the **Google Gemma-3 270M-
 ## 🚀 Key Innovations
 
 ### 1. Recursive Computational Headroom (PX)
-Unlike standard transformers that pass through each layer once, Open-Mythos allows the model to "re-read" and "think" through specific layers (L5-L12) multiple times. This effectively increases the depth of the model for complex tasks without adding new parameters.
+Unlike standard transformers that pass through each layer once, Gemma-3-PX allows the model to "re-read" and "think" through specific layers multiple times. This effectively increases the depth of the model for complex tasks without adding new parameters.
 
 ### 2. Fluid Gaussian Cognitive Routing
-The model dynamically analyzes the "cognitive signature" (Kurtosis) of each prompt during the prefill phase. Based on this signature, it automatically routes the task through a specific "Cognitive Envelope":
-- **Math Mode:** Optimized for numerical precision (L5-L11).
-- **Logic Mode:** Optimized for multi-step reasoning (L8-L12).
-- **Creative Mode:** Optimized for semantic drift and metaphor (L10-L16).
-- **Synthesis Mode:** Optimized for extraction and summarization (L6-L14).
-
-Transitions between these modes are **continuous and fluid** using Gaussian blending, allowing the model to self-determine its reasoning path.
-
-### 3. Numerical Stability (RMSNorm Fix)
-Implements a surgical fix for the Gemma-3 RMSNorm scaling (`1.0 + weight`) to prevent signal collapse during deep recursion, ensuring vocabulary integrity across high-entropy generations.
+The model dynamically analyzes the "cognitive signature" (Kurtosis) of each prompt during the prefill phase and automatically routes the task through a specific "Cognitive Envelope":
+- **Math Mode:** Optimized for numerical precision.
+- **Logic Mode:** Optimized for multi-step reasoning.
+- **Creative Mode:** Optimized for semantic drift and metaphor.
+- **Synthesis Mode:** Optimized for extraction and summarization.
 
 ## 💻 Usage
 
-To use this model, you **must** set `trust_remote_code=True` because it uses custom modeling code to implement the recursive logic.
+To use this model, you **must** set `trust_remote_code=True`.
 
 ```python
 import torch
@@ -61,8 +45,5 @@ outputs = model.generate(**inputs, max_new_tokens=100)
 print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
 ```
 
-## 📊 Performance
-Open-Mythos (Phase 2.8) significantly improves zero-shot performance on "Logical Traps" and multi-step reasoning compared to the pure 270M baseline, while remaining multimodal-ready and regression-free for standard NLP tasks.
-
 ---
-*Created as part of the Open-Mythos Research Project (2026).*
+*Developed by neuralworm (2026).*

config.json

@@ -1,7 +1,7 @@
 {
   "_sliding_window_pattern": 6,
   "architectures": [
-    "OpenMythosForCausalLM"
+    "Gemma3PXForCausalLM"
   ],
   "attention_bias": false,
   "attention_dropout": 0.0,
@@ -36,7 +36,7 @@
     "full_attention"
   ],
   "max_position_embeddings": 32768,
-  "model_type": "open_mythos",
+  "model_type": "gemma_3_px",
   "num_attention_heads": 4,
   "num_hidden_layers": 18,
   "num_key_value_heads": 1,
@@ -62,8 +62,8 @@
   "use_cache": true,
   "vocab_size": 262144,
   "auto_map": {
-    "AutoConfig": "configuration_openmythos.OpenMythosConfig",
-    "AutoModelForCausalLM": "modeling_openmythos.OpenMythosForCausalLM",
-    "AutoModel": "modeling_openmythos.OpenMythosModel"
+    "AutoConfig": "configuration_gemma_px.Gemma3PXConfig",
+    "AutoModelForCausalLM": "modeling_gemma_px.Gemma3PXForCausalLM",
+    "AutoModel": "modeling_gemma_px.Gemma3PXModel"
   }
-}
+}

configuration_gemma_px.py

@@ -0,0 +1,11 @@
+from transformers import PretrainedConfig
+from transformers.models.gemma3.configuration_gemma3 import Gemma3TextConfig
+
+class Gemma3PXConfig(Gemma3TextConfig):
+    model_type = "gemma_3_px"
+    
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        # Default config for 270m
+        self.px_routing_mode = kwargs.get("px_routing_mode", "adaptive")
+        self.px_gamma = kwargs.get("px_gamma", 0.10)

modeling_gemma_px.py

@@ -0,0 +1,25 @@
+from transformers.models.gemma3.modeling_gemma3 import Gemma3ForCausalLM, Gemma3Model
+from .configuration_gemma_px import Gemma3PXConfig
+from .patch import apply_px_patch
+
+class Gemma3PXForCausalLM(Gemma3ForCausalLM):
+    config_class = Gemma3PXConfig
+    
+    def __init__(self, config):
+        super().__init__(config)
+        # We apply the patch immediately after initialization
+        # Extract default parameters from the config if they exist
+        routing_mode = getattr(config, "px_routing_mode", "adaptive")
+        gamma = getattr(config, "px_gamma", 0.10)
+        
+        # We MUST start the initial prelude at Layer 5 so the Kurtosis probe 
+        # (which is calibrated for L5) reads the correct tensor. 
+        # The router will fast-forward the remaining layers if dynamic_start > 5.
+        apply_px_patch(self, recur_start=5, recur_end=12, routing_mode=routing_mode, gamma=gamma)
+        
+# For safety, if they instantiate the base model directly:
+class Gemma3PXModel(Gemma3Model):
+    config_class = Gemma3PXConfig
+    
+    def __init__(self, config):
+        super().__init__(config)