Upload 11 files

.gitattributes

@@ -0,0 +1 @@
+model.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,103 @@
+---
+license: agpl-3.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- pytorch
+- causal-lm
+- linear-attention
+- long-context
+- recurrent-memory
+- o1-memory
+- hfp
+- custom_code
+language:
+- en
+---
+
+# HFP — Hyper-Flux Projection (O(1)-Memory Causal LM)
+
+> **Status: research preview — architecture only, weights are UNTRAINED.**
+> This repository ships the model *code* and a randomly-initialized checkpoint so
+> the architecture can be loaded, inspected and trained. It is not a usable
+> language model yet. Canonical source & experiments:
+> **[github.com/kayra-hn/HFP](https://github.com/kayra-hn/HFP)**
+
+HFP is an experimental causal LM that pairs **windowed local attention** with a
+**per-layer recurrent linear-attention memory** (`M ∈ ℝ^{key_dim×H}`, `z ∈ ℝ^{key_dim}`).
+The inference-time state is **constant in context length** (O(1) memory instead of
+a growing KV-cache); long-range information must flow through the recurrent memory.
+
+Its distinguishing feature is a selectable **retention law** for that memory:
+
+- `decay_mode="exp"` — standard geometric decay (the RetNet/GLA/Mamba family baseline).
+- `decay_mode="cubic_flux"` — an exact discretization of the cubic relaxation
+  `dθ/dτ = −η·θ³`: a **state-magnitude-dependent** decay
+  `λ_t = 1/√(1+2η·z_t²)`. Empty channels barely decay (plateau); full channels
+  forget actively (self-limiting).
+
+Two further independent axes: a **binding convolution** on the Q/K path
+(`conv_kernel`, ablate with 1) and a **capacity axis** via DPFP key feature maps
+(`key_feature_map="dpfp"`).
+
+## Honest status of results
+
+Full multi-seed record: [RESULTS.md on GitHub](https://github.com/kayra-hn/HFP/blob/main/RESULTS.md).
+Highlights (small scale, synthetic recall; patterns seed-robust across 3 seeds):
+
+- **Length generalization**: trained at 160 tokens, the model transfers to
+  1280-token streams (8x), with fixed-gap recall *improving* as fact density falls —
+  train-short / infer-long is the supported deployment mode of the O(1) state.
+- **DPFP capacity axis** (`key_feature_map="dpfp"`): first mechanism with a clear
+  advantage — 2-6x baseline accuracy at long gaps under high interference, plus
+  more stable training. Recommended: `exp` + additive + `dpfp` + `ffn_type="standard"`.
+- `cubic_flux` currently trails the exponential baseline at this scale (parked as
+  a long-horizon hypothesis; exact parallel form implemented). No LM-benchmark
+  claims are made. Weights in this repo remain untrained/architecture-only.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained(
+    "kayrahan35/HFP-O1-Memory-Model",
+    trust_remote_code=True,          # custom architecture (HFPForCausalLM)
+)
+
+# Streaming inference with constant memory:
+past = None
+for chunk in token_chunks:                    # e.g. 256-token chunks
+    out = model(chunk, past_key_values=past, use_cache=True)
+    past = out.past_key_values                # fixed-size state, does not grow
+```
+
+Switch the retention law / capacity axis at construction:
+
+```python
+from transformers import AutoConfig, AutoModelForCausalLM
+cfg = AutoConfig.from_pretrained("kayrahan35/HFP-O1-Memory-Model", trust_remote_code=True)
+cfg.decay_mode = "cubic_flux"        # or "exp"
+cfg.key_feature_map = "dpfp"         # or "elu"
+model = AutoModelForCausalLM.from_config(cfg, trust_remote_code=True)
+```
+
+Note: `cubic_flux` uses a sequential scan (O(L)) and is ~2–3× slower than the
+parallel `exp` path.
+
+## Files
+
+`modeling_hfp.py` / `configuration_hfp.py` — HF-compatible model & config;
+`hfp_bulk_state.py` — the recurrent memory (retention laws, binding conv, DPFP);
+`bulk_trigger_decoder.py` — decoder layer (windowed attention + shared-bulk FFN).
+Training scripts, regression tests (`smoke_test.py`) and the retention/recall
+experiment suite live in the [GitHub repository](https://github.com/kayra-hn/HFP).
+
+## Links & license
+
+Theory preprint: [OSF](https://osf.io/xc7e4) (inspiration for the retention law;
+the model neither validates nor is validated by the physics).
+
+**GNU AGPL-3.0.** Network deployment of this architecture or derivatives
+requires open-sourcing modifications under the same license.

bulk_trigger_decoder.py

@@ -0,0 +1,264 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import math
+from .hfp_config import config as hfp_config
+from .hfp_utils import compute_curvature, compute_entropy_map, magnitude_defect_flag, coherence_score, conservation_check, holographic_information_bound
+from .hfp_bulk_state import HFPBulkState
+
+class HFPLinear(nn.Module):
+    def __init__(self, in_features, out_features):
+        super(HFPLinear, self).__init__()
+        self.linear = nn.Linear(in_features, out_features)
+
+    def forward(self, x):
+        return self.linear(x)
+
+# [FIX K3] TunnelingDropout KALDIRILDI: 3 forward onceki, FARKLI batch'e ait
+# detached aktivasyonlari simdiki ciktiya ekliyordu -> batch'ler arasi sizinti +
+# train/eval davranis farki. Ne dropout ne fizik; standart Dropout kullanilir.
+# (Eski kod referans icin _legacy_reference/ altinda.)
+
+class EntangledLinear(nn.Module):
+    """Tek Bulk agirligindan (W_bulk) iki projeksiyon (P_A, P_B) - physics-inspired
+    parametre baglama. Analoji: Paper II'nin 'tek Bulk vektorunun iki golgesi';
+    izomorfizm/simulasyon iddiasi degildir."""
+    def __init__(self, in_features_A, out_features_A, in_features_B, out_features_B, bulk_dim=128):
+        super(EntangledLinear, self).__init__()
+        self.max_in = max(in_features_A, in_features_B)
+        self.W_bulk = nn.Parameter(torch.randn(bulk_dim, self.max_in) / math.sqrt(self.max_in))
+
+        self.P_A = nn.Parameter(torch.randn(out_features_A, bulk_dim) / math.sqrt(bulk_dim))
+        self.P_B = nn.Parameter(torch.randn(out_features_B, bulk_dim) / math.sqrt(bulk_dim))
+
+        self.bias_A = nn.Parameter(torch.zeros(out_features_A))
+        self.bias_B = nn.Parameter(torch.zeros(out_features_B))
+
+    def get_orthogonality_loss(self):
+        dot = self.P_A @ self.P_B.t()
+        return torch.norm(dot, p='fro')
+
+    def forward_A(self, x):
+        if not self.training:
+            if not hasattr(self, 'W_A_cache'):
+                self.W_A_cache = self.P_A @ self.W_bulk[:, :x.size(-1)]
+            W_A = self.W_A_cache
+        else:
+            if hasattr(self, 'W_A_cache'):
+                del self.W_A_cache
+            W_A = self.P_A @ self.W_bulk[:, :x.size(-1)]
+        return F.linear(x, W_A, self.bias_A)
+
+    def forward_B(self, x):
+        if not self.training:
+            if not hasattr(self, 'W_B_cache'):
+                self.W_B_cache = self.P_B @ self.W_bulk[:, :x.size(-1)]
+            W_B = self.W_B_cache
+        else:
+            if hasattr(self, 'W_B_cache'):
+                del self.W_B_cache
+            W_B = self.P_B @ self.W_bulk[:, :x.size(-1)]
+        return F.linear(x, W_B, self.bias_B)
+
+class EntangledFFN(nn.Module):
+    def __init__(self, hidden_size, feedforward_dim, bulk_dim=128, dropout_p=0.1):
+        super(EntangledFFN, self).__init__()
+        self.entangled = EntangledLinear(hidden_size, feedforward_dim, feedforward_dim, hidden_size, bulk_dim)
+        self.gelu = nn.GELU()
+        # [FIX K3] Standart dropout (TunnelingDropout'un yerine)
+        self.dropout = nn.Dropout(dropout_p)
+
+    def forward(self, x):
+        mid = self.entangled.forward_A(x)
+        mid = self.gelu(mid)
+        mid = self.dropout(mid)
+        out = self.entangled.forward_B(mid)
+        return out
+
+    def get_orthogonality_loss(self):
+        return self.entangled.get_orthogonality_loss()
+
+class StandardFFN(nn.Module):
+    """[HFP-SCALE] Rank kisiti olmayan standart Transformer FFN'i.
+    EntangledFFN paylasilan W_bulk yuzunden rank<=bulk_dim darbogazi tasir
+    (or. bulk_dim=128, H=768'de FFN rank-128'e sikisir). Olcekleme kosulari
+    icin ffn_type="standard" bu darbogazi kaldirir. Parametre sayisi
+    EntangledFFN'den fazladir; A/B kiyaslarinda parametre esitligine dikkat."""
+    def __init__(self, hidden_size, feedforward_dim, dropout_p=0.1):
+        super().__init__()
+        self.fc1 = nn.Linear(hidden_size, feedforward_dim)
+        self.fc2 = nn.Linear(feedforward_dim, hidden_size)
+        self.gelu = nn.GELU()
+        self.dropout = nn.Dropout(dropout_p)
+
+    def forward(self, x):
+        return self.fc2(self.dropout(self.gelu(self.fc1(x))))
+
+    def get_orthogonality_loss(self):
+        return torch.zeros((), device=self.fc1.weight.device)
+
+class BulkTriggerDecoderLayer(nn.Module):
+    """
+    BulkTriggerDecoderLayer V3: Lokal (pencereli) attention + recurrent Bulk hafiza.
+
+    Mimari niyet (eski V2 yorumundaki 'Local Attention over Brane ONLY') artik
+    gercekten uygulanir: [FIX K5]
+    - local_window=None -> tam causal attention (eski davranis, geriye uyumlu).
+    - local_window=w    -> her sorgu yalnizca son w tokeni gorur; uzun menzil
+      bilgi YALNIZCA Bulk hafizadan (M, z) akabilir. Bellek iddialarini test
+      etmek icin bu mod sarttir (aksi halde attention tum baglami gorur ve
+      bellek olculmez).
+    - Ring buffer'in yazilmamis (sifir) slotlari artik MASKELENIR (eski D2 sorunu).
+    """
+    def __init__(self, hidden_size, num_heads, feedforward_dim, bulk_dim=128,
+                 vocab_size=None, return_aux=False, local_window=None, dropout_p=0.1,
+                 ffn_type="entangled"):
+        super(BulkTriggerDecoderLayer, self).__init__()
+        self.hidden_size = hidden_size
+        self.num_heads = num_heads
+        self.local_window = local_window
+
+        self.cross_attention = nn.MultiheadAttention(embed_dim=hidden_size, num_heads=num_heads, batch_first=True, dropout=0.1)
+        self.return_aux = return_aux
+        self.norm1 = nn.LayerNorm(hidden_size)
+
+        # [HFP-SCALE] ffn_type: "entangled" (parametre-bagli, rank<=bulk_dim) |
+        # "standard" (kisitsiz, olcekleme onerilen)
+        if ffn_type == "standard":
+            self.ffn = StandardFFN(hidden_size, feedforward_dim, dropout_p=dropout_p)
+        else:
+            self.ffn = EntangledFFN(hidden_size, feedforward_dim, bulk_dim=bulk_dim, dropout_p=dropout_p)
+        self.norm2 = nn.LayerNorm(hidden_size)
+
+        self.vocab_size = vocab_size
+        if vocab_size is not None:
+            self.lm_head = HFPLinear(hidden_size, vocab_size)
+        else:
+            self.lm_head = None
+
+    def _build_mask(self, seq_len, n_past, valid_past, device):
+        """True = maskeli. Sutunlar: [simdiki chunk (seq_len) | ring buffer (n_past)]."""
+        ii = torch.arange(seq_len, device=device).view(-1, 1)
+        jj = torch.arange(seq_len, device=device).view(1, -1)
+        causal = jj > ii
+        if self.local_window is not None:
+            # [K5] Sliding window: yalnizca son w token gorulur
+            causal = causal | (jj <= ii - self.local_window)
+        if n_past > 0:
+            # [K5/D2] Yazilmamis (sifir) slotlar maskelenir. Buffer dolana kadar
+            # yazim sirasi 0,1,2,... oldugundan gecerli slotlar ilk valid_past tanedir.
+            past_cols = (torch.arange(n_past, device=device) >= valid_past).view(1, -1)
+            past_mask = past_cols.expand(seq_len, n_past)
+            return torch.cat([causal, past_mask], dim=1)
+        return causal
+
+    def forward(self, x, bulk_state, past_state=None, return_past_state=False,
+                return_aux=None, detach_state=True):
+        if return_aux is None:
+            return_aux = getattr(self, 'return_aux', False)
+
+        # 1. Recurrent Bulk hafiza guncelle + oku ([K2] artik gradyanli yol)
+        short_mem, retrieved_memory, new_past_state = bulk_state.update(
+            x, past_state=past_state, detach_state=detach_state)
+
+        aux_losses = []
+
+        # Opsiyonel physics-inspired aux teshisleri (default kapali)
+        if hfp_config.ENABLE_RYU_TAKAYANAGI:
+            gate_entropy_tensor = bulk_state.gate_entropy_loss() / hfp_config.REG_WEIGHT if hfp_config.ENABLE_ENTROPY_MAP else torch.tensor(0.0, device=x.device)
+            M_matrix = new_past_state[1]
+            rt_loss = holographic_information_bound(gate_entropy_tensor, M_matrix)
+            aux_losses.append(rt_loss.mean().unsqueeze(0))
+
+        if hfp_config.ENABLE_ENTROPY_MAP:
+            aux_losses.append(bulk_state.gate_entropy_loss())
+
+        if hfp_config.ENABLE_5D_CURVATURE or hfp_config.ENABLE_CURVATURE:
+            aux_losses.append(compute_curvature(short_mem).unsqueeze(0))
+
+        if hfp_config.ENABLE_DEFECT_FLAG:
+            aux_losses.append(magnitude_defect_flag(short_mem).mean().unsqueeze(0))
+        if hfp_config.ENABLE_COHERENCE:
+            aux_losses.append(coherence_score(short_mem).unsqueeze(0))
+        if hfp_config.ENABLE_CONSERVATION:
+            aux_losses.append(torch.tensor(1.0 if conservation_check(short_mem) else 0.0, device=short_mem.device))
+
+        # 2. Lokal attention: simdiki chunk + (varsa) onceki chunk'larin ring buffer'i
+        seq_len = x.size(1)
+        if past_state is not None and past_state[0] is not None:
+            past_short_mem = past_state[0]
+            # [K5] state'teki token_count (index 3) gecerli slot sayisini verir
+            valid_past = min(int(past_state[3]), past_short_mem.size(1))
+        else:
+            past_short_mem = None
+            valid_past = 0
+
+        if past_short_mem is not None and valid_past > 0:
+            memory_bank = torch.cat([x, past_short_mem], dim=1)
+            n_past = past_short_mem.size(1)
+        else:
+            memory_bank = x
+            n_past = 0
+
+        dual_mask = self._build_mask(seq_len, n_past, valid_past, x.device)
+        attn_out, _ = self.cross_attention(query=x, key=memory_bank, value=memory_bank, attn_mask=dual_mask)
+
+        # 3. Bulk hafizadan okunan icerik eklenir
+        attn_out = attn_out + retrieved_memory
+
+        x = self.norm1(x + attn_out)
+
+        # 4. FFN
+        ffn_out = self.ffn(x)
+        x = self.norm2(x + ffn_out)
+
+        if return_aux:
+            aux_losses.append(self.ffn.get_orthogonality_loss().unsqueeze(0))
+
+        # 5. Logits
+        if self.lm_head is not None:
+            logits = self.lm_head(x)
+        else:
+            logits = x
+
+        if return_aux:
+            return logits, bulk_state, new_past_state, aux_losses
+        if return_past_state:
+            return logits, bulk_state, new_past_state
+        return logits, bulk_state
+
+if __name__ == "__main__":
+    batch_size = 2
+    hidden_size = 256
+    num_heads = 8
+    feedforward_dim = 1024
+    vocab_size = 50000
+
+    layer = BulkTriggerDecoderLayer(
+        hidden_size=hidden_size,
+        num_heads=num_heads,
+        feedforward_dim=feedforward_dim,
+        vocab_size=vocab_size
+    )
+    memory_system = HFPBulkState(hidden_size=hidden_size)
+    current_token = torch.randn(batch_size, 1, hidden_size)
+    logits, updated_memory = layer(current_token, memory_system)
+
+    print(f"Girdi Boyutu: {current_token.shape}")
+    print(f"Logits Çıktı Boyutu: {logits.shape}")

config.json

@@ -0,0 +1,40 @@
+{
+  "ENABLE_COHERENCE": false,
+  "architectures": [
+    "HFPForCausalLM"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_hfp.HFPConfig",
+    "AutoModelForCausalLM": "modeling_hfp.HFPForCausalLM"
+  },
+  "aux_gate_entropy_weight": 0.0,
+  "aux_ortho_weight": 0.0,
+  "bos_token_id": 1,
+  "bptt_across_chunks": false,
+  "bulk_dim": 128,
+  "conv_kernel": 3,
+  "decay_mode": "exp",
+  "dpfp_nu": 2,
+  "dropout_p": 0.1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "ffn_type": "entangled",
+  "hidden_size": 768,
+  "intermediate_size": 3072,
+  "key_feature_map": "elu",
+  "local_window": 64,
+  "max_position_embeddings": 4096,
+  "max_short_len": null,
+  "model_type": "hfp",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pe_scale": 0.3,
+  "rec_block": 64,
+  "short_len": 8,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.13.0",
+  "tunnel_decay": 0.8,
+  "tunnel_depth": 3,
+  "vocab_size": 50257,
+  "write_rule": "additive"
+}

configuration_hfp.py

@@ -0,0 +1,107 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+from transformers import PretrainedConfig
+
+class HFPConfig(PretrainedConfig):
+    model_type = "hfp"
+
+    def __init__(
+        self,
+        vocab_size=5000,
+        hidden_size=256,
+        num_hidden_layers=4,
+        num_attention_heads=8,
+        intermediate_size=512,
+        bulk_dim=128,
+        tunnel_depth=3,
+        tunnel_decay=0.8,
+        dropout_p=0.1,
+        short_len=8,
+        max_short_len=None,             # [FIX K4] ring buffer kapasitesi; None -> max(short_len, 32)
+        local_window=None,              # [FIX K5] None=tam causal attention; w=yalnizca son w token
+                                        #   (bellek testleri icin sart: aksi halde attention tum baglami gorur)
+        pe_scale=0.3,                   # [FIX K7] pozisyonel kodlama olcegi; ham PE (1.0) token
+                                        #   icerigini ~35x boguyordu -> recall imkansizdi. 0.3 = dengeli.
+        rec_block=64,                   # [K2] chunk-ici recurrence blok boyutu (hiz/bellek; sonucu degistirmez)
+        decay_mode="exp",               # [HFP-CORE] "exp"=geometrik decay baseline; "cubic_flux"=
+                                        #   makalenin dth/dtau=-eta*th^3 kubik-plato retention'i (ayirt edici);
+                                        #   "cubic_flux_chunked"=[HFP-SCALE] iki-gecisli TAM paralel form
+                                        #   (z-taramasi + GLA-tarzi chunkwise M; her rec_block'ta birebir)
+        conv_kernel=3,                  # [FIX K8] binding conv kernel'i (Q/K token-karisimi). 1=kapali (ablasyon)
+        key_feature_map="elu",          # [HFP-CAP] bellek anahtar ozellik-haritasi. "elu"=elu+1 (baseline,
+                                        #   D=H). "dpfp"=Deterministic Parameter-Free Projection (D=2H*nu):
+                                        #   efektif boyutu buyutur -> rank-collapse'i geciktirir, KAPASITE artar.
+        dpfp_nu=2,                      # [HFP-CAP] dpfp genisleme faktoru (key_dim = 2*hidden_size*nu)
+        bptt_across_chunks=False,       # [K2] True -> chunk'lar arasi state detach edilmez (TBPTT)
+        max_position_embeddings=8192,   # [FIX A1] onceden tanimsizdi -> from_1b_profile cokuyordu
+        aux_gate_entropy_weight=0.0,    # [C1] opsiyonel gate-entropy duzenleyici; 0.0 = kapali (durust baseline)
+        write_rule="additive",          # [HFP-DELTA] "additive"=k v^T toplama (baseline);
+                                        #   "delta"=olcum-guncelleme yazimi (girisim-dirençli, sirali)
+        ffn_type="entangled",         # [HFP-SCALE] "entangled"=paylasilan-bulk FFN (rank<=bulk_dim);
+                                        #   "standard"=kisitsiz FFN (olcekleme icin onerilir)
+        aux_ortho_weight=0.0,           # [ORTHO] EntangledFFN P_A/P_B ortogonallik cezasi; 0.0 = kapali
+        bos_token_id=1,
+        eos_token_id=2,
+        **kwargs
+    ):
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.intermediate_size = intermediate_size
+        self.bulk_dim = bulk_dim
+        self.tunnel_depth = tunnel_depth
+        self.tunnel_decay = tunnel_decay
+        self.dropout_p = dropout_p
+        self.short_len = short_len
+        self.max_short_len = max_short_len
+        self.local_window = local_window
+        self.pe_scale = pe_scale
+        self.rec_block = rec_block
+        self.decay_mode = decay_mode
+        self.conv_kernel = conv_kernel
+        self.key_feature_map = key_feature_map
+        self.dpfp_nu = dpfp_nu
+        self.bptt_across_chunks = bptt_across_chunks
+        self.max_position_embeddings = max_position_embeddings
+        self.aux_gate_entropy_weight = aux_gate_entropy_weight
+        self.write_rule = write_rule
+        self.ffn_type = ffn_type
+        self.aux_ortho_weight = aux_ortho_weight
+        # [TEMIZLIK B1] medium_freq / long_freq / medium_momentum kaldirildi:
+        # mimari lineer-attention (M, z + decay) oldugundan bunlarin hicbir entegrasyonu yok.
+        self.ENABLE_COHERENCE = kwargs.pop("ENABLE_COHERENCE", False)
+        # [D1] Weight tying resmen bildirilir: transformers v5 bunu gormezse
+        # from_pretrained sonrasi lm_head/embedding baglantisini KURMAZ.
+        kwargs.setdefault("tie_word_embeddings", True)
+        super().__init__(bos_token_id=bos_token_id, eos_token_id=eos_token_id, **kwargs)
+
+    @classmethod
+    def from_1b_profile(cls, vocab_size=50257):
+        """Creates a 1 Billion Parameter configuration for Cloud Training."""
+        return cls(
+            vocab_size=vocab_size,
+            hidden_size=2048,
+            num_hidden_layers=24,
+            num_attention_heads=16,
+            intermediate_size=8192,
+            bulk_dim=512,
+            short_len=64,
+            max_short_len=64,                # [FIX K4] eskiden sessizce 32'ye kirpiliyordu
+            max_position_embeddings=32768,   # [FIX A1] 1B profili icin pozisyon tavani
+            ENABLE_COHERENCE=False
+        )

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "transformers_version": "5.13.0"
+}

hfp_bulk_state.py

@@ -0,0 +1,426 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import logging
+
+logging.basicConfig(level=logging.WARNING, format='[%(levelname)s] %(message)s')
+from .hfp_utils import LandmarkBuffer, compute_gate_entropy, coherence_score
+from .hfp_config import config as hfp_config
+
+class HFPBulkState(nn.Module):
+    """
+    HFPBulkState V4 (Recurrent Edition): Causal chunkwise linear attention.
+
+    [FIX K2 - GRADYAN AKISI] Onceki surumde retrieval, M guncellemesinden ONCE
+    yapiliyordu; tek-parca egitimde M=0 oldugundan retrieval hep sifirdi ve
+    W_k/W_v/decay/importance_gate LM loss'tan HIC gradyan alamiyordu (bellek
+    egitimde olu agirlikti). Bu surum gercek causal lineer attention'dir:
+    her token, o ana KADARKI kumulatif M/z'den okur (kendi KV'si dahil),
+    per-token decay ile. Boylece bellek yolu ayni forward icinde ciktiya
+    baglanir ve TUM bellek parametreleri gradyan alir.
+
+    Matematik (RetNet/GLA tarzi chunkwise form, tum usler >= 0 -> stabil):
+        lam = sigmoid(decay)                          (K-kanali basina, 0..1)
+        M_t = lam (.) M_{t-1} + k_t v_t^T ,  z_t = lam (.) z_{t-1} + k_t
+        out_t = (q_t M_t) / (q_t . z_t)
+    Blok ici (m token):
+        cross:  (q_i * lam^i) M_0            intra:  S_ij = q_i . (lam^{i-j} (.) k_j), j<=i
+    Uretim yolu (1 token/cagri) ayni formulun m=1 halidir -> egitim/uretim
+    decay semantigi artik TUTARLI (eski surum decay'i cagri basina 1 kez
+    uyguluyordu; 256-token chunk ile 1-token generate farkli davraniyordu).
+
+    Onceki yapisal duzeltmeler korunur:
+    - Matrix blowup: decay init sigmoid(2.19)~0.9 + retrieval LayerNorm.
+    - Gate collapse: importance_gate bias -2.0.
+    - Ring buffer: sabit boyutlu, vektorize yazim (Python token-dongusu kaldirildi).
+    [FIX K4] max_short_len artik parametre: config.short_len > 32 sessizce
+    kirpilmiyor (1B profili short_len=64 gercekten 64 slot alir).
+    [FIX D3] batch>16'da sessiz half() donusumu kaldirildi.
+    """
+
+    def __init__(self, hidden_size, short_len=8, max_short_len=None,
+                 rec_block=64, use_mixed_precision=False, clip_value=1.0,
+                 decay_mode="exp", conv_kernel=3, key_feature_map="elu", dpfp_nu=2,
+                 write_rule="additive"):
+        super(HFPBulkState, self).__init__()
+        self.hidden_size = hidden_size
+        self.base_short_len = short_len
+        # [HFP-CAP] Anahtar ozellik-haritasi ve efektif anahtar boyutu (key_dim).
+        # "elu": elu(x)+1, key_dim=H (baseline). "dpfp": Deterministic Parameter-Free
+        # Projection, key_dim=2*H*nu -> daha yuksek efektif boyut, rank-collapse
+        # geciktirilir, bellek KAPASITESI (kac ayri olgu) artar. M artik (key_dim, H),
+        # z (key_dim); deger (V) boyutu H olarak kalir. Retention (exp/cubic) ve
+        # binding conv'dan BAGIMSIZ eksen.
+        self.key_feature_map = key_feature_map
+        self.dpfp_nu = max(1, dpfp_nu)
+        self.key_dim = hidden_size if key_feature_map != "dpfp" else 2 * hidden_size * self.dpfp_nu
+        # [HFP-CORE] Retention yasasi. "exp" = standart geometrik decay (RetNet/GLA
+        # ailesi, baseline). "cubic_flux" = makalenin dth/dtau=-eta*th^3 kubik
+        # akisinin birebir ayriklastirmasi: state-buyuklugune bagli, plato+power-law
+        # unutma. Ailedeki hicbir modelde olmayan ayirt edici mekanizma.
+        self.decay_mode = decay_mode
+        # [HFP-DELTA] Yazim kurali. "additive" = M += k(x)v^T (baseline; ayni anahtara
+        # tekrarli yazimlar GIRISIM yapar). "delta" = DeltaNet-tarzi olcum-guncelleme:
+        #     M~ = lam (.) M ;  v_old = k^T M~ ;  M = M~ + beta * k (v - v_old)^T
+        # Eski iliskiyi okuyup FARKI yazar -> ayni anahtarin eski degeri silinir,
+        # girisim birikmez. k L2-normalize edilir, beta=sigmoid(gate) in (0,1) ->
+        # (I - beta k k^T) kontraksiyon, state patlamaz. Payda (q.z) kullanilmaz
+        # (delta'da kutle birikimi anlamsiz); cikti q.M -> retrieval_norm olcekler.
+        # Sirali O(L) (WY/chunkwise formu ileriki is; GPU olcek icin gerekli).
+        self.write_rule = write_rule
+
+        # [FIX K4] Kapasite en az short_len; eskisi gibi sessizce 32'ye kirpma yok.
+        if max_short_len is None:
+            max_short_len = max(short_len, getattr(hfp_config, 'MAX_SHORT_LEN', 32))
+        assert max_short_len >= short_len, \
+            f"max_short_len ({max_short_len}) < short_len ({short_len})"
+        self.max_short_len = max_short_len
+
+        # [K2] Chunk-ici recurrence blok boyutu (dogruluk degil hiz/bellek dengesi;
+        # sonuc blok boyutundan BAGIMSIZDIR - bkz. smoke_test.py tutarlilik testi).
+        self.rec_block = max(1, rec_block)
+
+        self.landmark_max = hfp_config.LANDMARK_MAX
+        self.gate_temperature = nn.Parameter(torch.tensor(1.0), requires_grad=False)
+        self.use_mixed_precision = use_mixed_precision  # [D3] no-op; geriye uyumluluk icin duruyor
+        self.clip_value = clip_value
+        self.dynamic_short_thresh = hfp_config.ENTROPY_THRESH
+
+        # Selective Scan Gating (Information Bottleneck)
+        self.importance_gate = nn.Linear(hidden_size, hidden_size)
+        # [GATE COLLAPSE FIX] sigmoid(-2.0) ~ 0.12 baslangici
+        nn.init.constant_(self.importance_gate.bias, -2.0)
+        self.gate_dropout = nn.Dropout(0.1)
+
+        # [FIX K8 - KISA CAUSAL CONV / BINDING] Lineer-attention BELLEGININ
+        # associative-recall yapabilmesi icin sart olan token-karisimi. Onceki
+        # surumde her token bellege KENDI key(x_t)⊗value(x_t)'sini yaziyordu;
+        # v1'in anahtari onu ONCELEYEN k1'i kodlamadigindan sorgu=k1 ile v1
+        # GETIRILEMIYORDU (MQAR loss ln(val_space)'te sabit, full-attention %100).
+        # Depthwise causal conv (kernel=3) Q/K yoluna uygulanir -> K[v1-pozisyonu]
+        # artik onceki token k1'i kodlar, Q[k1] ile eslesir. V ORIJINAL x'ten
+        # (temiz deger). Mamba/H3/Based hepsi bu kisa conv'u icerir. Retention
+        # yasasindan (exp/cubic_flux) BAGIMSIZ - kimlige dokunmaz. Chunk-tutarlilik
+        # icin conv state chunk'lar arasi tasinir (T4 korunur).
+        self.conv_kernel = max(1, conv_kernel)
+        self.short_conv = nn.Conv1d(hidden_size, hidden_size, kernel_size=self.conv_kernel,
+                                    groups=hidden_size, bias=True, padding=0)
+
+        # Linear Attention Projections
+        self.W_q = nn.Linear(hidden_size, hidden_size, bias=False)
+        self.W_k = nn.Linear(hidden_size, hidden_size, bias=False)
+        self.W_v = nn.Linear(hidden_size, hidden_size, bias=False)
+
+        # [FIX K2b - COK-OLCEKLI DECAY] Eskiden tum kanallar sigmoid(2.19)~0.9
+        # ile TEK olcekte baslardi -> bellek ufku ~1/(1-0.9)=10 token; 100 token
+        # geriden recall matematiksel olarak imkansizdi (lam^100~2e-5). Simdi
+        # kanal basina lam 0.90..0.999 arasi lineer dagilir (RetNet/GLA-tarzi
+        # multi-timescale): bazi kanallar ~10 token, bazilari ~1000 token tutar.
+        # Sigmoid ciktisi (0,1) oldugundan matrix-blowup korumasi korunur; tum
+        # usler >= 0 stabilite degismez. decay hala LM loss'tan gradyan alir.
+        # [HFP-CAP] decay/eta artik anahtar-kanali basina -> key_dim boyutunda.
+        lam_min = getattr(hfp_config, 'DECAY_LAM_MIN', 0.90)
+        lam_max = getattr(hfp_config, 'DECAY_LAM_MAX', 0.999)
+        lam_init = torch.linspace(lam_min, lam_max, self.key_dim)
+        decay_logit = torch.log(lam_init / (1.0 - lam_init))  # sigmoid^{-1}
+        self.decay = nn.Parameter(decay_logit)
+
+        # [HFP-CORE] Kubik-flux esnekligi eta (per-kanal, >0). Tek-adim kararli
+        # cozumden lam_t = 1/sqrt(1 + 2*eta*s_t^2), s_t = anlik state buyuklugu.
+        # Gecis olcegi t* ~ 1/sqrt(2*eta): eta buyuk -> kisa plato, kucuk -> uzun.
+        # Kanallar arasi 1e-4..1e-2 log-dagilir -> plato ~7..70 token, ogrenilebilir.
+        eta_init = torch.logspace(-4.0, -2.0, self.key_dim)
+        self.log_eta = nn.Parameter(torch.log(eta_init))
+
+        # [HFP-DELTA] per-token yazim siddeti beta (0,1); bias +1 -> ~0.73 baslangic
+        self.beta_gate = nn.Linear(hidden_size, 1)
+        nn.init.constant_(self.beta_gate.bias, 1.0)
+
+        self.retrieval_norm = nn.LayerNorm(hidden_size)
+        self.landmark_buffer = LandmarkBuffer(max_size=hfp_config.LANDMARK_MAX)
+
+    def _feat(self, u):
+        """[HFP-CAP] Anahtar/sorgu ozellik-haritasi -> (..., key_dim), hep >= 0."""
+        if self.key_feature_map == "dpfp":
+            u = torch.cat([F.relu(u), F.relu(-u)], dim=-1)                   # (..., 2H)
+            parts = [u * torch.roll(u, shifts=i + 1, dims=-1) for i in range(self.dpfp_nu)]
+            return torch.cat(parts, dim=-1)                                  # (..., 2H*nu) >= 0
+        return F.elu(u) + 1.0                                               # (..., H) > 0
+
+    def get_initial_state(self, batch_size, device, dtype):
+        M = torch.zeros(batch_size, self.key_dim, self.hidden_size, device=device, dtype=dtype)
+        z = torch.zeros(batch_size, self.key_dim, device=device, dtype=dtype)
+        short_memory = torch.zeros(batch_size, self.max_short_len, self.hidden_size, device=device, dtype=dtype)
+        # [FIX K8] conv_state: onceki chunk'in son (kernel-1) girdisi (causal conv icin)
+        conv_state = torch.zeros(batch_size, self.conv_kernel - 1, self.hidden_size, device=device, dtype=dtype)
+        # state: (short_memory, M, z, token_count, short_len_dynamic, write_idx, conv_state)
+        return (short_memory, M, z, 0, self.base_short_len, 0, conv_state)
+
+    def reset_state(self):
+        self.landmark_buffer.clear()
+        if hasattr(self, "_last_gate"):
+            del self._last_gate
+        if hasattr(self, "_gate_entropy_live"):
+            del self._gate_entropy_live
+
+    def gate_entropy_loss(self):
+        if not hasattr(self, "_last_gate"):
+            return torch.tensor(0.0, device=next(self.parameters()).device)
+        if hfp_config.ENABLE_ENTROPY_MAP:
+            return compute_gate_entropy(self._last_gate) * hfp_config.REG_WEIGHT
+        else:
+            return torch.tensor(0.0, device=next(self.parameters()).device)
+
+    def _write_ring_buffer(self, short_memory, x, write_idx):
+        """[K6] Vektorize ring-buffer yazimi (eski per-token Python dongusu yerine).
+        clone(): detach edilmemis state ile in-place autograd hatasini onler."""
+        B, L, H = x.shape
+        cap = self.max_short_len
+        short_memory = short_memory.clone()
+        if L >= cap:
+            # yalnizca son 'cap' token buffer'da kalir
+            tail = x[:, L - cap:, :]
+            idx = (write_idx + (L - cap) + torch.arange(cap, device=x.device)) % cap
+            short_memory[:, idx, :] = tail
+        else:
+            idx = (write_idx + torch.arange(L, device=x.device)) % cap
+            short_memory[:, idx, :] = x
+        new_write_idx = (write_idx + L) % cap
+        return short_memory, new_write_idx
+
+    def update(self, x, past_state=None, detach_state=True):
+        if x.dim() == 2:
+            x = x.unsqueeze(1)
+        batch_size, seq_len, _ = x.size()
+        device = x.device
+        dtype = x.dtype
+
+        if past_state is not None:
+            (short_memory, M, z, token_count, short_len_dynamic, write_idx, conv_state) = past_state
+            if short_memory is not None and short_memory.size(0) != batch_size:
+                (short_memory, M, z, token_count, short_len_dynamic, write_idx, conv_state) = self.get_initial_state(batch_size, device, dtype)
+        else:
+            (short_memory, M, z, token_count, short_len_dynamic, write_idx, conv_state) = self.get_initial_state(batch_size, device, dtype)
+
+        # [K2] detach_state artik cagiran tarafindan kontrol edilir (TBPTT icin False).
+        if detach_state:
+            if short_memory is not None: short_memory = short_memory.detach()
+            if M is not None: M = M.detach()
+            if z is not None: z = z.detach()
+            if conv_state is not None: conv_state = conv_state.detach()
+
+        # 1. Ring buffer (vektorize)
+        short_memory, write_idx = self._write_ring_buffer(short_memory, x, write_idx)
+        token_count += seq_len
+        active_len = min(token_count, short_len_dynamic)
+
+        # 2. [FIX K8] Binding conv: Q/K'yi conv'lanmis girdiden hesapla (komsu token
+        #    karisimi -> anahtar onceki token'i kodlar), V'yi ORIJINAL x'ten (temiz deger).
+        kk = self.conv_kernel
+        if kk > 1:
+            if conv_state is None:
+                conv_state = torch.zeros(batch_size, kk - 1, self.hidden_size, device=device, dtype=dtype)
+            x_pad = torch.cat([conv_state, x], dim=1)                       # (B, kk-1+L, H)
+            x_qk = self.short_conv(x_pad.transpose(1, 2)).transpose(1, 2)   # (B, L, H) causal
+            new_conv_state = x_pad[:, x_pad.size(1) - (kk - 1):, :]         # son kk-1 girdi
+        else:
+            x_qk = x
+            new_conv_state = conv_state
+
+        Q = self._feat(self.W_q(x_qk))   # (B,L,key_dim) >= 0  [HFP-CAP]
+        K = self._feat(self.W_k(x_qk))   # (B,L,key_dim) >= 0
+        V_raw = self.W_v(x)              # (B,L,H) temiz deger
+
+        # 3. Gating (retrieval'dan ONCE: gate'li V hem intra-chunk okumaya
+        #    hem M guncellemesine girer -> gate gradyan alir)
+        gate_logits = self.importance_gate(x) / self.gate_temperature
+        gate = torch.sigmoid(self.gate_dropout(gate_logits))
+        gate = gate.to(dtype)
+        self._last_gate = gate.clone().detach()
+        # [C1] Gradyanli gate-entropy - modeling opsiyonel olarak loss'a ekler.
+        self._gate_entropy_live = compute_gate_entropy(gate)
+
+        gate_entropy = None
+        if hfp_config.ENABLE_ENTROPY_MAP or hfp_config.ENABLE_DEFECT_FLAG or hfp_config.ENABLE_RYU_TAKAYANAGI:
+            gate_entropy = compute_gate_entropy(gate)
+
+        V = V_raw * gate
+
+        # 4. Retention recurrence — mod secilir (exp baseline / cubic_flux HFP-core).
+        outputs = []
+
+        if self.write_rule == "delta":
+            # [HFP-DELTA] Sirali delta-yazim; decay_mode lam'i belirler (exp/cubic).
+            beta = torch.sigmoid(self.beta_gate(x)).to(dtype)                # (B,L,1)
+            if self.decay_mode == "exp":
+                lam_exp = torch.sigmoid(self.decay).to(dtype).unsqueeze(0)   # (1,D)
+            else:
+                eta = torch.exp(self.log_eta).to(dtype).unsqueeze(0)         # (1,D)
+            for t in range(seq_len):
+                kt = K[:, t]; vt = V[:, t]; qt = Q[:, t]                     # (B,D)/(B,H)
+                kn = kt / (kt.norm(dim=-1, keepdim=True) + 1e-6)             # ||k||=1
+                if self.decay_mode == "exp":
+                    lam_t = lam_exp
+                else:
+                    lam_t = 1.0 / torch.sqrt(1.0 + 2.0 * eta * z * z)        # (B,D)
+                Mt = M * lam_t.unsqueeze(-1)
+                v_old = torch.einsum('bd,bdh->bh', kn, Mt)                   # mevcut iliski
+                M = Mt + beta[:, t].unsqueeze(-1) * torch.einsum('bd,bh->bdh', kn, vt - v_old)
+                z = z * lam_t + kn
+                outputs.append(torch.einsum('bd,bdh->bh', qt, M).unsqueeze(1))
+            retrieved = torch.cat(outputs, dim=1)                           # (B,L,H)
+
+        elif self.decay_mode == "cubic_flux":
+            # [HFP-CORE] Makalenin dth/dtau = -eta*th^3 kubik akisinin birebir
+            # ayriklastirmasi. Tek-adim kararli cozum -> per-kanal decay faktoru:
+            #     lam_t = 1/sqrt(1 + 2*eta*z_{t-1}^2)   (z = anahtar-akumulatoru, per-kanal)
+            #     M_t = lam_t (.) M_{t-1} + k_t v_t^T ;  z_t = lam_t (.) z_{t-1} + k_t
+            #     out_t = (q_t M_t)/(q_t . z_t)        (causal-inclusive, kendi KV dahil)
+            # NOT: decay M'in degil Z'nin (anahtar kutlesi) buyuklugune baglidir.
+            # z bos iken lam~1 (PLATO, unutma yok); z buyudukce lam<1 (aktif, buyukluge
+            # bagli unutma) -> plato + power-law kuyruk. Kendini-sinirlayan: decay
+            # buyuklukle arttigindan state patlamaz. Sirali (O(L)); mod default degil.
+            # Saf recurrence oldugundan chunk-tutarli (full == state-tasiyan chunked).
+            eta = torch.exp(self.log_eta).to(dtype).unsqueeze(0)             # (1,H) > 0
+            for t in range(seq_len):
+                kt = K[:, t]; vt = V[:, t]; qt = Q[:, t]                     # (B,H)
+                lam_t = 1.0 / torch.sqrt(1.0 + 2.0 * eta * z * z)           # (B,H)
+                M = M * lam_t.unsqueeze(-1) + torch.einsum('bh,bg->bhg', kt, vt)
+                z = z * lam_t + kt
+                num = torch.einsum('bh,bhg->bg', qt, M)                      # (B,H)
+                den = (qt * z).sum(-1, keepdim=True) + 1e-6                  # (B,1)
+                outputs.append((num / den).unsqueeze(1))                     # (B,1,H)
+            retrieved = torch.cat(outputs, dim=1)                           # (B,L,H)
+
+        elif self.decay_mode == "cubic_flux_chunked":
+            # [HFP-SCALE] cubic_flux'in IKI-GECISLI TAM paralel formu (yaklasim DEGIL).
+            # Gozlem: lam_t yalnizca z_{t-1}'e baglidir ve z'nin recurrence'i M'siz,
+            # elementwise-ucuzdur. O halde:
+            #   GECIS 1: z-taramasi (sirali ama per-adim O(B*D) elementwise) ->
+            #            per-token lam_t TAM olarak bilinir.
+            #   GECIS 2: lam_t bilindiginde M-recurrence, GLA/Mamba2-tarzi
+            #            chunkwise-paralel cozulur (log-uzayda kumulatif carpim;
+            #            tum katsayilar <= 1 -> stabil).
+            # Sonuc her rec_block icin sirali cubic_flux ile birebir aynidir
+            # (bkz. review_scripts/scaling_checks.py); rec_block yalnizca hiz/bellek
+            # dengesidir. Bellek: intra-blok tensoru (B,m,m,key_dim).
+            eta = torch.exp(self.log_eta).to(dtype).unsqueeze(0)                 # (1,D)
+            lam_list = []
+            z_run = z
+            for t in range(seq_len):                                            # GECIS 1 (ucuz)
+                lam_t = 1.0 / torch.sqrt(1.0 + 2.0 * eta * z_run * z_run)        # (B,D)
+                lam_list.append(lam_t)
+                z_run = z_run * lam_t + K[:, t]
+            lam_seq = torch.stack(lam_list, dim=1)                               # (B,L,D)
+            loglam = torch.log(lam_seq.clamp_min(1e-12))
+
+            for s0 in range(0, seq_len, self.rec_block):                         # GECIS 2
+                Qb = Q[:, s0:s0 + self.rec_block]
+                Kb = K[:, s0:s0 + self.rec_block]
+                Vb = V[:, s0:s0 + self.rec_block]
+                m = Qb.size(1)
+                cs = torch.cumsum(loglam[:, s0:s0 + m], dim=1)                   # (B,m,D): log A_i, A_i = prod_{j<=i} lam_j
+                A = torch.exp(cs)                                                # (B,m,D) <= 1
+
+                # cross-block: M_0/z_0 katkisi A_i ile soner
+                Q_dec = Qb * A                                                   # (B,m,D)
+                num_cross = torch.bmm(Q_dec, M)                                  # (B,m,H)
+                den_cross = (Q_dec * z.unsqueeze(1)).sum(-1)                     # (B,m)
+
+                # intra-blok: pair (i,j<=i) katsayisi prod_{s=j+1..i} lam_s = exp(cs_i - cs_j) <= 1
+                ii = torch.arange(m, device=device).view(m, 1)
+                jj = torch.arange(m, device=device).view(1, m)
+                causal = (ii >= jj).to(dtype)                                    # (m,m)
+                Dm = torch.exp(cs.unsqueeze(2) - cs.unsqueeze(1))                # (B,m,m,D): exp(cs_i - cs_j)
+                Dm = Dm * causal.view(1, m, m, 1)
+
+                S = torch.einsum('bih,bijh,bjh->bij', Qb, Dm, Kb)                # (B,m,m)
+                num_intra = torch.bmm(S, Vb)                                     # (B,m,H)
+                den_intra = S.sum(dim=2)                                         # (B,m)
+
+                den = (den_cross + den_intra + 1e-6).unsqueeze(-1)
+                outputs.append((num_cross + num_intra) / den)
+
+                # state guncelle (blok sonu): A_m = tum blok carpimi
+                A_m = A[:, -1]                                                   # (B,D)
+                K_dec = Kb * torch.exp(cs[:, -1:] - cs)                          # (B,m,D): prod_{s=j+1..m}
+                M = M * A_m.unsqueeze(-1) + torch.bmm(K_dec.transpose(1, 2), Vb)
+                z = z * A_m + K_dec.sum(dim=1)
+            retrieved = torch.cat(outputs, dim=1)                              # (B,L,H)
+
+        else:
+            # [K2] exp mod: paralel chunkwise (per-token geometrik decay, causal-inclusive)
+            lam = torch.sigmoid(self.decay).to(dtype)  # (H,), 0..1
+            for s in range(0, seq_len, self.rec_block):
+                Qb = Q[:, s:s + self.rec_block]
+                Kb = K[:, s:s + self.rec_block]
+                Vb = V[:, s:s + self.rec_block]
+                m = Qb.size(1)
+
+                p = torch.arange(1, m + 1, device=device, dtype=dtype)          # 1..m
+                lam_i = lam.unsqueeze(0).pow(p.unsqueeze(1))                     # (m,H): lam^i
+                lam_rev = lam.unsqueeze(0).pow((m - p).unsqueeze(1))             # (m,H): lam^{m-i}
+
+                # cross-block: eski state'ten oku
+                Q_dec = Qb * lam_i.unsqueeze(0)                                  # (B,m,H)
+                num_cross = torch.bmm(Q_dec, M)                                  # (B,m,H)
+                den_cross = (Q_dec * z.unsqueeze(1)).sum(-1)                     # (B,m)
+
+                # intra-block: D_ij = lam^{i-j} (i>=j), tum usler >= 0 -> stabil
+                ii = torch.arange(m, device=device).view(m, 1)
+                jj = torch.arange(m, device=device).view(1, m)
+                e = (ii - jj).clamp_min(0).to(dtype)                             # (m,m)
+                causal = (ii >= jj).to(dtype)
+                D = lam.view(1, 1, -1).pow(e.unsqueeze(-1)) * causal.unsqueeze(-1)  # (m,m,H)
+
+                S = torch.einsum('bih,ijh,bjh->bij', Qb, D, Kb)                  # (B,m,m)
+                num_intra = torch.bmm(S, Vb)                                     # (B,m,H)
+                den_intra = S.sum(dim=2)                                         # (B,m) > 0 (Q,K>0)
+
+                den = (den_cross + den_intra + 1e-6).unsqueeze(-1)
+                outputs.append((num_cross + num_intra) / den)
+
+                # state guncelle (blok sonu)
+                lam_m = lam.pow(float(m))
+                K_dec = Kb * lam_rev.unsqueeze(0)
+                M = M * lam_m.view(1, -1, 1) + torch.bmm(K_dec.transpose(1, 2), Vb)
+                z = z * lam_m.view(1, -1) + K_dec.sum(dim=1)
+            retrieved = torch.cat(outputs, dim=1)                              # (B,L,H)
+
+        retrieved_memory = self.retrieval_norm(retrieved)                      # (B,L,H)
+
+        # 5. Dynamic Context Windowing & Landmarks (opsiyonel teshis yollari)
+        if gate_entropy is not None:
+            if gate_entropy < self.dynamic_short_thresh and short_len_dynamic < self.max_short_len:
+                short_len_dynamic = min(short_len_dynamic + 4, self.max_short_len)
+
+        if hfp_config.ENABLE_DEFECT_FLAG:
+            coherence = None
+            if hfp_config.ENABLE_COHERENCE:
+                coherence = coherence_score(short_memory)
+            if gate_entropy is not None and coherence is not None:
+                priority = coherence.item() * gate_entropy.item()
+            else:
+                priority = gate.mean().item()
+            self.landmark_buffer.push(priority, x.mean(dim=1))
+
+        new_past_state = (short_memory, M, z, token_count, short_len_dynamic, write_idx, new_conv_state)
+
+        active_short_view = short_memory[:, :active_len, :]
+        return active_short_view, retrieved_memory, new_past_state

hfp_config.py

@@ -0,0 +1,49 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+import dataclasses
+
+@dataclasses.dataclass
+class HFPConfig:
+    """Feature flags and hyper-parameters for optional physics-inspired analogues.
+
+    All flags are *disabled* by default to keep the baseline model clean, fast
+    and honest: these physics-inspired aux terms are experimental hooks, NOT
+    load-bearing parts of the trained model. Enable one only to *test* whether
+    it adds value; when off, no wasted compute and no false "physics" claim.
+    (Onceki surumde hepsi True idi ama modeling bunlari loss'a hic baglamiyordu
+    -> olu hesap. Durust baseline icin kapatildi; ilham olarak deneye acik kalir.)
+    """
+    # Feature toggles - deneysel, default kapali (opt-in)
+    ENABLE_CURVATURE: bool = False
+    ENABLE_ENTROPY_MAP: bool = False
+    ENABLE_DEFECT_FLAG: bool = False
+    ENABLE_COHERENCE: bool = False
+    ENABLE_CONSERVATION: bool = False
+    ENABLE_RYU_TAKAYANAGI: bool = False
+    ENABLE_5D_CURVATURE: bool = False
+
+    # Hyper-parameters (used when the feature is enabled)
+    REG_WEIGHT: float = 0.01               # gate-entropy regularisation weight
+    LANDMARK_MAX: int = 49                 # max entries in landmark buffer
+    ENTROPY_THRESH: float = 0.25           # threshold for dynamic short-memory expansion
+    MAX_SHORT_LEN: int = 32                # maximum short-memory length (tokens)
+    GRAD_CLIP_VAL: float = 0.5             # gradient-clipping value per memory block
+    MIXED_PRECISION: bool = True           # use torch.float16 for gate logits only
+    WARP_K: float = 0.5                    # Witten propagator warp factor
+
+# Global singleton configuration used throughout the package
+config = HFPConfig()

hfp_utils.py

@@ -0,0 +1,116 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+import heapq
+import torch
+
+def compute_gate_entropy(gate_tensor):
+    """Compute entropy of gate probabilities.
+    gate_tensor is expected to be in [0,1] range (after sigmoid).
+    Returns scalar tensor (float). DIFFERENTIABLE.
+    """
+    eps = 1e-8
+    p = torch.clamp(gate_tensor, min=eps, max=1 - eps)
+    entropy = - (p * torch.log(p) + (1 - p) * torch.log(1 - p))
+    return entropy.mean()
+
+class LandmarkBuffer:
+    """Priority buffer that keeps the top-k token summaries based on gate strength.
+    Uses a min-heap of size max_size; lower strengths are popped.
+    """
+    def __init__(self, max_size=49):
+        self.max_size = max_size
+        self.heap = []  # each entry: (strength, counter, tensor)
+        self.counter = 0
+
+    def clear(self):
+        self.heap.clear()
+        self.counter = 0
+
+    def push(self, strength, token_summary):
+        entry = (strength, self.counter, token_summary.clone().detach())
+        self.counter += 1
+        if len(self.heap) < self.max_size:
+            heapq.heappush(self.heap, entry)
+        else:
+            if strength > self.heap[0][0]:
+                heapq.heapreplace(self.heap, entry)
+
+    def get_buffer(self):
+        """Return a tensor of stacked token summaries sorted by strength descending."""
+        if not self.heap:
+            return None
+        sorted_entries = sorted(self.heap, key=lambda e: e[0], reverse=True)
+        tensors = [e[2] for e in sorted_entries]
+        return torch.stack(tensors, dim=1)  # shape: (batch, slots, hidden)
+
+def compute_curvature(vector: torch.Tensor) -> torch.Tensor:
+    """Discrete geometric curvature via second-order finite differences across time.
+    vector shape: (batch, seq_len, hidden_dim).
+    NOT: kaynak tensor detach edilmisse gradyan tasimaz; regularizer olarak kullanilacaksa
+    gradyanli bir tensore (or. katman girisi) uygulanmalidir.
+    """
+    if vector.size(1) < 3:
+        return torch.tensor(0.0, device=vector.device)
+    second_deriv = vector[:, 2:, :] - 2 * vector[:, 1:-1, :] + vector[:, :-2, :]
+    return torch.norm(second_deriv, dim=-1).mean()
+
+def compute_entropy_map(gates: torch.Tensor) -> torch.Tensor:
+    """Per-gate entropy map, shape (batch, seq_len)."""
+    eps = 1e-8
+    p = torch.clamp(gates, min=eps, max=1 - eps)
+    entropy = - (p * torch.log(p) + (1 - p) * torch.log(1 - p))
+    return entropy.mean(dim=-1)
+
+def magnitude_defect_flag(vector: torch.Tensor, threshold: float = 1.0) -> torch.Tensor:
+    """[DIAGNOSTIC ONLY - NON-DIFFERENTIABLE] norm(vector) > threshold.
+    Bir '>' karsilastirmasi -> gradyan TASIMAZ. Loss olarak kullanmayin; teshis metrigidir.
+    """
+    norm = torch.norm(vector, dim=-1)
+    return (norm > threshold).float()
+
+def coherence_score(memory_states: torch.Tensor) -> torch.Tensor:
+    """Average cosine similarity between consecutive memory states along the sequence dim.
+    Returns a scalar tensor.
+    """
+    if memory_states.size(1) < 2:
+        return torch.tensor(0.0, device=memory_states.device)
+    sims = torch.nn.functional.cosine_similarity(
+        memory_states[:, :-1, :], memory_states[:, 1:, :], dim=-1
+    )
+    return sims.mean()
+
+def conservation_check(state: torch.Tensor) -> bool:
+    """[DIAGNOSTIC ONLY - NON-DIFFERENTIABLE] Python bool dondurur -> gradyan TASIMAZ.
+    Temporal 'korunum' teshisi: gizli boyuttaki toplam zaman icinde suruklenmiyor mu?
+    Loss olarak kullanmayin; yalnizca izleme metrigidir. state shape: (batch, seq_len, hidden)
+    """
+    if state.size(1) < 2:
+        return True
+    eps = 1e-2
+    temporal_sum = state.sum(dim=-1)
+    drift = torch.abs(temporal_sum[:, 1:] - temporal_sum[:, :-1])
+    return torch.all(drift < eps).item()
+
+def holographic_information_bound(entropy_val: torch.Tensor, memory_matrix: torch.Tensor) -> torch.Tensor:
+    """Holographic Information Bound (V2.1): soft penalty ensuring current attention entropy
+    does not exceed the Frobenius-norm capacity of the [hidden, hidden] memory matrix.
+    DIFFERENTIABLE (softplus).
+    """
+    matrix_capacity = torch.linalg.matrix_norm(memory_matrix, ord='fro', dim=(-2, -1)).mean()
+    ratio = entropy_val / (matrix_capacity + 1e-8)
+    bound_violation = torch.nn.functional.softplus(ratio - 1.0)
+    return bound_violation

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91261fe89cfc09c74dfe52b94dfe88d80d77e4f3d6cc254f0eea62c85e0ccd3b
+size 436820144

modeling_hfp.py

@@ -0,0 +1,232 @@
+# Hyper Flux Projection (HFP) — O(1)-memory causal language model
+# Copyright (C) 2026 Kayrahan Yılmaz
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+import torch
+import torch.nn as nn
+import math
+from transformers import PreTrainedModel
+from transformers.modeling_outputs import CausalLMOutputWithPast, BaseModelOutputWithPast
+from .configuration_hfp import HFPConfig
+
+from .hfp_bulk_state import HFPBulkState
+from .bulk_trigger_decoder import BulkTriggerDecoderLayer
+
+class SinusoidalPositionalEncoding(nn.Module):
+    def __init__(self, hidden_size, max_len=5000, pe_scale=0.3):
+        super().__init__()
+        self.max_len = max_len
+        # [FIX K7] PE olcegi. Onceden ham (norm=sqrt(d/2)=8) eklenirken embedding
+        # normu 0.02*sqrt(d)=0.23 idi -> PE, token icerigini ~35x BOGUYORDU;
+        # anahtar/deger'ler ~%97 pozisyon oluyor, icerik-tabanli recall imkansiz
+        # (MQAR loss ln(val_space)'te sabitlenip binding hic ogrenilmiyordu).
+        # embed *sqrt(d) (bkz. HFPModel) + PE *0.3 ile normlar dengelenir.
+        self.pe_scale = pe_scale
+        pe = torch.zeros(max_len, hidden_size)
+        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
+        div_term = torch.exp(torch.arange(0, hidden_size, 2).float() * (-math.log(10000.0) / hidden_size))
+        pe[:, 0::2] = torch.sin(position * div_term)
+        pe[:, 1::2] = torch.cos(position * div_term)
+        self.register_buffer('pe', pe.unsqueeze(0))
+
+    def forward(self, x, offset: int = 0):
+        # [FIX A2] Streaming/chunked prefill'de her chunk'a 0..L degil, GLOBAL pozisyon eklenir.
+        seq_len = x.size(1)
+        if offset + seq_len > self.max_len:
+            offset = max(0, self.max_len - seq_len)
+        return x + self.pe_scale * self.pe[:, offset:offset + seq_len, :].to(x.device)
+
+class HFPPreTrainedModel(PreTrainedModel):
+    config_class = HFPConfig
+    base_model_prefix = "hfp"
+    _supports_cache_class = False
+
+    def _init_weights(self, module):
+        if isinstance(module, nn.Linear):
+            module.weight.data.normal_(mean=0.0, std=0.02)
+            if module.bias is not None:
+                # importance_gate bias'i bilerek -2.0 (gate-collapse onlemi); ezme.
+                if torch.all(module.bias.data == -2.0):
+                    pass
+                else:
+                    module.bias.data.zero_()
+        elif isinstance(module, nn.Embedding):
+            module.weight.data.normal_(mean=0.0, std=0.02)
+            if module.padding_idx is not None:
+                module.weight.data[module.padding_idx].zero_()
+        elif isinstance(module, nn.LayerNorm):
+            module.bias.data.zero_()
+            module.weight.data.fill_(1.0)
+
+class HFPModel(HFPPreTrainedModel):
+    def __init__(self, config):
+        super().__init__(config)
+        self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size)
+        # [FIX K7] Vaswani sqrt(d) olcegi: emb (init std 0.02) tek basina norm~0.23;
+        # sqrt(d) ile ~2.56'ya cikar ve sonuchennmus PE (*0.3, norm~2.4) ile dengelenir.
+        self.embed_scale = math.sqrt(config.hidden_size)
+        self.pos_encoder = SinusoidalPositionalEncoding(
+            config.hidden_size, max_len=config.max_position_embeddings,
+            pe_scale=getattr(config, "pe_scale", 0.3))
+
+        self.layers = nn.ModuleList([
+            BulkTriggerDecoderLayer(
+                hidden_size=config.hidden_size,
+                num_heads=config.num_attention_heads,
+                feedforward_dim=config.intermediate_size,
+                bulk_dim=config.bulk_dim,
+                vocab_size=None,
+                local_window=getattr(config, "local_window", None),   # [FIX K5]
+                dropout_p=getattr(config, "dropout_p", 0.1),          # [FIX K3]
+                ffn_type=getattr(config, "ffn_type", "entangled")     # [HFP-SCALE]
+            )
+            for _ in range(config.num_hidden_layers)
+        ])
+
+        # Katman-basi recurrent bellek (physics-inspired 'Bulk' analojisi;
+        # teknik olarak: decay'li lineer-attention state'i M, z)
+        # [TEMIZLIK B1] medium_freq/long_freq/medium_momentum kaldirildi.
+        self.bulk_states = nn.ModuleList([
+            HFPBulkState(
+                hidden_size=config.hidden_size,
+                short_len=config.short_len,
+                max_short_len=getattr(config, "max_short_len", None), # [FIX K4]
+                rec_block=getattr(config, "rec_block", 64),           # [K2]
+                decay_mode=getattr(config, "decay_mode", "exp"),      # [HFP-CORE]
+                conv_kernel=getattr(config, "conv_kernel", 3),        # [FIX K8] binding
+                key_feature_map=getattr(config, "key_feature_map", "elu"),  # [HFP-CAP]
+                dpfp_nu=getattr(config, "dpfp_nu", 2),                # [HFP-CAP]
+                write_rule=getattr(config, "write_rule", "additive")  # [HFP-DELTA]
+            )
+            for _ in range(config.num_hidden_layers)
+        ])
+        # [K2] TBPTT: True ise chunk'lar arasi state detach edilmez
+        self._detach_state = not getattr(config, "bptt_across_chunks", False)
+
+        self.norm = nn.LayerNorm(config.hidden_size)
+        self.post_init()
+
+    @staticmethod
+    def _offset_from_state(past_key_values_list):
+        # state tuple: (short_memory, M, z, token_count, short_len_dynamic, write_idx, conv_state)
+        first = past_key_values_list[0]
+        if first is not None and len(first) >= 4 and isinstance(first[3], int):
+            return int(first[3])
+        return 0
+
+    def forward(self, input_ids, attention_mask=None, past_key_values=None, use_cache=False, **kwargs):
+        x = self.embed_tokens(input_ids) * self.embed_scale  # [FIX K7]
+
+        if past_key_values is None or not isinstance(past_key_values, (tuple, list)):
+            past_key_values_list = [None] * len(self.layers)
+        else:
+            past_key_values_list = past_key_values
+
+        # [FIX A2] Global pozisyon offseti (chunked prefill icin)
+        offset = self._offset_from_state(past_key_values_list)
+        x = self.pos_encoder(x, offset=offset)
+
+        new_past_key_values = []
+        gate_entropies = []
+        for i, (layer, bulk_state) in enumerate(zip(self.layers, self.bulk_states)):
+            x, _, new_past_state = layer(x, bulk_state, past_state=past_key_values_list[i],
+                                         return_past_state=True, detach_state=self._detach_state)
+
+            # [C1] Gradyanli gate-entropy topla (yalnizca agirlik > 0 iken loss'a katilir)
+            if self.training and getattr(bulk_state, "_gate_entropy_live", None) is not None:
+                gate_entropies.append(bulk_state._gate_entropy_live)
+
+            if use_cache:
+                new_past_key_values.append(new_past_state)
+
+        x = self.norm(x)
+
+        out = BaseModelOutputWithPast(
+            last_hidden_state=x,
+            past_key_values=new_past_key_values if use_cache else None
+        )
+        out.gate_entropy = (torch.stack(gate_entropies).mean() if gate_entropies else None)
+        return out
+
+from transformers.generation import GenerationMixin
+
+class HFPForCausalLM(HFPPreTrainedModel, GenerationMixin):
+    # [D1] lm_head, embed_tokens'a bagli (weight tying) — safetensors kaydinin
+    # paylasilan tensoru dogru ele almasi icin bildirilmeli.
+    # transformers v5: dict (hedef -> kaynak); v4'te de sorunsuz calisir.
+    _tied_weights_keys = {"lm_head.weight": "hfp.embed_tokens.weight"}
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.hfp = HFPModel(config)
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+        # [D1] Weight tying (GPT standardi): embedding ve lm_head paylasimi
+        self.lm_head.weight = self.hfp.embed_tokens.weight
+        self.post_init()
+
+    # [D1] tie_weights()'in from_pretrained SONRASI baglantiyi yeniden kurabilmesi
+    # icin standart erisimciler (bunlarsiz yukleme tying'i sessizce koparir).
+    def get_input_embeddings(self):
+        return self.hfp.embed_tokens
+
+    def set_input_embeddings(self, value):
+        self.hfp.embed_tokens = value
+
+    def get_output_embeddings(self):
+        return self.lm_head
+
+    def set_output_embeddings(self, value):
+        self.lm_head = value
+
+    def forward(self, input_ids, attention_mask=None, labels=None, past_key_values=None, use_cache=False, **kwargs):
+        outputs = self.hfp(input_ids, attention_mask=attention_mask, past_key_values=past_key_values, use_cache=use_cache, **kwargs)
+        hidden_states = outputs.last_hidden_state
+        logits = self.lm_head(hidden_states)
+
+        loss = None
+        if labels is not None:
+            shift_logits = logits[..., :-1, :].contiguous()
+            shift_labels = labels[..., 1:].contiguous()
+            loss_fct = nn.CrossEntropyLoss()
+            loss = loss_fct(shift_logits.view(-1, self.config.vocab_size), shift_labels.view(-1))
+
+            # [C1] Opsiyonel gate-entropy duzenleyici - default kapali (weight=0.0).
+            w = float(getattr(self.config, "aux_gate_entropy_weight", 0.0))
+            if w > 0.0 and getattr(outputs, "gate_entropy", None) is not None:
+                loss = loss + w * outputs.gate_entropy
+
+            # [ORTHO] EntangledFFN ortogonallik duzenleyicisi. Paylasilan W_bulk'tan
+            # turetilen P_A/P_B'nin farkli ("tek bulk'in iki ayri golgesi") kalmasi
+            # icin baski. Onceden implement edilmis ama loss'a HIC bagli degildi
+            # (olu kod); artik opsiyonel + default kapali (weight=0.0 => baseline degismez).
+            w_o = float(getattr(self.config, "aux_ortho_weight", 0.0))
+            if w_o > 0.0:
+                ortho = sum(layer.ffn.get_orthogonality_loss() for layer in self.hfp.layers)
+                loss = loss + w_o * ortho
+
+        return CausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values
+        )
+
+    def prepare_inputs_for_generation(self, input_ids, past_key_values=None, **kwargs):
+        if past_key_values:
+            input_ids = input_ids[:, -1:]
+        return {
+            "input_ids": input_ids,
+            "past_key_values": past_key_values,
+            "use_cache": kwargs.get("use_cache", True)
+        }