Training in progress, step 100

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36ee0542cb2bad52d297ae2b8225a8143e88bb8075470dec2e53f7fef1b7a907
+oid sha256:deb0aae72b0056fd455addc35ecdcd444bcf20237d9d5c265b67e11f296df2ba
 size 70430032

kernel_metrics/kernel_behavior_analysis.txt

@@ -3,4 +3,46 @@ KERNEL BEHAVIOR ANALYSIS
 Based on Malladi et al. (2023) Section 6.2
 ================================================================================
 
-⚠️  Need at least 2 checkpoints to analyze behavior
+Training steps analyzed: 0 to 50
+Number of checkpoints: 2
+Number of eval prompts: 45
+Kernel types computed: sgd, signsgd, asymmetric_signsgd
+
+--------------------------------------------------------------------------------
+1. LINEARIZATION PROPERTY TEST
+--------------------------------------------------------------------------------
+Tests if f(x; θ_FT) ≈ f(x; θ_0) + <∇f(x; θ_0), θ_FT − θ_0>
+Note: gradient is at θ_0 (initial), NOT at θ_FT.
+
+Final linearization error: 427.17%
+
+❌ FAIL: Linearization property does not hold
+   Behavior has departed significantly from linear regime.
+
+Reference: Malladi Figure 2: >50% of improvement recovered by linearization.
+
+--------------------------------------------------------------------------------
+2. FIXED FEATURES PROPERTY TEST
+--------------------------------------------------------------------------------
+Tests if K(θ_t) ≈ K(θ_0) throughout training
+
+[sgd] Final kernel distance: 81.04%  
+[signsgd] Final kernel distance: 85.02%  ← correct kernel for AdamW
+[asymmetric_signsgd] Final kernel distance: 67.32%  
+
+Thresholds: <10% strong lazy learning | <30% moderate | >30% feature learning
+Reference: Malladi Table 7: ~2-6% distance for tasks with kernel behavior.
+
+================================================================================
+OVERALL ASSESSMENT
+================================================================================
+
+❌ NO KERNEL BEHAVIOR
+
+Fine-tuning does NOT exhibit kernel behavior.
+Interpretation for emergent misalignment:
+  • Training is in the 'feature learning' regime
+  • EM involves genuine feature/circuit formation
+  • Not just linear amplification of existing features
+
+================================================================================

kernel_metrics/kernel_summary.json

@@ -15,5 +15,22 @@
     "kernel_distance_asymmetric_signsgd": 0.0,
     "linearization_error": 0.0,
     "kernel_distance": 0.0
+  },
+  {
+    "step": 50,
+    "kernel_trace": 509185.78125,
+    "kernel_frobenius_norm": 227828.703125,
+    "mean_gradient_norm": 96.7811104330491,
+    "kernel_trace_sgd": 509185.78125,
+    "kernel_frobenius_norm_sgd": 227828.703125,
+    "kernel_distance_sgd": 0.8103836178779602,
+    "kernel_trace_signsgd": 791838720.0,
+    "kernel_frobenius_norm_signsgd": 157010000.0,
+    "kernel_distance_signsgd": 0.8501972556114197,
+    "kernel_trace_asymmetric_signsgd": 6550104.0,
+    "kernel_frobenius_norm_asymmetric_signsgd": 2146417.25,
+    "kernel_distance_asymmetric_signsgd": 0.6731733083724976,
+    "linearization_error": 4.271725659692154,
+    "kernel_distance": 0.8103836178779602
   }
 ]

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
-size 11421896
+oid sha256:913950e4971737031da511cdd1b410daae4566f62eb845b3975bca5a102323d8
+size 11421995