Upload SAE model weights, config, and training state

README.md

@@ -0,0 +1,96 @@
+---
+license: mit
+tags:
+- physical-ai-interpretability-sae
+- LeRobot
+- Robotics
+datasets:
+- villekuosmanen/build_block_tower
+- villekuosmanen/dAgger_build_block_tower_1.0.0
+- villekuosmanen/dAgger_build_block_tower_1.1.0
+- villekuosmanen/dAgger_build_block_tower_1.2.0
+- villekuosmanen/dAgger_build_block_tower_1.3.0
+- villekuosmanen/dAgger_build_block_tower_1.4.0
+- villekuosmanen/fail_build_block_tower_stationary
+- villekuosmanen/fail_build_block_tower_autonomous_interaction
+library_name: physical-ai-interpretability
+---
+
+# Sparse Autoencoder (SAE) Model
+
+This model is a Sparse Autoencoder trained for interpretability analysis of robotics policies using the LeRobot framework.
+
+## Model Details
+
+- **Architecture**: Multi-modal Sparse Autoencoder
+- **Training Dataset**: `villekuosmanen/build_block_tower`, `villekuosmanen/dAgger_build_block_tower_1.0.0`, `villekuosmanen/dAgger_build_block_tower_1.1.0`, `villekuosmanen/dAgger_build_block_tower_1.2.0`, `villekuosmanen/dAgger_build_block_tower_1.3.0`, `villekuosmanen/dAgger_build_block_tower_1.4.0`, `villekuosmanen/fail_build_block_tower_stationary`, `villekuosmanen/fail_build_block_tower_autonomous_interaction`
+- **Base Policy**: LeRobot ACT policy
+- **Layer Target**: `model.encoder.layers.3.norm2`
+- **Tokens**: 77
+- **Token Dimension**: 128
+- **Feature Dimension**: 12320
+- **Expansion Factor**: 1.25
+
+## Training Configuration
+
+- **Learning Rate**: 0.0001
+- **Batch Size**: 16
+- **L1 Penalty**: 0.3
+- **Epochs**: 20
+- **Optimizer**: adam
+
+## Usage
+
+```python
+from physical_ai_interpretability.sae.trainer import load_sae_from_hub
+
+# Load model from Hub
+model = load_sae_from_hub("villekuosmanen/build_block_tower_all_small_sae")
+
+# Or load using builder
+from physical_ai_interpretability.sae.builder import SAEBuilder
+builder = SAEBuilder(device='cuda')
+model = builder.load_from_hub("villekuosmanen/build_block_tower_all_small_sae")
+```
+
+## Out-of-Distribution Detection
+
+This SAE model can be used for OOD detection with LeRobot policies:
+
+```python
+from physical_ai_interpretability.ood import OODDetector
+
+# Create OOD detector with Hub-loaded SAE
+ood_detector = OODDetector(
+    policy=your_policy,
+    sae_hub_repo_id="villekuosmanen/build_block_tower_all_small_sae"
+)
+
+# Fit threshold and use for detection
+ood_detector.fit_ood_threshold_to_validation_dataset(validation_dataset)
+is_ood, error = ood_detector.is_out_of_distribution(observation)
+```
+
+## Files
+
+- `model.safetensors`: The trained SAE model weights
+- `config.json`: Training and model configuration
+- `training_state.pt`: Complete training state (optimizer, scheduler, metrics)
+- `ood_params.json`: OOD detection parameters (if fitted)
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@misc{sae_model,
+  title={Sparse Autoencoder for Build Block Tower},
+  author={Your Name},
+  year={2024},
+  url={https://huggingface.co/villekuosmanen/build_block_tower_all_small_sae}
+}
+```
+
+## Framework
+
+This model was trained using the [physical-ai-interpretability](https://github.com/your-repo/physical-ai-interpretability) framework with [LeRobot](https://github.com/huggingface/lerobot).

config.json

@@ -0,0 +1,36 @@
+{
+  "num_tokens": 77,
+  "token_dim": 128,
+  "expansion_factor": 1.25,
+  "activation_fn": "relu",
+  "use_token_sampling": true,
+  "fixed_tokens": [
+    0,
+    1
+  ],
+  "sampling_strategy": "block_average",
+  "sampling_stride": 8,
+  "max_sampled_tokens": 200,
+  "block_size": 8,
+  "batch_size": 16,
+  "learning_rate": 0.0001,
+  "num_epochs": 20,
+  "validation_split": 0.1,
+  "l1_penalty": 0.3,
+  "optimizer": "adam",
+  "weight_decay": 1e-05,
+  "lr_schedule": "constant",
+  "warmup_epochs": 2,
+  "gradient_clip_norm": 1.0,
+  "early_stopping_patience": 10,
+  "early_stopping_min_delta": 1e-05,
+  "log_every": 5,
+  "save_every": 1000,
+  "validate_every": 500,
+  "device": "cuda",
+  "repo_id": "[villekuosmanen/build_block_tower, villekuosmanen/dAgger_build_block_tower_1.0.0, villekuosmanen/dAgger_build_block_tower_1.1.0, villekuosmanen/dAgger_build_block_tower_1.2.0, villekuosmanen/dAgger_build_block_tower_1.3.0, villekuosmanen/dAgger_build_block_tower_1.4.0, villekuosmanen/fail_build_block_tower_stationary, villekuosmanen/fail_build_block_tower_autonomous_interaction]",
+  "repo_hash": "881292d6",
+  "layer_name": "model.encoder.layers.3.norm2",
+  "activation_cache_path": "/home/ville/.cache/physical_ai_interpretability/sae_activations",
+  "experiment_name": "sae_fail_build_block_tow_881292d6"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0211c1fe33e69f654dd269531f5aee5e04b26a56a7552ab4e5a7f14ea2449906
+size 971496408

training_state.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d142098a493441e4ab7adbb34caebce779e5659f678299accf1253b46743bd21
+size 1942998303