File size: 2,118 Bytes
eea471e 00b2b8b eea471e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | # Minimal Action Model
This is the first modeling baseline for the public Xperience-10M sample released by Ropedia.
The script is:
```text
scripts/train_min_action_model.py
```
It trains a small Numpy-only softmax classifier:
```text
annotation.hdf5
-> hand/body/IMU/camera/contact windows
-> action or subtask labels from captions
-> stratified train/test split
-> multinomial logistic regression
-> metrics and predictions
```
Run:
```bash
cd /path/to/Ropedia
source .venv/bin/activate
python scripts/train_min_action_model.py
```
Default output:
```text
outputs/min_action_model/
```
Important artifacts:
- `metrics.json`: accuracy, balanced accuracy, macro-F1, weighted-F1, majority baseline.
- `per_class_metrics.csv`: precision/recall/F1 per action class.
- `confusion_matrix.csv`: true label vs predicted label matrix.
- `predictions.csv`: one row per test window.
- `feature_dataset.npz`: processed numeric features and labels.
- `model.npz`: fitted scaler and softmax weights.
This is a learning baseline, not a publishable benchmark. The public sample is only one episode, so stratified windows from one episode are correlated. For serious evaluation, use many episodes and split by held-out episodes or held-out task instances.
## Current Sample Results
Action-label model:
```text
outputs/min_action_model/
accuracy: 0.9828
balanced_accuracy: 0.9644
macro_f1: 0.9688
weighted_f1: 0.9824
majority_baseline: 0.1375
classes: 18
test_windows: 291
```
Subtask-label model:
```text
outputs/min_subtask_model/
accuracy: 0.9759
balanced_accuracy: 0.9784
macro_f1: 0.9528
weighted_f1: 0.9779
majority_baseline: 0.1448
classes: 14
test_windows: 290
```
Why the numbers are high:
- This is one public sample episode.
- Windows are stratified randomly, so train/test windows can be close in time.
- The result proves the pipeline works; it does not prove cross-episode generalization.
Next serious evaluation:
```text
many episodes -> split by episode -> train on some episodes -> test on unseen episodes
```
|