Upload two_stream_attn_v1_finetune_20260514T122537Z

README.md

@@ -17,7 +17,7 @@ metrics:
 - accuracy
 - f1
 model-index:
-- name: two_stream_attn_v1_finetune_20260514T013537Z
+- name: two_stream_attn_v1_finetune_20260514T122537Z
   results:
   - task:
       type: gesture-recognition
@@ -26,12 +26,12 @@ model-index:
       type: IPN-Hand
     metrics:
     - type: accuracy
-      value: 0.9566
+      value: 0.9584
     - type: f1
-      value: 0.9561
+      value: 0.9573
 ---
 
-# two_stream_attn_v1_finetune_20260514T013537Z
+# two_stream_attn_v1_finetune_20260514T122537Z
 
 A real-time hand gesture classifier trained on
 a Hybrid Jester+IPN gesture dataset (Jester dynamic classes + IPN pointing classes).
@@ -43,10 +43,10 @@ standard webcam using MediaPipe for landmark extraction.
 ## Model Description
 
 - **Architecture**: EnhancedTwoStreamLSTM (BiLSTM h=96×2, MHA 8 heads, proj=96, mean+max pool, MLP gate)
-- **Parameters**: 2,099,820
+- **Parameters**: 2,099,434
 - **Input**: `(batch, 16, 147)`
     — 16-frame sliding window at 30 FPS ≈ 533 ms
-- **Output**: Softmax logits over 12 gesture classes
+- **Output**: Softmax logits over 10 gesture classes
 - **Inference latency**: < 1 ms per call (CPU, single sample)
 - **Feature schema**: `feature-schema-v1`
 
@@ -78,7 +78,7 @@ Input (B, T=32, 147)
     │   ctx_b  = LN(ctx_b × gate_b + ctx_b)
     │
     └─ cat(ctx_a, ctx_b) → (384,)
-       LN → Linear(384→192) → GELU → Dropout → Linear(192→12)
+       LN → Linear(384→192) → GELU → Dropout → Linear(192→10)
 ```
 
 **Design rationale:**
@@ -92,8 +92,6 @@ Input (B, T=32, 147)
 | Class | Description |
 |-------|-------------|
 | `palm` | Open palm held flat toward camera (static hand shape) |
-| `pinch` | Thumb and index finger brought together (pinch grip) |
-| `click` | Brief tap / click gesture — index finger snaps toward thumb |
 | `swiping_right` | Horizontal swipe from left to right |
 | `swiping_left` | Horizontal swipe from right to left |
 | `swiping_down` | Vertical swipe downward |
@@ -109,8 +107,6 @@ Input (B, T=32, 147)
 | Class | Mode | Command | Runtime handling |
 |-------|------|---------|------------------|
 | `palm` | `discrete` | `erase_annotations` | Discrete command via GestureActivationController → CommandDispatcher |
-| `pinch` | `discrete` | `no_action` | No-op background class |
-| `click` | `discrete` | `no_action` | No-op background class |
 | `swiping_right` | `discrete` | `next_slide` | Discrete command via GestureActivationController → CommandDispatcher |
 | `swiping_left` | `discrete` | `previous_slide` | Discrete command via GestureActivationController → CommandDispatcher |
 | `swiping_down` | `discrete` | `stop_presentation` | Discrete command via GestureActivationController → CommandDispatcher |
@@ -143,7 +139,7 @@ from maestro.infrastructure.model.checkpoint_loader import load_inference_artifa
 # Download the artifact (cached after first call)
 local_path = hf_hub_download(
     repo_id="ntsrigaud/maestro-lstm-hybrid",
-    filename="two_stream_attn_v1_finetune_20260514T013537Z_inference.pt",
+    filename="two_stream_attn_v1_finetune_20260514T122537Z_inference.pt",
 )
 
 # Load the artifact (includes model, class labels, and feature schema)
@@ -165,7 +161,7 @@ with torch.no_grad():
 ## Training Dataset
 
 - **Source**: Hybrid merge of Jester and IPN-Hand windows: Jester provides unknown/swiping/zoom/stop_sign classes; IPN-Hand provides point_one and point_two
-- **Used classes**: 12 (11 active gestures + `unknown` background)
+- **Used classes**: 10 (9 active gestures + `unknown` background)
 - **Dataset split**: 70% train / 15% val / 15% test (stratified by class)
 - **Augmentation**: temporal scale ±20%, spatial jitter σ=0.005
 
@@ -175,7 +171,7 @@ Two-phase transfer learning pipeline:
 - **Phase 1 (pretraining):** backbone pretrained on external checkpoint `two_stream_attn_v1_20260513T155730Z.pt` to learn generic gesture dynamics.
 - **Phase 2 (fine-tuning):** head replaced and model adapted on Hybrid Jester+IPN 10-gesture vocabulary.
 - **Stage A (frozen backbone):** 10 epoch(s) head-only warmup.
-- **Stage B (full model):** up to 51 epoch(s) joint fine-tuning with scheduler/early stopping.
+- **Stage B (full model):** up to 60 epoch(s) joint fine-tuning with scheduler/early stopping.
 - **Stage B retention defences:** replay_max_samples_per_class=500, distillation_weight=0.0, replay_ce_weight=0.0, backbone_lr_multiplier=0.1, ewc_weight=2000.0, gpm_components=0, forgetting_penalty_weight=0.5.
 
 ## Training Configuration
@@ -203,25 +199,23 @@ Two-phase transfer learning pipeline:
 
 | Metric | Value |
 |--------|-------|
-| Accuracy | 95.7% |
-| Macro F1 | 95.6% |
+| Accuracy | 95.8% |
+| Macro F1 | 95.7% |
 
 ### Per-Class Recall
 
 | Class | Recall |
 |-------|--------|
-| `palm` | 98.7% |
-| `pinch` | 98.4% |
-| `click` | 94.0% |
+| `palm` | 98.4% |
 | `swiping_right` | 95.3% |
-| `swiping_left` | 98.3% |
-| `swiping_down` | 96.3% |
-| `swiping_up` | 97.6% |
-| `zooming_in_full_hand` | 96.8% |
-| `zooming_out_full_hand` | 92.4% |
+| `swiping_left` | 98.7% |
+| `swiping_down` | 96.8% |
+| `swiping_up` | 97.8% |
+| `zooming_in_full_hand` | 97.3% |
+| `zooming_out_full_hand` | 92.8% |
 | `point_one` | 97.1% |
-| `point_two` | 93.2% |
-| `unknown` | 89.4% |
+| `point_two` | 94.3% |
+| `unknown` | 88.9% |
 
 ## Comparison with Previous Architecture
 
@@ -232,7 +226,7 @@ Two-phase transfer learning pipeline:
 | Feature projection | No | **Yes (→96)** |
 | Temporal pooling | Mean only | **Mean + Max** |
 | Cross-stream fusion | Concat only | **2-layer MLP gate** |
-| Parameters | ~182 K | ~2,099,820 |
+| Parameters | ~182 K | ~2,099,434 |
 
 ## Limitations and Risks
 

config.json

@@ -1,12 +1,12 @@
 {
-  "model_version": "two_stream_attn_v1_finetune_20260514T013537Z",
+  "model_version": "two_stream_attn_v1_finetune_20260514T122537Z",
   "model_config": {
     "model_name": "two_stream_attn_v1",
     "input_size": 147,
     "hidden_size": 96,
     "num_layers": 4,
     "dropout": 0.35,
-    "num_classes": 12
+    "num_classes": 10
   },
   "feature_schema": {
     "feature_schema_version": "feature-schema-v1",
@@ -32,43 +32,37 @@
     }
   },
   "evaluation": {
-    "test_accuracy": 0.9566010951125532,
-    "test_macro_f1": 0.9560781251156708,
-    "test_loss": 0.4304363657823322,
-    "calibration_ece": 0.029200464802868007,
+    "test_accuracy": 0.9584274740171712,
+    "test_macro_f1": 0.9573115163058359,
+    "test_loss": 0.41317880455315303,
+    "calibration_ece": 0.03046416184897343,
     "per_class_recall": {
-      "palm": 0.9869109947643979,
-      "pinch": 0.984313725490196,
-      "click": 0.94,
+      "palm": 0.9842931937172775,
       "swiping_right": 0.9525959367945824,
-      "swiping_left": 0.983177570093458,
-      "swiping_down": 0.9625246548323472,
-      "swiping_up": 0.9755600814663951,
-      "zooming_in_full_hand": 0.9681818181818181,
-      "zooming_out_full_hand": 0.9240506329113924,
+      "swiping_left": 0.9869158878504672,
+      "swiping_down": 0.9684418145956607,
+      "swiping_up": 0.9775967413441955,
+      "zooming_in_full_hand": 0.9727272727272728,
+      "zooming_out_full_hand": 0.9282700421940928,
       "point_one": 0.9711286089238845,
-      "point_two": 0.9320652173913043,
-      "unknown": 0.8938271604938272
+      "point_two": 0.9429347826086957,
+      "unknown": 0.8888888888888888
     },
     "per_class_precision": {
-      "palm": 0.9792207792207792,
-      "pinch": 0.9471698113207547,
-      "click": 0.9832635983263598,
-      "swiping_right": 0.9634703196347032,
-      "swiping_left": 0.9813432835820896,
-      "swiping_down": 0.9606299212598425,
-      "swiping_up": 0.9676767676767677,
-      "zooming_in_full_hand": 0.9240780911062907,
-      "zooming_out_full_hand": 0.9668874172185431,
-      "point_one": 0.8958837772397095,
-      "point_two": 0.9607843137254902,
-      "unknown": 0.9501312335958005
+      "palm": 0.9868766404199475,
+      "swiping_right": 0.9723502304147466,
+      "swiping_left": 0.9795918367346939,
+      "swiping_down": 0.958984375,
+      "swiping_up": 0.97165991902834,
+      "zooming_in_full_hand": 0.928416485900217,
+      "zooming_out_full_hand": 0.9649122807017544,
+      "point_one": 0.9002433090024331,
+      "point_two": 0.9665738161559888,
+      "unknown": 0.9498680738786279
     }
   },
   "class_labels": [
     "palm",
-    "pinch",
-    "click",
     "swiping_right",
     "swiping_left",
     "swiping_down",
@@ -79,7 +73,7 @@
     "point_two",
     "unknown"
   ],
-  "created_at": "2026-05-14T01:38:18.988584+00:00",
+  "created_at": "2026-05-14T12:30:37.922314+00:00",
   "gesture_command_mapping": {
     "commands": {
       "swiping_up": "start_presentation",
@@ -89,10 +83,8 @@
       "zooming_in_full_hand": "zoom_in_view",
       "zooming_out_full_hand": "zoom_out_view",
       "palm": "erase_annotations",
-      "fist": "no_action",
-      "thumb_up": "no_action",
-      "pinch": "no_action",
-      "click": "no_action",
+      "pinch": "activate_laser_pointer",
+      "click": "mouse_click",
       "unknown": "no_action"
     },
     "modes": {
@@ -103,8 +95,6 @@
       "zooming_in_full_hand": "discrete",
       "zooming_out_full_hand": "discrete",
       "palm": "discrete",
-      "fist": "discrete",
-      "thumb_up": "discrete",
       "pinch": "discrete",
       "click": "discrete",
       "point_one": "continuous",