tingyuzhang commited on
Commit
d931f51
·
1 Parent(s): 1137531

Add detailed model card

Browse files
Files changed (1) hide show
  1. README.md +76 -0
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: nvidia/Cosmos-Reason2-2B
3
+ license: other
4
+ tags:
5
+ - egonormia
6
+ - social-norms
7
+ - video-understanding
8
+ - sft
9
+ - multitask
10
+ - short-cot
11
+ ---
12
+
13
+ # EgoNormia-Cosmos-Reason2-2B-v5-shortcot
14
+
15
+ Multi-task SFT fine-tune of [nvidia/Cosmos-Reason2-2B](https://huggingface.co/nvidia/Cosmos-Reason2-2B) on the [EgoNormia](https://opensocial.world) social norm benchmark. This v5 variant keeps the same 3-task setup as v4, but compresses the reasoning traces into short 1-sentence CoT supervision.
16
+
17
+ ## Training
18
+
19
+ | Parameter | Value |
20
+ |-----------|-------|
21
+ | Base model | nvidia/Cosmos-Reason2-2B (Qwen3-VL-2B) |
22
+ | Tasks | Action + Justification + Sensibility (multi-task) |
23
+ | Train samples | 4959 |
24
+ | Training file | `data/egonormia_llava_shortcot_train.json` |
25
+ | CoT style | Short CoT, 1-sentence distilled traces |
26
+ | CoT length | median ~25 words (compressed from ~64 words) |
27
+ | Epochs | 3 |
28
+ | Global batch | 64 (8 replicas x 8 per replica) |
29
+ | Learning rate | 1e-5 (cosine decay, 3% warmup) |
30
+ | Context length | 8192 |
31
+ | Video input | `video_prev.mp4`, 8 frames |
32
+ | Hardware | 8x A100-SXM4-80GB |
33
+ | Seed 1 run dir | `outputs/egonormia_sft/20260228141559/` |
34
+ | Seed 2 run dir | `outputs/egonormia_sft/20260301002022/` |
35
+ | Uploaded checkpoint | seed2 `step_150` |
36
+
37
+ ## Evaluation (200 verified test samples)
38
+
39
+ | Model | Action | Justification | Both | S-IoU |
40
+ |-------|--------|---------------|------|-------|
41
+ | Zero-shot | 58.5% | 81.5% | 51.0% | 0.516 |
42
+ | v3 best (`step_175`) | 78.0% | **97.0%** | 77.0% | **0.664** |
43
+ | v5 seed1 (`step_155`) | 80.5% | 95.5% | 78.5% | 0.618 |
44
+ | **v5 seed2 (`step_150`)** | **82.0%** | 95.5% | **78.5%** | 0.634 |
45
+
46
+ Average over the two seed-wise best checkpoints:
47
+ - Action: 81.25%
48
+ - Justification: 95.5%
49
+ - Both: 78.5%
50
+ - S-IoU: 0.626
51
+
52
+ ## Robustness (option shuffle)
53
+
54
+ | Checkpoint | Delta Action | Delta S-IoU | Sign test p | Verdict |
55
+ |------------|--------------|-------------|-------------|---------|
56
+ | seed1 `step_155` | -2.0pt | -0.035 | 0.585 | pass |
57
+ | seed2 `step_150` | -5.0pt | -0.027 | 0.076 | pass |
58
+
59
+ ## Notes
60
+
61
+ - v5 recovers the robustness lost in v4 while keeping stronger action accuracy than v3.
62
+ - Best S-IoU still trails v3 (`0.634` vs `0.664`), so the gain is mainly in action / joint accuracy tradeoff rather than sensibility quality.
63
+ - On this run family, explicit think-mode inference hurts performance: for seed2, no-think `step_150` reaches 82.0% action / 78.5% both, while think mode peaks lower at 78.0% action / 72.5% both.
64
+
65
+ ## Usage
66
+
67
+ ```python
68
+ from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
69
+
70
+ model = Qwen3VLForConditionalGeneration.from_pretrained(
71
+ "robertzty/EgoNormia-Cosmos-Reason2-2B-v5-shortcot",
72
+ torch_dtype="bfloat16",
73
+ device_map="auto",
74
+ )
75
+ processor = AutoProcessor.from_pretrained("robertzty/EgoNormia-Cosmos-Reason2-2B-v5-shortcot")
76
+ ```