sangramrout commited on
Commit
11a9ac7
·
verified ·
1 Parent(s): f98243d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Wan-AI/Wan2.1-I2V-14B-480P-Diffusers
3
+ tags:
4
+ - lora
5
+ - qlora
6
+ - egocentric
7
+ - exo-to-ego
8
+ - video-generation
9
+ - egoX
10
+ ---
11
+
12
+ # EgoX mix LoRA — rank-128 QLoRA (single 24 GB GPU reproduction)
13
+
14
+ A **LoRA adapter** for **Wan2.1-I2V-14B-480P** reproducing **EgoX** (exocentric → egocentric video
15
+ generation) on a **single 24 GB GPU** (vs the paper's 4× H200). Trained with the paper's
16
+ Geometry-Guided Self-Attention (GGA), scaled down to fit the hardware budget.
17
+
18
+ ![joker in-the-wild](joker_mix_ckpt1984_infer.gif)
19
+
20
+ *exo input | our vitS ego-prior | ego pred (our prior) | ego pred (shipped prior)*
21
+
22
+ ## Training
23
+ - **Adapter:** LoRA rank 128 / α 128 on the Wan-14B transformer (NF4 4-bit base + 8-bit AdamW)
24
+ - **Data:** 552 clips, **all 17 EgoX domains** (Ego-Exo4D); 495 train / 57 val
25
+ - **Resolution:** 49×176×704 · **LR** 2e-5, constant-with-warmup · effective batch 4
26
+ - **16 epochs / 1,984 steps**, held-out val loss **0.247 → 0.173** (checkpoint-1984)
27
+
28
+ ## Use
29
+ Load as an unfused PEFT adapter onto an NF4-quantized Wan2.1-I2V-14B with GGA enabled
30
+ (see the [reproduction repo](https://github.com/skr3178/egoX): `EgoX/infer_nf4.py --use_GGA`).
31
+
32
+ ## Caveats
33
+ Scaled-down repro: 176-res output, NF4-r128, 552 clips (vs the paper's ~3,510). Image-criteria
34
+ scores trail the paper (~11 vs 16 PSNR) due to resolution + budget, not pipeline fidelity.