ZeyuLing commited on
Commit
63d431b
Β·
verified Β·
1 Parent(s): a735cb5

Normalize Motius Pipeline.from_pretrained metadata

Browse files
Files changed (2) hide show
  1. README.md +21 -13
  2. model_index.json +23 -0
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- library_name: hftrainer
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
@@ -13,10 +13,10 @@ license: other
13
 
14
  # MDM β€” Human Motion Diffusion Model
15
 
16
- Text-to-motion baseline integrated into the hftrainer Model Zoo. Our
17
  reproduction is **fully self-contained and independent of `ref_repo`**: the
18
  network, the Gaussian-diffusion schedule, the classifier-free-guidance sampler
19
- and the collate are all vendored into `hftrainer.models.motion.mdm._mdm`, and verified
20
  to be **bit-identical** to the released checkpoint (`max-abs-diff = 0.0` for the
21
  same seed/input).
22
 
@@ -24,7 +24,7 @@ same seed/input).
24
  |---|---|
25
  | **Task** | Text-to-Motion (T2M) |
26
  | **Bundle / Pipeline** | `MDMBundle` / `MDMPipeline` |
27
- | **Processed HF artifact** | [`ZeyuLing/hftrainer-mdm-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-mdm-humanml3d) |
28
  | **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
29
  | **Text encoder** | CLIP ViT-B/32 (frozen) |
30
  | **Paper** | *Human Motion Diffusion Model*, Tevet et al., ICLR 2023 β€” [arXiv:2209.14916](https://arxiv.org/abs/2209.14916) |
@@ -34,26 +34,26 @@ same seed/input).
34
 
35
  ## Weights
36
 
37
- Current hftrainer artifact (diffusers-style `from_pretrained`):
38
 
39
  | Artifact | Location | Contents | Status |
40
  |---|---|---|---|
41
- | MDM HumanML3D | [`ZeyuLing/hftrainer-mdm-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-mdm-humanml3d) | `model.safetensors` + `mdm_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact; complete CLIP packaging pending |
42
  | local mirror | `checkpoints/mdm/humanml_trans_enc_512` | same layout | optional local cache |
43
 
44
  **Use directly from the Hub:**
45
 
46
  ```python
47
- from hftrainer.pipelines.mdm import MDMPipeline
48
 
49
- pipe = MDMPipeline.from_pretrained("ZeyuLing/hftrainer-mdm-humanml3d", device="cuda")
50
  motions = pipe.infer_t2m(["a person walks forward then sits down"], [120]) # list of (T, 263)
51
  ```
52
 
53
  **Or download to disk first:**
54
 
55
  ```bash
56
- huggingface-cli download ZeyuLing/hftrainer-mdm-humanml3d \
57
  --local-dir checkpoints/mdm/humanml_trans_enc_512
58
  ```
59
 
@@ -82,7 +82,7 @@ CLIP ViT-B/32 is currently resolved by name rather than stored inside the repo.
82
  | `local_vel` | 66 | local joint velocities (22Γ—3) |
83
  | `foot_contact` | 4 | binary foot-contact labels |
84
 
85
- Convert to/from other spaces with `hftrainer.motion.representation.convert`
86
  (e.g. `hml263_to_joints`, `hml263_to_motion135`, `hml263_to_motion272`).
87
 
88
  ---
@@ -90,7 +90,7 @@ Convert to/from other spaces with `hftrainer.motion.representation.convert`
90
  ## Evaluation
91
 
92
  Generation under the **official HumanML3D protocol** (standard test split, native
93
- 263-dim @ 20 fps, first caption) and scoring with the two persisted hftrainer
94
  evaluators. Reproduce with:
95
 
96
  ```bash
@@ -103,7 +103,7 @@ python3 scripts/eval/verify_evaluators.py --which hml263 \
103
 
104
  ### HumanML3D-263 evaluator (native space, n=3970)
105
 
106
- | Metric | hftrainer | MDM paper | Note |
107
  |---|---|---|---|
108
  | **FID** ↓ | **0.509** | 0.544 | βœ… reproduced (within noise) |
109
  | **Diversity** β†’ | **9.563** | 9.559 | βœ… matches |
@@ -136,10 +136,18 @@ is the `263β†’272` representation mismatch.
136
 
137
  ## Implementation notes
138
 
139
- - **Vendored, ref_repo-independent**: `hftrainer/models/mdm/_mdm/` holds the
140
  network (`network.py`), diffusion (`diffusion/`), CFG sampler and collate.
141
  Training-only deps are stubbed (inference-only).
142
  - **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
143
  HumanML3D *training* stats (not the evaluator stats) and are embedded in the
144
  artifact, eliminating the recurring "wrong Mean/Std β†’ forward drift" bug.
145
  - **Guidance**: classifier-free, default scale `2.5`.
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: motius
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
 
13
 
14
  # MDM β€” Human Motion Diffusion Model
15
 
16
+ Text-to-motion baseline integrated into the motius Model Zoo. Our
17
  reproduction is **fully self-contained and independent of `ref_repo`**: the
18
  network, the Gaussian-diffusion schedule, the classifier-free-guidance sampler
19
+ and the collate are all vendored into `motius.models.motion.mdm._mdm`, and verified
20
  to be **bit-identical** to the released checkpoint (`max-abs-diff = 0.0` for the
21
  same seed/input).
22
 
 
24
  |---|---|
25
  | **Task** | Text-to-Motion (T2M) |
26
  | **Bundle / Pipeline** | `MDMBundle` / `MDMPipeline` |
27
+ | **Processed HF artifact** | [`ZeyuLing/Motius-MDM-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MDM-HumanML3D) |
28
  | **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
29
  | **Text encoder** | CLIP ViT-B/32 (frozen) |
30
  | **Paper** | *Human Motion Diffusion Model*, Tevet et al., ICLR 2023 β€” [arXiv:2209.14916](https://arxiv.org/abs/2209.14916) |
 
34
 
35
  ## Weights
36
 
37
+ Current motius artifact (diffusers-style `from_pretrained`):
38
 
39
  | Artifact | Location | Contents | Status |
40
  |---|---|---|---|
41
+ | MDM HumanML3D | [`ZeyuLing/Motius-MDM-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MDM-HumanML3D) | `model.safetensors` + `mdm_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact; complete CLIP packaging pending |
42
  | local mirror | `checkpoints/mdm/humanml_trans_enc_512` | same layout | optional local cache |
43
 
44
  **Use directly from the Hub:**
45
 
46
  ```python
47
+ from motius.pipelines.mdm import MDMPipeline
48
 
49
+ pipe = MDMPipeline.from_pretrained("ZeyuLing/Motius-MDM-HumanML3D", device="cuda")
50
  motions = pipe.infer_t2m(["a person walks forward then sits down"], [120]) # list of (T, 263)
51
  ```
52
 
53
  **Or download to disk first:**
54
 
55
  ```bash
56
+ huggingface-cli download ZeyuLing/Motius-MDM-HumanML3D \
57
  --local-dir checkpoints/mdm/humanml_trans_enc_512
58
  ```
59
 
 
82
  | `local_vel` | 66 | local joint velocities (22Γ—3) |
83
  | `foot_contact` | 4 | binary foot-contact labels |
84
 
85
+ Convert to/from other spaces with `motius.motion.representation.convert`
86
  (e.g. `hml263_to_joints`, `hml263_to_motion135`, `hml263_to_motion272`).
87
 
88
  ---
 
90
  ## Evaluation
91
 
92
  Generation under the **official HumanML3D protocol** (standard test split, native
93
+ 263-dim @ 20 fps, first caption) and scoring with the two persisted motius
94
  evaluators. Reproduce with:
95
 
96
  ```bash
 
103
 
104
  ### HumanML3D-263 evaluator (native space, n=3970)
105
 
106
+ | Metric | motius | MDM paper | Note |
107
  |---|---|---|---|
108
  | **FID** ↓ | **0.509** | 0.544 | βœ… reproduced (within noise) |
109
  | **Diversity** β†’ | **9.563** | 9.559 | βœ… matches |
 
136
 
137
  ## Implementation notes
138
 
139
+ - **Vendored, ref_repo-independent**: `motius/models/mdm/_mdm/` holds the
140
  network (`network.py`), diffusion (`diffusion/`), CFG sampler and collate.
141
  Training-only deps are stubbed (inference-only).
142
  - **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
143
  HumanML3D *training* stats (not the evaluator stats) and are embedded in the
144
  artifact, eliminating the recurring "wrong Mean/Std β†’ forward drift" bug.
145
  - **Guidance**: classifier-free, default scale `2.5`.
146
+
147
+ ## Direct Loading
148
+
149
+ ```python
150
+ from motius import Pipeline
151
+
152
+ pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MDM-HumanML3D")
153
+ ```
model_index.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "MDMPipeline",
3
+ "_library_name": "motius",
4
+ "format_version": 1,
5
+ "pipeline_class": "motius.pipelines.mdm.MDMPipeline",
6
+ "bundle_class": "motius.models.mdm.MDMBundle",
7
+ "tasks": [
8
+ "text_to_motion"
9
+ ],
10
+ "required_files": [
11
+ "Mean.npy",
12
+ "Std.npy",
13
+ "mdm_config.json",
14
+ "model.safetensors"
15
+ ],
16
+ "api": {
17
+ "loader": "motius.Pipeline.from_pretrained",
18
+ "task_methods": [
19
+ "infer_text_to_motion"
20
+ ]
21
+ },
22
+ "artifact_format": "motius-mdm-v1"
23
+ }