ZeyuLing commited on
Commit
5568c23
Β·
verified Β·
1 Parent(s): 6f35e65

Normalize Motius Pipeline.from_pretrained metadata

Browse files
Files changed (2) hide show
  1. README.md +22 -14
  2. model_index.json +28 -0
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- library_name: hftrainer
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
@@ -13,11 +13,11 @@ license: other
13
 
14
  # MotionStreamer
15
 
16
- Streaming/autoregressive text-to-motion baseline integrated into the hftrainer
17
  Model Zoo. Our reproduction is **fully self-contained and independent of
18
  `ref_repo`**: the causal TAE, the LLaMA autoregressive transformer, the
19
  per-token diffusion head and the OpenAI-style Gaussian-diffusion sampler are all
20
- vendored into `hftrainer.models.motion.motionstreamer._ms`. The `save_pretrained` /
21
  `from_pretrained` round-trip is **bit-identical** (`max-abs-diff = 0.0` for both
22
  the TAE and the AR weights).
23
 
@@ -25,7 +25,7 @@ the TAE and the AR weights).
25
  |---|---|
26
  | **Task** | Text-to-Motion (T2M) |
27
  | **Bundle / Pipeline** | `MotionStreamerBundle` / `MotionStreamerPipeline` |
28
- | **Processed HF artifact** | [`ZeyuLing/hftrainer-motionstreamer-humanml272`](https://huggingface.co/ZeyuLing/hftrainer-motionstreamer-humanml272) |
29
  | **Motion representation** | **MotionStreamer-272** (272-dim, 30 fps) |
30
  | **Text encoder** | SentenceT5-XXL (`sentence-transformers/sentence-t5-xxl`, frozen) |
31
  | **Paper** | *MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model*, 2025 β€” [arXiv:2503.15451](https://arxiv.org/abs/2503.15451) |
@@ -35,20 +35,20 @@ the TAE and the AR weights).
35
 
36
  ## Weights
37
 
38
- Current hftrainer artifact (diffusers-style `from_pretrained`):
39
 
40
  | Artifact | Location | Contents | Status |
41
  |---|---|---|---|
42
- | MotionStreamer HumanML3D-272 | [`ZeyuLing/hftrainer-motionstreamer-humanml272`](https://huggingface.co/ZeyuLing/hftrainer-motionstreamer-humanml272) | `tae.safetensors` + `ar.safetensors` + `ms_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact; complete SentenceT5 packaging pending |
43
  | local mirror | `checkpoints/motionstreamer/t2m_humanml272` | same layout | optional local cache |
44
 
45
  **Use directly from the Hub:**
46
 
47
  ```python
48
- from hftrainer.pipelines.motionstreamer import MotionStreamerPipeline
49
 
50
  pipe = MotionStreamerPipeline.from_pretrained(
51
- "ZeyuLing/hftrainer-motionstreamer-humanml272",
52
  device="cuda",
53
  )
54
  motions = pipe.infer_t2m(["a person walks forward then turns around"], [120]) # list of (T, 272)
@@ -62,7 +62,7 @@ resolved by name rather than stored inside the repo.
62
  **Or download to disk first:**
63
 
64
  ```bash
65
- huggingface-cli download ZeyuLing/hftrainer-motionstreamer-humanml272 \
66
  --local-dir checkpoints/motionstreamer/t2m_humanml272
67
  ```
68
 
@@ -79,7 +79,7 @@ text -> SentenceT5-XXL -> LLaMA AR (CFG, per-token diffusion sampling)
79
  -> latent tokens (dim 16) -> causal TAE decoder (Γ—4 upsample) -> 272-dim motion
80
  ```
81
 
82
- Convert to/from HumanML3D-263 with `hftrainer.motion.representation.convert`
83
  (`hml263_to_motion272`, etc.).
84
 
85
  ---
@@ -100,14 +100,14 @@ python3 scripts/eval/eval_ms_h3d272.py --pred_dir outputs/evaluation/ms_h3d272/m
100
 
101
  ### MotionStreamer-272 evaluator (native space)
102
 
103
- The hftrainer `MotionStreamer272Evaluator` is the same TMR-style evaluator used
104
  in the paper (matching feature scale: MM-Dist β‰ˆ 15, Diversity β‰ˆ 27). Paper
105
  numbers below are from the ICCV 2025 HumanML3D test-set table.
106
 
107
- > _Full-set generation (7412 pairs, 8 GPUs) is in progress; the `hftrainer`
108
  > column is filled in once scoring completes._
109
 
110
- | Metric | hftrainer | MotionStreamer paper (ICCV'25) |
111
  |---|---|---|
112
  | FID ↓ | _pending_ | 11.790 |
113
  | R-Precision Top-1 / 2 / 3 ↑ | _pending_ | 0.631 / 0.802 / 0.859 |
@@ -122,10 +122,18 @@ confirming the evaluator; the model row follows once generation finishes.
122
 
123
  ## Implementation notes
124
 
125
- - **Vendored, ref_repo-independent**: `hftrainer/models/motionstreamer/_ms/` holds
126
  `tae.py` / `causal_cnn.py` / `resnet.py` (causal TAE), `llama_model.py` (LLaMA
127
  AR), `diffloss.py` + `diffusion/` (per-token diffusion head). Only relative
128
  imports were changed from the upstream source.
129
  - **Text encoder reloaded by name**: SentenceT5-XXL is frozen and not duplicated
130
  into the artifact (like CLIP for MDM).
131
  - **Guidance**: classifier-free, default scale `4.0`, token unit length `4`.
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: motius
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
 
13
 
14
  # MotionStreamer
15
 
16
+ Streaming/autoregressive text-to-motion baseline integrated into the motius
17
  Model Zoo. Our reproduction is **fully self-contained and independent of
18
  `ref_repo`**: the causal TAE, the LLaMA autoregressive transformer, the
19
  per-token diffusion head and the OpenAI-style Gaussian-diffusion sampler are all
20
+ vendored into `motius.models.motion.motionstreamer._ms`. The `save_pretrained` /
21
  `from_pretrained` round-trip is **bit-identical** (`max-abs-diff = 0.0` for both
22
  the TAE and the AR weights).
23
 
 
25
  |---|---|
26
  | **Task** | Text-to-Motion (T2M) |
27
  | **Bundle / Pipeline** | `MotionStreamerBundle` / `MotionStreamerPipeline` |
28
+ | **Processed HF artifact** | [`ZeyuLing/Motius-MotionStreamer-HumanML272`](https://huggingface.co/ZeyuLing/Motius-MotionStreamer-HumanML272) |
29
  | **Motion representation** | **MotionStreamer-272** (272-dim, 30 fps) |
30
  | **Text encoder** | SentenceT5-XXL (`sentence-transformers/sentence-t5-xxl`, frozen) |
31
  | **Paper** | *MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model*, 2025 β€” [arXiv:2503.15451](https://arxiv.org/abs/2503.15451) |
 
35
 
36
  ## Weights
37
 
38
+ Current motius artifact (diffusers-style `from_pretrained`):
39
 
40
  | Artifact | Location | Contents | Status |
41
  |---|---|---|---|
42
+ | MotionStreamer HumanML3D-272 | [`ZeyuLing/Motius-MotionStreamer-HumanML272`](https://huggingface.co/ZeyuLing/Motius-MotionStreamer-HumanML272) | `tae.safetensors` + `ar.safetensors` + `ms_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact; complete SentenceT5 packaging pending |
43
  | local mirror | `checkpoints/motionstreamer/t2m_humanml272` | same layout | optional local cache |
44
 
45
  **Use directly from the Hub:**
46
 
47
  ```python
48
+ from motius.pipelines.motionstreamer import MotionStreamerPipeline
49
 
50
  pipe = MotionStreamerPipeline.from_pretrained(
51
+ "ZeyuLing/Motius-MotionStreamer-HumanML272",
52
  device="cuda",
53
  )
54
  motions = pipe.infer_t2m(["a person walks forward then turns around"], [120]) # list of (T, 272)
 
62
  **Or download to disk first:**
63
 
64
  ```bash
65
+ huggingface-cli download ZeyuLing/Motius-MotionStreamer-HumanML272 \
66
  --local-dir checkpoints/motionstreamer/t2m_humanml272
67
  ```
68
 
 
79
  -> latent tokens (dim 16) -> causal TAE decoder (Γ—4 upsample) -> 272-dim motion
80
  ```
81
 
82
+ Convert to/from HumanML3D-263 with `motius.motion.representation.convert`
83
  (`hml263_to_motion272`, etc.).
84
 
85
  ---
 
100
 
101
  ### MotionStreamer-272 evaluator (native space)
102
 
103
+ The motius `MotionStreamer272Evaluator` is the same TMR-style evaluator used
104
  in the paper (matching feature scale: MM-Dist β‰ˆ 15, Diversity β‰ˆ 27). Paper
105
  numbers below are from the ICCV 2025 HumanML3D test-set table.
106
 
107
+ > _Full-set generation (7412 pairs, 8 GPUs) is in progress; the `motius`
108
  > column is filled in once scoring completes._
109
 
110
+ | Metric | motius | MotionStreamer paper (ICCV'25) |
111
  |---|---|---|
112
  | FID ↓ | _pending_ | 11.790 |
113
  | R-Precision Top-1 / 2 / 3 ↑ | _pending_ | 0.631 / 0.802 / 0.859 |
 
122
 
123
  ## Implementation notes
124
 
125
+ - **Vendored, ref_repo-independent**: `motius/models/motionstreamer/_ms/` holds
126
  `tae.py` / `causal_cnn.py` / `resnet.py` (causal TAE), `llama_model.py` (LLaMA
127
  AR), `diffloss.py` + `diffusion/` (per-token diffusion head). Only relative
128
  imports were changed from the upstream source.
129
  - **Text encoder reloaded by name**: SentenceT5-XXL is frozen and not duplicated
130
  into the artifact (like CLIP for MDM).
131
  - **Guidance**: classifier-free, default scale `4.0`, token unit length `4`.
132
+
133
+ ## Direct Loading
134
+
135
+ ```python
136
+ from motius import Pipeline
137
+
138
+ pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MotionStreamer-HumanML272")
139
+ ```
model_index.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "MotionStreamerPipeline",
3
+ "_library_name": "motius",
4
+ "format_version": 1,
5
+ "pipeline_class": "motius.pipelines.motionstreamer.MotionStreamerPipeline",
6
+ "bundle_class": "motius.models.motionstreamer.MotionStreamerBundle",
7
+ "tasks": [
8
+ "text_to_motion",
9
+ "temporal_motion_completion",
10
+ "sequential_text_to_motion"
11
+ ],
12
+ "required_files": [
13
+ "Mean.npy",
14
+ "Std.npy",
15
+ "ar.safetensors",
16
+ "ms_config.json",
17
+ "tae.safetensors"
18
+ ],
19
+ "api": {
20
+ "loader": "motius.Pipeline.from_pretrained",
21
+ "task_methods": [
22
+ "infer_text_to_motion",
23
+ "infer_temporal_motion_completion",
24
+ "infer_sequential_text_to_motion"
25
+ ]
26
+ },
27
+ "artifact_format": "motius-motionstreamer-v1"
28
+ }