xrenaa commited on
Commit
1b9b087
Β·
verified Β·
1 Parent(s): 3348c59

Update model card for v1.5 checkpoints

Browse files

Add v1.5 release news and refresh checkpoint tables to match docs/checkpoints.md.
Mark previous FLUX / FLUX.2 / Qwen-Image 2kto4k checkpoints as deprecated.

Files changed (1) hide show
  1. README.md +60 -37
README.md CHANGED
@@ -31,6 +31,15 @@ base_model:
31
  [Sanja Fidler](https://www.cs.utoronto.ca/~fidler/),
32
  [Xuanchi Ren](https://xuanchiren.com/) <br>
33
 
 
 
 
 
 
 
 
 
 
34
 
35
  PiD reformulates the latent-to-pixel decoder as a conditional pixel-space
36
  diffusion model, unifying decoding and upsampling into a single generative
@@ -38,8 +47,9 @@ module. It denoises directly in high-resolution pixel space and produces a
38
  super-resolved image in one pass. This repository hosts the released decoder
39
  checkpoints, plus the encoder/decoder ("VAE") weights they depend on.
40
 
41
- All `PiD_*` checkpoints in this repo are **4-step distilled**. The non-`PiD_*`
42
- entries (`ae.safetensors`, `flux2_ae.safetensors`, `sdxl_vae.safetensors`, `QwenImage_VAE_2d.pth`, `sd3_vae/`, `rae/`,
 
43
  `scale_rae/`) are **the corresponding encoder/decoder VAE weights** that PiD
44
  plugs into β€” they're not PiD checkpoints themselves.
45
 
@@ -52,38 +62,50 @@ Global
52
 
53
  ## PiD checkpoints
54
 
55
- Two variants are released for each diffusers-style backbone:
56
 
57
- - **`2k`** β€” trained at 2048px, used as a 4Γ— decoder (512 LDM β†’ 2048 px), or as
58
  an 8Γ— decoder for the Scale-RAE backbone (256 β†’ 2048).
59
- - **`2kto4k`** β€” trained with multi-resolution data bucketing 2048β†’4096 and an
60
- SD3-style dynamic shift; designed for 1024 LDM β†’ 4K (4096 px) decoding.
61
-
62
- Both checkpoint variants support multiple aspect ratios.
63
-
64
- | Path | Latent space | SR factor | Variant |
65
- |-----------------------------------------------------------------|--------------|-----------|---------|
66
- | `checkpoints/PiD_res2k_sr4x_official_flux_distill_4step` | Flux1-dev | 4Γ— | 2k |
67
- | `checkpoints/PiD_res2k_sr4x_official_flux2_distill_4step` | Flux2-dev | 4Γ— | 2k |
68
- | `checkpoints/PiD_res2k_sr4x_official_sd3_distill_4step` | SD3 medium | 4Γ— | 2k |
69
- | `checkpoints/PiD_res2k_sr4x_official_dinov2_distill_4step` | DINOv2-B | 4Γ— | 2k |
70
- | `checkpoints/PiD_res2k_sr8x_official_siglip_distill_4step` | SigLIP-2 | 8Γ— | 2k |
71
- | `checkpoints/PiD_res2kto4k_sr4x_official_flux_distill_4step` | Flux1-dev | 4Γ— | 2kto4k |
72
- | `checkpoints/PiD_res2kto4k_sr4x_official_flux2_distill_4step_2606` | Flux2-dev | 4Γ— | 2kto4k |
73
- | `checkpoints/PiD_res2kto4k_sr4x_official_sd3_distill_4step` | SD3 medium | 4Γ— | 2kto4k |
74
- | `checkpoints/PiD_res2kto4k_sr4x_official_sdxl_distill_4step` | SDXL | 4Γ— | 2kto4k |
75
- | `checkpoints/PiD_res2kto4k_sr4x_official_qwenimage_distill_4step` | Qwen-Image | 4Γ— | 2kto4k |
76
-
77
- Each directory contains a single file, `model_ema_bf16.pth`, which is the EMA
78
- weights cast to bfloat16 β€” the format the inference scripts load by default.
79
-
80
- > **⚠️ Flux2-dev `2kto4k` β€” use the new `_2606` checkpoint.** The previous
81
- > `PiD_res2kto4k_sr4x_official_flux2_distill_4step` (without the `_2606` suffix)
82
- > suffered from a color-drifting issue. The new
83
- > `PiD_res2kto4k_sr4x_official_flux2_distill_4step_2606` fixes it β€” please use it
84
- > and do **not** use the old one. See the
85
- > [comparison](https://github.com/nv-tlabs/pid/blob/main/docs/FLUX2_2kto4k_new_ckpt_compare.md)
86
- > for details.
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
  ### Latent space β†’ compatible LDMs
89
 
@@ -103,9 +125,9 @@ the latent space's checkpoint above.
103
  | SigLIP-2 | `checkpoints/scale_rae/` | `siglip` | [Scale-RAE](https://github.com/ZitengWangNYU/Scale-RAE) (text-conditional; nyu-visionx/Scale-RAE-Qwen1.5B_DiT2.4B) |
104
 
105
  For example, Z-Image and Z-Image-Turbo share Flux1-dev's VAE, so they reuse the
106
- `flux` checkpoints (both `2k` and `2kto4k`) β€” no separate `zimage` checkpoint is
107
- shipped. Likewise `qwenimage-2512` reuses the `qwenimage` decoder (same VAE,
108
- different transformer).
109
 
110
  ## Usage
111
 
@@ -129,7 +151,8 @@ PYTHONPATH=. python -m pid._src.inference.from_ldm --backbone flux \
129
  --pid_inference_steps 4
130
  ```
131
 
132
- Pick the `2kto4k` variant via `--pid_ckpt_type 2kto4k` when decoding at 4K.
 
133
 
134
  ## Citation
135
  ```
@@ -139,4 +162,4 @@ Pick the `2kto4k` variant via `--pid_ckpt_type 2kto4k` when decoding at 4K.
139
  journal={arXiv preprint arXiv:2605.23902},
140
  year={2026}
141
  }
142
- ```
 
31
  [Sanja Fidler](https://www.cs.utoronto.ca/~fidler/),
32
  [Xuanchi Ren](https://xuanchiren.com/) <br>
33
 
34
+ ## News
35
+
36
+ - [July 2026] PiD v1.5 checkpoints for **FLUX**, **FLUX.2**, and
37
+ **Qwen-Image** are released. See the
38
+ [comparison page](https://research.nvidia.com/labs/sil/projects/pid/comparison.html)
39
+ for the improvements:
40
+ - Improved decoding color fidelity
41
+ - Removed grid artifacts in image corners
42
+ - Improved anime and facial details
43
 
44
  PiD reformulates the latent-to-pixel decoder as a conditional pixel-space
45
  diffusion model, unifying decoding and upsampling into a single generative
 
47
  super-resolved image in one pass. This repository hosts the released decoder
48
  checkpoints, plus the encoder/decoder ("VAE") weights they depend on.
49
 
50
+ The distilled `PiD_*` checkpoints in this repo are **4-step distilled**. The
51
+ non-`PiD_*` entries (`ae.safetensors`, `flux2_ae.safetensors`,
52
+ `sdxl_vae.safetensors`, `QwenImage_VAE_2d.pth`, `sd3_vae/`, `rae/`,
53
  `scale_rae/`) are **the corresponding encoder/decoder VAE weights** that PiD
54
  plugs into β€” they're not PiD checkpoints themselves.
55
 
 
62
 
63
  ## PiD checkpoints
64
 
65
+ PiD checkpoint variants:
66
 
67
+ - **`2k`** - trained at 2048px, used as a 4Γ— decoder (512 LDM β†’ 2048 px), or as
68
  an 8Γ— decoder for the Scale-RAE backbone (256 β†’ 2048).
69
+ - **`2kto4k_v1pt5`** - the recommended up-to-4K decoder for **FLUX**,
70
+ **FLUX.2**, and **Qwen-Image** latent spaces.
71
+ - **`2kto4k`** - the legacy up-to-4K decoder still used for **SD3** and
72
+ **SDXL**. The previous FLUX / FLUX.2 / Qwen-Image `2kto4k` checkpoints are
73
+ deprecated and have been moved to `checkpoints_deprecated/`.
74
+
75
+ Each checkpoint directory contains a single file, `model_ema_bf16.pth`, which
76
+ is the EMA weights cast to bfloat16, the format the inference scripts load by
77
+ default.
78
+
79
+ ### Distilled checkpoints
80
+
81
+ | Backbone | decode 2k resolution only | decode 2k resolution to 4k resolution |
82
+ |----------|---------------------------|--------------------------------------|
83
+ | flux | `checkpoints/PiD_res2k_sr4x_official_flux_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux_distill_4step` |
84
+ | flux2 | `checkpoints/PiD_res2k_sr4x_official_flux2_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux2_distill_4step` |
85
+ | flux2-klein-4b | `checkpoints/PiD_res2k_sr4x_official_flux2_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux2_distill_4step` |
86
+ | flux2-klein-9b | `checkpoints/PiD_res2k_sr4x_official_flux2_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux2_distill_4step` |
87
+ | zimage | `checkpoints/PiD_res2k_sr4x_official_flux_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux_distill_4step` |
88
+ | zimage-turbo | `checkpoints/PiD_res2k_sr4x_official_flux_distill_4step` | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux_distill_4step` |
89
+ | qwenimage | - | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_qwenimage_distill_4step` |
90
+ | qwenimage-2512 | - | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_qwenimage_distill_4step` |
91
+ | sd3 | `checkpoints/PiD_res2k_sr4x_official_sd3_distill_4step` | `checkpoints/PiD_res2kto4k_sr4x_official_sd3_distill_4step` |
92
+ | sdxl | - | `checkpoints/PiD_res2kto4k_sr4x_official_sdxl_distill_4step` |
93
+ | dinov2 | `checkpoints/PiD_res2k_sr4x_official_dinov2_distill_4step` | - |
94
+ | siglip | `checkpoints/PiD_res2k_sr8x_official_siglip_distill_4step` | - |
95
+
96
+ ### Undistilled checkpoints
97
+
98
+ | VAE | decode 2k resolution to 4k resolution |
99
+ |-----|--------------------------------------|
100
+ | flux | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux_undistilled` |
101
+ | flux2 | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_flux2_undistilled` |
102
+ | qwenimage (wan2.1) | `checkpoints/PiD_v1pt5_res2kto4k_sr4x_official_qwenimage_undistilled` |
103
+
104
+ ### PixelDiT 2k-to-4k checkpoint
105
+
106
+ | Model | Checkpoint path |
107
+ |-------|-----------------|
108
+ | PixelDiT | `checkpoints/PixelDiT_finetune_2kto4k` |
109
 
110
  ### Latent space β†’ compatible LDMs
111
 
 
125
  | SigLIP-2 | `checkpoints/scale_rae/` | `siglip` | [Scale-RAE](https://github.com/ZitengWangNYU/Scale-RAE) (text-conditional; nyu-visionx/Scale-RAE-Qwen1.5B_DiT2.4B) |
126
 
127
  For example, Z-Image and Z-Image-Turbo share Flux1-dev's VAE, so they reuse the
128
+ `flux` checkpoints (both `2k` and `2kto4k_v1pt5`) β€” no separate `zimage`
129
+ checkpoint is shipped. Likewise `qwenimage-2512` reuses the `qwenimage`
130
+ decoder (same VAE, different transformer).
131
 
132
  ## Usage
133
 
 
151
  --pid_inference_steps 4
152
  ```
153
 
154
+ Pick `--pid_ckpt_type 2kto4k_v1pt5` for FLUX / FLUX.2 / Qwen-Image 4K
155
+ decoding. Use `--pid_ckpt_type 2kto4k` for SD3 / SDXL 4K decoding.
156
 
157
  ## Citation
158
  ```
 
162
  journal={arXiv preprint arXiv:2605.23902},
163
  year={2026}
164
  }
165
+ ```