tonera commited on
Commit
5139b65
·
verified ·
1 Parent(s): 4cb094d

Upload ./README_CN.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README_CN.md +25 -24
README_CN.md CHANGED
@@ -13,50 +13,51 @@ base_model_relation: quantized
13
  license: apache-2.0
14
  ---
15
 
16
- # 模型说明(SVDQuant
17
 
18
- > **文档语言**中文|[English](README.md)
19
 
20
  ![Chroma1-HD](chroma1-hd.png)
21
 
22
 
23
- ## 模型名称
24
 
25
- - **模型仓库**`tonera/Chroma1-HD-SVDQ`
26
- - **BaseDiffusers 权重路径)**`tonera/Chroma1-HD-SVDQ`(本仓库根目录)
27
- - **量化 Transformer 权重**`tonera/Chroma1-HD-SVDQ/svdq-<precision>_r32-Chroma1-HD.safetensors`
28
 
29
- ## 量化 / 推理技术
30
 
31
- - **推理引擎**Nunchaku`https://github.com/nunchaku-ai/nunchaku`
32
 
33
- Nunchaku 是一个面向 **4-bitFP4/INT4)低比特神经网络**的高性能推理引擎,核心目标是在尽量保持生成质量的同时显著降低显存占用并提升推理速度。它实现并工程化了 **SVDQuant** 等后训练量化方案,并通过算子/内核融合等优化减少低秩分支带来的额外开销。
34
 
35
- 本模型仓库中的 Chroma1-HD 量化权重(例如 `svdq-*_r32-*.safetensors`)用于配合 Nunchaku,在支持的 GPU 上进行高效推理。
36
 
 
37
 
38
- ## 使用前必须安装 Nunchaku
39
 
40
- - **官方安装文档**(建议以此为准):`https://nunchaku.tech/docs/nunchaku/installation/installation.html`
41
 
42
- ### (推荐)方式:安装官方预编译 Wheel
43
-
44
- - **前置条件**:安装 `PyTorch >= 2.5`(实际以对应 wheel 的要求为准)
45
- - **安装 nunchaku wheel**:从 GitHub Releases / HuggingFace / ModelScope 选择与你环境匹配的 wheel(注意 `cp311` 表示 Python 3.11):
46
  - `https://github.com/nunchaku-ai/nunchaku/releases`
47
 
48
  ```bash
49
- # 示例(请按你的 torch/cuda/python 版本选择正确的 wheel URL)
50
  pip install https://github.com/nunchaku-ai/nunchaku/releases/download/vX.Y.Z/nunchaku-X.Y.Z+torch2.9-cp311-cp311-linux_x86_64.whl
51
  ```
52
 
53
- - **提示(50 GPU)**:通常建议 `CUDA >= 12.8`,并优先使用 FP4 模型以获得更好的兼容性与性能(以官方文档为准)。
54
-
55
- ## 使用示例(Diffusers + Nunchaku Transformer)
56
- - 注意:我正在推动Nunchaku官方合并此PR:https://github.com/nunchaku-ai/nunchaku/pull/916
57
- 在此之前,如果您想体验,可将仓库中transformer_chroma.py
58
- 拷贝到nunchaku/models/transformers/transformer_chroma.py中。这是一个相对低性能版本,高性能版本仍然要等PR通过。
59
 
 
 
 
 
 
 
 
60
 
61
  ```python
62
  import torch
@@ -65,7 +66,7 @@ from diffusers import ChromaPipeline
65
  from nunchaku import NunchakuChromaTransformer2dModel
66
  from nunchaku.utils import get_precision
67
 
68
- MODEL = "Chroma1-HD-SVDQ"
69
  REPO_ID = f"tonera/{MODEL}"
70
 
71
  if __name__ == "__main__":
 
13
  license: apache-2.0
14
  ---
15
 
16
+ # Model Card (SVDQuant)
17
 
18
+ > **Language**: English | [中文](README_CN.md)
19
 
20
  ![Chroma1-HD](chroma1-hd.png)
21
 
22
 
23
+ ## Model name
24
 
25
+ - **Model repo**: `tonera/Chroma1-HD-SVDQ`
26
+ - **Base (Diffusers weights path)**: `tonera/Chroma1-HD-SVDQ` (repo root)
27
+ - **Quantized Transformer weights**: `tonera/Chroma1-HD-SVDQ/svdq-<precision>_r32-Chroma1-HD.safetensors`
28
 
29
+ ## Quantization / inference tech
30
 
31
+ - **Inference engine**: Nunchaku (`https://github.com/nunchaku-ai/nunchaku`)
32
 
33
+ Nunchaku is a high-performance inference engine for **4-bit (FP4/INT4) low-bit neural networks**. Its goal is to significantly reduce VRAM usage and improve inference speed while preserving generation quality as much as possible. It implements and productionizes post-training quantization methods such as **SVDQuant**, and uses operator/kernel fusion and other optimizations to reduce the extra overhead introduced by low-rank branches.
34
 
35
+ The Chroma1-HD quantized weights in this repository (e.g. `svdq-*_r32-*.safetensors`) are meant to be used with Nunchaku for efficient inference on supported GPUs.
36
 
37
+ ## You must install Nunchaku before use
38
 
39
+ - **Official installation docs** (recommended source of truth): `https://nunchaku.tech/docs/nunchaku/installation/installation.html`
40
 
41
+ ### (Recommended) Install the official prebuilt wheel
42
 
43
+ - **Prerequisite**: `PyTorch >= 2.5` (follow the wheel requirements as the source of truth)
44
+ - **Install the nunchaku wheel**: pick the wheel matching your environment from GitHub Releases / HuggingFace / ModelScope (note `cp311` means Python 3.11):
 
 
45
  - `https://github.com/nunchaku-ai/nunchaku/releases`
46
 
47
  ```bash
48
+ # Example (choose the correct wheel URL for your torch/cuda/python versions)
49
  pip install https://github.com/nunchaku-ai/nunchaku/releases/download/vX.Y.Z/nunchaku-X.Y.Z+torch2.9-cp311-cp311-linux_x86_64.whl
50
  ```
51
 
52
+ - **Tip (RTX 50 series GPUs)**: usually `CUDA >= 12.8` is recommended, and FP4 models are preferred for better compatibility and performance (follow the official docs).
 
 
 
 
 
53
 
54
+ ## Usage example (Diffusers + Nunchaku Transformer)
55
+ Note: I am pushing for the official Nunchaku PR to be merged: https://github.com/nunchaku-ai/nunchaku/pull/928
56
+ Until then, if you want to try it out, you can copy `transformer_chroma.py` from the repository to `nunchaku/models/transformers/transformer_chroma.py`.
57
+ Usage like this:
58
+ ```
59
+ from nunchaku.models.transformers.transformer_chroma import NunchakuChromaTransformer2dModel
60
+ ```
61
 
62
  ```python
63
  import torch
 
66
  from nunchaku import NunchakuChromaTransformer2dModel
67
  from nunchaku.utils import get_precision
68
 
69
+ MODEL = "Chroma1-HD-SVDQ"
70
  REPO_ID = f"tonera/{MODEL}"
71
 
72
  if __name__ == "__main__":