Upload folder using huggingface_hub

README.md

@@ -1,181 +1,79 @@
 ---
-license: apache-2.0
 pipeline_tag: text-to-image
+library_name: diffusers
+tags:
+  - Chroma
+  - quantization
+  - svdquant
+  - nunchaku
+  - fp4
+  - int4
+base_model: tonera/Chroma1-HD-SVDQ
+base_model_relation: quantized
+license: apache-2.0
 ---
-# Chroma1-HD
 
-Chroma1-HD is an **8.9B** parameter text-to-image foundational model based on **FLUX.1-schnell**. It is fully **Apache 2.0 licensed**, ensuring that anyone can use, modify, and build upon it.
+# Model Card (SVDQuant)
 
-As a **base model**, Chroma1 is intentionally designed to be an excellent starting point for **finetuning**. It provides a strong, neutral foundation for developers, researchers, and artists to create specialized models.
+> **Language**: English | [中文](README_CN.md)
 
-for the fast CFG "baked" version please go to [Chroma1-Flash](https://huggingface.co/lodestones/Chroma1-Flash).
+## Model name
 
-### Key Features
-*   **High-Performance Base:** 8.9B parameters, built on the powerful FLUX.1 architecture.
-*   **Easily Finetunable:** Designed as an ideal checkpoint for creating custom, specialized models.
-*   **Community-Driven & Open-Source:** Fully transparent with an Apache 2.0 license, and training history.
-*   **Flexible by Design:** Provides a flexible foundation for a wide range of generative tasks.
+- **Model repo**: `tonera/Chroma1-HD-SVDQ`
+- **Base (Diffusers weights path)**: `tonera/Chroma1-HD-SVDQ` (repo root)
+- **Quantized Transformer weights**: `tonera/Chroma1-HD-SVDQ/svdq-<precision>_r32-Chroma1-HD.safetensors`
 
-## Special Thanks
-A massive thank you to our supporters who make this project possible.
-*   **Anonymous donor** whose incredible generosity funded the pretraining run and data collections. Your support has been transformative for open-source AI.
-*   **Fictional.ai** for their fantastic support and for helping push the boundaries of open-source AI. You can try Chroma on their platform:
+## Quantization / inference tech
 
-[![FictionalChromaBanner_1.png](./images/FictionalChromaBanner_1.png)](https://fictional.ai/?ref=chroma_hf)
+- **Inference engine**: Nunchaku (`https://github.com/nunchaku-ai/nunchaku`)
 
-## How to Use
+Nunchaku is a high-performance inference engine for **4-bit (FP4/INT4) low-bit neural networks**. Its goal is to significantly reduce VRAM usage and improve inference speed while preserving generation quality as much as possible. It implements and productionizes post-training quantization methods such as **SVDQuant**, and uses operator/kernel fusion and other optimizations to reduce the extra overhead introduced by low-rank branches.
 
-### `diffusers` Library
+The Chroma1-HD quantized weights in this repository (e.g. `svdq-*_r32-*.safetensors`) are meant to be used with Nunchaku for efficient inference on supported GPUs.
 
-install the requirements 
+## You must install Nunchaku before use
 
-`pip install transformers diffusers sentencepiece accelerate`
+- **Official installation docs** (recommended source of truth): `https://nunchaku.tech/docs/nunchaku/installation/installation.html`
 
-```python
-import torch
-from diffusers import ChromaPipeline
+### (Recommended) Install the official prebuilt wheel
 
-pipe = ChromaPipeline.from_pretrained("lodestones/Chroma1-HD", torch_dtype=torch.bfloat16)
-pipe.enable_model_cpu_offload()
-
-prompt = [
-    "A high-fashion close-up portrait of a blonde woman in clear sunglasses. The image uses a bold teal and red color split for dramatic lighting. The background is a simple teal-green. The photo is sharp and well-composed, and is designed for viewing with anaglyph 3D glasses for optimal effect. It looks professionally done."
-]
-negative_prompt =  ["low quality, ugly, unfinished, out of focus, deformed, disfigure, blurry, smudged, restricted palette, flat colors"]
-
-image = pipe(
-    prompt=prompt,
-    negative_prompt=negative_prompt,
-    generator=torch.Generator("cpu").manual_seed(433),
-    num_inference_steps=40,
-    guidance_scale=3.0,
-    num_images_per_prompt=1,
-).images[0]
-image.save("chroma.png")
-```
-
-Quantized inference using gemlite
-
-```py
-import torch
-from diffusers import ChromaPipeline
+- **Prerequisite**: `PyTorch >= 2.5` (follow the wheel requirements as the source of truth)
+- **Install the nunchaku wheel**: pick the wheel matching your environment from GitHub Releases / HuggingFace / ModelScope (note `cp311` means Python 3.11):
+  - `https://github.com/nunchaku-ai/nunchaku/releases`
 
-pipe = ChromaPipeline.from_pretrained("lodestones/Chroma1-HD", torch_dtype=torch.float16)
-#pipe.enable_model_cpu_offload()
-
-#######################################################
-import gemlite
-device = 'cuda:0'
-processor = gemlite.helper.A8W8_int8_dynamic
-#processor = gemlite.helper.A8W8_fp8_dynamic
-#processor = gemlite.helper.A16W4_MXFP
-
-for name, module in pipe.transformer.named_modules():
-    module.name = name
-
-def patch_linearlayers(model, fct):
-    for name, layer in model.named_children():
-        if isinstance(layer, torch.nn.Linear):
-            setattr(model, name, fct(layer, name))
-        else:
-            patch_linearlayers(layer, fct)
-
-def patch_linear_to_gemlite(layer, name):
-    layer = layer.to(device, non_blocking=True)
-    try:
-        return processor(device=device).from_linear(layer) 
-    except Exception as exception:
-        print('Skipping gemlite conversion for: ' + str(layer.name), exception)
-        return layer
-
-patch_linearlayers(pipe.transformer, patch_linear_to_gemlite)
-torch.cuda.synchronize()
-torch.cuda.empty_cache()
-
-pipe.to(device)
-pipe.transformer.forward = torch.compile(pipe.transformer.forward, fullgraph=True)
-pipe.vae.forward = torch.compile(pipe.vae.forward, fullgraph=True)
-#pipe.set_progress_bar_config(disable=True)
-#######################################################
-
-prompt = [
-    "A high-fashion close-up portrait of a blonde woman in clear sunglasses. The image uses a bold teal and red color split for dramatic lighting. The background is a simple teal-green. The photo is sharp and well-composed, and is designed for viewing with anaglyph 3D glasses for optimal effect. It looks professionally done."
-]
-negative_prompt =  ["low quality, ugly, unfinished, out of focus, deformed, disfigure, blurry, smudged, restricted palette, flat colors"]
-
-import time
-for _ in range(3):
-    t_start = time.time()
-    image = pipe(
-        prompt=prompt,
-        negative_prompt=negative_prompt,
-        generator=torch.Generator("cpu").manual_seed(433),
-        num_inference_steps=40,
-        guidance_scale=3.0,
-        num_images_per_prompt=1,
-    ).images[0]
-    t_end = time.time()
-    print(f"Took: {t_end - t_start} secs.") #66.1242527961731 -> 27.72 sec
-
-image.save("chroma.png")
+```bash
+# Example (choose the correct wheel URL for your torch/cuda/python versions)
+pip install https://github.com/nunchaku-ai/nunchaku/releases/download/vX.Y.Z/nunchaku-X.Y.Z+torch2.9-cp311-cp311-linux_x86_64.whl
 ```
 
-ComfyUI
-For advanced users and customized workflows, you can use Chroma with ComfyUI.
-
-**Requirements:**
-*   A working ComfyUI installation.
-*   [Chroma checkpoint](https://huggingface.co/lodestones/Chroma) (latest version).
-*   [T5 XXL Text Encoder](https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors).
-*   [FLUX VAE](https://huggingface.co/lodestones/Chroma/resolve/main/ae.safetensors).
-*   [Chroma Workflow JSON](https://huggingface.co/lodestones/Chroma1-HD/resolve/main/ComfyUI_Chroma1-HD_T2I-workflow.json).
-
-![Chroma Workflow](./ComfyUI_Chroma1-HD_T2I-sample.png)
-![Workflow Overview](./ComfyUI_Chroma1-HD_T2I-overview.png)
-
-**Setup:**
-1.  Place the `T5_xxl` model in your `ComfyUI/models/clip` folder.
-2.  Place the `FLUX VAE` in your `ComfyUI/models/vae` folder.
-3.  Place the `Chroma checkpoint` in your `ComfyUI/models/diffusion_models` folder.
-4.  Load the Chroma workflow file into ComfyUI and run.
-
-## Model Details
-*   **Architecture:** Based on the 8.9B parameter FLUX.1-schnell model.
-*   **Training Data:** Trained on a 5M sample dataset curated from a 20M pool, including artistic, photographic, and niche styles.
-*   **Technical Report:** A comprehensive technical paper detailing the architectural modifications and training process is forthcoming.
+- **Tip (RTX 50 series GPUs)**: usually `CUDA >= 12.8` is recommended, and FP4 models are preferred for better compatibility and performance (follow the official docs).
 
-## Intended Use
-Chroma is intended to be used as a **base model** for researchers and developers to build upon. It is ideal for:
-*   Finetuning on specific styles, concepts, or characters.
-*   Research into generative model behavior, alignment, and safety.
-*   As a foundational component in larger AI systems.
+## Usage example (Diffusers + Nunchaku Transformer)
 
-## Limitations and Bias Statement
-Chroma is trained on a broad, filtered dataset from the internet. As such, it may reflect the biases and stereotypes present in its training data. The model is released in a state as is and has not been aligned with a specific safety filter.
+```python
+import torch
+from diffusers import ChromaPipeline
 
-Users are responsible for their own use of this model. It has the potential to generate content that may be considered harmful, explicit, or offensive. I encourage developers to implement appropriate safeguards and ethical considerations in their downstream applications.
+from chroma_transformer import NunchakuChromaTransformer2dModel
+from nunchaku.utils import get_precision
 
-## Summary of Architectural Modifications
-*(For a full breakdown, tech report soon-ish.)*
+MODEL = "Chroma1-HD-SVDQ"
+REPO_ID = f"tonera/{MODEL}"
 
-*   **12B → 8.9B Parameters:**
-    *   **TL;DR:** I replaced a 3.3B parameter timestep-encoding layer with a more efficient 250M parameter FFN, as the original was vastly oversized for its task.
-*   **MMDiT Masking:**
-    *   **TL;DR:** Masking T5 padding tokens enhanced fidelity and increased training stability by preventing the model from focusing on irrelevant `<pad>` tokens.
-*   **Custom Timestep Distributions:**
-    *   **TL;DR:** I implemented a custom timestep sampling distribution (`-x^2`) to prevent loss spikes and ensure the model trains effectively on both high-noise and low-noise regions.
+if __name__ == "__main__":
+    transformer = NunchakuChromaTransformer2dModel.from_pretrained(
+        f"{REPO_ID}/svdq-{get_precision()}_r32-{MODEL}.safetensors"
+    )
 
-## P.S
-Chroma1-HD is not the old Chroma-v.50 it has been retrained from v.48
+    pipe = ChromaPipeline.from_pretrained(
+        f"{REPO_ID}",
+        transformer=transformer,
+        torch_dtype=torch.bfloat16,
+        use_safetensors=True,
+    ).to("cuda")
 
-## Citation
+    prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"
+    image = pipe(prompt=prompt, guidance_scale=2.5, num_inference_steps=40).images[0]
+    image.save("Chroma1.png")
 ```
-@misc{rock2025chroma,
-  author = {Lodestone Rock},
-  title = {Chroma1-HD},
-  year = {2025},
-  publisher = {Hugging Face},
-  journal = {Hugging Face repository},
-  howpublished = {\url{https://huggingface.co/lodestones/Chroma1-HD}},
-}
-```
+

README_CN.md

@@ -0,0 +1,81 @@
+---
+pipeline_tag: text-to-image
+library_name: diffusers
+tags:
+  - Chroma
+  - quantization
+  - svdquant
+  - nunchaku
+  - fp4
+  - int4
+base_model: tonera/Chroma1-HD-SVDQ
+base_model_relation: quantized
+license: apache-2.0
+---
+
+# 模型说明（SVDQuant）
+
+> **文档语言**：中文｜[English](README.md)
+
+## 模型名称
+
+- **模型仓库**：`tonera/Chroma1-HD-SVDQ`
+- **Base（Diffusers 权重路径）**：`tonera/Chroma1-HD-SVDQ`（本仓库根目录）
+- **量化 Transformer 权重**：`tonera/Chroma1-HD-SVDQ/svdq-<precision>_r32-Chroma1-HD.safetensors`
+
+## 量化 / 推理技术
+
+- **推理引擎**：Nunchaku（`https://github.com/nunchaku-ai/nunchaku`）
+
+Nunchaku 是一个面向 **4-bit（FP4/INT4）低比特神经网络**的高性能推理引擎，核心目标是在尽量保持生成质量的同时显著降低显存占用并提升推理速度。它实现并工程化了 **SVDQuant** 等后训练量化方案，并通过算子/内核融合等优化减少低秩分支带来的额外开销。
+
+本模型仓库中的 Chroma1-HD 量化权重（例如 `svdq-*_r32-*.safetensors`）用于配合 Nunchaku，在支持的 GPU 上进行高效推理。
+
+
+## 使用前必须安装 Nunchaku
+
+- **官方安装文档**（建议以此为准）：`https://nunchaku.tech/docs/nunchaku/installation/installation.html`
+
+### （推荐）方式：安装官方预编译 Wheel
+
+- **前置条件**：安装 `PyTorch >= 2.5`（实际以对应 wheel 的要求为准）
+- **安装 nunchaku wheel**：从 GitHub Releases / HuggingFace / ModelScope 选择与你环境匹配的 wheel（注意 `cp311` 表示 Python 3.11）：
+  - `https://github.com/nunchaku-ai/nunchaku/releases`
+
+```bash
+# 示例（请按你的 torch/cuda/python 版本选择正确的 wheel URL）
+pip install https://github.com/nunchaku-ai/nunchaku/releases/download/vX.Y.Z/nunchaku-X.Y.Z+torch2.9-cp311-cp311-linux_x86_64.whl
+```
+
+- **提示（50 系 GPU）**：通常建议 `CUDA >= 12.8`，并优先使用 FP4 模型以获得更好的兼容性与性能（以官方文档为准）。
+
+## 使用示例（Diffusers + Nunchaku Transformer）
+
+
+```python
+import torch
+from diffusers import ChromaPipeline
+
+from chroma_transformer import NunchakuChromaTransformer2dModel
+from nunchaku.utils import get_precision
+
+MODEL = "Chroma1-HD-SVDQ" 
+REPO_ID = f"tonera/{MODEL}"
+
+if __name__ == "__main__":
+    transformer = NunchakuChromaTransformer2dModel.from_pretrained(
+        f"{REPO_ID}/svdq-{get_precision()}_r32-{MODEL}.safetensors"
+    )
+
+    pipe = ChromaPipeline.from_pretrained(
+        f"{REPO_ID}",
+        transformer=transformer,
+        torch_dtype=torch.bfloat16,
+        use_safetensors=True,
+    ).to("cuda")
+
+    prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"
+    image = pipe(prompt=prompt, guidance_scale=2.5, num_inference_steps=40).images[0]
+    image.save("Chroma1.png")
+```
+