Text-to-Image
Diffusers
Safetensors
StableDiffusionPipeline
clover-image
diffusion
stable-diffusion
knowledge-distillation
compact
local-inference
Instructions to use neonforestmist/Clover-Image-Tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use neonforestmist/Clover-Image-Tiny with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("neonforestmist/Clover-Image-Tiny", torch_dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Publish validated Clover Image Tiny research preview
Browse files- README.md +92 -13
- checksums.json +3 -3
- examples/generate.py +200 -26
- research_preview.json +4 -4
README.md
CHANGED
|
@@ -14,15 +14,28 @@ tags:
|
|
| 14 |
|
| 15 |
# Clover Image Tiny — Research Preview
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
## Exact artifact identity
|
| 28 |
|
|
@@ -37,10 +50,11 @@ claim of uniformly better output than its starting model.
|
|
| 37 |
| Checkpoint-bundle SHA-256 | `384b6515f5f26838aea33ec9a941e06610a20764f0b8637c8b7b0667bfc0d447` |
|
| 38 |
| Resolved-config SHA-256 | `80cf9395d1f587dc0c1d440d9f5b55c55c20703187998509bb306d19d463f597` |
|
| 39 |
| Dataset-manifest SHA-256 | `50c1249f1cb0d8d690a9acc451ca10c9432eb5a7f4e26f34acb5462096e72322` |
|
| 40 |
-
|
|
|
|
|
| 41 |
| Package files | `30` |
|
| 42 |
| Validated Stage B source-package checksums SHA-256 | `d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc` |
|
| 43 |
-
| Builder source commit | `
|
| 44 |
|
| 45 |
The checkpoint completed 500 optimizer steps, 4,000 microsteps, and 4,000
|
| 46 |
sample presentations. All 500 recorded training rows were finite and had a
|
|
@@ -61,8 +75,10 @@ With the pinned Diffusers 0.39.0 scheduler, those 50 requested PNDM steps use
|
|
| 61 |
All eight Stage B gallery images were finite, nonblank, nonblack, and returned
|
| 62 |
clear from the packaged upstream safety checker in that measured run. This
|
| 63 |
small engineering subset is not a general safety or quality evaluation. The
|
| 64 |
-
|
| 65 |
-
|
|
|
|
|
|
|
| 66 |
|
| 67 |
## Run locally
|
| 68 |
|
|
@@ -72,6 +88,58 @@ space for the model itself. Mac MPS is the only local runtime measured for this
|
|
| 72 |
preview; Windows/Linux CUDA and CPU execution are supported by the runner but
|
| 73 |
their performance is not claimed here.
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
### macOS Apple silicon — MPS
|
| 76 |
|
| 77 |
These commands download a complete local copy, install the pinned runtime in
|
|
@@ -191,6 +259,9 @@ python examples/generate.py \
|
|
| 191 |
--output clover-image-tiny.png
|
| 192 |
```
|
| 193 |
|
|
|
|
|
|
|
|
|
|
| 194 |
Seeded generation is repeatable only within the limits of the selected runtime.
|
| 195 |
Different devices, dtypes, kernels, and dependency builds can produce different
|
| 196 |
pixels. The bundled Mac example below was measured locally on an Apple M4 Pro
|
|
@@ -236,6 +307,8 @@ RAIL-M terms, and the package checksums for the portable evidence included here.
|
|
| 236 |
|
| 237 |
- Research and inspection of compact Stable Diffusion knowledge distillation
|
| 238 |
- Reproducing the fixed conventional 50-step engineering examples
|
|
|
|
|
|
|
| 239 |
- Comparing this early calibration checkpoint with its pinned starting student
|
| 240 |
- Non-consequential creative experimentation subject to the model license
|
| 241 |
|
|
@@ -257,6 +330,12 @@ gallery run, but it is not a complete safety system and can miss harmful output
|
|
| 257 |
or over-filter benign output. Applications should add appropriate prompt and
|
| 258 |
output controls, human review, and policy enforcement.
|
| 259 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 260 |
Do not use this preview for consequential decisions, identity claims, medical
|
| 261 |
or legal conclusions, harassment, exploitation, illegal activity, or any use
|
| 262 |
prohibited by CreativeML OpenRAIL-M. Review outputs before sharing them.
|
|
|
|
| 14 |
|
| 15 |
# Clover Image Tiny — Research Preview
|
| 16 |
|
| 17 |
+
**Clover Image Tiny** is a compact, consumer-runnable text-to-image model with
|
| 18 |
+
a 323,384,964-parameter denoiser and a complete Diffusers package of about
|
| 19 |
+
1.67 GB. It generates locally on Apple silicon, NVIDIA CUDA systems, or CPU;
|
| 20 |
+
no hosted inference API is required. Its examples have the recognizable,
|
| 21 |
+
playful fidelity that can reasonably be described informally as
|
| 22 |
+
**DALL·E mini-ish**. That phrase is a qualitative analogy, not a benchmark or
|
| 23 |
+
claim of equivalent architecture, training scale, or measured quality.
|
| 24 |
+
|
| 25 |
+
This exact artifact is a conventional knowledge-distillation checkpoint
|
| 26 |
+
released as a research preview. Clover trained the checkpoint for 500 optimizer
|
| 27 |
+
steps on an exact licensed 1,000-pair calibration set, so it is a genuinely
|
| 28 |
+
modified model rather than a rename of unchanged upstream weights. It was
|
| 29 |
+
**not trained from random initialization**: it starts from BK-SDM-Tiny-2M and
|
| 30 |
+
was distilled with a frozen Stable Diffusion v1.4 teacher. Calling it "trained
|
| 31 |
+
from scratch" would therefore be inaccurate. The validated reference recipe
|
| 32 |
+
uses 50-step PNDM, 512×512 output, classifier-free guidance 7.5, and an empty
|
| 33 |
+
negative prompt.
|
| 34 |
+
|
| 35 |
+
The checkpoint is a production-data quality-refresh pilot, not a release-ready
|
| 36 |
+
model or a controlled quality benchmark. Its compact size, coherent example
|
| 37 |
+
outputs, exact lineage, and local runner make it useful today for creative
|
| 38 |
+
experimentation and compact-model research.
|
| 39 |
|
| 40 |
## Exact artifact identity
|
| 41 |
|
|
|
|
| 50 |
| Checkpoint-bundle SHA-256 | `384b6515f5f26838aea33ec9a941e06610a20764f0b8637c8b7b0667bfc0d447` |
|
| 51 |
| Resolved-config SHA-256 | `80cf9395d1f587dc0c1d440d9f5b55c55c20703187998509bb306d19d463f597` |
|
| 52 |
| Dataset-manifest SHA-256 | `50c1249f1cb0d8d690a9acc451ca10c9432eb5a7f4e26f34acb5462096e72322` |
|
| 53 |
+
| Denoiser parameters | `323,384,964` |
|
| 54 |
+
| Package bytes | `1671502657` |
|
| 55 |
| Package files | `30` |
|
| 56 |
| Validated Stage B source-package checksums SHA-256 | `d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc` |
|
| 57 |
+
| Builder source commit | `647bac81f3713a0238eb21abf1844a8cd8d48f11` |
|
| 58 |
|
| 59 |
The checkpoint completed 500 optimizer steps, 4,000 microsteps, and 4,000
|
| 60 |
sample presentations. All 500 recorded training rows were finite and had a
|
|
|
|
| 75 |
All eight Stage B gallery images were finite, nonblank, nonblack, and returned
|
| 76 |
clear from the packaged upstream safety checker in that measured run. This
|
| 77 |
small engineering subset is not a general safety or quality evaluation. The
|
| 78 |
+
examples demonstrate recognizable objects, people, animals, landscapes,
|
| 79 |
+
interiors, food, and night scenes. Results are not uniformly better than the
|
| 80 |
+
starting model: the bottle example loses prompt fidelity, hands remain weak,
|
| 81 |
+
and some details change without clear improvement.
|
| 82 |
|
| 83 |
## Run locally
|
| 84 |
|
|
|
|
| 88 |
preview; Windows/Linux CUDA and CPU execution are supported by the runner but
|
| 89 |
their performance is not claimed here.
|
| 90 |
|
| 91 |
+
### Generation controls
|
| 92 |
+
|
| 93 |
+
The bundled runner exposes the ordinary Diffusers controls below. Its defaults
|
| 94 |
+
exactly preserve the measured gallery recipe.
|
| 95 |
+
|
| 96 |
+
| Argument | Accepted values | Default | Effect |
|
| 97 |
+
|---|---|---|---|
|
| 98 |
+
| `--steps` | 4–100 | `50` | Conventional diffusion inference steps; more steps usually take longer |
|
| 99 |
+
| `--guidance-scale` | 0.0–20.0 | `7.5` | Strength of text guidance |
|
| 100 |
+
| `--negative-prompt` | Text, or empty | Empty | Content or traits to discourage |
|
| 101 |
+
| `--width` | 256–768, divisible by 64 | `512` | Output width |
|
| 102 |
+
| `--height` | 256–768, divisible by 64 | `512` | Output height |
|
| 103 |
+
| `--scheduler` | `pndm`, `ddim`, `euler`, `euler-a`, `dpmpp-2m` | `pndm` | Sampling algorithm |
|
| 104 |
+
| `--num-images` | 1–4 | `1` | Images generated in one invocation |
|
| 105 |
+
| `--seed` | 0–(2⁶³−1) | `1337` | Starting deterministic seed |
|
| 106 |
+
|
| 107 |
+
Only **PNDM + 50 steps + guidance 7.5 + 512×512 + an empty negative
|
| 108 |
+
prompt** is the validated gallery and local-MPS recipe. Other combinations are
|
| 109 |
+
deliberately available for exploration, but no comparative quality or speed
|
| 110 |
+
claim is attached to them. These are conventional diffusion steps, **not** the
|
| 111 |
+
separate 1–4 Clover Leaf Steps system, which is not part of this package. In
|
| 112 |
+
particular, selecting four or eight inference steps does not turn this
|
| 113 |
+
checkpoint into a distilled four- or eight-step model.
|
| 114 |
+
|
| 115 |
+
For example, this asks for two 512×512 images using 30 Euler steps, stronger
|
| 116 |
+
guidance, and an explicit negative prompt:
|
| 117 |
+
|
| 118 |
+
```bash
|
| 119 |
+
python model/examples/generate.py \
|
| 120 |
+
--model model \
|
| 121 |
+
--device auto \
|
| 122 |
+
--local-files-only \
|
| 123 |
+
--prompt "a tiny glass greenhouse glowing in a moonlit garden, detailed photography" \
|
| 124 |
+
--negative-prompt "blurry, distorted, low detail" \
|
| 125 |
+
--steps 30 \
|
| 126 |
+
--guidance-scale 8.5 \
|
| 127 |
+
--scheduler euler \
|
| 128 |
+
--width 512 \
|
| 129 |
+
--height 512 \
|
| 130 |
+
--num-images 2 \
|
| 131 |
+
--seed 1337 \
|
| 132 |
+
--output greenhouse.png
|
| 133 |
+
```
|
| 134 |
+
|
| 135 |
+
For multiple images, the first is saved to the requested output and subsequent
|
| 136 |
+
images use numbered names such as `greenhouse-02.png`. Seeds advance
|
| 137 |
+
consecutively from the requested seed. One `greenhouse.png.json` sidecar records
|
| 138 |
+
the resolved controls and every output filename and seed. The runner refuses to
|
| 139 |
+
overwrite an existing planned image or sidecar. Resolution and batch size
|
| 140 |
+
multiply memory use; reduce `--num-images`, width, or height if a consumer GPU
|
| 141 |
+
runs out of memory.
|
| 142 |
+
|
| 143 |
### macOS Apple silicon — MPS
|
| 144 |
|
| 145 |
These commands download a complete local copy, install the pinned runtime in
|
|
|
|
| 259 |
--output clover-image-tiny.png
|
| 260 |
```
|
| 261 |
|
| 262 |
+
Run `python examples/generate.py --help` for the complete control reference and
|
| 263 |
+
accepted ranges.
|
| 264 |
+
|
| 265 |
Seeded generation is repeatable only within the limits of the selected runtime.
|
| 266 |
Different devices, dtypes, kernels, and dependency builds can produce different
|
| 267 |
pixels. The bundled Mac example below was measured locally on an Apple M4 Pro
|
|
|
|
| 307 |
|
| 308 |
- Research and inspection of compact Stable Diffusion knowledge distillation
|
| 309 |
- Reproducing the fixed conventional 50-step engineering examples
|
| 310 |
+
- Exploring step count, guidance, negative prompting, resolution, schedulers,
|
| 311 |
+
and small batches through the bundled local runner
|
| 312 |
- Comparing this early calibration checkpoint with its pinned starting student
|
| 313 |
- Non-consequential creative experimentation subject to the model license
|
| 314 |
|
|
|
|
| 330 |
or over-filter benign output. Applications should add appropriate prompt and
|
| 331 |
output controls, human review, and policy enforcement.
|
| 332 |
|
| 333 |
+
The bundled runner keeps that checker enabled and does not expose a bypass
|
| 334 |
+
flag. A flagged result may be returned as a black placeholder; its sidecar sets
|
| 335 |
+
`nsfw_content_detected` for the corresponding image so the condition is not
|
| 336 |
+
silent. Developers integrating the raw Diffusers pipeline are responsible for
|
| 337 |
+
providing an appropriate moderation system for their application.
|
| 338 |
+
|
| 339 |
Do not use this preview for consequential decisions, identity claims, medical
|
| 340 |
or legal conclusions, harassment, exploitation, illegal activity, or any use
|
| 341 |
prohibited by CreativeML OpenRAIL-M. Review outputs before sharing them.
|
checksums.json
CHANGED
|
@@ -4,16 +4,16 @@
|
|
| 4 |
"LICENSE-CODE": "e26a0375b714267325c27905ba947fdc41ebc0b7a36940a49ae12379ed6208d6",
|
| 5 |
"LICENSE-MODEL-CREATIVEML-OPENRAIL-M.txt": "be351ebe7ac01bcdbb018639aadcfd38f136b7dc3f2a3d4d3a24db51d1b210ef",
|
| 6 |
"MODEL_DATA_LICENSES.md": "8f6b712826f9904561fb32038635612722be61cfd776da58fb61a4848a03148e",
|
| 7 |
-
"README.md": "
|
| 8 |
"assets/clover-image-tiny-local-mps-library-seed-1469.png": "f8830346f2a9c2b9a8c2a01d8f90e6925c93d667c1bcf998aa904a150589a742",
|
| 9 |
"assets/clover-image-tiny-paired-contact-sheet.png": "8653d1105b7b0c56e2385a127336dbe9b3a1a03c8518edb4934353a9d8e13bfa",
|
| 10 |
"evidence/clover-image-tiny-local-mps-library-seed-1469.json": "1ddd86de0be992987214266720e0f341f25791402aae637065d6d09a6ea67b27",
|
| 11 |
"evidence/stage-b-inference-checksums.json": "d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc",
|
| 12 |
-
"examples/generate.py": "
|
| 13 |
"feature_extractor/preprocessor_config.json": "205d45875859846dc1ca211b9a19f5db72f47e953b396954ee34d48d8e95c3c6",
|
| 14 |
"model_index.json": "792073e269e5caff40b3f8c553387540c17a5d900ed0e9cbfab309d650e708dd",
|
| 15 |
"requirements.txt": "9dbb74869a34e3b610f737bce8a9d2a20691fd0fa416cf6bb42354b78e0a3861",
|
| 16 |
-
"research_preview.json": "
|
| 17 |
"safety_checker/config.json": "241bd8fb1ba6feff0d7421f7af0f85743711c5d76353626f71f7e2cc8009ac7c",
|
| 18 |
"safety_checker/model.safetensors": "57ecdfa243b170f9b4cb3eefaf0f64552ef78fc0bf0eb1c5b9675308447184f6",
|
| 19 |
"scheduler/scheduler_config.json": "184bc33ed887951dd86e37f20c878710c4e18e1c5fd4461256a3825679b53d15",
|
|
|
|
| 4 |
"LICENSE-CODE": "e26a0375b714267325c27905ba947fdc41ebc0b7a36940a49ae12379ed6208d6",
|
| 5 |
"LICENSE-MODEL-CREATIVEML-OPENRAIL-M.txt": "be351ebe7ac01bcdbb018639aadcfd38f136b7dc3f2a3d4d3a24db51d1b210ef",
|
| 6 |
"MODEL_DATA_LICENSES.md": "8f6b712826f9904561fb32038635612722be61cfd776da58fb61a4848a03148e",
|
| 7 |
+
"README.md": "020e0c921afca2a8cd4378e20032dc80c9974fc39351f74e1b5762ff28bbd07f",
|
| 8 |
"assets/clover-image-tiny-local-mps-library-seed-1469.png": "f8830346f2a9c2b9a8c2a01d8f90e6925c93d667c1bcf998aa904a150589a742",
|
| 9 |
"assets/clover-image-tiny-paired-contact-sheet.png": "8653d1105b7b0c56e2385a127336dbe9b3a1a03c8518edb4934353a9d8e13bfa",
|
| 10 |
"evidence/clover-image-tiny-local-mps-library-seed-1469.json": "1ddd86de0be992987214266720e0f341f25791402aae637065d6d09a6ea67b27",
|
| 11 |
"evidence/stage-b-inference-checksums.json": "d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc",
|
| 12 |
+
"examples/generate.py": "fcea598655f7e144bc1ccce9be6d4e9e166915a3ad3c26fc0aa6c8ade00b49f1",
|
| 13 |
"feature_extractor/preprocessor_config.json": "205d45875859846dc1ca211b9a19f5db72f47e953b396954ee34d48d8e95c3c6",
|
| 14 |
"model_index.json": "792073e269e5caff40b3f8c553387540c17a5d900ed0e9cbfab309d650e708dd",
|
| 15 |
"requirements.txt": "9dbb74869a34e3b610f737bce8a9d2a20691fd0fa416cf6bb42354b78e0a3861",
|
| 16 |
+
"research_preview.json": "e9906a1cbac490e08c8a72cc4669cacffb4fab0b1a944334a6bd74424d13416c",
|
| 17 |
"safety_checker/config.json": "241bd8fb1ba6feff0d7421f7af0f85743711c5d76353626f71f7e2cc8009ac7c",
|
| 18 |
"safety_checker/model.safetensors": "57ecdfa243b170f9b4cb3eefaf0f64552ef78fc0bf0eb1c5b9675308447184f6",
|
| 19 |
"scheduler/scheduler_config.json": "184bc33ed887951dd86e37f20c878710c4e18e1c5fd4461256a3825679b53d15",
|
examples/generate.py
CHANGED
|
@@ -1,14 +1,44 @@
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
-
"""Generate
|
| 3 |
|
| 4 |
from __future__ import annotations
|
| 5 |
|
| 6 |
import argparse
|
|
|
|
|
|
|
| 7 |
import json
|
|
|
|
| 8 |
from pathlib import Path
|
|
|
|
| 9 |
|
| 10 |
import torch
|
| 11 |
-
from diffusers import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
|
| 14 |
def _runtime(requested: str) -> tuple[torch.device, torch.dtype]:
|
|
@@ -31,62 +61,206 @@ def _runtime(requested: str) -> tuple[torch.device, torch.dtype]:
|
|
| 31 |
|
| 32 |
def _parser() -> argparse.ArgumentParser:
|
| 33 |
parser = argparse.ArgumentParser(
|
| 34 |
-
description=
|
|
|
|
|
|
|
|
|
|
| 35 |
)
|
| 36 |
parser.add_argument("--model", required=True, help="Hub repository ID or local directory")
|
| 37 |
parser.add_argument("--prompt", required=True)
|
|
|
|
| 38 |
parser.add_argument("--seed", type=int, default=1337)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
parser.add_argument("--output", type=Path, default=Path("clover-image-tiny.png"))
|
| 40 |
parser.add_argument("--device", choices=("auto", "cuda", "mps", "cpu"), default="auto")
|
| 41 |
parser.add_argument("--local-files-only", action="store_true")
|
| 42 |
return parser
|
| 43 |
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
def main() -> int:
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
device, dtype = _runtime(args.device)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
pipe = DiffusionPipeline.from_pretrained(
|
| 49 |
args.model,
|
| 50 |
torch_dtype=dtype,
|
| 51 |
local_files_only=args.local_files_only,
|
| 52 |
)
|
| 53 |
-
|
|
|
|
| 54 |
pipe = pipe.to(device)
|
| 55 |
generator_device = "cuda" if device.type == "cuda" else "cpu"
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
| 57 |
with torch.inference_mode():
|
| 58 |
result = pipe(
|
| 59 |
-
prompt=args.prompt,
|
| 60 |
-
negative_prompt=
|
| 61 |
-
num_inference_steps=
|
| 62 |
-
guidance_scale=
|
| 63 |
-
height=
|
| 64 |
-
width=
|
|
|
|
| 65 |
generator=generator,
|
| 66 |
)
|
| 67 |
-
if not result.images:
|
| 68 |
-
raise RuntimeError("the pipeline returned
|
| 69 |
-
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
metadata = {
|
| 72 |
"model": args.model,
|
| 73 |
-
"prompt": args.prompt,
|
| 74 |
-
"negative_prompt":
|
| 75 |
"seed": args.seed,
|
| 76 |
"scheduler": type(pipe.scheduler).__name__,
|
| 77 |
-
"
|
| 78 |
-
"
|
| 79 |
-
"
|
| 80 |
-
"
|
|
|
|
|
|
|
|
|
|
| 81 |
"device": device.type,
|
| 82 |
"dtype": str(dtype).removeprefix("torch."),
|
| 83 |
-
"nsfw_content_detected":
|
|
|
|
|
|
|
| 84 |
"cross_device_pixel_identity_claimed": False,
|
| 85 |
"usage_label": "RESEARCH PREVIEW - PRODUCTION PILOT - NOT RELEASE-READY",
|
| 86 |
}
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
return 0
|
| 91 |
|
| 92 |
|
|
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
+
"""Generate configurable conventional Clover Image Tiny preview images."""
|
| 3 |
|
| 4 |
from __future__ import annotations
|
| 5 |
|
| 6 |
import argparse
|
| 7 |
+
import hashlib
|
| 8 |
+
import io
|
| 9 |
import json
|
| 10 |
+
import math
|
| 11 |
from pathlib import Path
|
| 12 |
+
from typing import Any
|
| 13 |
|
| 14 |
import torch
|
| 15 |
+
from diffusers import (
|
| 16 |
+
DDIMScheduler,
|
| 17 |
+
DiffusionPipeline,
|
| 18 |
+
DPMSolverMultistepScheduler,
|
| 19 |
+
EulerAncestralDiscreteScheduler,
|
| 20 |
+
EulerDiscreteScheduler,
|
| 21 |
+
PNDMScheduler,
|
| 22 |
+
)
|
| 23 |
+
|
| 24 |
+
DEFAULT_STEPS = 50
|
| 25 |
+
DEFAULT_GUIDANCE_SCALE = 7.5
|
| 26 |
+
DEFAULT_RESOLUTION = 512
|
| 27 |
+
MIN_STEPS = 4
|
| 28 |
+
MAX_STEPS = 100
|
| 29 |
+
MIN_RESOLUTION = 256
|
| 30 |
+
MAX_RESOLUTION = 768
|
| 31 |
+
RESOLUTION_MULTIPLE = 64
|
| 32 |
+
MAX_IMAGES = 4
|
| 33 |
+
MAX_TEXT_LENGTH = 2_000
|
| 34 |
+
MAX_SEED = 2**63 - 1
|
| 35 |
+
SCHEDULERS: dict[str, type[Any]] = {
|
| 36 |
+
"pndm": PNDMScheduler,
|
| 37 |
+
"ddim": DDIMScheduler,
|
| 38 |
+
"euler": EulerDiscreteScheduler,
|
| 39 |
+
"euler-a": EulerAncestralDiscreteScheduler,
|
| 40 |
+
"dpmpp-2m": DPMSolverMultistepScheduler,
|
| 41 |
+
}
|
| 42 |
|
| 43 |
|
| 44 |
def _runtime(requested: str) -> tuple[torch.device, torch.dtype]:
|
|
|
|
| 61 |
|
| 62 |
def _parser() -> argparse.ArgumentParser:
|
| 63 |
parser = argparse.ArgumentParser(
|
| 64 |
+
description=(
|
| 65 |
+
"Run Clover Image Tiny with configurable conventional Diffusers settings. "
|
| 66 |
+
"The defaults reproduce the validated 50-step preview recipe."
|
| 67 |
+
)
|
| 68 |
)
|
| 69 |
parser.add_argument("--model", required=True, help="Hub repository ID or local directory")
|
| 70 |
parser.add_argument("--prompt", required=True)
|
| 71 |
+
parser.add_argument("--negative-prompt", default="")
|
| 72 |
parser.add_argument("--seed", type=int, default=1337)
|
| 73 |
+
parser.add_argument(
|
| 74 |
+
"--steps",
|
| 75 |
+
type=int,
|
| 76 |
+
default=DEFAULT_STEPS,
|
| 77 |
+
help=f"requested inference steps ({MIN_STEPS}..{MAX_STEPS})",
|
| 78 |
+
)
|
| 79 |
+
parser.add_argument(
|
| 80 |
+
"--guidance-scale",
|
| 81 |
+
type=float,
|
| 82 |
+
default=DEFAULT_GUIDANCE_SCALE,
|
| 83 |
+
help="classifier-free guidance scale (0.0..20.0)",
|
| 84 |
+
)
|
| 85 |
+
parser.add_argument(
|
| 86 |
+
"--width",
|
| 87 |
+
type=int,
|
| 88 |
+
default=DEFAULT_RESOLUTION,
|
| 89 |
+
help=(
|
| 90 |
+
f"output width ({MIN_RESOLUTION}..{MAX_RESOLUTION}, multiple of {RESOLUTION_MULTIPLE})"
|
| 91 |
+
),
|
| 92 |
+
)
|
| 93 |
+
parser.add_argument(
|
| 94 |
+
"--height",
|
| 95 |
+
type=int,
|
| 96 |
+
default=DEFAULT_RESOLUTION,
|
| 97 |
+
help=(
|
| 98 |
+
f"output height ({MIN_RESOLUTION}..{MAX_RESOLUTION}, multiple of {RESOLUTION_MULTIPLE})"
|
| 99 |
+
),
|
| 100 |
+
)
|
| 101 |
+
parser.add_argument(
|
| 102 |
+
"--scheduler",
|
| 103 |
+
choices=tuple(SCHEDULERS),
|
| 104 |
+
default="pndm",
|
| 105 |
+
)
|
| 106 |
+
parser.add_argument(
|
| 107 |
+
"--num-images",
|
| 108 |
+
type=int,
|
| 109 |
+
default=1,
|
| 110 |
+
help=f"number of images (1..{MAX_IMAGES}); seeds increment from --seed",
|
| 111 |
+
)
|
| 112 |
parser.add_argument("--output", type=Path, default=Path("clover-image-tiny.png"))
|
| 113 |
parser.add_argument("--device", choices=("auto", "cuda", "mps", "cpu"), default="auto")
|
| 114 |
parser.add_argument("--local-files-only", action="store_true")
|
| 115 |
return parser
|
| 116 |
|
| 117 |
|
| 118 |
+
def _validate_args(args: argparse.Namespace) -> None:
|
| 119 |
+
if not isinstance(args.prompt, str) or not args.prompt.strip():
|
| 120 |
+
raise ValueError("--prompt must not be empty")
|
| 121 |
+
if len(args.prompt) > MAX_TEXT_LENGTH:
|
| 122 |
+
raise ValueError(f"--prompt must contain at most {MAX_TEXT_LENGTH} characters")
|
| 123 |
+
if len(args.negative_prompt) > MAX_TEXT_LENGTH:
|
| 124 |
+
raise ValueError(f"--negative-prompt must contain at most {MAX_TEXT_LENGTH} characters")
|
| 125 |
+
if not MIN_STEPS <= args.steps <= MAX_STEPS:
|
| 126 |
+
raise ValueError(f"--steps must be between {MIN_STEPS} and {MAX_STEPS}")
|
| 127 |
+
if not math.isfinite(args.guidance_scale) or not 0.0 <= args.guidance_scale <= 20.0:
|
| 128 |
+
raise ValueError("--guidance-scale must be finite and between 0.0 and 20.0")
|
| 129 |
+
for name in ("width", "height"):
|
| 130 |
+
value = int(getattr(args, name))
|
| 131 |
+
if not MIN_RESOLUTION <= value <= MAX_RESOLUTION or value % RESOLUTION_MULTIPLE != 0:
|
| 132 |
+
raise ValueError(
|
| 133 |
+
f"--{name} must be between {MIN_RESOLUTION} and {MAX_RESOLUTION} "
|
| 134 |
+
f"and divisible by {RESOLUTION_MULTIPLE}"
|
| 135 |
+
)
|
| 136 |
+
if not 1 <= args.num_images <= MAX_IMAGES:
|
| 137 |
+
raise ValueError(f"--num-images must be between 1 and {MAX_IMAGES}")
|
| 138 |
+
if not 0 <= args.seed <= MAX_SEED or args.seed + args.num_images - 1 > MAX_SEED:
|
| 139 |
+
raise ValueError("--seed and all incremented image seeds must be in [0, 2**63)")
|
| 140 |
+
if args.output.suffix.lower() != ".png":
|
| 141 |
+
raise ValueError("--output must use a .png filename")
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def _output_paths(output: Path, count: int) -> tuple[list[Path], Path]:
|
| 145 |
+
images = [output]
|
| 146 |
+
images.extend(
|
| 147 |
+
output.with_name(f"{output.stem}-{index:02d}{output.suffix}")
|
| 148 |
+
for index in range(2, count + 1)
|
| 149 |
+
)
|
| 150 |
+
return images, output.with_suffix(output.suffix + ".json")
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
def _write_new(path: Path, payload: bytes) -> None:
|
| 154 |
+
with path.open("xb") as handle:
|
| 155 |
+
handle.write(payload)
|
| 156 |
+
|
| 157 |
+
|
| 158 |
def main() -> int:
|
| 159 |
+
parser = _parser()
|
| 160 |
+
args = parser.parse_args()
|
| 161 |
+
try:
|
| 162 |
+
_validate_args(args)
|
| 163 |
+
except ValueError as exc:
|
| 164 |
+
parser.error(str(exc))
|
| 165 |
device, dtype = _runtime(args.device)
|
| 166 |
+
output_paths, metadata_path = _output_paths(args.output, args.num_images)
|
| 167 |
+
args.output.parent.mkdir(parents=True, exist_ok=True)
|
| 168 |
+
for path in (*output_paths, metadata_path):
|
| 169 |
+
if path.exists() or path.is_symlink():
|
| 170 |
+
raise FileExistsError(f"planned output already exists: {path}")
|
| 171 |
pipe = DiffusionPipeline.from_pretrained(
|
| 172 |
args.model,
|
| 173 |
torch_dtype=dtype,
|
| 174 |
local_files_only=args.local_files_only,
|
| 175 |
)
|
| 176 |
+
scheduler_type = SCHEDULERS[args.scheduler]
|
| 177 |
+
pipe.scheduler = scheduler_type.from_config(pipe.scheduler.config)
|
| 178 |
pipe = pipe.to(device)
|
| 179 |
generator_device = "cuda" if device.type == "cuda" else "cpu"
|
| 180 |
+
seeds = [args.seed + index for index in range(args.num_images)]
|
| 181 |
+
generators = [torch.Generator(device=generator_device).manual_seed(seed) for seed in seeds]
|
| 182 |
+
generator: torch.Generator | list[torch.Generator]
|
| 183 |
+
generator = generators[0] if len(generators) == 1 else generators
|
| 184 |
with torch.inference_mode():
|
| 185 |
result = pipe(
|
| 186 |
+
prompt=args.prompt.strip(),
|
| 187 |
+
negative_prompt=args.negative_prompt,
|
| 188 |
+
num_inference_steps=args.steps,
|
| 189 |
+
guidance_scale=args.guidance_scale,
|
| 190 |
+
height=args.height,
|
| 191 |
+
width=args.width,
|
| 192 |
+
num_images_per_prompt=args.num_images,
|
| 193 |
generator=generator,
|
| 194 |
)
|
| 195 |
+
if not isinstance(result.images, list) or len(result.images) != args.num_images:
|
| 196 |
+
raise RuntimeError("the pipeline returned an unexpected number of images")
|
| 197 |
+
safety = getattr(result, "nsfw_content_detected", None)
|
| 198 |
+
if not isinstance(safety, list) or len(safety) != args.num_images:
|
| 199 |
+
raise RuntimeError("the pipeline returned an unexpected safety-check result")
|
| 200 |
+
image_records: list[dict[str, Any]] = []
|
| 201 |
+
image_payloads: list[bytes] = []
|
| 202 |
+
for image, output_path, seed, nsfw in zip(
|
| 203 |
+
result.images,
|
| 204 |
+
output_paths,
|
| 205 |
+
seeds,
|
| 206 |
+
safety,
|
| 207 |
+
strict=True,
|
| 208 |
+
):
|
| 209 |
+
buffer = io.BytesIO()
|
| 210 |
+
image.save(buffer, format="PNG")
|
| 211 |
+
payload = buffer.getvalue()
|
| 212 |
+
image_payloads.append(payload)
|
| 213 |
+
image_records.append(
|
| 214 |
+
{
|
| 215 |
+
"filename": output_path.name,
|
| 216 |
+
"seed": seed,
|
| 217 |
+
"sha256": hashlib.sha256(payload).hexdigest(),
|
| 218 |
+
"nsfw_content_detected": nsfw,
|
| 219 |
+
}
|
| 220 |
+
)
|
| 221 |
+
validated_gallery_recipe = (
|
| 222 |
+
args.scheduler == "pndm"
|
| 223 |
+
and args.steps == DEFAULT_STEPS
|
| 224 |
+
and args.guidance_scale == DEFAULT_GUIDANCE_SCALE
|
| 225 |
+
and args.negative_prompt == ""
|
| 226 |
+
and args.width == DEFAULT_RESOLUTION
|
| 227 |
+
and args.height == DEFAULT_RESOLUTION
|
| 228 |
+
and args.num_images == 1
|
| 229 |
+
)
|
| 230 |
metadata = {
|
| 231 |
"model": args.model,
|
| 232 |
+
"prompt": args.prompt.strip(),
|
| 233 |
+
"negative_prompt": args.negative_prompt,
|
| 234 |
"seed": args.seed,
|
| 235 |
"scheduler": type(pipe.scheduler).__name__,
|
| 236 |
+
"scheduler_key": args.scheduler,
|
| 237 |
+
"num_inference_steps": args.steps,
|
| 238 |
+
"guidance_scale": args.guidance_scale,
|
| 239 |
+
"height": args.height,
|
| 240 |
+
"width": args.width,
|
| 241 |
+
"num_images": args.num_images,
|
| 242 |
+
"images": image_records,
|
| 243 |
"device": device.type,
|
| 244 |
"dtype": str(dtype).removeprefix("torch."),
|
| 245 |
+
"nsfw_content_detected": safety,
|
| 246 |
+
"validated_gallery_recipe": validated_gallery_recipe,
|
| 247 |
+
"leaf_steps": None,
|
| 248 |
"cross_device_pixel_identity_claimed": False,
|
| 249 |
"usage_label": "RESEARCH PREVIEW - PRODUCTION PILOT - NOT RELEASE-READY",
|
| 250 |
}
|
| 251 |
+
for output_path, payload in zip(output_paths, image_payloads, strict=True):
|
| 252 |
+
_write_new(output_path, payload)
|
| 253 |
+
_write_new(metadata_path, (json.dumps(metadata, indent=2) + "\n").encode("utf-8"))
|
| 254 |
+
print(
|
| 255 |
+
json.dumps(
|
| 256 |
+
{
|
| 257 |
+
"output": str(output_paths[0]),
|
| 258 |
+
"outputs": [str(path) for path in output_paths],
|
| 259 |
+
"metadata": str(metadata_path),
|
| 260 |
+
**metadata,
|
| 261 |
+
}
|
| 262 |
+
)
|
| 263 |
+
)
|
| 264 |
return 0
|
| 265 |
|
| 266 |
|
research_preview.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"build": {
|
| 3 |
-
"repository_commit": "
|
| 4 |
"sources": {
|
| 5 |
"LICENSE-CODE": {
|
| 6 |
"sha256": "e26a0375b714267325c27905ba947fdc41ebc0b7a36940a49ae12379ed6208d6",
|
|
@@ -27,11 +27,11 @@
|
|
| 27 |
"tracked_at_commit": true
|
| 28 |
},
|
| 29 |
"hf/stage_b_preview/README.template.md": {
|
| 30 |
-
"sha256": "
|
| 31 |
"tracked_at_commit": true
|
| 32 |
},
|
| 33 |
"hf/stage_b_preview/examples/generate.py": {
|
| 34 |
-
"sha256": "
|
| 35 |
"tracked_at_commit": true
|
| 36 |
},
|
| 37 |
"hf/stage_b_preview/requirements.txt": {
|
|
@@ -116,7 +116,7 @@
|
|
| 116 |
"human_review": "not_measured",
|
| 117 |
"quality_evaluation": "not_measured"
|
| 118 |
},
|
| 119 |
-
"metadata_sha256": "
|
| 120 |
"release_ready": false,
|
| 121 |
"runtime_contract": {
|
| 122 |
"denoiser_calls": 51,
|
|
|
|
| 1 |
{
|
| 2 |
"build": {
|
| 3 |
+
"repository_commit": "647bac81f3713a0238eb21abf1844a8cd8d48f11",
|
| 4 |
"sources": {
|
| 5 |
"LICENSE-CODE": {
|
| 6 |
"sha256": "e26a0375b714267325c27905ba947fdc41ebc0b7a36940a49ae12379ed6208d6",
|
|
|
|
| 27 |
"tracked_at_commit": true
|
| 28 |
},
|
| 29 |
"hf/stage_b_preview/README.template.md": {
|
| 30 |
+
"sha256": "1a38562b255d984609eb38764a5c0f46fdcaaede9abd6c3dc37fa07234529938",
|
| 31 |
"tracked_at_commit": true
|
| 32 |
},
|
| 33 |
"hf/stage_b_preview/examples/generate.py": {
|
| 34 |
+
"sha256": "fcea598655f7e144bc1ccce9be6d4e9e166915a3ad3c26fc0aa6c8ade00b49f1",
|
| 35 |
"tracked_at_commit": true
|
| 36 |
},
|
| 37 |
"hf/stage_b_preview/requirements.txt": {
|
|
|
|
| 116 |
"human_review": "not_measured",
|
| 117 |
"quality_evaluation": "not_measured"
|
| 118 |
},
|
| 119 |
+
"metadata_sha256": "3c4ef00fbdff6bf23dc24feadd71965b520920bef4e61f9cf2f3632c756334c0",
|
| 120 |
"release_ready": false,
|
| 121 |
"runtime_contract": {
|
| 122 |
"denoiser_calls": 51,
|