Publish validated Clover Image Tiny research preview

629ce55 verified 9 days ago

10.9 kB

library_name: diffusers
pipeline_tag: text-to-image
inference: false
base_model: nota-ai/bk-sdm-tiny-2m
license: creativeml-openrail-m
tags:
  - text-to-image
  - diffusion
  - stable-diffusion
  - knowledge-distillation
  - research-preview

Clover Image Tiny — Research Preview

This is Clover Image Tiny, an early conventional knowledge-distillation checkpoint released as a research preview. It is a production-data pilot, not a release-ready model. It uses the compact BK-SDM-Tiny-2M architecture, a conventional 50-step PNDM schedule, 512×512 output, and classifier-free guidance 7.5.

The checkpoint is useful for inspecting an early, licensed-data quality-refresh experiment. It is not a finished Clover release, a quality benchmark, or a claim of uniformly better output than its starting model.

Exact artifact identity

Field	Value
Repository	`neonforestmist/Clover-Image-Tiny`
Status	RESEARCH PREVIEW · PRODUCTION PILOT · NOT RELEASE-READY
Experiment	`clover-kd-20260712T050925Z-01KXABNHP0`
Training stage	Stage B knowledge distillation
Optimizer step	500
Checkpoint SHA-256	`4a5b99ff18478742528a0d31c97dcee939b166a51be858721d40ad5984110893`
Checkpoint-bundle SHA-256	`384b6515f5f26838aea33ec9a941e06610a20764f0b8637c8b7b0667bfc0d447`
Resolved-config SHA-256	`80cf9395d1f587dc0c1d440d9f5b55c55c20703187998509bb306d19d463f597`
Dataset-manifest SHA-256	`50c1249f1cb0d8d690a9acc451ca10c9432eb5a7f4e26f34acb5462096e72322`
Package bytes	`1671492371`
Package files	`30`
Validated Stage B source-package checksums SHA-256	`d9a28d5fe6f5b675ee1b9db52e6d0493c8d3d357bb824eac590911acbd5c3ebc`
Builder source commit	`e6079eec2b91c5d026bfb461fd6c844a98b888ec`

The checkpoint completed 500 optimizer steps, 4,000 microsteps, and 4,000 sample presentations. All 500 recorded training rows were finite and had a nonzero gradient. Those are training-integrity observations, not image-quality scores.

Example gallery

Each row uses the same prompt and seed. The left column is the pinned BK-SDM-Tiny-2M baseline; only the right column is this Stage B checkpoint. The gallery was generated on an NVIDIA L4 in bfloat16 with 50 PNDM steps, guidance 7.5, an empty negative prompt, and 512×512 output. With the pinned Diffusers 0.39.0 scheduler, those 50 requested PNDM steps use 51 U-Net invocations because PLMS repeats its first retained timestep.

All eight Stage B gallery images were finite, nonblank, nonblack, and returned clear from the packaged upstream safety checker in that measured run. This small engineering subset is not a general safety or quality evaluation. The changes are modest and not uniformly better: the bottle example loses prompt fidelity, hands remain weak, and some details change without clear improvement.

Run locally

This is a conventional PyTorch/Diffusers model, so it can run on macOS, Windows, or Linux. Use Python 3.11 or 3.12 and allow about 2 GB of free disk space for the model itself. Mac MPS is the only local runtime measured for this preview; Windows/Linux CUDA and CPU execution are supported by the runner but their performance is not claimed here.

macOS Apple silicon — MPS

These commands download a complete local copy, install the pinned runtime in an isolated environment, and generate through MPS:

mkdir clover-image-tiny-local
cd clover-image-tiny-local

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"

hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model/requirements.txt

python model/examples/generate.py \
  --model model \
  --device mps \
  --local-files-only \
  --prompt "a compact modern library with arched windows" \
  --seed 1469 \
  --output clover-image-tiny.png

open clover-image-tiny.png

If python3.12 is not installed but Python 3.11 is, use python3.11 instead. After the first download, --local-files-only keeps inference offline. The script writes the resolved runtime settings beside the PNG as clover-image-tiny.png.json. This is the PyTorch/Diffusers model; no Core ML or iPhone package is required or included.

Windows — PowerShell

On Windows, the same runner automatically selects an available NVIDIA CUDA GPU and otherwise falls back to CPU:

mkdir clover-image-tiny-local
cd clover-image-tiny-local

py -3.12 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install "huggingface-hub==0.36.2"

hf download "neonforestmist/Clover-Image-Tiny" --local-dir model
python -m pip install -r model\requirements.txt

python model\examples\generate.py `
  --model model `
  --device auto `
  --local-files-only `
  --prompt "a compact modern library with arched windows" `
  --seed 1469 `
  --output clover-image-tiny.png

Invoke-Item .\clover-image-tiny.png

Use py -3.11 if that is the installed supported Python. Check python -c "import torch; print(torch.cuda.is_available())" after installation; False means --device auto will use CPU. A Windows AMD/DirectML path is not included or validated.

Linux

Use the macOS shell flow with python3.11 or python3.12, and replace --device mps with --device auto. It selects CUDA when PyTorch can see an NVIDIA GPU and otherwise uses CPU.

Python API

The equivalent conventional configuration, with automatic CUDA/MPS/CPU selection, is:

import torch
from diffusers import DiffusionPipeline, PNDMScheduler

model_id = "neonforestmist/Clover-Image-Tiny"
if torch.cuda.is_available():
    device = "cuda"
elif torch.backends.mps.is_available():
    device = "mps"
else:
    device = "cpu"
dtype = torch.float16 if device in {"cuda", "mps"} else torch.float32

pipe = DiffusionPipeline.from_pretrained(model_id, torch_dtype=dtype)
pipe.scheduler = PNDMScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to(device)

generator_device = "cuda" if device == "cuda" else "cpu"
generator = torch.Generator(device=generator_device).manual_seed(1337)
result = pipe(
    prompt="a tiny greenhouse glowing in a moonlit garden",
    negative_prompt="",
    num_inference_steps=50,
    guidance_scale=7.5,
    height=512,
    width=512,
    generator=generator,
)
result.images[0].save("clover-image-tiny.png")

The bundled examples/generate.py chooses CUDA, MPS, or CPU safely and writes the resolved settings next to the PNG:

python examples/generate.py \
  --model "neonforestmist/Clover-Image-Tiny" \
  --prompt "a tiny greenhouse glowing in a moonlit garden" \
  --output clover-image-tiny.png

Seeded generation is repeatable only within the limits of the selected runtime. Different devices, dtypes, kernels, and dependency builds can produce different pixels. The bundled Mac example below was measured locally on an Apple M4 Pro with MPS and fp16: 18.21 seconds, exactly 51 U-Net calls, and 631,341,056 bytes of process-lifetime maximum RSS. It used the prompt “a compact modern library with arched windows” and seed 1469. The packaged safety checker ran and returned clear. This does not promise identical pixels or performance on another Mac.

Its machine-readable evidence is bundled at evidence/clover-image-tiny-local-mps-library-seed-1469.json; the image SHA-256 is f8830346f2a9c2b9a8c2a01d8f90e6925c93d667c1bcf998aa904a150589a742.

Training lineage

Starting student: nota-ai/bk-sdm-tiny-2m@aad3e0e8ba61b7cb9f64869dc4e586f8ad9d3665
Frozen teacher: CompVis/stable-diffusion-v1-4@133a221b8aa7292a167afc5127cb63fb5005638b
Fine-tuning data: exactly 1,000 accepted image-caption pairs from Spawning/PD3M@2a5eb24a8dccf245acd8e56341761aee06da0bdf
Split: 973 train, 17 validation, and 10 test records in one checksummed shard
Data gate: CDLA-Permissive-2.0; accepted items retain CC0-1.0 or Public Domain Mark 1.0 provenance
Preprocessing: deterministic center crop and 512×512 JPEG conversion, version clover-pd3m-center-crop-512-jpeg95-v1

The 1,000 records describe only this Clover fine-tuning run. The starting student and teacher already contain knowledge learned from much larger upstream corpora. Their pinned model cards and weight licenses were verified, but the project does not have complete per-record provenance for all foundational pretraining behind those weights. In particular, the BK-SDM model card names LAION-Aesthetics V2 6.25+ and 2,256,472 pairs without the per-record evidence required by Clover's full release policy. This inherited gap is a material reason the package is labeled a research preview and not release-ready.

See DATA_PROVENANCE.md, MODEL_DATA_LICENSES.md, the bundled CreativeML Open RAIL-M terms, and the package checksums for the portable evidence included here.

Intended use

Research and inspection of compact Stable Diffusion knowledge distillation
Reproducing the fixed conventional 50-step engineering examples
Comparing this early calibration checkpoint with its pinned starting student
Non-consequential creative experimentation subject to the model license

Limitations

The model was fine-tuned for only 500 optimizer steps on 1,000 pairs.
The eight-prompt gallery is an engineering-health subset, not a representative quality, alignment, diversity, bias, or human-preference evaluation.
Results may omit requested objects, lose relationships or counts, produce malformed anatomy and hands, render text poorly, or preserve/amplify biases from upstream models and data.
The Stage B changes are modest and can make individual prompts worse.
The included pipeline is the conventional PyTorch/Diffusers path only.

Safety and out-of-scope use

The upstream safety checker is included and was exercised in the recorded gallery run, but it is not a complete safety system and can miss harmful output or over-filter benign output. Applications should add appropriate prompt and output controls, human review, and policy enforcement.

Do not use this preview for consequential decisions, identity claims, medical or legal conclusions, harassment, exploitation, illegal activity, or any use prohibited by CreativeML OpenRAIL-M. Review outputs before sharing them.

Licenses

The model weights are a derivative under CreativeML OpenRAIL-M. The small example and packaging code is licensed separately under Apache-2.0. The PD3M dataset declaration and each accepted item's public-domain status remain separate from both licenses. Read the bundled license files and MODEL_DATA_LICENSES.md; this summary is not legal advice.