Add repo card explaining Stage-2 generator adapters

README.md

@@ -0,0 +1,95 @@
+---
+base_model: Qwen/Qwen2.5-VL-3B-Instruct
+library_name: peft
+pipeline_tag: image-text-to-text
+license: other
+language:
+  - en
+tags:
+  - color-grading
+  - lut
+  - cube-lut
+  - image-editing
+  - instruction-following
+  - qlora
+  - lora
+  - peft
+  - qwen2-vl
+  - vlm
+---
+
+# LUT-SLM — Stage-2 Generator Adapters (QLoRA over Qwen2.5-VL-3B)
+
+QLoRA adapters for the **Stage-2 generator** of the LUT-SLM project: a small vision-language model
+that turns *(source image + natural-language photo-editing instruction)* into a single global color
+Look-Up Table (LUT). Given *"make it warmer and lift the shadows"* the model emits the tokens of a
+17³ `.cube` LUT that bakes in exactly that look; given a request a single global LUT physically
+**cannot** satisfy (e.g. *"remove the person on the left"*) it emits `<unsupported>` and refuses.
+
+These adapters are trained on the companion dataset
+**[`ericrcwu/LUT_SLM`](https://huggingface.co/datasets/ericrcwu/LUT_SLM)** (see that card for the full
+data story). The Stage-1 request router lives in
+**[`ericrcwu/LUT_SLM_interpreter`](https://huggingface.co/ericrcwu/LUT_SLM_interpreter)**.
+
+> **Status — research artifacts, work in progress.** These are smoke-scale / bilevel-search run
+> outputs, not a finalized release. Treat them as reproducible checkpoints from the collapse-fix and
+> two-stage experiments.
+
+## What's in this repo
+
+Each subfolder is a self-contained PEFT adapter (adapter weights + tokenizer + chat template +
+`adapter_manifest.json`), **except** `distill_r1_distilled_corpus/`, which holds a distilled data
+corpus rather than weights.
+
+| Subfolder | What it is |
+|---|---|
+| `p6_twostage_d0f9c744_smokefull/` | **Deployed generator.** P6 two-stage run adapter used by the webapp / Modal deploy (`deploy/modal_app.py`). mean train loss ≈ 1.677, 182 steps, lr 2e-4. |
+| `bl_63cd1bf7_smokefull/` | One-stage full-run winner from the bilevel-over-SFT search. mean train loss ≈ 1.747, 162 steps, lr 3e-4. |
+| `bl_a0ccbcff_smokefull/` | Bilevel baseline adapter (full smoke run). |
+| `bl_a0ccbcff_smoke600/` | Bilevel baseline adapter (600-example smoke run). |
+| `distill_r1_smokefull/` | Distillation round-1 adapter. |
+| `distill_r1_distilled_corpus/` | Distilled corpus (`active_rows.jsonl`, `active_manifest.json`, `harvest_cache.jsonl`) — **data, not weights.** |
+
+## Shared architecture & training recipe
+
+All adapters share the same shape (per `adapter_manifest.json`):
+
+- **Base model:** `Qwen/Qwen2.5-VL-3B-Instruct`, with the output embedding **resized to 151,924
+  tokens** — the base vocab plus 259 special LUT tokens (`<lut_bos>`, `<lut_eos>`, `<unsupported>`,
+  `<lut_000>`…`<lut_255>`). Embeddings are **tied**.
+- **LoRA:** `r = 16`, `alpha = 32`, `dropout = 0.05`, targets
+  `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`.
+- **Quantization:** 4-bit QLoRA — `nf4`, double-quant, bf16 compute dtype.
+- **Optim:** effective batch size 32 (per-device 1 × grad-accum 32), cosine schedule, 3% warmup,
+  grad-checkpointing, 2 epochs, seed 0.
+- **Frozen VQ tokenizer:** the 64 LUT code tokens decode via
+  `tokenizer_version = vq_v2_srgbres_17to4_cb256_t64…` (encoder 17³ → 4³ latent → 64 codes over a
+  256-entry codebook; decoder → a residual LUT added to the sRGB identity grid → `.cube`). The
+  tokenizer artifacts themselves ship with the [`LUT_SLM`](https://huggingface.co/datasets/ericrcwu/LUT_SLM)
+  corpus shards.
+
+**Output grammar:** supported → `<lut_bos> <lut_###> ×64 <lut_eos>`; unsupported → `<unsupported>`.
+
+## How to load
+
+The adapter targets a **vocab-resized** base, so you must resize the base embeddings to 151,924 and
+add the 259 special tokens *before* attaching the adapter (the `adapter_config.json` `base_model`
+field points at a local `models/base_resized`, i.e. the resized base — not a Hub repo).
+
+```python
+from huggingface_hub import snapshot_download
+d = snapshot_download("ericrcwu/LUT_SLM_sft_adapters",
+                      allow_patterns=["p6_twostage_d0f9c744_smokefull/*"])
+# 1) load Qwen/Qwen2.5-VL-3B-Instruct, 2) add the 259 special tokens + resize embeddings to 151924,
+# 3) PeftModel.from_pretrained(base, f"{d}/p6_twostage_d0f9c744_smokefull").
+# See notebooks/colab_lut_slm_inference.ipynb in the source repo for a runnable end-to-end example
+# (vocab is reconstructed in memory, LUT codes decoded with the frozen tokenizer, image rendered).
+```
+
+## Licensing & provenance
+
+`license: other`. The base model is governed by its own Qwen license; these adapters are derived from
+the mixed-provenance **[`LUT_SLM`](https://huggingface.co/datasets/ericrcwu/LUT_SLM)** corpus, several
+sources of which are **personal-use / non-redistribution** (see that dataset's licensing section).
+This repository makes **no license claim** over the underlying LUTs or images. Research use;
+verify each source family's original terms before any redistribution or commercial use.