24yearsold/flux2-klein-9b-character-transfer-lora-r128

FLUX.2-klein-9B — Character Transfer LoRA (r=128, gaussian, 6 epochs)

LoRA adapter for black-forest-labs/FLUX.2-klein-9B that performs character-transfer: given a source image (composition, pose) and a reference image (target character/style), generate a new image that re-renders the source's composition with the reference's character.

This is the production model from the see-through-dev character-transfer sweep — best on every val-loss u-bucket and visually preferred on color fidelity, face cleanness, and figure separation.

Quick start (diffusers)

import torch
from diffusers import Flux2KleinPipeline

pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-9B",
    torch_dtype=torch.bfloat16,
).to("cuda")

# Load this LoRA
pipe.load_lora_weights(
    "24yearsold/flux2-klein-9b-character-transfer-lora-r128",
    weight_name="pytorch_lora_weights.safetensors",
)

# Build the conditioning image: source (960x544, left) + reference (544x544, right)
# stitched horizontally to 1504x544. See the input-stitching section below.
from PIL import Image
cond_image = build_stitched_conditioning(source_path, reference_path)  # 1504x544

prompt = open("character_transfer_prompt.txt", encoding="utf-8").read()  # ~300 CJK chars
out = pipe(
    image=cond_image,
    prompt=prompt,
    num_inference_steps=28,
    guidance_scale=1.0,
).images[0]
# out is 960x544 — the generated source half
out.save("transferred.png")

You can also adjust LoRA strength on the fly:

pipe.set_adapters("default", adapter_weights=0.85)  # any value 0.0..1.0+

Input stitching (how the conditioning image is built)

The model was trained on a horizontally-stitched conditioning image:

+--------------------+----------+
|     source 960x544 | ref 544² |    -> 1504 × 544 RGB
|  (center-cropped)  |  (white- |
|                    |  padded) |
+--------------------+----------+

Reference is PAD-to-square with white fill (composites RGBA over white if the reference has alpha), then resized to 544×544. Source is center-cropped (or letterboxed) to 960×544. Concatenate left-to-right.

Output: model generates a 960×544 image — the regenerated source half. (The output has the source's composition with the reference's character identity transferred onto it.)

Usage in ComfyUI

Works directly with current ComfyUI (≥ v0.10.0, released 2026-01-20).

The diffusers fused-projection key naming this LoRA uses (to_qkv_mlp_proj on single-stream blocks, to_q/k/v/out.0 on dual-stream blocks) is mapped to ComfyUI's internal Flux2 linear1/linear2/etc. by the converter in comfy/utils.py:flux_to_diffusers after PR #11981.

Workflow:

1. Base model: load FLUX.2-klein-9B (downloaded automatically by recent ComfyUI versions, or place the safetensors in models/checkpoints/).
2. Load LoRA node: place this pytorch_lora_weights.safetensors in models/loras/ and select it. Strength: trained at α=1.0 (lora_alpha=lora_r=128); recommended ComfyUI strength 0.8–1.0.
3. Conditioning image: build the 1504×544 stitched image (source 960×544 left + reference 544×544 right) outside ComfyUI or with Image Resize
   • Image Pad + Image Concat nodes, then feed it into the standard FLUX.2-klein image-to-image graph.
4. Output: the model generates 960×544; crop/keep the left half as the final result if needed.

If your ComfyUI is older than v0.10.0 (Jan 2026) you may see key-mismatch warnings — update ComfyUI or apply PR #11981 manually to comfy/utils.py.

Training recipe

Base	black-forest-labs/FLUX.2-klein-9B
Rank	128 (alpha=128)
Init	gaussian (NOT PiSSA)
Target modules	to_k/q/v/out.0 + to_qkv_mlp_proj + single_transformer_blocks.{0..23}.attn.to_out
Trainable params	~106 M
Schedule	cosine to 0.25× peak, 4056 steps over 6 epochs
LR	2e-4 (warmup 50, num_cycles=0.3333)
Effective batch	4 (2 GPU × bsz 2)
Precision	bf16
Weighting	none (NOT logit_normal)
Hardware	2× RTX 4090 48 GB
Wall clock	~10 hours
Prompt	Single ~300-char Chinese prompt, padded to 512 tokens via Qwen3 tokenizer

Held-out val_loss (per-σ MSE, 8 triplets)

u	mean MSE
0.10 (texture)	0.3236
0.25	0.2276
0.50	0.2476
0.75	0.3563
0.90 (struct)	0.5419
mean	0.3394

Best of the full sweep (P5–P12 across r∈{32,64,128} × init∈{gauss,PiSSA} × lr∈{1e-4,2e-4,3e-4} × epochs∈{3,6}). Trajectory was monotonically descending at endpoint — could likely improve with 8 epochs.

Citation / context

Trained by the see-through-dev project as part of the character-transfer LoRA stack. Other production candidates from the same sweep:

• 24yearsold/flux2-klein-9b-character-transfer-pissa-r64 — earlier PiSSA r=64 baseline (full merged base + LoRA). Superseded by this LoRA.

License

Apache 2.0 (matches base model). The base model black-forest-labs/FLUX.2-klein-9B has its own license — please follow theirs when using.