hamzi275/pk-fashion-lora-sd15

Pakistani Fashion LoRA — Stable Diffusion v1.5

LoRA fine-tuned from Stable Diffusion v1.5, specialized for generating images of traditional Pakistani garments, particularly women's shalwar kameez.

Scope

This is a garment image generation LoRA, not a virtual try-on model. It generates clothing images from text prompts and does not combine a person image with a garment image. Virtual try-on architectures such as IDM-VTON were explored but were not used due to their high computational requirements on free-tier Google Colab GPUs.

Training Details

Parameter	Value
Base Model	runwayml/stable-diffusion-v1-5
Method	LoRA (PEFT), Rank=4, Alpha=4
Target Modules	UNet Attention (to_q, to_k, to_v, to_out.0)
Resolution	512 × 512 (Pad-to-Square preprocessing)
Training Steps	1500
Checkpoint	lora_step_1500.safetensors
Optimizer	AdamW
Learning Rate	1e-4
Caption Strategy	Class-based prompts

Dataset

The model was trained on a custom-curated Pakistani fashion dataset containing traditional women's garments.

The preprocessing pipeline included:

• Removal of corrupt images
• Duplicate detection using perceptual hashing
• Aspect ratio inspection
• Pad-to-square resizing
• Image quality verification

Usage

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

pipe.unet.load_attn_procs("hamzi275/pk-fashion-lora-sd15")

image = pipe(
    "a photo of a woman wearing traditional Pakistani shalwar kameez"
).images[0]

image.save("output.png")

Example Prompt

a photo of a woman wearing traditional Pakistani shalwar kameez

Limitations

• Not a virtual try-on model
• No person-image conditioning
• Uses class-level captions instead of detailed per-image captions
• No quantitative evaluation metrics (FID, LPIPS, SSIM) have been computed yet

Future Work

• Add person-image and garment-image conditioning for virtual try-on
• Improve caption quality using per-image annotations
• Evaluate using FID, LPIPS, and CLIP Score
• Expand the dataset with additional Pakistani clothing styles

Citation

If you use this model in your research or project, please cite this repository.