flavianv/qwen4b-reward-pairwise-v1

DeepShopper Reward V1 — pairwise Bradley-Terry outfit reward (Qwen3-4B)

Reward V1: a Qwen3-4B AutoModelForSequenceClassification (scalar value head) warm-started from V0 and trained with margin Bradley-Terry loss on gold-vs-corrupted preference pairs — to discriminate good outfits, fixing V0's gender/structure blindness.

• Data: flavianv/deepshopper-reward-pairs — 93k train pairs, 5 corruption types (gender_flip, item_swap, duplicate_role, count_drop, cross_need); larger margin for gender_flip.
• Held-out ranking accuracy: overall 0.995, gender_flip 0.999 (V0 was flat on gender), duplicate_role 1.0, count_drop 0.998, cross_need 0.993, item_swap 0.987.
• Use: score an outfit (system bundle prompt + need + ### Item i: role\ntitle) → scalar reward (higher = better).
• Note: "good" = resembling observed/sold bundles (purchase-likeness), the intended objective; still a learned proxy for real sales.

Code: https://github.com/clijo/reco-rl (branch outfit_bundle).