uam-rl/qwen35-9b-muon-lora-r16

Qwen3.5 9B Typst SFT LoRA

This repository contains a PEFT LoRA adapter trained from Qwen/Qwen3.5-9B. It does not include merged base-model weights.

Contents

• adapter_model.safetensors: LoRA adapter weights
• adapter_config.json: PEFT adapter configuration
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer sidecars from the training run

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen3.5-9B"
adapter = "uam-rl/qwen35-9b-muon-lora-r16"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Training

• Method: supervised fine-tuning with TRL SFTTrainer
• Adapter: LoRA, rank 16, alpha 32, dropout 0.05
• Optimizer: Muon
• Base model: Qwen/Qwen3.5-9B

The adapter was trained for internal evaluation on the Typst Universe scrape.