opsd-llama32-3b-ft-nonthink-gen1024-step100-jsdclip1e-6-20260515
This public repository contains LoRA adapter checkpoints from an OPSD run.
Method
- Base model:
meta-llama/Llama-3.2-3B-Instruct - Method: OPSD fixed-teacher, non-thinking mode
- Teacher: fixed base policy with LoRA adapters disabled during teacher forward pass
- Loss: full-vocabulary forward KL/JSD (
beta=0) - Per-token JSD clipping:
1e-06 - Student/teacher thinking flags:
False / False - Dataset:
siyanzhao/Openthoughts_math_30k_opsd - Train budget:
max_steps=100,max_completion_length=1024 - Batch:
per_device_train_batch_size=1,gradient_accumulation_steps=2, effective batch8 - vLLM:
colocate, GPU memory utilization0.35 - GPUs: 4
Only adapter/checkpoint artifacts and logs are uploaded; optimizer states are intentionally omitted.
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for yuxiaoyang/opsd-llama32-3b-ft-nonthink-gen1024-step100-jsdclip1e-6-20260515
Base model
meta-llama/Llama-3.2-3B-Instruct