| ---
|
| library_name: peft
|
| base_model: LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct
|
| tags:
|
| - lora
|
| - qa
|
| - f1-regulations
|
| - led
|
| pipeline_tag: text-generation
|
| ---
|
|
|
| # LED QA LoRA — EXAONE-3.0-7.8B-Instruct
|
|
|
| LoRA adapter fine-tuned on **F1 technical regulations QA**.
|
|
|
| ## Base model
|
|
|
| `LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct`
|
|
|
| ## Training
|
|
|
| - Dataset: `filtered/full` Proposed QA SFT split
|
| - LoRA r=16, alpha=32, target=all linear projections
|
| - Epochs: 3
|
| - Max seq len: 2048
|
|
|
| ## Load
|
|
|
| ```python
|
| from peft import PeftModel
|
| from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
| base = "LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct"
|
| adapter = "REPO_ID"
|
|
|
| tokenizer = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
|
| model = AutoModelForCausalLM.from_pretrained(
|
| base,
|
| device_map="auto",
|
| torch_dtype="auto",
|
| trust_remote_code=True,
|
| )
|
| model = PeftModel.from_pretrained(model, adapter)
|
| ```
|
|
|
