File size: 1,345 Bytes
98388ae | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
library_name: peft
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- lora
- peft
- unsloth
- qwen
- qwen-2.5
- cyberspace
- presales
---
# Cyberspace Presales Co-Pilot — Qwen 2.5 7B LoRA Adapter
LoRA adapter for Cyberspace Limited's Vertical AI Presales Co-Pilot.
## Training Summary
- **Base model:** `Qwen/Qwen2.5-7B-Instruct`
- **Method:** LoRA (QLoRA 4-bit) via Unsloth
- **Dataset:** 132 Cyberspace proposals (ChatML format, zero template bleed)
- **Final loss:** 1.4795 (target: 1.2–1.6 ✅)
- **Training time:** ~4.9 min on A100 80GB
## LoRA Config
- Rank: 16
- Alpha: 16
- Dropout: 0.05
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Epochs: 3
- LR: 2e-4 (cosine scheduler)
- `train_on_responses_only`: True (ChatML markers)
## Usage
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "JupiterJil/cyberspace-qwen25-7b-lora")
```
## Recommended Inference Settings
- Temperature: 0.4
- Top-p: 0.9
- Repeat penalty: 1.1
- Use detailed category-specific system prompts
## Production Artifact
For inference, use the GGUF Q4_K_M version: [JupiterJil/cyberspace-qwen25-7b-gguf](https://huggingface.co/JupiterJil/cyberspace-qwen25-7b-gguf)
|