| --- |
| library_name: peft |
| base_model: Qwen/Qwen2.5-7B-Instruct |
| tags: |
| - lora |
| - peft |
| - unsloth |
| - qwen |
| - qwen-2.5 |
| - cyberspace |
| - presales |
| --- |
| |
| # Cyberspace Presales Co-Pilot — Qwen 2.5 7B LoRA Adapter |
|
|
| LoRA adapter for Cyberspace Limited's Vertical AI Presales Co-Pilot. |
|
|
| ## Training Summary |
| - **Base model:** `Qwen/Qwen2.5-7B-Instruct` |
| - **Method:** LoRA (QLoRA 4-bit) via Unsloth |
| - **Dataset:** 132 Cyberspace proposals (ChatML format, zero template bleed) |
| - **Final loss:** 1.4795 (target: 1.2–1.6 ✅) |
| - **Training time:** ~4.9 min on A100 80GB |
|
|
| ## LoRA Config |
| - Rank: 16 |
| - Alpha: 16 |
| - Dropout: 0.05 |
| - Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| - Epochs: 3 |
| - LR: 2e-4 (cosine scheduler) |
| - `train_on_responses_only`: True (ChatML markers) |
|
|
| ## Usage |
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct") |
| model = PeftModel.from_pretrained(base, "JupiterJil/cyberspace-qwen25-7b-lora") |
| ``` |
|
|
| ## Recommended Inference Settings |
| - Temperature: 0.4 |
| - Top-p: 0.9 |
| - Repeat penalty: 1.1 |
| - Use detailed category-specific system prompts |
|
|
| ## Production Artifact |
| For inference, use the GGUF Q4_K_M version: [JupiterJil/cyberspace-qwen25-7b-gguf](https://huggingface.co/JupiterJil/cyberspace-qwen25-7b-gguf) |
|
|
