Cyberspace Presales Co-Pilot — Llama 3.2 3B LoRA Adapter
LoRA adapter for Cyberspace Limited's Vertical AI Presales Co-Pilot.
Training Summary
- Base model:
meta-llama/Llama-3.2-3B-Instruct - Method: LoRA (QLoRA 4-bit) via Unsloth
- Dataset: 132 Cyberspace proposals (ChatML format, zero template bleed)
- Final loss: 1.6964 (target: 1.0–1.8 ✅)
- Training time: ~2.7 min on A100 80GB
LoRA Config
- Rank: 8
- Alpha: 8
- Dropout: 0.05
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Epochs: 3
- LR: 2e-4 (cosine scheduler)
train_on_responses_only: True
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
model = PeftModel.from_pretrained(base, "JupiterJil/cyberspace-llama32-3b-lora")
Production Artifact
For inference, use the GGUF Q5_K_M version: JupiterJil/cyberspace-llama32-3b-gguf
- Downloads last month
- 2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for JupiterJil/cyberspace-llama32-3b-lora
Base model
meta-llama/Llama-3.2-3B-Instruct