Elinnos/elinnos-sv-v7-ahb-GGUF

ELINNOS SV-v7-AHB — GGUF Quantized

GGUF quantized versions of Elinnos/elinnos-sv-v7-ahb-merged, the merged ELINNOS SV-v7-AHB model fine-tuned on AMBA AHB hardware design tasks.

Available Quantizations

File	Size	Quant	Description
elinnos-sv-v7-ahb-Q4_K_M.gguf	~4.4 GB	Q4_K_M	Recommended — best quality/size trade-off
elinnos-sv-v7-ahb-Q8_0.gguf	~7.6 GB	Q8_0	Near-lossless, use if VRAM allows
elinnos-sv-v7-ahb-f16.gguf	~15 GB	F16	Full precision reference

Quick Start (Ollama)

ollama run pkelinnos/elinnos-sv-v7-ahb

Quick Start (llama.cpp)

./llama-cli -m elinnos-sv-v7-ahb-Q4_K_M.gguf \
  --ctx-size 8192 \
  --temp 0 \
  -p "Design an AHB-Lite slave with 4 read/write registers."

Quick Start (Python — llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="elinnos-sv-v7-ahb-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,
)
output = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are Elinnos, a hardware design assistant..."},
        {"role": "user",   "content": "Design an AHB bus matrix for 2 masters and 3 slaves."},
    ],
    temperature=0,
    max_tokens=2048,
)
print(output["choices"][0]["message"]["content"])

Model Details

See the full model card at Elinnos/elinnos-sv-v7-ahb.

• Base: Qwen2.5-7B-Instruct
• Adapter chain: v3 → v4 → v5 → v6 → v7 (merged)
• LoRA: r=96, α=192, target: q/k/v/o/gate/up/down proj
• Training: 4 epochs, best eval_loss=0.5408
• Conversion: llama.cpp convert_hf_to_gguf.py + llama-quantize