--- license: apache-2.0 base_model: - amalia-llm/AMALIA-9B-0626-SFT base_model_relation: quantized library_name: llama.cpp pipeline_tag: text-generation tags: - gguf - quantized - llama-cpp - imatrix - llama - portuguese - european-portuguese - kld-measured - conversational - endpoints_compatible language: - pt --- ![banner](banner.png)

KIKOCIS // EUROPEAN-PORTUGUESE LLM // IMATRIX GGUF + KLD
   ╔═══════════════╗
   ║  A M A L I A  ║   pt-PT
   ╚═══════════════╝
   ┌──┐ ┌──┐ ┌──┐ ┌──┐
   │Q3│ │Q4│ │Q5│ │Q8│  ← ladder
   └──┘ └──┘ └──┘ └──┘
    ●────●────●────●
      KLD vs F16
AMALIA-9B · GGUF
llama · 9B · 32K ctx · imatrix (pt) · KLD-measured

    
      FORMAT
GGUF (imatrix)

      SIZES
3.1 – 9.1 GB

      ARCH
Llama · 42L

      CONTEXT
32768

    

      IMATRIX
pt-PT corpus

      VALIDATION
KLD vs F16

      LANGUAGE
Português (EU)

      LICENSE
Apache-2.0

    

# AMALIA-9B-0626-SFT — GGUF (imatrix + KLD) > imatrix-quantized GGUFs of **AMALIA-9B**, an open **European-Portuguese** instruction model — one of the first open LLMs developed in **Portugal**. Runs from **~3.1 GB** (IQ2_M) to **~9.1 GB** (Q8_0). Calibrated (imatrix) **on Portuguese text**, and every quant is measured against the full-precision model with **KLD** so you can pick a size with the fidelity cost in front of you. Credit: this is **the AMALIA-LLM project's** model — [`amalia-llm/AMALIA-9B-0626-SFT`](https://huggingface.co/amalia-llm/AMALIA-9B-0626-SFT); ours is the quant ladder + metrics. ## ✅ Recommended files | Use case | File | Notes | |---|---|---| | ⭐ Best value | `amalia-9b-Q5_K_M.gguf` | The pick — **lower KLD than Q6_K** at 1 GB less (0.051 vs 0.078). ~6.1 GB. | | Safe default | `amalia-9b-Q4_K_M.gguf` | Standard K-quant, ~5.2 GB; noticeably less faithful than Q5 (KLD 0.22 vs 0.05). | | Max fidelity | `amalia-9b-Q8_0.gguf` | Lowest KLD (0.006) — near-lossless. ~9.1 GB. | | Tightest RAM | `amalia-9b-Q3_K_M.gguf` | ~4.2 GB — runs on 8 GB machines, still coherent. | | Smallest (aggressive) | `amalia-9b-IQ2_M.gguf` | ~3.1 GB — fits very tight RAM, at a real quality cost (top-1 61%). | ## 📦 Files (the full ladder) | Quant | Bits | File size | Notes | |---|---|---:|---| | IQ2_M | ~2.7 | ~3.1 GB | Smallest — aggressive i-quant, real quality trade-off. | | Q3_K_M | 3 | ~4.2 GB | Smallest *comfortable* quant. | | IQ4_XS | ~4.3 | ~4.7 GB | Compact i-quant (imatrix). | | Q4_K_M | 4 | ~5.2 GB | Standard K-quant. | | Q5_K_M | 5 | ~6.1 GB | ⭐ Best quality/size. | | Q6_K | 6 | ~7.0 GB | Near-lossless. | | Q8_0 | 8 | ~9.1 GB | Highest precision. | ## 📊 Metrics — objective quality vs the F16 reference **KLD** (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the full-precision model — the gold-standard fidelity metric (lower = closer). **Top-1 match** = how often the quant's top token agrees with F16. Measured with `llama-perplexity --kl-divergence` over a **Portuguese** corpus at **ctx 2048**, vs the F16 GGUF. | Model | Size GB | PPL | ΔPPL vs F16 | KLD mean | KLD p95 | Top-1 match | |---|---:|---:|---:|---:|---:|---:| | **F16 (reference)** | 17.1 | 20.04 | 0.000 | 0.0000 | 0.0000 | 100.0% | | Q8_0 | 9.06 | 20.20 | +0.177 | 0.0057 | 0.0198 | 96.38% | | Q6_K | 7.00 | 19.93 | -0.091 | 0.0779 | 0.1740 | 89.06% | | Q5_K_M | 6.08 | 19.80 | -0.227 | 0.0511 | 0.1747 | 88.89% | | Q4_K_M | 5.20 | 20.23 | +0.203 | 0.2230 | 0.7429 | 81.39% | | IQ4_XS | 4.70 | 20.78 | +0.757 | 0.2173 | 0.7600 | 80.55% | | Q3_K_M | 4.25 | 22.24 | +2.215 | 0.3258 | 1.1486 | 73.58% | | IQ2_M | 3.11 | 21.99 | +1.968 | 0.8211 | 2.9052 | 60.69% | _{Full per-quant reports in [`reports/`](reports); machine-readable summary in [`metrics/quant-summary.csv`](metrics/quant-summary.csv); SHA-256 of every file in [`reports/artifact-sha256sums.txt`](reports/artifact-sha256sums.txt). Note: this is an **instruction-tuned** model measured on **raw** text, so the absolute PPL is higher than a base model's — what matters here is the **relative** KLD/top-1 across quants.} > **Why does Q5_K_M beat Q6_K on KLD here?** It's real and reproducible for this model: Q5_K_M's imatrix-weighted bit allocation is very effective, so it lands closer to F16 on mean-KLD than Q6_K (which stays marginally ahead on top-1). Both are near-lossless — so **Q5_K_M is the better value**, and Q8_0 is the true fidelity ceiling. > **A note on i-quants:** we tested the i-quants too. For *this* model the K-quants win at low bit — e.g. IQ3_M came out *worse* than Q3_K_M on KLD, so it isn't shipped. **IQ2_M is included** as the smallest option for very tight RAM, but with an honest quality cost (top-1 61%); if you can spare the space, Q3_K_M is a big step up. ### 📈 Charts ![kld vs size](metrics/chart-kld-vs-size.png) ![top-1 match](metrics/chart-top1.png) ## 🧮 Will it fit? (RAM/VRAM cheat-sheet) Memory ≈ weights + KV-cache (KV grows with context). Rough guide: | you have | comfortable quant | context | |---|---|---| | 6 GB | IQ2_M / Q3_K_M | ~8–16K | | 8 GB | Q3_K_M / Q4_K_M | ~16–32K | | 12 GB | Q5_K_M / Q6_K | ~32K | | 16 GB+ | Q6_K / Q8_0 | ~32K (native) | ## 🚀 How to run it ```bash # ollama (pulls straight from this repo) ollama run hf.co/KikoCis/AMALIA-9B-0626-SFT-GGUF:Q5_K_M # llama.cpp llama-server -m amalia-9b-Q5_K_M.gguf -c 32768 --jinja llama-cli -m amalia-9b-Q5_K_M.gguf -p "Explica em português o que é a saudade." -ngl 99 ``` **Recommended sampling**: temperature ~0.7, top_p ~0.9. It's a **chat/instruct** model (uses its built-in chat template) — prompt it in **European Portuguese** for best results. ## ⚠️ Good to know - **Strengths**: European-Portuguese fluency and instruction-following; one of the first open LLMs built in Portugal. - **Limits**: specialised for Portuguese — not a coding/agentic model, and not tuned for English. Low-bit quants (IQ2/Q3) trade real fidelity for size (see KLD). - Absolute perplexity looks high only because it's measured on raw text with an instruct model; the KLD table is the fair quant-quality signal. ## 📊 Evaluation methodology (how the numbers were measured) - **What**: quantization **fidelity** vs the F16 GGUF — not a task benchmark. `llama-perplexity --kl-divergence` (KLD mean/p95/max, ΔPPL, top-1 agreement). - **Corpus**: European-Portuguese text (Wikipedia + diverse), **ctx 2048**, same corpus family used for the imatrix calibration. - **Reference**: the model's own **F16** GGUF (KLD = 0 by definition). - **Date**: 2026-07. *Caveat: relative fidelity ranking across quants of this model; not comparable across different models/corpora.* ## 🔁 Provenance & reproducibility - **Scripts**: [`scripts/`](scripts) — exact convert → imatrix → quant → KLD commands. - **imatrix**: [`amalia-9b-pt.imatrix`](amalia-9b-pt.imatrix) — importance matrix (Portuguese calibration) used for the IQ/K quants. - **Checksums**: [`reports/artifact-sha256sums.txt`](reports/artifact-sha256sums.txt) — `shasum -a 256 -c` to verify. - **Source**: [`amalia-llm/AMALIA-9B-0626-SFT`](https://huggingface.co/amalia-llm/AMALIA-9B-0626-SFT) — weights **unmodified** (faithful quantization). ## 🗒️ Changelog - 2026-07 v1: imatrix GGUF ladder (IQ2_M → Q8_0) + KLD/PPL metrics, Portuguese-calibrated. ## 📚 Credit & license Model, weights, training data: **© the AMALIA-LLM project** — [model](https://huggingface.co/amalia-llm/AMALIA-9B-0626-SFT) · [website](https://amaliallm.pt/) · [paper (PROPOR 2026)](https://aclanthology.org/2026.propor-1.38/). Quant ladder + imatrix (pt) + KLD/PPL metrics: KikoCis. **Apache-2.0** (same as upstream). No weights modified.

FORMAT GGUF (imatrix)	SIZES 3.1 – 9.1 GB	ARCH Llama · 42L	CONTEXT 32768
IMATRIX pt-PT corpus	VALIDATION KLD vs F16	LANGUAGE Português (EU)	LICENSE Apache-2.0