README.md

metadata

license: apache-2.0
base_model:
  - amalia-llm/AMALIA-9B-0626-SFT
base_model_relation: quantized
library_name: llama.cpp
pipeline_tag: text-generation
tags:
  - gguf
  - quantized
  - llama-cpp
  - imatrix
  - llama
  - portuguese
  - european-portuguese
  - kld-measured
  - conversational
  - endpoints_compatible
language:
  - pt

KIKOCIS // EUROPEAN-PORTUGUESE LLM // IMATRIX GGUF + KLD

   ╔═══════════════╗
   ║  A M A L I A  ║   pt-PT
   ╚═══════════════╝
   ┌──┐ ┌──┐ ┌──┐ ┌──┐
   │Q3│ │Q4│ │Q5│ │Q8│  ← ladder
   └──┘ └──┘ └──┘ └──┘
    ●────●────●────●
      KLD vs F16

AMALIA-9B · GGUF

llama · 9B · 32K ctx · imatrix (pt) · KLD-measured

FORMAT GGUF (imatrix)	SIZES 3.1 – 9.1 GB	ARCH Llama · 42L	CONTEXT 32768
IMATRIX pt-PT corpus	VALIDATION KLD vs F16	LANGUAGE Português (EU)	LICENSE Apache-2.0

AMALIA-9B-0626-SFT — GGUF (imatrix + KLD)

imatrix-quantized GGUFs of AMALIA-9B, an open European-Portuguese instruction model — one of the first open LLMs developed in Portugal. Runs from ~3.1 GB (IQ2_M) to ~9.1 GB (Q8_0). Calibrated (imatrix) on Portuguese text, and every quant is measured against the full-precision model with KLD so you can pick a size with the fidelity cost in front of you. Credit: this is the AMALIA-LLM project's model — amalia-llm/AMALIA-9B-0626-SFT; ours is the quant ladder + metrics.

✅ Recommended files

Use case	File	Notes
⭐ Best value	amalia-9b-Q5_K_M.gguf	The pick — lower KLD than Q6_K at 1 GB less (0.051 vs 0.078). ~6.1 GB.
Safe default	amalia-9b-Q4_K_M.gguf	Standard K-quant, ~5.2 GB; noticeably less faithful than Q5 (KLD 0.22 vs 0.05).
Max fidelity	amalia-9b-Q8_0.gguf	Lowest KLD (0.006) — near-lossless. ~9.1 GB.
Tightest RAM	amalia-9b-Q3_K_M.gguf	~4.2 GB — runs on 8 GB machines, still coherent.
Smallest (aggressive)	amalia-9b-IQ2_M.gguf	~3.1 GB — fits very tight RAM, at a real quality cost (top-1 61%).

📦 Files (the full ladder)

Quant	Bits	File size	Notes
IQ2_M	~2.7	~3.1 GB	Smallest — aggressive i-quant, real quality trade-off.
Q3_K_M	3	~4.2 GB	Smallest comfortable quant.
IQ4_XS	~4.3	~4.7 GB	Compact i-quant (imatrix).
Q4_K_M	4	~5.2 GB	Standard K-quant.
Q5_K_M	5	~6.1 GB	⭐ Best quality/size.
Q6_K	6	~7.0 GB	Near-lossless.
Q8_0	8	~9.1 GB	Highest precision.

📊 Metrics — objective quality vs the F16 reference

KLD (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the full-precision model — the gold-standard fidelity metric (lower = closer). Top-1 match = how often the quant's top token agrees with F16. Measured with llama-perplexity --kl-divergence over a Portuguese corpus at ctx 2048, vs the F16 GGUF.

Model	Size GB	PPL	ΔPPL vs F16	KLD mean	KLD p95	Top-1 match
F16 (reference)	17.1	20.04	0.000	0.0000	0.0000	100.0%
Q8_0	9.06	20.20	+0.177	0.0057	0.0198	96.38%
Q6_K	7.00	19.93	-0.091	0.0779	0.1740	89.06%
Q5_K_M	6.08	19.80	-0.227	0.0511	0.1747	88.89%
Q4_K_M	5.20	20.23	+0.203	0.2230	0.7429	81.39%
IQ4_XS	4.70	20.78	+0.757	0.2173	0.7600	80.55%
Q3_K_M	4.25	22.24	+2.215	0.3258	1.1486	73.58%
IQ2_M	3.11	21.99	+1.968	0.8211	2.9052	60.69%

_{Full per-quant reports in reports/; machine-readable summary in metrics/quant-summary.csv; SHA-256 of every file in reports/artifact-sha256sums.txt. Note: this is an instruction-tuned model measured on raw text, so the absolute PPL is higher than a base model's — what matters here is the relative KLD/top-1 across quants.}

Why does Q5_K_M beat Q6_K on KLD here? It's real and reproducible for this model: Q5_K_M's imatrix-weighted bit allocation is very effective, so it lands closer to F16 on mean-KLD than Q6_K (which stays marginally ahead on top-1). Both are near-lossless — so Q5_K_M is the better value, and Q8_0 is the true fidelity ceiling.

A note on i-quants: we tested the i-quants too. For this model the K-quants win at low bit — e.g. IQ3_M came out worse than Q3_K_M on KLD, so it isn't shipped. IQ2_M is included as the smallest option for very tight RAM, but with an honest quality cost (top-1 61%); if you can spare the space, Q3_K_M is a big step up.

📈 Charts

🧮 Will it fit? (RAM/VRAM cheat-sheet)

Memory ≈ weights + KV-cache (KV grows with context). Rough guide:

you have	comfortable quant	context
6 GB	IQ2_M / Q3_K_M	~8–16K
8 GB	Q3_K_M / Q4_K_M	~16–32K
12 GB	Q5_K_M / Q6_K	~32K
16 GB+	Q6_K / Q8_0	~32K (native)

🚀 How to run it

# ollama (pulls straight from this repo)
ollama run hf.co/KikoCis/AMALIA-9B-0626-SFT-GGUF:Q5_K_M

# llama.cpp
llama-server -m amalia-9b-Q5_K_M.gguf -c 32768 --jinja
llama-cli   -m amalia-9b-Q5_K_M.gguf -p "Explica em português o que é a saudade." -ngl 99

Recommended sampling: temperature ~0.7, top_p ~0.9. It's a chat/instruct model (uses its built-in chat template) — prompt it in European Portuguese for best results.

⚠️ Good to know

• Strengths: European-Portuguese fluency and instruction-following; one of the first open LLMs built in Portugal.
• Limits: specialised for Portuguese — not a coding/agentic model, and not tuned for English. Low-bit quants (IQ2/Q3) trade real fidelity for size (see KLD).
• Absolute perplexity looks high only because it's measured on raw text with an instruct model; the KLD table is the fair quant-quality signal.

📊 Evaluation methodology (how the numbers were measured)

• What: quantization fidelity vs the F16 GGUF — not a task benchmark. llama-perplexity --kl-divergence (KLD mean/p95/max, ΔPPL, top-1 agreement).
• Corpus: European-Portuguese text (Wikipedia + diverse), ctx 2048, same corpus family used for the imatrix calibration.
• Reference: the model's own F16 GGUF (KLD = 0 by definition).
• Date: 2026-07. Caveat: relative fidelity ranking across quants of this model; not comparable across different models/corpora.

🔁 Provenance & reproducibility

• Scripts: scripts/ — exact convert → imatrix → quant → KLD commands.
• imatrix: amalia-9b-pt.imatrix — importance matrix (Portuguese calibration) used for the IQ/K quants.
• Checksums: reports/artifact-sha256sums.txt — shasum -a 256 -c to verify.
• Source: amalia-llm/AMALIA-9B-0626-SFT — weights unmodified (faithful quantization).

🗒️ Changelog

• 2026-07 v1: imatrix GGUF ladder (IQ2_M → Q8_0) + KLD/PPL metrics, Portuguese-calibrated.

📚 Credit & license

Model, weights, training data: © the AMALIA-LLM project — model · website · paper (PROPOR 2026). Quant ladder + imatrix (pt) + KLD/PPL metrics: KikoCis. Apache-2.0 (same as upstream). No weights modified.