Instructions to use eaddario/gemma-4-E4B-it-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use eaddario/gemma-4-E4B-it-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use Docker

docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16

LM Studio
Jan
Ollama
How to use eaddario/gemma-4-E4B-it-GGUF with Ollama:
```
ollama run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```
Unsloth Desktop

How to use eaddario/gemma-4-E4B-it-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/gemma-4-E4B-it-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use eaddario/gemma-4-E4B-it-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```

Lemonade

How to use eaddario/gemma-4-E4B-it-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/gemma-4-E4B-it-GGUF:F16

Run and chat with the model

lemonade run user.gemma-4-E4B-it-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use eaddario/gemma-4-E4B-it-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/gemma-4-E4B-it-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use eaddario/gemma-4-E4B-it-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "eaddario/gemma-4-E4B-it-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

eaddario commited on Apr 19

Commit

2c7e1e2

verified ·

1 Parent(s): de75350

Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores

Browse files

Files changed (42) hide show

scores/gemma-4-E4B-it-q2_k.arc +23 -0
scores/gemma-4-E4B-it-q2_k.hsw +22 -0
scores/gemma-4-E4B-it-q2_k.mmlu +23 -0
scores/gemma-4-E4B-it-q2_k.ppx +56 -0
scores/gemma-4-E4B-it-q2_k.tqa +23 -0
scores/gemma-4-E4B-it-q2_k.wng +21 -0
scores/gemma-4-E4B-it-q3_k.arc +23 -0
scores/gemma-4-E4B-it-q3_k.hsw +22 -0
scores/gemma-4-E4B-it-q3_k.mmlu +23 -0
scores/gemma-4-E4B-it-q3_k.ppx +56 -0
scores/gemma-4-E4B-it-q3_k.tqa +23 -0
scores/gemma-4-E4B-it-q3_k.wng +21 -0
scores/gemma-4-E4B-it-q4_k.arc +21 -0
scores/gemma-4-E4B-it-q4_k.hsw +20 -0
scores/gemma-4-E4B-it-q4_k.mmlu +21 -0
scores/gemma-4-E4B-it-q4_k.ppx +54 -0
scores/gemma-4-E4B-it-q4_k.tqa +21 -0
scores/gemma-4-E4B-it-q4_k.wng +19 -0
scores/gemma-4-E4B-it-q5_k.arc +18 -0
scores/gemma-4-E4B-it-q5_k.hsw +17 -0
scores/gemma-4-E4B-it-q5_k.mmlu +18 -0
scores/gemma-4-E4B-it-q5_k.ppx +51 -0
scores/gemma-4-E4B-it-q5_k.tqa +18 -0
scores/gemma-4-E4B-it-q5_k.wng +16 -0
scores/gemma-4-E4B-it-q6_k.arc +17 -0
scores/gemma-4-E4B-it-q6_k.hsw +16 -0
scores/gemma-4-E4B-it-q6_k.mmlu +17 -0
scores/gemma-4-E4B-it-q6_k.ppx +50 -0
scores/gemma-4-E4B-it-q6_k.tqa +17 -0
scores/gemma-4-E4B-it-q6_k.wng +15 -0
scores/gemma-4-E4B-it-q7_k.arc +16 -0
scores/gemma-4-E4B-it-q7_k.hsw +15 -0
scores/gemma-4-E4B-it-q7_k.mmlu +16 -0
scores/gemma-4-E4B-it-q7_k.ppx +49 -0
scores/gemma-4-E4B-it-q7_k.tqa +16 -0
scores/gemma-4-E4B-it-q7_k.wng +14 -0
scores/gemma-4-E4B-it-q8_0.arc +15 -0
scores/gemma-4-E4B-it-q8_0.hsw +14 -0
scores/gemma-4-E4B-it-q8_0.mmlu +15 -0
scores/gemma-4-E4B-it-q8_0.ppx +48 -0
scores/gemma-4-E4B-it-q8_0.tqa +15 -0
scores/gemma-4-E4B-it-q8_0.wng +13 -0

scores/gemma-4-E4B-it-q2_k.arc ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 34.8000 +/- 1.7405
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q2_k.hsw ADDED Viewed

	@@ -0,0 +1,22 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	42.53333333%	[39.0419%, 46.1008%]

scores/gemma-4-E4B-it-q2_k.mmlu ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 34.1333 +/- 1.7325
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q2_k.ppx ADDED Viewed

	@@ -0,0 +1,56 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   : 158.513459 ±   1.991659
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  70.11%
+Mean ln(PPL(Q)/PPL(base))     :   1.101352 ±   0.009377
+Mean PPL(Q)/PPL(base)         :   3.008232 ±   0.028209
+Mean PPL(Q)-PPL(base)         : 105.820225 ±   1.623290
+====== KL divergence statistics ======
+Mean    KLD:   2.371872 ±   0.006399
+Maximum KLD:  30.074741
+99.9%   KLD:  16.281898
+99.0%   KLD:  10.876118
+95.0%   KLD:   7.153347
+90.0%   KLD:   5.582164
+Median  KLD:   1.691821
+10.0%   KLD:   0.054692
+ 5.0%   KLD:   0.010447
+ 1.0%   KLD:   0.000600
+ 0.1%   KLD:   0.000037
+Minimum KLD:   0.000000
+====== Token probability statistics ======
+Mean    Δp: -13.185 ± 0.082 %
+Maximum Δp: 99.835%
+99.9%   Δp: 89.775%
+99.0%   Δp: 59.588%
+95.0%   Δp: 19.010%
+90.0%   Δp:  4.926%
+75.0%   Δp:  0.030%
+Median  Δp: -0.543%
+25.0%   Δp: -18.065%
+10.0%   Δp: -66.818%
+ 5.0%   Δp: -88.176%
+ 1.0%   Δp: -99.296%
+ 0.1%   Δp: -99.987%
+Minimum Δp: -100.000%
+RMS Δp    : 33.940 ± 0.093 %
+Same top p: 44.998 ± 0.130 %

scores/gemma-4-E4B-it-q2_k.tqa ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 28.1333 +/- 1.6430
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q2_k.wng ADDED Viewed

	@@ -0,0 +1,21 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q2_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:    7 tensors
+llama_model_loader: - type q2_K:  112 tensors
+llama_model_loader: - type q3_K:   36 tensors
+llama_model_loader: - type q4_K:   15 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type iq2_xxs:   94 tensors
+llama_model_loader: - type iq2_xs:   38 tensors
+llama_model_loader: - type iq3_xxs:   38 tensors
+llama_model_loader: - type iq4_xs:    2 tensors
+llama_model_loader: - type iq1_m:   36 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ2_S - 2.5 bpw
+print_info: file size   = 2.19 GiB (2.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 51.8667 +/- 1.8257

scores/gemma-4-E4B-it-q3_k.arc ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 54.0000 +/- 1.8211
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q3_k.hsw ADDED Viewed

	@@ -0,0 +1,22 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	61.46666667%	[57.9336%, 64.8829%]

scores/gemma-4-E4B-it-q3_k.mmlu ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 64.6667 +/- 1.7466
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q3_k.ppx ADDED Viewed

	@@ -0,0 +1,56 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  53.460778 ±   0.637355
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  93.43%
+Mean ln(PPL(Q)/PPL(base))     :   0.014461 ±   0.004273
+Mean PPL(Q)/PPL(base)         :   1.014566 ±   0.004336
+Mean PPL(Q)-PPL(base)         :   0.767544 ±   0.227712
+====== KL divergence statistics ======
+Mean    KLD:   0.368276 ±   0.001632
+Maximum KLD:  16.233898
+99.9%   KLD:   6.639840
+99.0%   KLD:   2.925996
+95.0%   KLD:   1.350669
+90.0%   KLD:   0.914818
+Median  KLD:   0.166417
+10.0%   KLD:   0.001494
+ 5.0%   KLD:   0.000202
+ 1.0%   KLD:   0.000008
+ 0.1%   KLD:   0.000000
+Minimum KLD:  -0.000030
+====== Token probability statistics ======
+Mean    Δp: -1.997 ± 0.038 %
+Maximum Δp: 99.669%
+99.9%   Δp: 75.373%
+99.0%   Δp: 40.206%
+95.0%   Δp: 14.981%
+90.0%   Δp:  6.041%
+75.0%   Δp:  0.341%
+Median  Δp: -0.002%
+25.0%   Δp: -1.599%
+10.0%   Δp: -13.659%
+ 5.0%   Δp: -27.491%
+ 1.0%   Δp: -61.928%
+ 0.1%   Δp: -94.101%
+Minimum Δp: -99.998%
+RMS Δp    : 14.827 ± 0.072 %
+Same top p: 75.953 ± 0.111 %

scores/gemma-4-E4B-it-q3_k.tqa ADDED Viewed

	@@ -0,0 +1,23 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 33.3333 +/- 1.7225
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q3_k.wng ADDED Viewed

	@@ -0,0 +1,21 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q3_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    1 tensors
+llama_model_loader: - type q4_1:   11 tensors
+llama_model_loader: - type q5_1:    6 tensors
+llama_model_loader: - type q8_0:    1 tensors
+llama_model_loader: - type q2_K:   11 tensors
+llama_model_loader: - type q3_K:  118 tensors
+llama_model_loader: - type q4_K:   41 tensors
+llama_model_loader: - type q5_K:    6 tensors
+llama_model_loader: - type iq3_xxs:   41 tensors
+llama_model_loader: - type iq3_s:   49 tensors
+llama_model_loader: - type iq4_xs:   96 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = IQ4_XS - 4.25 bpw
+print_info: file size   = 3.06 GiB (3.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 66.6667 +/- 1.7225

scores/gemma-4-E4B-it-q4_k.arc ADDED Viewed

	@@ -0,0 +1,21 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 58.1333 +/- 1.8026
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q4_k.hsw ADDED Viewed

	@@ -0,0 +1,20 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	66.66666667%	[63.2155%, 69.9479%]

scores/gemma-4-E4B-it-q4_k.mmlu ADDED Viewed

	@@ -0,0 +1,21 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 66.2667 +/- 1.7276
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q4_k.ppx ADDED Viewed

	@@ -0,0 +1,54 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  56.460364 ±   0.691942
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  97.35%
+Mean ln(PPL(Q)/PPL(base))     :   0.069052 ±   0.002823
+Mean PPL(Q)/PPL(base)         :   1.071492 ±   0.003025
+Mean PPL(Q)-PPL(base)         :   3.767129 ±   0.170411
+====== KL divergence statistics ======
+Mean    KLD:   0.095300 ±   0.000557
+Maximum KLD:  15.853424
+99.9%   KLD:   2.527996
+99.0%   KLD:   0.863295
+95.0%   KLD:   0.356812
+90.0%   KLD:   0.229792
+Median  KLD:   0.035697
+10.0%   KLD:   0.000216
+ 5.0%   KLD:   0.000025
+ 1.0%   KLD:   0.000001
+ 0.1%   KLD:  -0.000001
+Minimum KLD:  -0.000043
+====== Token probability statistics ======
+Mean    Δp: -0.421 ± 0.020 %
+Maximum Δp: 99.633%
+99.9%   Δp: 54.652%
+99.0%   Δp: 23.879%
+95.0%   Δp:  8.503%
+90.0%   Δp:  3.458%
+75.0%   Δp:  0.202%
+Median  Δp: -0.000%
+25.0%   Δp: -0.392%
+10.0%   Δp: -5.060%
+ 5.0%   Δp: -11.030%
+ 1.0%   Δp: -29.030%
+ 0.1%   Δp: -62.118%
+Minimum Δp: -98.611%
+RMS Δp    :  7.866 ± 0.054 %
+Same top p: 87.367 ± 0.087 %

scores/gemma-4-E4B-it-q4_k.tqa ADDED Viewed

	@@ -0,0 +1,21 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 32.5333 +/- 1.7119
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q4_k.wng ADDED Viewed

	@@ -0,0 +1,19 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:    2 tensors
+llama_model_loader: - type q4_1:    5 tensors
+llama_model_loader: - type q5_1:   24 tensors
+llama_model_loader: - type q8_0:    2 tensors
+llama_model_loader: - type q3_K:    1 tensors
+llama_model_loader: - type q4_K:   95 tensors
+llama_model_loader: - type q5_K:  101 tensors
+llama_model_loader: - type q6_K:    2 tensors
+llama_model_loader: - type iq4_xs:  149 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q4_K - Medium
+print_info: file size   = 3.94 GiB (4.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 65.4667 +/- 1.7374

scores/gemma-4-E4B-it-q5_k.arc ADDED Viewed

	@@ -0,0 +1,18 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 57.4667 +/- 1.8065
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q5_k.hsw ADDED Viewed

	@@ -0,0 +1,17 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	68.13333333%	[64.7134%, 71.3685%]

scores/gemma-4-E4B-it-q5_k.mmlu ADDED Viewed

	@@ -0,0 +1,18 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 67.4667 +/- 1.7119
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q5_k.ppx ADDED Viewed

	@@ -0,0 +1,51 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  58.576787 ±   0.727545
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  98.32%
+Mean ln(PPL(Q)/PPL(base))     :   0.105851 ±   0.002352
+Mean PPL(Q)/PPL(base)         :   1.111657 ±   0.002615
+Mean PPL(Q)-PPL(base)         :   5.883553 ±   0.169353
+====== KL divergence statistics ======
+Mean    KLD:   0.021173 ±   0.000200
+Maximum KLD:   8.555525
+99.9%   KLD:   0.762584
+99.0%   KLD:   0.206188
+95.0%   KLD:   0.075586
+90.0%   KLD:   0.047285
+Median  KLD:   0.007055
+10.0%   KLD:   0.000044
+ 5.0%   KLD:   0.000005
+ 1.0%   KLD:  -0.000000
+ 0.1%   KLD:  -0.000002
+Minimum KLD:  -0.000057
+====== Token probability statistics ======
+Mean    Δp: -0.211 ± 0.010 %
+Maximum Δp: 95.665%
+99.9%   Δp: 29.084%
+99.0%   Δp: 10.904%
+95.0%   Δp:  3.685%
+90.0%   Δp:  1.450%
+75.0%   Δp:  0.063%
+Median  Δp: -0.000%
+25.0%   Δp: -0.216%
+10.0%   Δp: -2.289%
+ 5.0%   Δp: -4.984%
+ 1.0%   Δp: -12.990%
+ 0.1%   Δp: -30.230%
+Minimum Δp: -92.949%
+RMS Δp    :  3.844 ± 0.042 %
+Same top p: 94.144 ± 0.061 %

scores/gemma-4-E4B-it-q5_k.tqa ADDED Viewed

	@@ -0,0 +1,18 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 34.2667 +/- 1.7342
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q5_k.wng ADDED Viewed

	@@ -0,0 +1,16 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q5_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   13 tensors
+llama_model_loader: - type q5_1:   28 tensors
+llama_model_loader: - type q8_0:    6 tensors
+llama_model_loader: - type q4_K:    3 tensors
+llama_model_loader: - type q5_K:  158 tensors
+llama_model_loader: - type q6_K:  173 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q5_K - Medium
+print_info: file size   = 4.81 GiB (5.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 67.3333 +/- 1.7137

scores/gemma-4-E4B-it-q6_k.arc ADDED Viewed

	@@ -0,0 +1,17 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 57.4667 +/- 1.8065
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q6_k.hsw ADDED Viewed

	@@ -0,0 +1,16 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	68.66666667%	[65.2590%, 71.8841%]

scores/gemma-4-E4B-it-q6_k.mmlu ADDED Viewed

	@@ -0,0 +1,17 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 66.2667 +/- 1.7276
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q6_k.ppx ADDED Viewed

	@@ -0,0 +1,50 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  57.395025 ±   0.709393
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  98.51%
+Mean ln(PPL(Q)/PPL(base))     :   0.085471 ±   0.002205
+Mean PPL(Q)/PPL(base)         :   1.089229 ±   0.002402
+Mean PPL(Q)-PPL(base)         :   4.701791 ±   0.150722
+====== KL divergence statistics ======
+Mean    KLD:   0.006872 ±   0.000100
+Maximum KLD:   3.890319
+99.9%   KLD:   0.332454
+99.0%   KLD:   0.064667
+95.0%   KLD:   0.022811
+90.0%   KLD:   0.013986
+Median  KLD:   0.002055
+10.0%   KLD:   0.000015
+ 5.0%   KLD:   0.000002
+ 1.0%   KLD:  -0.000000
+ 0.1%   KLD:  -0.000002
+Minimum KLD:  -0.000105
+====== Token probability statistics ======
+Mean    Δp: -0.077 ± 0.006 %
+Maximum Δp: 75.114%
+99.9%   Δp: 15.320%
+99.0%   Δp:  6.360%
+95.0%   Δp:  2.144%
+90.0%   Δp:  0.862%
+75.0%   Δp:  0.043%
+Median  Δp: -0.000%
+25.0%   Δp: -0.097%
+10.0%   Δp: -1.173%
+ 5.0%   Δp: -2.602%
+ 1.0%   Δp: -6.889%
+ 0.1%   Δp: -17.803%
+Minimum Δp: -88.628%
+RMS Δp    :  2.229 ± 0.037 %
+Same top p: 96.672 ± 0.047 %

scores/gemma-4-E4B-it-q6_k.tqa ADDED Viewed

	@@ -0,0 +1,17 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 33.8667 +/- 1.7292
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q6_k.wng ADDED Viewed

	@@ -0,0 +1,15 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q6_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   32 tensors
+llama_model_loader: - type q5_1:    8 tensors
+llama_model_loader: - type q8_0:   91 tensors
+llama_model_loader: - type q5_K:    2 tensors
+llama_model_loader: - type q6_K:  248 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q6_K
+print_info: file size   = 5.69 GiB (6.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 66.5333 +/- 1.7242

scores/gemma-4-E4B-it-q7_k.arc ADDED Viewed

	@@ -0,0 +1,16 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 57.2000 +/- 1.8079
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q7_k.hsw ADDED Viewed

	@@ -0,0 +1,15 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	68.00000000%	[64.5771%, 71.2395%]

scores/gemma-4-E4B-it-q7_k.mmlu ADDED Viewed

	@@ -0,0 +1,16 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 66.5333 +/- 1.7242
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q7_k.ppx ADDED Viewed

	@@ -0,0 +1,49 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  57.909227 ±   0.717870
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  98.55%
+Mean ln(PPL(Q)/PPL(base))     :   0.094390 ±   0.002196
+Mean PPL(Q)/PPL(base)         :   1.098988 ±   0.002413
+Mean PPL(Q)-PPL(base)         :   5.215993 ±   0.155827
+====== KL divergence statistics ======
+Mean    KLD:   0.003237 ±   0.000086
+Maximum KLD:   8.411139
+99.9%   KLD:   0.152386
+99.0%   KLD:   0.029975
+95.0%   KLD:   0.010175
+90.0%   KLD:   0.006270
+Median  KLD:   0.000917
+10.0%   KLD:   0.000005
+ 5.0%   KLD:   0.000001
+ 1.0%   KLD:  -0.000001
+ 0.1%   KLD:  -0.000006
+Minimum KLD:  -0.000048
+====== Token probability statistics ======
+Mean    Δp: -0.062 ± 0.004 %
+Maximum Δp: 74.921%
+99.9%   Δp: 10.826%
+99.0%   Δp:  4.042%
+95.0%   Δp:  1.411%
+90.0%   Δp:  0.565%
+75.0%   Δp:  0.027%
+Median  Δp: -0.000%
+25.0%   Δp: -0.072%
+10.0%   Δp: -0.796%
+ 5.0%   Δp: -1.764%
+ 1.0%   Δp: -4.686%
+ 0.1%   Δp: -11.661%
+Minimum Δp: -87.893%
+RMS Δp    :  1.534 ± 0.034 %
+Same top p: 97.743 ± 0.039 %

scores/gemma-4-E4B-it-q7_k.tqa ADDED Viewed

	@@ -0,0 +1,16 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 33.7333 +/- 1.7276
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q7_k.wng ADDED Viewed

	@@ -0,0 +1,14 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q7_K.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   52 tensors
+llama_model_loader: - type q5_1:    1 tensors
+llama_model_loader: - type q8_0:  274 tensors
+llama_model_loader: - type q6_K:   54 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 6.56 GiB (7.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 66.6667 +/- 1.7225

scores/gemma-4-E4B-it-q8_0.arc ADDED Viewed

	@@ -0,0 +1,15 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+multiple_choice_score: there are 869 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 869 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 56.9333 +/- 1.8093
+Random chance: 25.0083 +/- 1.5824

scores/gemma-4-E4B-it-q8_0.hsw ADDED Viewed

	@@ -0,0 +1,14 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+hellaswag_score : loaded 10042 tasks from prompt.
+hellaswag_score : selecting 750 randomized tasks.
+hellaswag_score : calculating hellaswag score over selected tasks.
+750	68.00000000%	[64.5771%, 71.2395%]

scores/gemma-4-E4B-it-q8_0.mmlu ADDED Viewed

	@@ -0,0 +1,15 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+multiple_choice_score: there are 5362 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 5362 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 66.5333 +/- 1.7242
+Random chance: 25.0000 +/- 1.5822

scores/gemma-4-E4B-it-q8_0.ppx ADDED Viewed

	@@ -0,0 +1,48 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+====== Perplexity statistics ======
+Mean PPL(Q)                   :  57.324375 ±   0.708968
+Mean PPL(base)                :  52.693235 ±   0.610088
+Cor(ln(PPL(Q)), ln(PPL(base))):  98.58%
+Mean ln(PPL(Q)/PPL(base))     :   0.084239 ±   0.002164
+Mean PPL(Q)/PPL(base)         :   1.087889 ±   0.002354
+Mean PPL(Q)-PPL(base)         :   4.631141 ±   0.148444
+====== KL divergence statistics ======
+Mean    KLD:   0.001798 ±   0.000059
+Maximum KLD:   4.473496
+99.9%   KLD:   0.106441
+99.0%   KLD:   0.015963
+95.0%   KLD:   0.005257
+90.0%   KLD:   0.003187
+Median  KLD:   0.000441
+10.0%   KLD:   0.000003
+ 5.0%   KLD:   0.000000
+ 1.0%   KLD:  -0.000001
+ 0.1%   KLD:  -0.000007
+Minimum KLD:  -0.000047
+====== Token probability statistics ======
+Mean    Δp: -0.020 ± 0.003 %
+Maximum Δp: 71.373%
+99.9%   Δp:  8.424%
+99.0%   Δp:  3.038%
+95.0%   Δp:  1.051%
+90.0%   Δp:  0.443%
+75.0%   Δp:  0.026%
+Median  Δp: -0.000%
+25.0%   Δp: -0.037%
+10.0%   Δp: -0.510%
+ 5.0%   Δp: -1.171%
+ 1.0%   Δp: -3.218%
+ 0.1%   Δp: -8.574%
+Minimum Δp: -88.102%
+RMS Δp    :  1.155 ± 0.038 %
+Same top p: 98.371 ± 0.033 %

scores/gemma-4-E4B-it-q8_0.tqa ADDED Viewed

	@@ -0,0 +1,15 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+multiple_choice_score: there are 817 tasks in prompt
+multiple_choice_score: selecting 750 random tasks from 817 tasks available
+multiple_choice_score : calculating TruthfulQA score over 750 tasks.
+Final result: 34.0000 +/- 1.7309
+Random chance: 19.8992 +/- 1.4588

scores/gemma-4-E4B-it-q8_0.wng ADDED Viewed

	@@ -0,0 +1,13 @@

+llama_model_loader: loaded meta data with 49 key-value pairs and 720 tensors from gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q8_0.gguf (version GGUF V3 (latest))
+llama_model_loader: - type  f32:  339 tensors
+llama_model_loader: - type  f16:   60 tensors
+llama_model_loader: - type q8_0:  309 tensors
+llama_model_loader: - type q6_K:   12 tensors
+print_info: file format = GGUF V3 (latest)
+print_info: file type   = Q8_0
+print_info: file size   = 7.44 GiB (8.50 BPW)
+winogrande_score : loaded 1267 tasks from prompt.
+winogrande_score : selecting 750 random tasks
+Final Winogrande score(750 tasks): 67.0667 +/- 1.7172