Instructions to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public",
	filename="gemma4-e2b-social-v12-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Use Docker

docker model run hf.co/apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

LM Studio
Jan

vLLM

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Ollama
How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Ollama:
```
ollama run hf.co/apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
```

Unsloth Studio

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public to start chatting

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Docker Model Runner:
```
docker model run hf.co/apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M
```

Lemonade

How to use apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull apol/gemma4-e2b-social-spain-v12-q4-k-m-gguf-public:Q4_K_M

Run and chat with the model

lemonade run user.gemma4-e2b-social-spain-v12-q4-k-m-gguf-public-Q4_K_M

List all available models

lemonade list

Gemma 4 E2B Social Spain v12 Q4_K_M GGUF

Public experimental GGUF export for local llama.cpp / LM Studio testing.

This is a merged and quantized checkpoint built from google/gemma-4-E2B-it plus the v12 Social Spain recovery LoRA, then quantized to Q4_K_M.

Status

This is not a production social-services model and must not be presented as legal, medical, or professional social-work advice.

The recommended architecture is:

local GGUF + dated RAG + privacy filter + deterministic validators + repair/abstention + human escalation

The public demo of that wrapper is:

https://huggingface.co/spaces/apol/gemma4-social-spain-demo

About Gemma 4 MTP

Google released Gemma 4 Multi-Token Prediction drafters as an inference-speed optimization. The MTP path for this project is currently documented and tested as a Transformers backend using:

google/gemma-4-E2B-it + apol/gemma4-e2b-social-spain-v12-recovery-lora-public
assistant: google/gemma-4-E2B-it-assistant

Do not assume this GGUF has MTP acceleration in LM Studio or llama.cpp until the exact runtime supports Gemma 4 MTP drafters and is benchmarked. The GGUF remains the simple local runtime; MTP is an additional experimental Transformers path.

Files

gemma4-e2b-social-v12-Q4_K_M.gguf: quantized model.
conversion_provenance_sanitized.json: conversion provenance.
merge_lora_provenance_sanitized.json: merge provenance.
MANIFEST.json: file sizes and SHA-256 hashes.
NOTICE: Gemma redistribution notice.

Local Use

In LM Studio, download gemma4-e2b-social-v12-Q4_K_M.gguf and load it as a local GGUF model.

Example llama.cpp smoke:

llama-cli \
  -m gemma4-e2b-social-v12-Q4_K_M.gguf \
  -sys "Eres un asistente social. Responde sin razonamiento oculto." \
  -p "Responde en espanol claro. Que datos personales no debe compartir una persona migrante por chat?" \
  -n 96 --ctx-size 512 --threads 8 --batch-size 32 \
  --single-turn --reasoning off --no-warmup --simple-io --log-disable

Safety Use

For social-assistance scenarios:

do not ask users to share full passport/NIE numbers, exact address, document scans, or third-party documents in chat;
do not invent phone numbers, addresses, URLs, deadlines, or requirements;
use dated official/local sources for mutable facts;
escalate minors, violence, exploitation, homelessness tonight, medical urgency, and self-harm to qualified human support;
keep human-in-the-loop review for any high-risk case.

License And Use Restrictions

This artifact is distributed under the Gemma Terms of Use and is subject to the Gemma Prohibited Use Policy:

This model is a modified derivative of Gemma. Modified files and generated artifacts are provided with prominent notices in this model card and NOTICE.

Do not use this model for prohibited uses, including unauthorized legal, medical, social-welfare, or other professional practice, automated decisions affecting material rights or well-being, harmful content, or processing private personal information without required rights and consent.