Instructions to use JBrussee/gemma-4-31B-caveman with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JBrussee/gemma-4-31B-caveman with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JBrussee/gemma-4-31B-caveman")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("JBrussee/gemma-4-31B-caveman")
model = AutoModelForMultimodalLM.from_pretrained("JBrussee/gemma-4-31B-caveman")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JBrussee/gemma-4-31B-caveman with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JBrussee/gemma-4-31B-caveman"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JBrussee/gemma-4-31B-caveman",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JBrussee/gemma-4-31B-caveman

SGLang

How to use JBrussee/gemma-4-31B-caveman with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JBrussee/gemma-4-31B-caveman" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JBrussee/gemma-4-31B-caveman",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JBrussee/gemma-4-31B-caveman" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JBrussee/gemma-4-31B-caveman",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JBrussee/gemma-4-31B-caveman with Docker Model Runner:
```
docker model run hf.co/JBrussee/gemma-4-31B-caveman
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

gemma-4-31B-caveman

Gemma 4 31B fine-tuned to speak caveman-mode natively. Full bf16 weights, drop-in replacement for google/gemma-4-31B-it.

Drops articles, filler, pleasantries, hedging. Allows fragments. Keeps code blocks, function names, error strings, and CLI commands byte-exact. Pattern: [thing] [action] [reason]. [next step].

For the lighter LoRA-adapter variant see JBrussee/gemma-4-31B-caveman-lora.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("JBrussee/gemma-4-31B-caveman")
model = AutoModelForCausalLM.from_pretrained(
    "JBrussee/gemma-4-31B-caveman",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

msgs = [{"role": "user", "content": "Explain database connection pooling."}]
ids = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(ids, max_new_tokens=300, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Example

Prompt: "Why does my React component re-render every time the parent updates?"

Output:

Parent re-render → child re-render by default. Props "change" each render
if inline obj/array/fn → new ref → child sees new props.

Fix:
- Wrap child: React.memo(Child)
- Stabilize props: useMemo for obj/array, useCallback for fn
- Lift state so parent re-render not needed

Check: if child memo'd but still re-renders, inline prop is culprit.

Training

Method: QLoRA NF4 + double-quant + bf16 compute (Unsloth), then merged to bf16
LoRA: rank 16, α 32, dropout 0, targets q,k,v,o,gate,up,down
Loss: assistant-only loss masking via TRL completion_only_loss=True
Epochs: 3, lr 2e-4 cosine, batch 2 × grad-accum 8 (effective 16)
Hardware: RunPod RTX PRO 6000 Blackwell 96 GB
Time: ~50 minutes total

Data

1750 train + 193 eval pairs, source-normal → caveman style transfer, drawn from six permissive datasets:

Source	License	Used
OpenAssistant/oasst2	Apache 2.0	dialogue
princeton-nlp/SWE-bench_Verified	research-permissive	debug
ronantakizawa/github-codereview	permissive subset	code review
bigcode/commitpackft (MIT/Apache subset)	MIT/Apache 2.0	refactor
theblackcat102/evol-codealpaca-v1	Apache 2.0	short Q&A
HuggingFaceH4/ultrachat_200k	MIT	short Q&A

Caveman side synthesized via Claude Code (claude -p) and Codex CLI (codex exec with GPT-5.5), routed through the SKILL.md ruleset.

Eval (n=193 holdout)

Category	n	compression	article density	code_fence_match	semantic_sim
dialogue	28	0.59	0.020	1.000	0.91
debug	34	0.92	0.009	0.995	0.98
refactor	27	0.92	0.005	0.963	0.98
qa	104	0.65	0.007	1.000	0.92

Final train_loss 0.024 · eval_loss 0.72 · eval_mean_token_accuracy 0.815.

Strengths: code preservation (96-100% fence-exact), low article density (0.5-2%), strong semantic preservation (91-98%). Weakness: compression ~10-40% rather than the 50-70% of the gold caveman pairs. To compress harder, retrain on a tighter-filtered dataset (filter upper bound 0.7 instead of 1.0).

Reproduce

Full code, data pipeline, and configs: https://github.com/JuliusBrussee/finetune-caveman

License

Inherits the Gemma Prohibited Use Policy. Apache 2.0 base + Gemma terms apply to all outputs. Repository code is MIT. The caveman style ruleset is MIT (https://github.com/JuliusBrussee/caveman).

Citing

@misc{brussee2026cavemanGemma,
  author = {Julius Brussee},
  title  = {Caveman-mode Gemma 4 31B},
  year   = {2026},
  url    = {https://huggingface.co/JBrussee/gemma-4-31B-caveman}
}

Downloads last month: 117

Safetensors

Model size

33B params

Tensor type

BF16

Model tree for JBrussee/gemma-4-31B-caveman

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Finetuned

(215)

this model

Quantizations

1 model