gemma4-mentalhealthbuddy-v1 (GGUF · Q4_K_M)

A small, locally-runnable chat model for REFRAME, a live CBT (cognitive behavioral therapy) studio built for the Build Small Hackathon. Fine-tuned from google/gemma-4-12B-it with QLoRA on a blend of mental-health counseling, empathetic-dialogue, and crisis-response datasets, merged, and exported to GGUF Q4_K_M for low-latency, fully-local inference via llama.cpp (Ollama or llama-cpp-python).

The REFRAME app layers CBT-style Socratic questioning (via its system prompt) on top of this mental-health-tuned base. It is a supportive conversationalist — not a clinician. See Limitations.

Model details

• Base: google/gemma-4-12B-it (12B)
• Method: QLoRA — LoRA r=16, α=16, dropout=0, context 2048
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Artifact: merged weights → GGUF, quantized Q4_K_M
• Runtime name: gemma4_mentalhealthbuddy_v1 (Ollama) · llama.cpp engine

Training

• Platform: Modal · single NVIDIA H100 80GB
• Libraries: unsloth, peft, trl, bitsandbytes, accelerate, torch 2.10
• Batch: per-device 2 × grad-accum 4 (effective 8)
• Data: a blend of mental-health datasets, chat-formatted via the tokenizer chat template (see Training data below)
• Storage: Modal persistent volume (adapters · merged model · GGUF export)

Training data

Fine-tuned on a blend of public mental-health corpora — counseling, empathetic dialogue, and crisis responses:

Dataset	Hugging Face	Source / GitHub
MentalChat16K	ShenLab/MentalChat16K	PennShenLab/MentalChat16K · arXiv:2503.13509
Mental Health Counseling Conversations	Amod/mental_health_counseling_conversations	sourced from counselchat.com
CounselChat	nbertagnolli/counsel-chat	nbertagnolli/counsel-chat
EmpatheticDialogues (LLM-formatted)	Estwld/empathetic_dialogues_llm	facebookresearch/EmpatheticDialogues
Mental-Health Crisis Responses (score-filtered)	arnaiztech/llms-mental-health-crisis-responses	ellisalicante/LLMs-Mental-Health-Crisis · arXiv:2509.24857

Plus synthetic crisis-response data where available.

How to run

Ollama

ollama run gemma4_mentalhealthbuddy_v1

llama-cpp-python

from llama_cpp import Llama
llm = Llama.from_pretrained(
    repo_id="emailvenky/gemma4-mentalhealthbuddy-v1",
    filename="*Q4_K_M.gguf",
    n_ctx=4096,
)
print(llm.create_chat_completion(messages=[{"role": "user", "content": "Hello"}]))

Intended use & limitations

• Intended: the REFRAME demo — supportive, reflective conversation that helps a person reframe an unhelpful thought via Socratic questioning.
• Not intended: medical advice, diagnosis, or crisis intervention. It is not a substitute for professional care. REFRAME surfaces crisis helplines when a message signals risk.
• Limitations: a 12B model fine-tuned (QLoRA) on mental-health data — expect the usual LLM limits (factual errors, limited reasoning depth). Quantization (Q4_K_M) trades some quality for speed/size.

License

Derived from Gemma — use is subject to the Gemma Terms of Use.

— Part of REFRAME 🍁