---
license: gemma
base_model: google/gemma-4-12B-it
datasets:
  - ShenLab/MentalChat16K
  - Amod/mental_health_counseling_conversations
  - nbertagnolli/counsel-chat
  - Estwld/empathetic_dialogues_llm
  - arnaiztech/llms-mental-health-crisis-responses
language:
  - en
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - qlora
  - unsloth
  - mental-health
  - cbt
  - build-small-hackathon
---

# gemma4-mentalhealthbuddy-v1 (GGUF · Q4_K_M)

A small, locally-runnable chat model for **REFRAME**, a live CBT (cognitive behavioral therapy) studio
built for the Build Small Hackathon. Fine-tuned from `google/gemma-4-12B-it` with **QLoRA** on a blend of
**mental-health counseling, empathetic-dialogue, and crisis-response datasets**, merged, and exported to
**GGUF Q4_K_M** for low-latency, fully-local inference via **llama.cpp** (Ollama or llama-cpp-python).

> The REFRAME app layers **CBT-style Socratic questioning** (via its system prompt) on top of this
> mental-health-tuned base. It is a supportive conversationalist — not a clinician. See Limitations.

## Model details
- **Base:** `google/gemma-4-12B-it` (12B)
- **Method:** QLoRA — LoRA `r=16`, `α=16`, `dropout=0`, context `2048`
- **Target modules:** `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
- **Artifact:** merged weights → **GGUF**, quantized **Q4_K_M**
- **Runtime name:** `gemma4_mentalhealthbuddy_v1` (Ollama) · llama.cpp engine

## Training
- **Platform:** [Modal](https://modal.com) · single **NVIDIA H100 80GB**
- **Libraries:** `unsloth`, `peft`, `trl`, `bitsandbytes`, `accelerate`, `torch 2.10`
- **Batch:** per-device 2 × grad-accum 4 (effective 8)
- **Data:** a blend of mental-health datasets, chat-formatted via the tokenizer chat template (see **Training data** below)
- **Storage:** Modal persistent volume (adapters · merged model · GGUF export)

## Training data
Fine-tuned on a blend of public mental-health corpora — counseling, empathetic dialogue, and crisis responses:

| Dataset | Hugging Face | Source / GitHub |
|---|---|---|
| MentalChat16K | [ShenLab/MentalChat16K](https://huggingface.co/datasets/ShenLab/MentalChat16K) | [PennShenLab/MentalChat16K](https://github.com/PennShenLab/MentalChat16K) · [arXiv:2503.13509](https://arxiv.org/abs/2503.13509) |
| Mental Health Counseling Conversations | [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations) | sourced from counselchat.com |
| CounselChat | [nbertagnolli/counsel-chat](https://huggingface.co/datasets/nbertagnolli/counsel-chat) | [nbertagnolli/counsel-chat](https://github.com/nbertagnolli/counsel-chat) |
| EmpatheticDialogues (LLM-formatted) | [Estwld/empathetic_dialogues_llm](https://huggingface.co/datasets/Estwld/empathetic_dialogues_llm) | [facebookresearch/EmpatheticDialogues](https://github.com/facebookresearch/EmpatheticDialogues) |
| Mental-Health Crisis Responses (score-filtered) | [arnaiztech/llms-mental-health-crisis-responses](https://huggingface.co/datasets/arnaiztech/llms-mental-health-crisis-responses) | [ellisalicante/LLMs-Mental-Health-Crisis](https://github.com/ellisalicante/LLMs-Mental-Health-Crisis) · [arXiv:2509.24857](https://arxiv.org/abs/2509.24857) |

Plus synthetic crisis-response data where available.

## How to run
**Ollama**
```bash
ollama run gemma4_mentalhealthbuddy_v1
```
**llama-cpp-python**
```python
from llama_cpp import Llama
llm = Llama.from_pretrained(
    repo_id="emailvenky/gemma4-mentalhealthbuddy-v1",
    filename="*Q4_K_M.gguf",
    n_ctx=4096,
)
print(llm.create_chat_completion(messages=[{"role": "user", "content": "Hello"}]))
```

## Intended use & limitations
- **Intended:** the REFRAME demo — supportive, reflective conversation that helps a person reframe an
  unhelpful thought via Socratic questioning.
- **Not intended:** medical advice, diagnosis, or crisis intervention. It is **not a substitute for
  professional care**. REFRAME surfaces crisis helplines when a message signals risk.
- **Limitations:** a 12B model fine-tuned (QLoRA) on mental-health data — expect the usual LLM limits
  (factual errors, limited reasoning depth). Quantization (Q4_K_M) trades some quality for speed/size.

## License
Derived from Gemma — use is subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).

— Part of **REFRAME** 🍁