GemmaThink
Collection
A collection of Gemma3-1b-it models that we post-trained using SFT and GRPO to enhance its reasoning capabilities, using Google's new Tunix library. • 7 items • Updated
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0# Run inference directly in the terminal:
llama cli -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0# Run inference directly in the terminal:
./llama-cli -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0# Run inference directly in the terminal:
./build/bin/llama-cli -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0Use Docker
docker model run hf.co/chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0Quick Links
GemmaThink-32k-Q8_0-GGUF (GRPO Trained Model)
This model was trained using GRPO (Group Relative Policy Optimization) to generate structured reasoning traces.
Training Details
- Base Model: google/gemma-3-1b-it
- Training Method: SFT + GRPO
- LoRA Rank: 32
- LoRA Alpha: 64.0
- Framework: Tunix (JAX)
- Hardware: v6e-1 TPU in Colab
Output Format
<reasoning>step-by-step thinking process</reasoning>
<answer>final answer</answer>
Quicklinks:
- SFT Base Model
- SFT Base Model Q8 GGUF
- GRPO Full Model
- Q8-GGUF <-- You're here
- Article
- Downloads last month
- 10
Hardware compatibility
Log In to add your hardware
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0# Run inference directly in the terminal: llama cli -hf chimbiwide/gemma-3-1b-it-thinking-32k-grpo-merged-Q8_0-GGUF:Q8_0