Gemma 4 31B — Fenrir Cybersecurity LoRA

LoRA fine-tune of google/gemma-4-31B on AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 by Alican Kiraz, trained on an HPE ProLiant Compute DL384 Gen12 (2× NVIDIA GH200 144GB HBM3e).

Training data

Dataset: AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 (99,870 rows; system / user / assistant turns)
Domain: offensive & defensive cybersecurity Q&A

Training setup

LoRA: r=16, alpha=32, dropout=0.05, target_modules = q/k/v/o + gate/up/down (language model only)
Trainable: ~~122M / 31.4B (~~0.39%)
Optimizer: AdamW (fused), lr=2e-4, cosine schedule, warmup_ratio=0.03
Per-device batch=4, grad-accum=2, world=2 → effective batch 16
Precision: bf16; attention: SDPA; gradient checkpointing on
Sequence length: 2048 (covers ~97% of dataset without truncation)
Epochs: 3

Hardware

HPE ProLiant Compute DL384 Gen12, 2× NVIDIA GH200 Grace-Hopper 144GB HBM3e (aarch64)

Final metrics

{
  "train_runtime": 145390.0266,
  "train_samples_per_second": 2.061,
  "train_steps_per_second": 0.129,
  "total_flos": 6.327548262623268e+19,
  "train_loss": 0.4692629598424079
}

Use

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer

base = AutoModelForImageTextToText.from_pretrained("google/gemma-4-31B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "ygzgrdmr/gemma-4-31B-fenrir-cybersec-hpe-dl384")
tok = AutoTokenizer.from_pretrained("google/gemma-4-31B")

Credits

Dataset: Alican Kiraz — AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1
Base model: Google Gemma 4 31B

Downloads last month: 12

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ygzgrdmr/gemma-4-31B-fenrir-cybersec-hpe-dl384

Base model

google/gemma-4-31B

Adapter

(2)

this model

ygzgrdmr
/

gemma-4-31B-fenrir-cybersec-hpe-dl384