noir-verdict-nemotron-4b-lora

How to use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/NVIDIA-Nemotron-3-Nano-4B"
adapter = "sankalphs/noir-verdict-nemotron-4b-lora"

tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)
model.eval()

For inference through llama.cpp, use the companion merged checkpoint (...-merged) or the Q4_K_M GGUF (...-gguf).

How it was trained

Image: nvidia/cuda:12.8.1-devel-ubuntu22.04 + Python 3.13
Pip deps: torch>=2.8.0, triton>=3.4.0, unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo, unsloth[base] @ git+https://github.com/unslothai/unsloth, --torch-backend=cu128
Native: causal-conv1d==1.6.2.post1 and mamba-ssm==2.3.2.post1 compiled from source with --no-build-isolation, CC=gcc, CXX=g++ (no prebuilt cu128 + Py3.13 wheel exists)
Trainer: TRL SFTTrainer, packing, bf16, Unsloth LoRA (r=16, alpha=32, lr=2e-4 cosine, bs=2 grad_accum=8, 240 steps)
Orchestrator: train/modal_finetune.py

5-case smoke results (A10G, `--n-gpu-layers 99`)

case	suspect	personality	truth_mode	failure_flags
0	Greta Lindholm	nervous	lie	none
37	(37, 1)	helpful	partial_truth	none
113	(113, 2)	arrogant	truth	none
241	(241, 3)	evasive	deflect	none
497	Greta Lindholm	nervous	lie	none