---
license: apache-2.0
library_name: peft
base_model: unsloth/NVIDIA-Nemotron-3-Nano-4B
tags:
  - noir
  - nemotron
  - unsloth
  - modal
  - sankalphs
pipeline_tag: text-generation
---

# noir-verdict-nemotron-4b-lora

## How to use

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/NVIDIA-Nemotron-3-Nano-4B"
adapter = "sankalphs/noir-verdict-nemotron-4b-lora"

tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)
model.eval()
```

For inference through llama.cpp, use the companion merged checkpoint
(`...-merged`) or the Q4_K_M GGUF (`...-gguf`).


## How it was trained

- **Image**: `nvidia/cuda:12.8.1-devel-ubuntu22.04` + Python 3.13
- **Pip deps**: `torch>=2.8.0`, `triton>=3.4.0`, `unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo`, `unsloth[base] @ git+https://github.com/unslothai/unsloth`, `--torch-backend=cu128`
- **Native**: `causal-conv1d==1.6.2.post1` and `mamba-ssm==2.3.2.post1` compiled from source with `--no-build-isolation`, `CC=gcc`, `CXX=g++` (no prebuilt cu128 + Py3.13 wheel exists)
- **Trainer**: TRL `SFTTrainer`, packing, bf16, Unsloth LoRA (r=16, alpha=32, lr=2e-4 cosine, bs=2 grad_accum=8, 240 steps)
- **Orchestrator**: `train/modal_finetune.py`

## 5-case smoke results (A10G, `--n-gpu-layers 99`)

| case | suspect | personality | truth_mode | failure_flags |
|---|---|---|---|---|
| 0   | Greta Lindholm | nervous  | lie          | none |
| 37  | (37, 1)        | helpful  | partial_truth | none |
| 113 | (113, 2)       | arrogant | truth         | none |
| 241 | (241, 3)       | evasive  | deflect       | none |
| 497 | Greta Lindholm | nervous  | lie          | none |

Pace: ~125 tokens/sec on A10G. No role-token leaks, no leaked `<think>` blocks, no overlong generations.

## Companion artifacts

- **LoRA**: [sankalphs/noir-verdict-nemotron-4b-lora](https://huggingface.co/sankalphs/noir-verdict-nemotron-4b-lora)
- **Merged BF16**: [sankalphs/noir-verdict-nemotron-4b-merged](https://huggingface.co/sankalphs/noir-verdict-nemotron-4b-merged) (7.95 GB)
- **Q4_K_M GGUF**: [sankalphs/noir-verdict-nemotron-4b-gguf](https://huggingface.co/sankalphs/noir-verdict-nemotron-4b-gguf) (2.84 GB)
- **Traces**: [sankalphs/noir-verdict-traces](https://huggingface.co/datasets/sankalphs/noir-verdict-traces)
- **App**: [build-small-hackathon/noir-verdict](https://huggingface.co/spaces/build-small-hackathon/noir-verdict)

## License

Apache-2.0. The base Nemotron 3 Nano weights are governed by NVIDIA's
[model license](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-4B);
the adapter and training code in this repo are Apache-2.0.