piyushptiwari/ebrm-v4-qwen3-4b

EBRM v4 — Qwen3-4B Energy-Based Verifier (heads-only, hard-negative trained)

Honest summary. This is the best heads-only EBRM verifier we trained (Qwen3-4B encoder frozen, 46M trainable parameters across pooler / projector / cross-attention energy head). It was trained with hard-negative mining (6,273 contrastive pairs) on top of ebrm-v2-qwen3-4b.

It does not beat self-consistency majority voting at k=6. It is published as a research artifact and as a transfer-learning starting point — not as a drop-in runtime verifier that improves accuracy over majority voting.

What it is

A lightweight energy-based reward model that scores (problem, candidate solution) pairs. Encoder is a frozen 4-bit Qwen3-4B. Trainable parameters:

Module	Params
WeightedPooler (4 heads)	8
GatedProjector (hidden → 768)	12
CrossAttentionEnergy	38
Total trainable	46M

Architecture is identical to v2/v3; only the heads were further trained on hard-negative pairs. Files:

• ebrm_inference.pt — flat-key state dict, 174 MB, step 400 (early-stopped on held-out BBH eval).

Benchmarks

Head-to-head against ebrm-v3-general (previous release) and the single-shot / majority-voting baselines. Selection rule: pick the candidate with the lowest energy.

Bench	n	k	Single-shot	Majority @k	v3 best-of-k	v4 best-of-k
GSM8K	150	6	0.933	0.947	0.940	0.933
BBH (4 tasks)	182	6	0.923	0.962	0.923	0.951

BBH tasks: logical_deduction_five_objects, date_understanding, navigate, reasoning_about_colored_objects. causal_judgement excluded due to upstream content-filter pollution in our generation pipeline.

Verifier-only accuracy on the held-out BBH eval set (n=33, ranking the gold solution above hard negatives): 0.7576.

What changed vs v3

• v4 beats v3 by +2.8 pp on BBH best-of-6 (0.951 vs 0.923) — real signal from hard-negative mining.
• v4 ties v3 / single-shot on GSM8K best-of-6 (0.933).
• v4 still loses to majority voting on both benchmarks (−1.4 pp on GSM8K, −1.1 pp on BBH at k=6).

Intended use

• ✅ Transfer-learning starting point for further EBRM training (e.g. domain fine-tuning, larger encoders, joint encoder+head training).
• ✅ Research baseline for "small frozen-encoder reward heads vs. self-consistency" comparisons.
• ❌ Not recommended as a runtime verifier when majority voting is feasible.

Known limitations

• Architectural ceiling: 46M heads on a frozen 4B encoder cannot recover the signal that majority voting extracts from k=6 samples on closed-form QA.
• Trained on math/general-reasoning hard negatives; out-of-distribution performance is not characterized.
• No safety / bias evaluation.

Loading

from ebrm_system.verifiers import EBRMScorer

scorer = EBRMScorer.from_pretrained(
    repo_id="piyushptiwari/ebrm-v4-qwen3-4b",
    checkpoint_file="ebrm_inference.pt",
)
energy = scorer.score(problem="...", solution="...")

Requires ebrm-system >= 0.31.0.

Training recipe (reproducible)

• Warm-start from ebrm-v2-qwen3-4b (58 tensors loaded; nested-dict state unwrapped at load time).
• Pairwise hinge loss on (gold, hard-negative) energies, margin 1.0.
• Energy clamp ±50, grad-clip 0.5, lr 3e-5, NaN-skip guard.
• Dataset: 6,273 hard-neg pairs mined from GSM8K + BBH-style traces.
• Best checkpoint: step 400 (early-stopped on held-out verifier accuracy).

Citation

@misc{ebrm_v4_2026,
  author       = {Piyush Tiwari},
  title        = {EBRM v4: Hard-negative trained energy verifier on Qwen3-4B},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/piyushptiwari/ebrm-v4-qwen3-4b}},
}

License

Apache-2.0. Base encoder (Qwen/Qwen3-4B) is subject to its own license.