ShyFoo/CountHallu-quality_cls_model-RealHand

CountHallu — RealHand Quality Classifier

RealHand quality classifier from Counting Hallucinations in Diffusion Models (arXiv:2510.13080). It is the first stage of the RealHand evaluation pipeline: it decides whether a generated hand image is clean enough to be counted, filtering out visually failed images before the finger detector runs. Without this gate, malformed images would be miscounted rather than flagged as visual failures. Therefore, you need this reproduce the non-counting failure rates (NCFR) in the RealHand dataset.

Architecture & checkpoint

• A MaxViT binary classifier (maxvit_small_tf_224 from timm, ImageNet init).
• Two classes; index 1 = clean / countable, index 0 = failed. A softmax probability p(class 1) ≥ 0.5 marks the image as good.
• Ships a single model.pth (a plain state_dict saved from the bare timm model — keep that format when re-saving).

Usage

See the CountHallu repository for the full evaluation protocol.

Inputs are RGB images resized to 224 and normalised with ImageNet statistics (Resize(224), ToTensor, Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])).

import timm, torch
from huggingface_hub import hf_hub_download

ckpt = hf_hub_download("ShyFoo/CountHallu-quality_cls_model-RealHand", "model.pth")
model = timm.create_model("maxvit_small_tf_224", pretrained=False, num_classes=2)
model.load_state_dict(torch.load(ckpt, map_location="cpu"))
model.eval()

# is_clean = torch.softmax(model(x), dim=1)[:, 1] >= 0.5

Or let the evaluation protocol fetch it for you:

from counthallu.utils import load_quality_cls_model
model = load_quality_cls_model(
    "realhand", use_hub_model=True,
    repo_id="ShyFoo/CountHallu-quality_cls_model-RealHand"
)

Citation

@article{fu2025counting,
  title={Counting Hallucinations in Diffusion Models},
  author={Fu, Shuai and Zhou, Jian and Chen, Qi and Jing, Huang and Nguyen, Huy Anh and Liu, Xiaohan and Zeng, Zhixiong and Ma, Lin and Zhang, Quanshi and Wu, Qi},
  journal={arXiv preprint arXiv:2510.13080},
  year={2025}
}