Hugging Face's logo

Spaces:
build-small-hackathon
/
kirana-detective
Sleeping

App Files Community
kirana-detective / README.md
naazimsnh02's picture
naazimsnh02
Update Readme.md
9b7c4b9 verified
preview code
|
Raw
History Blame Contribute Delete
13.8 kB

A newer version of the Gradio SDK is available: 6.19.0

Upgrade
metadata 
sdk: gradio
sdk_version: 6.16.0
app_file: app.py
title: Kirana Detective
short_description: AI invoice auditor for Indian kirana stores
license: mit
tags:
  - invoice-audit
  - llm
  - yolo
  - gguf
  - gradio
  - indian-fmcg
  - kirana
  - minicpm
  - multimodal
  - backyard-ai
  - local-first
  - fine-tuned
  - custom-ui
  - llama.cpp
  - open-trace
  - blog-post
  - openbmb
  - modal.com
  - track:backyard
  - sponsor:openbmb
  - sponsor:modal
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:llama
  - achievement:sharing
  - achievement:fieldnotes
colorFrom: purple
colorTo: pink
pinned: true
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/639548cc276ff8643fab34ac/p5tj50-FcFouVAlbUk8KJ.png

🔍 Kirana Detective

AI-Powered Invoice & Inventory Auditor for Indian Kirana Stores

Find where money is being lost — in under 60 seconds

Try It Live Watch Demo Blog Post X Post License: MIT Python 3.11 Training: Modal Models: OpenBMB Hackathon: Build Small 2026

What It Does

Indian kirana store owners receive 3–5 distributor invoices every week via WhatsApp, printed bills, or Tally exports. Verifying them manually is impossible.

Kirana Detective uploads an invoice + delivery photos and finds:

Finding Example
Price overcharge Surf Excel 1kg charged ₹255 — historical price ₹220 (+15.9%)
Delivery shortage Invoice says 24 Coke bottles — photo shows 20
Duplicate charge Parle-G 80g appears twice on the same invoice
GST mismatch Aashirvaad Atta billed at 12% instead of 5%

Every finding converts to a rupee leakage number with an actionable follow-up step.

Demo 

Upload Invoice (photo/PDF/WhatsApp) + Delivery Photos (up to 5)
        ↓
Agent 1 — MiniCPM-V 4.6 extracts structured JSON from invoice image
Agent 2 — MiniCPM5-1B normalises "SURF XL 1K" → "Surf Excel Washing Powder 1kg"
Agent 3 — Rule engine checks price vs. stored invoice history
Agent 4 — YOLO26n counts products in delivery photos
Agent 5 — Reconciliation: invoice qty vs. counted qty → shortage flags
Agent 6 — MiniCPM5-1B generates rupee savings report + action items
        ↓
₹ TOTAL LEAKAGE DETECTED: ₹858

Six-Agent Pipeline 

┌──────────────────────────────────┐
│  Agent 1 — Invoice Extractor     │  MiniCPM-V 4.6 (merged, bfloat16)
│  Invoice image/PDF → JSON        │  OCR + structured field extraction
└──────────────┬───────────────────┘
               ↓
┌──────────────────────────────────┐
│  Agent 2 — Product Matcher       │  MiniCPM5-1B (GGUF Q4_K_M)
│  Raw names → canonical SKU IDs   │  "MAGGI NDL" → Nestle Maggi 70g
└──────────────┬───────────────────┘
               ↓
┌──────────────────────────────────┐
│  Agent 3 — Pricing Agent         │  Rule-based (SQLite history)
│  Normalized invoice → price flags│  Detects overcharges & GST errors
└──────────────┬───────────────────┘
               ↓
┌──────────────────────────────────┐
│  Agent 4 — Visual Counter        │  YOLO26n (ONNX, 1,831 classes)
│  Delivery photos → product counts│  mAP50 = 0.428 on merged dataset
└──────────────┬───────────────────┘
               ↓
┌──────────────────────────────────┐
│  Agent 5 — Reconciliation Agent  │  Rule-based
│  Invoice qty vs. counted qty     │  Shortage flags + ₹ loss
└──────────────┬───────────────────┘
               ↓
┌──────────────────────────────────┐
│  Agent 6 — Savings Agent         │  MiniCPM5-1B (GGUF Q4_K_M)
│  All flags → ₹ report + actions  │  "Call HUL rep. Request credit note."
└──────────────────────────────────┘

Every agent run is stored locally in SQLite for audit history.

Fine-Tuned Models

All three models were trained from scratch on Modal A10G GPUs and published to HuggingFace. Total training cost: ~$5.80.

Model 1 — MiniCPM-V 4.6 (Invoice Extractor)

Repo build-small-hackathon/minicpm-v-4-6-indian-invoice-extraction-merged
Base openbmb/MiniCPM-V-4.6
Method QLoRA rank 16 (PEFT + bitsandbytes), then merged to full bfloat16 weights
Data 500 synthetic Indian invoices — printed GST, Tally PDF, handwritten, WhatsApp
Eval loss 0.212 (epoch 3 / 3)
Training 51 min 50 sec on A10G · 87 steps · 9.5M trainable params (0.72%)
Inference transformers AutoModel · model.chat() — no PEFT at runtime

Model 2 — MiniCPM5-1B (Product Normalizer)

Repo build-small-hackathon/minicpm5-1b-indian-fmcg-normalizer
Base openbmb/MiniCPM5-1B
Method QLoRA rank 16 via Unsloth, exported to GGUF Q4_K_M
Data 2,000 synthetic (raw_name → canonical_name) pairs · 200 Indian FMCG SKUs
Training ~1 hour on A10G
Inference llama-cpp-python · create_chat_completion()

Model 3 — YOLO26n (Product Detector)

Repo build-small-hackathon/yolo26n-indian-fmcg-detection
Base Ultralytics YOLO26n
Method Supervised fine-tuning on 3 merged Roboflow datasets
Data ~11,400 images · 1,831 unified classes
Metrics mAP50 = 0.428 · mAP50-95 = 0.302 · 100 epochs (A10G)
Inference ONNX Runtime · CPU or GPU

Training Dataset

Repo build-small-hackathon/kirana-invoice-train-data
Contents 500 synthetic invoice images (450 train / 50 eval) with structured JSON annotations

Running Locally 

git clone https://github.com/naazimsnh02/kirana-detective.git
cd kirana-detective
pip install -r requirements.txt
python app.py

First run: downloads ~3 GB of model weights (cached after that).
Requirements: ~6 GB RAM · Python 3.11 · optional CUDA GPU for faster MiniCPM-V inference.

Environment Variables

Variable Required Purpose
HF_TOKEN Optional Faster downloads from HF Hub (avoids rate limits)

Re-Training the Models

All training scripts are in finetune/. Training is orchestrated on Modal.

export HF_TOKEN=<your-token>
export ROBOFLOW_API_KEY=<your-key>

modal run finetune/generate_invoices.py        # ~10 min — generate 500 synthetic invoices
modal run finetune/train_minicpm_v.py          # ~52 min — fine-tune invoice extractor
modal run finetune/export_minicpm_v_gguf.py   # ~10 min — merge LoRA → push HF weights
modal run finetune/train_minicpm5_1b.py        # ~1 hour — fine-tune product normalizer
modal run finetune/train_yolo26n.py            # ~2 hours — fine-tune YOLO26n detector

Scripts publish to naazimsnh02/ first; transfer to build-small-hackathon/ manually after.
See finetune/README.md for the full workflow.

Model Architecture

Component Model Parameters Runtime
Invoice OCR & extraction MiniCPM-V 4.6 (merged) 1.3B transformers
Product normalisation + report MiniCPM5-1B (GGUF Q4_K_M) 1.08B llama-cpp-python
Product detection & counting YOLO26n (ONNX) ~2.4M onnxruntime
Total — ~2.38B —

Comfortably within the Tiny Titan ≤4B threshold. Zero cloud API calls — fully local inference.

Hackathon Badges

Badge How
🎯 Well-Tuned 3 custom fine-tuned models published on HF Hub
🔌 Off the Grid 100% local inference — MiniCPM-V (transformers) + MiniCPM5-1B (GGUF) + YOLO26n (ONNX)
🦙 Llama Champion MiniCPM5-1B served via llama-cpp-python (GGUF Q4_K_M)
🎨 Off-Brand Custom Gradio UI — rupee savings cards, colour-coded anomaly flags
📡 Sharing is Caring Claude Code build sessions (11 JSONL sessions) published as a public trace dataset
📓 Field Notes "How I built an AI auditor for India's 12 million kirana stores"
🏋️ Tiny Titan ~2.38B total parameters — OCR + normalization + counting + report

Sharing is Caring — Build Trace Dataset

The 11 raw Claude Code (Sonnet 4.6) JSONL sessions used to design, code, debug, and document this entire project — from blank repo to hackathon submission. Viewable in HF Data Studio's native agent trace viewer.

Dataset Contents Format
build-small-hackathon/kirana-detective-build-traces 11 Claude Code build sessions · ~8.9 MB Native JSONL trace viewer

To upload build traces:

export HF_TOKEN=<your-token>
python finetune/upload_build_traces.py

Project Structure 

kirana-detective/
├── app.py                      # Gradio + FastAPI server
├── pipeline.py                 # AuditOrchestrator (6-agent runner)
├── models.py                   # Dataclasses: InvoiceJSON, LeakageReport, etc.
├── catalog.py                  # FMCG product catalog + alias lookup
├── storage.py                  # SQLite price history + audit log
├── tracer.py                   # Agent trace logging → HF Hub
├── agents/
│   ├── invoice_extractor.py    # Agent 1 — MiniCPM-V 4.6
│   ├── product_matcher.py      # Agent 2 — MiniCPM5-1B (alias + LLM)
│   ├── pricing_agent.py        # Agent 3 — rule-based price checks
│   ├── visual_counter.py       # Agent 4 — YOLO26n ONNX
│   ├── reconciliation_agent.py # Agent 5 — invoice vs. photo reconciliation
│   └── savings_agent.py        # Agent 6 — MiniCPM5-1B report generator
├── finetune/
│   ├── README.md               # Training workflow guide
│   ├── generate_invoices.py    # Synthetic invoice generator
│   ├── train_minicpm_v.py      # Fine-tune MiniCPM-V
│   ├── train_minicpm5_1b.py    # Fine-tune MiniCPM5-1B
│   ├── train_yolo26n.py        # Fine-tune YOLO26n
│   ├── export_minicpm_v_gguf.py# Merge LoRA → push HF weights
│   └── upload_build_traces.py  # Upload Claude Code sessions → HF Hub
├── data/
│   └── fmcg_catalog.json       # 200 canonical SKU names + GST rates
└── MODEL_CARD.md               # Full training + evaluation documentation

Links

License

  • Code: MIT
  • MiniCPM-V / MiniCPM5-1B: Apache 2.0 (OpenBMB)
  • YOLO26n: AGPL-3.0 (Ultralytics)

HuggingFace Build Small Hackathon 2026 · Track 1: Backyard AI · naazimsnh02