README.md

metadata

title: DocAgent Backend
emoji: 📄
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 8000
pinned: false

DocAgent — Document Processing AI Agent

Hugging Face Space note: this Space runs the FastAPI backend (Docker). The UI is deployed separately on Vercel and talks to this Space's URL. Health check: GET /api/health. On the free tier there is no persistent disk, so uploads/DB reset on restart. See docs/DEPLOY.md.

A production-grade, chat-based document processing agent built for Monkhub Innovations. Upload any document → it parses, classifies, extracts structured data, summarizes, answers grounded questions, flags anomalies, and exports clean data.

40 / 60 UI: a persistent streaming chat (40%) beside a functional workspace (60%) — document viewer with bounding-box citation highlights, extracted fields & tables, classification, summary, and export.

Built on the findings in docs/RESEARCH.md — a composed stack: Docling (local extraction) · Gemini Flash (free default LLM) ⇄ GPT-4o-mini (fallback) · agentic-RAG orchestration · FastAPI + Next.js.

📘 New here? Read docs/OVERVIEW.md — one document covering the research, what we chose and why, the comparison table, the feature/function set, and exactly how the agent works (architecture + both flows + a walkthrough).

---

✨ Features (the 7 core functions)

Function	What it does
Ingest	PDF, DOCX, PPTX, XLSX, HTML, images — parsed with Docling (layout, tables, OCR)
Extract	Schema-driven fields, tables, key-value pairs, entities — each with a confidence score
Classify	Zero-shot document type detection with confidence + candidates
Summarize	Map-reduce summary (TL;DR + key points)
Q&A	Conversational, agentic-RAG answers grounded with citations you can click to highlight on the page
Flag anomalies	Missing required fields, low-confidence values, arithmetic checks, LLM consistency review
Export	JSON · CSV · Excel

---

🏗️ Architecture

See docs/ARCHITECTURE.md for the full diagram and rationale.

Next.js (40/60 UI)  ──REST + SSE──▶  FastAPI
                                      ├─ agent/      orchestrator (tool-calling loop)
                                      ├─ llm/        Gemini ⇄ OpenAI provider abstraction
                                      ├─ services/   ingestion(Docling) · extraction ·
                                      │              classify · summary · qa(RAG) ·
                                      │              anomaly · export · vectorstore · storage
                                      └─ schemas/    Pydantic contracts + extraction templates
                                   Docling (local) · ChromaDB (embedded) · SQLite

Design highlights

• Single orchestrator agent with 5 tools (query, classify, get-extracted, summarize, flag) — the simplest design that covers the brief, per the research.
• Provider abstraction: nothing imports a vendor SDK except llm/*. Switch Gemini⇄OpenAI by env, per-request, or in the UI.
• Provenance-aware chunks: every chunk carries page + normalised bbox, so RAG answers and extracted fields highlight their exact source.
• Local-first & ~zero marginal cost: Docling runs locally; Gemini free tier; ChromaDB + SQLite need no external services.

---

🚀 Quick start

Prerequisites

• Python 3.10–3.12 (3.13 works for the API; some ML wheels lag — see notes)
• Node.js 18+
• A free Gemini API key → https://aistudio.google.com/apikey

1. Backend

cd backend
python -m venv .venv
# Windows:  .venv\Scripts\activate
# macOS/Linux:  source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env          # then put your GEMINI_API_KEY in .env
uvicorn app.main:app --reload --port 8000

API now at http://localhost:8000 — interactive docs at http://localhost:8000/docs.

2. Frontend

cd frontend
npm install
cp .env.local.example .env.local   # default points at :8000
npm run dev

App now at http://localhost:3000.

---

🔧 Configuration (backend/.env)

Var	Default	Notes
GEMINI_API_KEY	—	Required. Free tier.
OPENAI_API_KEY	—	Optional fallback (GPT-4o-mini).
DEFAULT_LLM_PROVIDER	gemini	gemini or openai.
GEMINI_MODEL	gemini-2.0-flash
OPENAI_MODEL	gpt-4o-mini
ENABLE_OCR	true	Docling OCR for scanned docs.

The app runs with only GEMINI_API_KEY. If both keys are present you can switch providers live from the header toggle.

---

🧭 How to use

1. Drag a document into the left rail (or click New document).
2. Watch it process — pages render, then classification, fields, summary, and anomaly flags stream into the workspace tabs.
3. Chat on the left: "Summarize this", "What's the total?", "Any missing fields?" Click any citation chip to highlight its source on the page.
4. Export from the Export tab (JSON / CSV / Excel).

---

📁 Project layout

Document-agent/
├─ docs/
│  ├─ RESEARCH.md        # deep research: 5 systems, scoring, winner, SOTA, UX
│  └─ ARCHITECTURE.md    # architecture decisions + diagram
├─ backend/
│  ├─ app/
│  │  ├─ api/            # documents, chat(SSE), actions, meta
│  │  ├─ agent/          # orchestrator, tools, prompts
│  │  ├─ llm/            # base, gemini, openai, registry
│  │  ├─ services/       # ingestion, extraction, classify, summary, qa, anomaly, export, storage, vectorstore, pipeline
│  │  ├─ schemas/        # documents + extraction templates
│  │  └─ core/           # config, logging
│  └─ requirements.txt
└─ frontend/
   ├─ app/               # layout, page (40/60 shell)
   ├─ components/        # Chat, Workspace, Viewer, Fields, Classify, Summary, Export, Sidebar
   └─ lib/               # api client (+SSE), types

---

📝 Notes & next steps

• Python 3.13: the API layer runs fine; if a Docling/Torch wheel isn't yet published for 3.13 on your platform, use a 3.11/3.12 venv for the backend.
• Designed to scale: swap SQLite→Postgres and ChromaDB→pgvector by editing only services/storage.py and services/vectorstore.py.
• Auth, rate limiting, and multi-tenant workspaces are natural follow-ons.