--- title: DocAgent Backend emoji: πŸ“„ colorFrom: blue colorTo: indigo sdk: docker app_port: 8000 pinned: false --- # DocAgent β€” Document Processing AI Agent > **Hugging Face Space note:** this Space runs the **FastAPI backend** (Docker). > The UI is deployed separately on Vercel and talks to this Space's URL. Health > check: `GET /api/health`. On the free tier there is no persistent disk, so > uploads/DB reset on restart. See [`docs/DEPLOY.md`](docs/DEPLOY.md). A production-grade, **chat-based document processing agent** built for Monkhub Innovations. Upload any document β†’ it parses, classifies, extracts structured data, summarizes, answers grounded questions, flags anomalies, and exports clean data. > **40 / 60 UI:** a persistent streaming chat (40%) beside a functional workspace > (60%) β€” document viewer with bounding-box citation highlights, extracted fields & > tables, classification, summary, and export. Built on the findings in [`docs/RESEARCH.md`](docs/RESEARCH.md) β€” a composed stack: **Docling** (local extraction) Β· **Gemini Flash** (free default LLM) ⇄ **GPT-4o-mini** (fallback) Β· agentic-RAG orchestration Β· **FastAPI** + **Next.js**. > πŸ“˜ **New here? Read [`docs/OVERVIEW.md`](docs/OVERVIEW.md)** β€” one document covering the > research, what we chose and why, the comparison table, the feature/function set, and > exactly how the agent works (architecture + both flows + a walkthrough). --- ## ✨ Features (the 7 core functions) | Function | What it does | |---|---| | **Ingest** | PDF, DOCX, PPTX, XLSX, HTML, images β€” parsed with Docling (layout, tables, OCR) | | **Extract** | Schema-driven fields, tables, key-value pairs, entities β€” each with a confidence score | | **Classify** | Zero-shot document type detection with confidence + candidates | | **Summarize** | Map-reduce summary (TL;DR + key points) | | **Q&A** | Conversational, agentic-RAG answers **grounded with citations** you can click to highlight on the page | | **Flag anomalies** | Missing required fields, low-confidence values, arithmetic checks, LLM consistency review | | **Export** | JSON Β· CSV Β· Excel | --- ## πŸ—οΈ Architecture See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the full diagram and rationale. ``` Next.js (40/60 UI) ──REST + SSE──▢ FastAPI β”œβ”€ agent/ orchestrator (tool-calling loop) β”œβ”€ llm/ Gemini ⇄ OpenAI provider abstraction β”œβ”€ services/ ingestion(Docling) Β· extraction Β· β”‚ classify Β· summary Β· qa(RAG) Β· β”‚ anomaly Β· export Β· vectorstore Β· storage └─ schemas/ Pydantic contracts + extraction templates Docling (local) Β· ChromaDB (embedded) Β· SQLite ``` **Design highlights** - **Single orchestrator agent** with 5 tools (query, classify, get-extracted, summarize, flag) β€” the simplest design that covers the brief, per the research. - **Provider abstraction**: nothing imports a vendor SDK except `llm/*`. Switch Gemini⇄OpenAI by env, per-request, or in the UI. - **Provenance-aware chunks**: every chunk carries page + normalised bbox, so RAG answers and extracted fields highlight their exact source. - **Local-first & ~zero marginal cost**: Docling runs locally; Gemini free tier; ChromaDB + SQLite need no external services. --- ## πŸš€ Quick start ### Prerequisites - **Python 3.10–3.12** (3.13 works for the API; some ML wheels lag β€” see notes) - **Node.js 18+** - A **free Gemini API key** β†’ https://aistudio.google.com/apikey ### 1. Backend ```bash cd backend python -m venv .venv # Windows: .venv\Scripts\activate # macOS/Linux: source .venv/bin/activate pip install -r requirements.txt cp .env.example .env # then put your GEMINI_API_KEY in .env uvicorn app.main:app --reload --port 8000 ``` API now at http://localhost:8000 β€” interactive docs at http://localhost:8000/docs. ### 2. Frontend ```bash cd frontend npm install cp .env.local.example .env.local # default points at :8000 npm run dev ``` App now at **http://localhost:3000**. --- ## πŸ”§ Configuration (`backend/.env`) | Var | Default | Notes | |---|---|---| | `GEMINI_API_KEY` | β€” | **Required.** Free tier. | | `OPENAI_API_KEY` | β€” | Optional fallback (GPT-4o-mini). | | `DEFAULT_LLM_PROVIDER` | `gemini` | `gemini` or `openai`. | | `GEMINI_MODEL` | `gemini-2.0-flash` | | | `OPENAI_MODEL` | `gpt-4o-mini` | | | `ENABLE_OCR` | `true` | Docling OCR for scanned docs. | The app runs with **only** `GEMINI_API_KEY`. If both keys are present you can switch providers live from the header toggle. --- ## 🧭 How to use 1. Drag a document into the left rail (or click **New document**). 2. Watch it process β€” pages render, then classification, fields, summary, and anomaly flags stream into the workspace tabs. 3. **Chat** on the left: *"Summarize this"*, *"What's the total?"*, *"Any missing fields?"* Click any **citation chip** to highlight its source on the page. 4. **Export** from the Export tab (JSON / CSV / Excel). --- ## πŸ“ Project layout ``` Document-agent/ β”œβ”€ docs/ β”‚ β”œβ”€ RESEARCH.md # deep research: 5 systems, scoring, winner, SOTA, UX β”‚ └─ ARCHITECTURE.md # architecture decisions + diagram β”œβ”€ backend/ β”‚ β”œβ”€ app/ β”‚ β”‚ β”œβ”€ api/ # documents, chat(SSE), actions, meta β”‚ β”‚ β”œβ”€ agent/ # orchestrator, tools, prompts β”‚ β”‚ β”œβ”€ llm/ # base, gemini, openai, registry β”‚ β”‚ β”œβ”€ services/ # ingestion, extraction, classify, summary, qa, anomaly, export, storage, vectorstore, pipeline β”‚ β”‚ β”œβ”€ schemas/ # documents + extraction templates β”‚ β”‚ └─ core/ # config, logging β”‚ └─ requirements.txt └─ frontend/ β”œβ”€ app/ # layout, page (40/60 shell) β”œβ”€ components/ # Chat, Workspace, Viewer, Fields, Classify, Summary, Export, Sidebar └─ lib/ # api client (+SSE), types ``` --- ## πŸ“ Notes & next steps - **Python 3.13:** the API layer runs fine; if a Docling/Torch wheel isn't yet published for 3.13 on your platform, use a 3.11/3.12 venv for the backend. - Designed to scale: swap SQLiteβ†’Postgres and ChromaDBβ†’pgvector by editing only `services/storage.py` and `services/vectorstore.py`. - Auth, rate limiting, and multi-tenant workspaces are natural follow-ons.