# Architecture — Document Processing AI Agent

**Decision record from research (see `RESEARCH.md`). Stack chosen: custom composed stack.**

## High-level

```
┌──────────────────────────────────────────────────────────────┐
│  FRONTEND  (Next.js + React + TypeScript + Tailwind)          │
│  40% Chat  │  60% Workspace                                   │
│  ────────  │  ── Tabs: Viewer · Fields · Classify · Summary · │
│  streaming │     Export ──                                    │
│  agent     │  Document viewer w/ bounding-box highlights      │
└─────┬────────────────────────────────────────────────────────┘
      │  REST + SSE (streaming)
┌─────▼────────────────────────────────────────────────────────┐
│  BACKEND  (FastAPI, async)                                    │
│                                                              │
│  api/        routes: documents, chat(SSE), extract,          │
│              classify, summary, export, health               │
│  agent/      orchestrator (tool-calling loop) + prompts      │
│  llm/        provider abstraction: Gemini(default)⇄OpenAI     │
│  services/   ingestion(Docling) · extraction · classification│
│              · summary · qa(RAG) · anomaly · export ·         │
│              vectorstore · storage                           │
│  schemas/    Pydantic models (incl. invoice/contract)        │
│  core/       config · logging · deps                         │
└─────┬───────────────────────────┬────────────────────────────┘
      │                           │
┌─────▼─────┐            ┌─────────▼──────────┐
│ Docling   │            │ Storage            │
│ (local    │            │  • files (disk/S3) │
│  extract) │            │  • SQLite/Postgres │
└───────────┘            │  • vectors (chroma/│
                         │    pgvector)       │
                         └────────────────────┘
```

## The 7 core functions → implementation

| Function | How | Tool exposed to agent |
|---|---|---|
| 1. Upload & ingest | `POST /documents` → Docling parse → DoclingDocument JSON + page images | `ingest_document` |
| 2. Parse & extract structured data | Docling structure + LLM structured-output against Pydantic schema (fields, tables, KV, entities) + confidence | `extract_fields` |
| 3. Classify | LLM zero-shot classification → type + confidence | `classify_document` |
| 4. Summarize | Map-reduce summarization over chunks | `summarize_document` |
| 5. Conversational Q&A | Agentic RAG: retrieve chunks → answer **with citations** (page + bbox) | `query_document` |
| 6. Flag anomalies | Rule + LLM checks: missing required fields, inconsistencies, low-confidence | `flag_anomalies` |
| 7. Export | Serialize extracted data → JSON / CSV / Excel | `export_data` |

## Agent design
Single **orchestrator agent** with the tools above (tool-calling loop), agentic-RAG retrieval, structured outputs (Pydantic), human-in-the-loop on low-confidence fields. Pattern mirrors LlamaIndex ADW: *parse → maintain state → retrieve → reason → surface for validation*.

## LLM strategy
- **Default (free):** Gemini Flash (1M context, native vision for scans).
- **Fallback (switchable):** GPT-4o-mini.
- `llm/base.py` defines a `LLMProvider` interface; runtime switch via config/env or per-request header. Cheap model for classify, stronger for reasoning.

## Tech choices (pragmatic defaults for a runnable, impressive build)
- **Backend:** FastAPI, Pydantic v2, Docling, `google-generativeai`, `openai`, ChromaDB (embedded vector store — no external service), SQLite (dev) with a path to Postgres/pgvector.
- **Frontend:** Next.js (App Router) + React + TypeScript + Tailwind + a PDF/image viewer with overlay layer for bounding boxes; SSE for streaming chat.
- **Local-first:** runs with only a free Gemini API key. No paid services required.

## Separation of concerns
Routes are thin → call services. Agent orchestrates services as tools. LLM access only via `llm/`. Storage only via `services/storage` + `services/vectorstore`. Schemas shared. This is the Dify-grade discipline we borrowed.