Spaces:

hetanshwaghela
/

courtmitra

Sleeping

App Files Files Community

courtmitra / HANDOFF.md

Hetansh Waghela

handoff doc

ad76a88 about 2 months ago

preview code

Raw

History Blame Contribute Delete

19.4 kB

CourtMitra MissionOS — Handoff Document

1. What This Is

CourtMitra is a citizen-side grievance/legal-action autopilot for India. A user describes a problem once (WhatsApp / Telegram / browser); the system turns it into a mission, builds the evidence file, grounds it in Indian law/procedure, picks an official route, drafts + sends approved actions, follows up, and records the outcome.

Architecture

apps/
  web/         → Next.js App Router (port 3000)
  api/         → NestJS + Fastify (port 3001)
  worker/      → Inngest functions (port 3002)
packages/
  domain/      → Pure business logic (no framework imports)
  ports/       → Interfaces only (hexagonal boundary)
  adapters/    → Concrete implementations (LLM, email, channels, etc.)
  db/          → Drizzle schema, migrations, seeds
  contracts/   → Zod schemas (single source of truth)
  prompts/     → Versioned prompt templates
  ui/          → Shared React components
infra/
  docker-compose.yml  → Postgres, Inngest dev server, API, Worker, Web
  Dockerfile.*        → Multi-stage builds

How It Works

Intake: User sends a message via WhatsApp/Telegram/browser → system extracts structured facts via LLM
Evidence: User uploads screenshots/PDFs/audio → vision-LLM or ASR extracts data
Facts: User confirms extracted facts
Legal grounding: pgvector + FTS retrieves relevant Indian law citations
Route plan: LLM generates a step-by-step plan grounded in law
Action preview + consent: System drafts an action, user approves with hash-bound consent
Execution: Resend sends email, portal fills forms, etc.
Follow-up: System tracks replies, schedules reminders
Resolution: Outcome recorded, anonymized ledger entry

2. Environment Variables — Complete Reference

Copy infra/.env.example to infra/.env and fill in the values below.

Required (system won't start without these)

Variable	Where to Get	Notes
`DATABASE_URL`	Auto-set by Docker compose	`postgres://courtmitra:courtmitra@localhost:5432/courtmitra` for local dev
`DIRECT_DATABASE_URL`	Same as DATABASE_URL	Used by Drizzle for direct connections
`AI_PROVIDER`	Choose one	`openrouter` (default) or `opencode`
`OPENROUTER_API_KEY`	https://openrouter.ai/keys	Free tier available. Used for LLM calls.
`OPENAI_API_KEY`	https://platform.openai.com/api-keys	Required for Whisper ASR (voice notes). Free tier: $5 credit.

Required for Email Sending

Variable	Where to Get	Notes
`RESEND_API_KEY`	https://resend.com/api-keys	Free tier: 100 emails/day, 3000/month
`RESEND_FROM`	Set in Resend dashboard	e.g. `noreply@yourdomain.com`
`RESEND_REPLY_DOMAIN`	Your domain	e.g. `reply.yourdomain.com` — used for reply-token threading

Required for Vector Search

Variable	Where to Get	Notes
`QDRANT_URL`	https://cloud.qdrant.io	Free tier: 1GB storage, 1 cluster
`QDRANT_API_KEY`	Qdrant Cloud dashboard	Created when you set up the cluster
`QDRANT_COLLECTION`	Just set	`courtmitra_legal` (default)

Optional — WhatsApp

Variable	Where to Get	Notes
`WHATSAPP_VERIFY_TOKEN`	You create this	Any random string for webhook verification
`WHATSAPP_APP_SECRET`	Meta Developer Portal	For HMAC signature verification
`WHATSAPP_ACCESS_TOKEN`	Meta Developer Portal	Permanent token from App Dashboard
`WHATSAPP_PHONE_NUMBER_ID`	Meta Developer Portal	Found in WhatsApp > API Setup
`WHATSAPP_BUSINESS_ACCOUNT_ID`	Meta Developer Portal	Found in WhatsApp > API Setup

Optional — Telegram

Variable	Where to Get	Notes
`TELEGRAM_BOT_TOKEN`	@BotFather on Telegram	`/newbot` command

Optional — Portal Automation

Variable	Where to Get	Notes
`BROWSERBASE_API_KEY`	https://www.browserbase.com	Free tier: 5 hours/month
`BROWSERBASE_PROJECT_ID`	Browserbase dashboard	Created when you sign up
`ENABLE_PORTAL_AUTOMATION`	Set to `1`	Without this, portal assist degrades to manual packet

Optional — eCourts CNR Lookup

Variable	Where to Get	Notes
`SUREPASS_API_KEY`	https://surepass.io	Paid API for Indian court case data

Optional — Observability

Variable	Where to Get	Notes
`LANGFUSE_PUBLIC_KEY`	https://langfuse.com	Free tier available
`LANGFUSE_SECRET_KEY`	Langfuse dashboard

Defaults (usually fine as-is)

Variable	Default	Notes
`APP_BASE_URL`	`http://localhost:3000`	Web app URL
`API_BASE_URL`	`http://localhost:3001`	API URL
`WEBHOOK_BASE_URL`	`http://localhost:3001`	For Resend/WhatsApp webhook callbacks
`LOCAL_STORAGE_DIR`	`.storage`	File storage path (relative to repo root)
`INNGEST_BASE_URL`	`http://localhost:8288`	Inngest dev server
`INNGEST_DEV`	`1`	Enable Inngest dev mode
`PII_HASH_SALT`	`dev-salt-change-me`	For PII hashing
`JWT_SECRET`	`dev-jwt-secret-change-me`	For session tokens
`AI_MODEL`	`google/gemini-2.0-flash-exp:free`	LLM model (free on OpenRouter)
`AI_VISION_MODEL`	`google/gemini-2.0-flash-exp:free`	Vision model for evidence parsing

3. Step-by-Step Setup

Prerequisites

Node.js >= 20
pnpm 10.20.0+ (corepack enable && corepack prepare pnpm@10 --activate)
Docker + Docker Compose
Git

Step 1: Clone and Install

git clone <repo-url> hackathon
cd hackathon
pnpm install

Step 2: Start Infrastructure

# Start Postgres + Inngest dev server
pnpm infra:up

# Wait for Postgres to be healthy (check with:)
docker ps
# Should show courtmitra_pg as "healthy"

Step 3: Set Up Environment

cp infra/.env.example infra/.env
# Edit infra/.env with your API keys (see table above)

Minimum viable .env for local dev:

DATABASE_URL=postgres://courtmitra:courtmitra@localhost:5432/courtmitra
DIRECT_DATABASE_URL=postgres://courtmitra:courtmitra@localhost:5432/courtmitra
AI_PROVIDER=openrouter
OPENROUTER_API_KEY=<your-key>
OPENAI_API_KEY=<your-key>
RESEND_API_KEY=<your-key>
RESEND_FROM=noreply@yourdomain.com
RESEND_REPLY_DOMAIN=reply.yourdomain.com
QDRANT_URL=<your-qdrant-cloud-url>
QDRANT_API_KEY=<your-qdrant-key>
QDRANT_COLLECTION=courtmitra_legal
LOCAL_STORAGE_DIR=.storage
INNGEST_BASE_URL=http://localhost:8288
INNGEST_DEV=1
PII_HASH_SALT=dev-salt-change-me
JWT_SECRET=dev-jwt-secret-change-me
AI_MODEL=google/gemini-2.0-flash-exp:free
AI_VISION_MODEL=google/gemini-2.0-flash-exp:free

Step 4: Run Database Migrations

pnpm db:migrate

This creates all 30+ tables including: users, missions, actions, evidence, legal_chunks, followups, social_posts, case_lookups, portal_assist_sessions, etc.

Step 5: Seed Legal Data + Qdrant

# Seed legal sources, chunks, and route rules into Postgres
pnpm db:seed

# Seed Qdrant with embeddings (requires OPENROUTER_API_KEY for embeddings)
npx tsx scripts/seed-qdrant.ts

Step 6: Start the Application

# Start all three services (API, Worker, Web)
pnpm dev

This starts:

Web on http://localhost:3000
API on http://localhost:3001
Worker on http://localhost:3002
Inngest Dev Server on http://localhost:8288 (from Docker)

Step 7: Verify Everything Works

# Run all checks
pnpm typecheck && pnpm lint && pnpm test && pnpm test:contracts

# Check API health
curl http://localhost:3001/api/admin/health

# Check capabilities (shows what's available/blocked)
curl http://localhost:3001/api/capabilities

4. How to Test the Product End-to-End

Test 1: Create a Mission from Browser

Open http://localhost:3000
Click "Create Mission"
Select issue domain (consumer/payment/cyber_fraud)
Enter title and description
Click create → mission appears on the board

Test 2: Send a Telegram Message (Easiest Channel Test)

Set TELEGRAM_BOT_TOKEN in .env
Restart the API
Send a message to your bot on Telegram
Check http://localhost:3000 — mission should appear
Check Inngest dev server at http://localhost:8288 — events should flow

Test 3: Full Loop (Email Send)

Create a mission from browser
Confirm facts (click "Confirm Facts" in the mission room)
Wait for route plan generation (check Inngest)
Preview an action → approve with consent
Execute → email sent via Resend
Check proof_state = sent_by_email
Reply to the email → reply ingested → mission advances

Test 4: WhatsApp (Requires Meta App)

Set WhatsApp env vars
Configure webhook URL in Meta Developer Portal: https://your-domain/webhooks/whatsapp
Set verify token to match WHATSAPP_VERIFY_TOKEN
Send a voice note or text message
Mission created, evidence parsed, facts extracted

Test 5: Voice Notes (Requires OPENAI_API_KEY)

Send a voice note via WhatsApp or Telegram
System downloads audio → transcribes via Whisper → includes transcript in intake
Check evidence_parsed_outputs for parserName = "whisper-asr"

5. What's Built (Phase 1 + Phase 2 Modules 1-8)

Phase 1 — Complete Automation Loop ✅

Component	Status
Mission CRUD from browser	✅
WhatsApp webhook + media download	✅
Telegram webhook + voice	✅
Intake (LLM extracts structured facts)	✅
Vision-LLM evidence parsing	✅
Whisper ASR transcription	✅
Fact confirmation UI	✅
Legal grounding (pgvector + FTS + RRF)	✅
Route plan generation with citations	✅
Claim verification (literal substring)	✅
Action preview + consent hash	✅
Defamation gate	✅
PII redaction (Aadhaar/PAN/UPI/IFSC/phone/email)	✅
Email send via Resend	✅
Reply-token threading	✅
Reply classification	✅
Follow-up scheduling	✅
Resolution capture	✅
Public case page (private link)	✅
DPDP data export + erase	✅
Capability registry	✅

Phase 2 — Modules 1-8 ✅

Module	What's Built
1. Complaint Swarm	Entity resolution, swarm clustering, dossier generation, opt-in with consent
2. ASR / Voice Notes	Whisper API transcription, WhatsApp audio download, Telegram voice, PII redaction
3. Indic Translation	LLM-based language detection + translation, preferredLanguage on users, localized_texts table
4. Hybrid RAG	Qdrant Cloud adapter (dense vectors + Postgres FTS + RRF), seed script
5. Portal Automation	Playwright adapter, portal_assist_sessions, consent gate, receipt capture, degradation
6. Company Resolver	Organizations, resolver threads, domain verification, masked case view
7. Social Publishing	Social accounts/posts tables, copy-ready fallback, social card image generator
8. eCourts / CNR	Surepass API adapter, case_lookups table, hearing reminder scheduling

Phase 2 — Modules 9-11 ❌ (Not Built)

Module	What's Needed
9. Intermediary Compliance	Takedown requests table, Grievance Officer contact, IT Rules 2021 timelines
10. AI Public Account	@CourtMitraWatch — requires modules 7+9, legal review
11. Advanced Ledger	Materialized views, route-success rates, repeat-offender clusters

6. Known Issues / Limitations

Architecture

No auth system — browser missions use a hardcoded "Demo User". No login, no session management.
Single-user demo — all browser missions attach to the same demo user.
Local file storage — evidence files stored on local filesystem (.storage/). Production needs R2/Supabase Storage.

Data

42 curated legal chunks across 3 domains. No automated PDF ingestion (Docling skipped).
3 route rules (consumer, payment, cyber_fraud). No employment/RTI/landlord domains.
All social posts are copy-ready — no Meta App Review completed, no X API payment.

Code Quality

26 lint warnings — mostly any types in existing code, unused imports.
No integration tests — only unit tests with mocked DB.
Some as never casts in Inngest event sends (type safety gap).

Docker Images

infra-api: ~404MB
infra-worker: ~384MB
infra-postgres: ~640MB (pgvector base)

7. Key Commands Reference

# Development
pnpm dev                    # Start all services (web + api + worker)
pnpm build                  # Build all packages
pnpm typecheck              # TypeScript type checking
pnpm lint                   # ESLint
pnpm test                   # Run all tests
pnpm test:contracts         # Run contract tests only

# Database
pnpm db:migrate             # Run Drizzle migrations
pnpm db:seed                # Seed legal data + route rules
npx tsx scripts/seed-qdrant.ts  # Seed Qdrant with embeddings

# Docker
pnpm infra:up               # Start Postgres + Inngest
pnpm infra:down             # Stop all containers
docker compose -f infra/docker-compose.yml up -d    # Same as above
docker compose -f infra/docker-compose.yml down     # Same as above

# Verification (run before committing)
pnpm typecheck && pnpm lint && pnpm test && pnpm test:contracts

8. File Structure (Key Files)

CLAUDE.md                          # Project invariants — READ THIS FIRST
PHASE_1_automation_loop.md         # Phase 1 spec + DoD
PHASE_2_scale_and_intelligence.md  # Phase 2 spec (11 modules)

packages/
  db/src/schema.ts                 # All 30+ table definitions
  db/src/migrations/               # 0000 through 0009
  db/src/seeds/                    # legal-data.ts, route-rules-data.ts, run.ts
  contracts/src/                   # All Zod schemas (enums, mission, action, swarm, asr, etc.)
  ports/src/index.ts               # All port interfaces
  adapters/src/                    # All adapter implementations
    legal/pgvector-fts.ts          # Original Postgres RAG adapter
    legal/qdrant-hybrid.ts         # Qdrant Cloud hybrid adapter
    audio/whisper-api.ts           # Whisper ASR adapter
    translation/llm-adapter.ts     # LLM translation adapter
    portal/playwright-adapter.ts   # Playwright portal adapter
    social/copy-ready-fallback.ts  # Social publishing fallback
    social/card-generator.ts       # Social card image generator
    case-lookup/surepass-adapter.ts # eCourts CNR adapter
    channel/whatsapp.ts            # WhatsApp Cloud API
    channel/telegram.ts            # Telegram Bot API
    submission/resend-email.ts     # Resend email adapter
    redaction/regex.ts             # PII regex redaction
    redaction/regex-llm.ts         # LLM-assisted redaction
    storage/local-fs.ts            # Local filesystem storage
    index.ts                       # Adapter exports + capability registry

apps/
  api/src/
    app.module.ts                  # Root NestJS module (all modules wired)
    workflow/inngest.module.ts     # Inngest client + WorkflowService
    actions/                       # Action preview, consent, execute
    missions/                      # Mission CRUD + reply ingestion
    evidence/                      # Evidence files + fact confirmation
    webhooks/                      # WhatsApp, Telegram, email webhook handlers
    conversations/                 # Message ingestion service
    route-plans/                   # Route plan generation trigger
    resolution/                    # Resolution capture
    ledger/                        # Anonymized ledger
    dpdp/                          # Data export + erase
    swarm/                         # Complaint swarm
    portal-assist/                 # Portal automation
    resolver/                      # Company resolver
    case-lookup/                   # eCourts CNR lookup
    capabilities/                  # GET /api/capabilities
  worker/src/
    inngest.ts                     # All registered Inngest functions
    deps.ts                        # Singleton dependencies
    functions/
      intake.ts                    # Run intake (extract facts)
      evidence.ts                  # Parse evidence (vision-LLM or ASR)
      route-plan.ts                # Generate route plan with citations
      action-execute.ts            # Execute actions via ActionExecutor
      followup-schedule.ts         # Schedule follow-up reminders
      followup-handler.ts          # Handle due follow-ups
      reply-classify.ts            # Classify inbound replies
      resolution-verify.ts         # Verify resolution
      swarm.ts                     # Entity resolution + clustering
      portal-assist.ts             # Portal automation flow
      case-lookup.ts               # eCourts CNR lookup
    lib/
      action-executor.ts           # The ONLY place side-effects happen
  web/src/
    app/page.tsx                   # Mission board (list + create)
    app/missions/[id]/page.tsx     # Mission room (full detail)
    app/p/[slug]/page.tsx          # Public case page (private link)
    app/ledger/page.tsx            # Anonymized ledger dashboard
    lib/api.ts                     # Typed API client

9. Troubleshooting

"DATABASE_URL is not set"

Make sure infra/.env exists and has DATABASE_URL set. The API and worker read from this file.

"Qdrant connection failed"

Check QDRANT_URL and QDRANT_API_KEY. For local Qdrant: docker run -p 6333:6333 qdrant/qdrant.

"OpenAI API key not set" (Whisper)

Set OPENAI_API_KEY for voice note transcription. Without it, audio evidence will be skipped.

"Resend API key not set" (Email)

Set RESEND_API_KEY and RESEND_FROM for email sending. Without it, email actions will fail with proof_state: failed.

"Inngest function not found"

Make sure the Inngest dev server is running (pnpm infra:up) and the worker is running (pnpm dev). Check http://localhost:8288 for the Inngest dashboard.

Tests failing

pnpm typecheck  # Check for type errors first
pnpm test       # Run tests

Docker build failing

# Rebuild from scratch
docker compose -f infra/docker-compose.yml down -v
docker compose -f infra/docker-compose.yml build --no-cache
docker compose -f infra/docker-compose.yml up -d

10. Architecture Invariants (from CLAUDE.md)

These are non-negotiable. Violating them breaks the system's trustworthiness:

Hexagonal — packages/domain has ZERO framework imports
Everything external is an Action — side-effects only happen in action-executor.ts
No fake submissions — proof_state must reflect reality
No legal claim without cited source — literal substring check
No public output without redaction + consent — Presidio redaction before share
Contracts-first — all Zod schemas in packages/contracts
Idempotent writes — every write endpoint accepts Idempotency-Key
Every schema change is a Drizzle migration — no ad-hoc DB edits