updates

.vscode/settings.json

@@ -1,5 +1,8 @@
 {
     "python.analysis.extraPaths": [
         "./ai_med_extract/utils"
+    ],
+    "cursorpyright.analysis.extraPaths": [
+        "./ai_med_extract/utils"
     ]
 }

API_PROMPT_RESPONSE_UPDATE.md

@@ -1,77 +0,0 @@
-# API Response Update: Added Full Prompt to LLM Responses
-
-## Overview
-Updated all API endpoints to include the full prompt that was passed to the LLM in the response, along with the summary and other values.
-
-## Changes Made
-
-### 1. GGUF Model Response (`routes_fastapi.py`)
-**Location**: Line 642
-**Change**: Added `"prompt": full_prompt` to the result dictionary
-**Prompt Source**: `full_prompt` variable (lines 551-573) - Contains the complete system prompt with patient data
-
-### 2. Text-Generation Model Response (`routes_fastapi.py`)
-**Location**: Line 726
-**Change**: Added `"prompt": prompt` to the result dictionary
-**Prompt Source**: `prompt` variable (line 699) - Built using `build_main_prompt(baseline, delta_text)`
-
-### 3. Summarization Model Response (`routes_fastapi.py`)
-**Location**: Line 785
-**Change**: Added `"prompt": context` to the result dictionary
-**Prompt Source**: `context` variable (line 755) - Contains "Patient Data:\nBaseline: {baseline}\nChanges: {delta_text}"
-
-### 4. Seq2Seq Model Response (`routes_fastapi.py`)
-**Location**: Line 833
-**Change**: Added `"prompt": context` to the result dictionary
-**Prompt Source**: `context` variable (line 807) - Contains "Patient Data:\nBaseline: {baseline}\nChanges: {delta_text}"
-
-### 5. OpenVINO Patient Summary Endpoint (`routes_fastapi.py`)
-**Location**: Line 1960
-**Change**: Added `"prompt": prompt` to the JSONResponse content
-**Prompt Source**: `prompt` variable (line 1919) - Built using `build_main_prompt(baseline, delta_text, patient_info)`
-
-### 6. Model Management API (`model_management_fastapi.py`)
-**Location**: Line 104
-**Change**: Added `"prompt": prompt` to the JSONResponse content
-**Prompt Source**: `prompt` variable from request data (line 78)
-
-## Response Structure
-
-All API responses now include the following structure:
-
-```json
-{
-  "summary": "Generated summary text...",
-  "baseline": "Baseline patient data...",
-  "delta": "Delta/changes data...",
-  "prompt": "Full prompt passed to LLM...",
-  "timing": {
-    "ehr_api": 0.8,
-    "generation": 15.2,
-    "total": 16.0
-  },
-  "model_used": "model_name (model_type)",
-  "timeout_mode_used": "normal"
-}
-```
-
-## Benefits
-
-1. **Transparency**: Users can see exactly what prompt was sent to the LLM
-2. **Debugging**: Easier to debug issues by examining the full prompt
-3. **Reproducibility**: Users can reproduce results by using the same prompt
-4. **Audit Trail**: Complete record of what was sent to the model
-5. **Quality Control**: Users can verify the prompt quality and make improvements
-
-## Affected Endpoints
-
-- `POST /generate_patient_summary` - Main patient summary endpoint
-- `POST /api/patient_summary_openvino` - OpenVINO-specific endpoint
-- `POST /api/models/generate` - Model management API
-- All model types: GGUF, text-generation, summarization, seq2seq
-
-## Backward Compatibility
-
-✅ **Fully backward compatible** - Existing fields remain unchanged, only added new `prompt` field
-✅ **No breaking changes** - All existing API consumers will continue to work
-✅ **Optional field** - The `prompt` field is additional information, not required for basic functionality

CONTAINER_OPTIMIZATION_SUMMARY.md

@@ -1,155 +0,0 @@
-# Container Log Diagnostic & Optimization Summary
-
-## Issues Identified and Fixed
-
-### 1️⃣ Transformers / Accelerate Dependency Mismatch ✅ FIXED
-**Problem**: Models like flan-t5-large and bart-large-cnn failed to load due to missing accelerate and pipeline API incompatibility.
-
-**Solution Applied**:
-- Updated `requirements.txt` to use compatible versions:
-  - `transformers>=4.42.0` (was 4.53.3)
-  - `accelerate>=0.30.0` (was 0.25.0)
-- Modified `model_manager.py` to handle pipeline creation with minimal parameters to avoid `assistant_model` issues
-
-### 2️⃣ GGUF Preload Path Missing ✅ FIXED
-**Problem**: GGUF model preload fails because the model directory doesn't exist yet.
-
-**Solution Applied**:
-- Disabled GGUF preload by default in `app.py`
-- Added proper fallback handling in GGUF model loader
-- Set `PRELOAD_GGUF=false` environment variable
-
-### 3️⃣ OpenVINO Telemetry Write Failure ✅ FIXED
-**Problem**: The OpenVINO runtime cannot write telemetry files to `/`.
-
-**Solution Applied**:
-- Added `OPENVINO_TELEMETRY_DIR=/tmp/openvino_telemetry` environment variable
-- Created writable directory `/tmp/openvino_telemetry` with proper permissions
-- Updated entrypoint script to set telemetry directory
-
-### 4️⃣ Invalid OMP_NUM_THREADS Setting ✅ FIXED
-**Problem**: OpenMP runtime throws "Invalid value for environment variable OMP_NUM_THREADS".
-
-**Solution Applied**:
-- Set `OMP_NUM_THREADS=4` in environment variables
-- Added dynamic setting in entrypoint script
-- Configured related threading variables: `MKL_NUM_THREADS=4`, `NUMEXPR_NUM_THREADS=4`
-
-### 5️⃣ /tmp Permission Denied During Cache Cleanup ✅ FIXED
-**Problem**: Entry script attempts to change /tmp permissions (chmod /tmp), not allowed in restricted environments.
-
-**Solution Applied**:
-- Modified entrypoint script to clean only specific cache directories
-- Removed `chmod /tmp` command that was causing permission errors
-- Changed to: `rm -rf /tmp/huggingface/* /tmp/torch/* || true`
-
-### 6️⃣ Redis and Database Not Configured ✅ FIXED
-**Problem**: App logs indicate Redis/DB unavailable, switching to fallback.
-
-**Solution Applied**:
-- Enhanced Redis fallback logic in `app.py` lifespan function
-- Added proper HF Spaces detection to skip Redis/DB initialization
-- Implemented graceful degradation when Redis is unavailable
-
-### 7️⃣ Matplotlib Cache Directory Issue ✅ FIXED
-**Problem**: Matplotlib fails to write config to `/.config/matplotlib`.
-
-**Solution Applied**:
-- Set `MPLCONFIGDIR=/tmp/matplotlib` environment variable
-- Created writable directory `/tmp/matplotlib` with proper permissions
-- Updated entrypoint script to prepare matplotlib cache directory
-
-### 8️⃣ Duplicate Route Registration (Multiple Init) ✅ FIXED
-**Problem**: Routes printed repeatedly due to Uvicorn reload or multiple startup triggers.
-
-**Solution Applied**:
-- Added `--no-reload` flag to uvicorn command in Dockerfile
-- Updated CMD to: `uvicorn app:app --host 0.0.0.0 --port 7860 --no-reload`
-
-### 9️⃣ Hugging Face Cache Redownloads Each Restart ✅ FIXED
-**Problem**: Each container start re-downloads 2GB+ GGUF model.
-
-**Solution Applied**:
-- Set persistent cache directory: `HF_HOME=/app/.cache/huggingface`
-- Created writable cache directory with proper permissions
-- Optimized cache cleanup to preserve downloaded models
-
-## Deliverables Created
-
-### 1. Optimized Dockerfile (`Dockerfile.optimized`)
-- Implements all environment variables and persistent cache setup
-- Installs fixed dependency versions
-- Pre-downloads or properly defers GGUF model load
-- Creates all necessary writable directories
-
-### 2. Improved Entrypoint Script (`entrypoint_optimized.sh`)
-- Cleans only specific caches (no chmod /tmp)
-- Prepares writable directories for OpenVINO and Matplotlib
-- Sets all required environment variables
-- Provides comprehensive startup logging
-
-### 3. Updated Requirements (`requirements.txt`)
-- Fixed Transformers and Accelerate version compatibility
-- Maintained all other dependencies
-
-### 4. Enhanced Model Manager (`model_manager.py`)
-- Fixed pipeline creation to avoid `assistant_model` issues
-- Improved error handling for newer transformers versions
-
-### 5. Updated Application Logic (`app.py`)
-- Disabled GGUF preload by default
-- Enhanced Redis/DB fallback logic
-- Improved HF Spaces detection
-
-## Performance Goals Achieved
-
-✅ **Model load under 20 seconds** (GGUF warm start)  
-✅ **No model redownloads after restarts** (persistent cache)  
-✅ **Clean startup logs with zero unhandled warnings**  
-✅ **Single set of route logs** (no duplicates)  
-✅ **All inference models load successfully** (GGUF, Transformers, OpenVINO)  
-✅ **GPU utilization optimized** (proper CUDA configuration)  
-
-## Environment Variables Set
-
-```bash
-HF_HOME=/app/.cache/huggingface
-XDG_CACHE_HOME=/tmp
-TORCH_HOME=/tmp/torch
-WHISPER_CACHE=/tmp/whisper
-PYTHONUNBUFFERED=1
-PYTHONPATH=/app
-GGUF_N_THREADS=4
-GGUF_N_BATCH=64
-OMP_NUM_THREADS=4
-MKL_NUM_THREADS=4
-NUMEXPR_NUM_THREADS=4
-OPENVINO_TELEMETRY_DIR=/tmp/openvino_telemetry
-MPLCONFIGDIR=/tmp/matplotlib
-PRELOAD_GGUF=false
-```
-
-## Success Criteria Met
-
-After applying all fixes, the container should:
-
-1. **Start once** with a single set of route logs
-2. **Load GGUF and Transformer models** without warnings
-3. **Have writable directories** for /tmp, /app/.cache, and /tmp/matplotlib
-4. **Gracefully disable Redis/DB** if missing
-5. **Function fully on GPU** with OpenVINO and Transformers pipelines
-6. **Show clean startup logs** with "Application startup complete — no warnings"
-
-## Usage
-
-To use the optimized container:
-
-```bash
-# Build with optimized Dockerfile
-docker build -f Dockerfile.optimized -t ai-service-optimized .
-
-# Run with optimized entrypoint
-docker run -p 7860:7860 ai-service-optimized
-```
-
-The container will now start cleanly with all identified issues resolved and optimal performance.

CRITICAL_DEPLOYMENT_FIX.md

@@ -1,288 +0,0 @@
-# ⚠️ CRITICAL DEPLOYMENT FIXES
-
-**Date:** $(date)
-**Status:** 🔴 URGENT - MUST APPLY BEFORE DEPLOYMENT
-
-## 🚨 Issues Found in Production Logs
-
-From your Hugging Face Spaces deployment logs, two critical issues were identified:
-
----
-
-## Issue 1: ASGI vs WSGI Error (BLOCKING DEPLOYMENT)
-
-### Error Message:
-```
-TypeError: FastAPI.__call__() missing 1 required positional argument: 'send'
-```
-
-### Problem:
-FastAPI is an **ASGI application**, but the Dockerfile was using **Gunicorn in WSGI mode**. This is incompatible and causes all requests to fail.
-
-### Root Cause:
-```dockerfile
-# OLD (WRONG):
-CMD ["gunicorn", "--bind", "0.0.0.0:7860", "--workers", "1", "--threads", "2", "--timeout", "0", "app:app"]
-```
-
-Gunicorn without ASGI workers cannot handle FastAPI applications.
-
-### Fix Applied:
-```dockerfile
-# NEW (CORRECT):
-CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860", "--workers", "1"]
-```
-
-**File Changed:** `Dockerfile` (line 227)
-
-### Alternative Fix (if you prefer Gunicorn):
-```dockerfile
-CMD ["gunicorn", "-k", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0:7860", "--workers", "1", "app:app"]
-```
-
----
-
-## Issue 2: Missing onnxruntime Dependency
-
-### Error Message:
-```
-ModuleNotFoundError: No module named 'onnxruntime'
-```
-
-### Problem:
-The `inference_service.py` imports `ORTModelForSeq2SeqLM` from `optimum.onnxruntime`, which requires `onnxruntime`, but it was not in `requirements.txt`.
-
-### Root Cause:
-- `optimum==1.27.0` was installed
-- But `onnxruntime` (required dependency) was missing
-- No error handling for optional ONNX optimization
-
-### Fixes Applied:
-
-#### Fix 1: Added onnxruntime to requirements.txt
-```diff
-# Model Optimization & Quantization
-optimum==1.27.0
-optimum-intel==1.25.2
-+ onnxruntime==1.16.3
-nncf==2.17.0
-```
-
-**File Changed:** `requirements.txt` (line 54)
-
-#### Fix 2: Added error handling in inference_service.py
-```python
-# Optional ONNX Runtime support
-try:
-    from optimum.onnxruntime import ORTModelForSeq2SeqLM
-    ONNX_AVAILABLE = True
-except (ImportError, ModuleNotFoundError) as e:
-    logging.warning(f"ONNX Runtime not available: {e}")
-    ORTModelForSeq2SeqLM = None
-    ONNX_AVAILABLE = False
-```
-
-**File Changed:** `services/ai-service/src/ai_med_extract/inference_service.py` (lines 9-16)
-
----
-
-## 📋 Summary of Changes
-
-### Files Modified: 3
-
-1. **Dockerfile**
-   - Changed from gunicorn (WSGI) to uvicorn (ASGI)
-   - ✅ Critical - Without this, the app cannot serve requests
-
-2. **requirements.txt**
-   - Added `onnxruntime==1.16.3`
-   - ✅ Critical - Routes fail to register without this
-
-3. **services/ai-service/src/ai_med_extract/inference_service.py**
-   - Added try-except for ONNX imports
-   - Added graceful fallback to standard transformers
-   - ✅ Important - Prevents import errors
-
----
-
-## 🚀 Deployment Steps
-
-### Step 1: Verify Changes
-```bash
-# Check Dockerfile CMD line
-grep "CMD" Dockerfile
-# Should show: CMD ["uvicorn", "app:app", ...]
-
-# Check onnxruntime in requirements
-grep "onnxruntime" requirements.txt
-# Should show: onnxruntime==1.16.3
-```
-
-### Step 2: Rebuild and Deploy
-```bash
-# Commit changes
-git add Dockerfile requirements.txt services/ai-service/src/ai_med_extract/inference_service.py
-git commit -m "Fix ASGI/WSGI error and add onnxruntime dependency"
-git push origin main
-```
-
-### Step 3: Verify Deployment
-```bash
-# Wait for rebuild (5-10 minutes)
-# Then test endpoints:
-
-# Test health
-curl https://your-space.hf.space/health
-
-# Test root
-curl https://your-space.hf.space/
-
-# Check logs for:
-# ✅ "Starting server with uvicorn"
-# ✅ "Application startup complete"
-# ❌ NO "TypeError: FastAPI.__call__() missing"
-```
-
----
-
-## 🎯 Expected Behavior After Fix
-
-### Startup Logs Should Show:
-```
-✅ Detected Hugging Face Spaces environment
-✅ Model manager imported successfully
-✅ Agents initialized successfully
-✅ App instance created successfully
-✅ Uvicorn running on http://0.0.0.0:7860
-✅ Application startup complete
-```
-
-### NOT:
-```
-❌ TypeError: FastAPI.__call__() missing 1 required positional argument: 'send'
-❌ ModuleNotFoundError: No module named 'onnxruntime'
-❌ Error handling request /
-```
-
----
-
-## 🔍 Why This Happened
-
-### ASGI vs WSGI Issue:
-- **WSGI** (Web Server Gateway Interface): Synchronous, used by Flask, Django
-- **ASGI** (Asynchronous Server Gateway Interface): Async, used by FastAPI, Starlette
-- Gunicorn is primarily a WSGI server
-- FastAPI requires ASGI, so we need Uvicorn (ASGI server) or Gunicorn with Uvicorn workers
-
-### Missing Dependency:
-- `optimum` package has optional dependencies
-- `optimum[onnxruntime]` would install onnxruntime automatically
-- But plain `optimum` doesn't include it
-- The inference_service.py assumed it would be there
-
----
-
-## 📊 Impact Assessment
-
-### Before Fixes:
-- ❌ **All API requests fail** with TypeError
-- ❌ Routes fail to register due to import error
-- ❌ App appears to start but cannot serve traffic
-- ❌ 100% failure rate
-
-### After Fixes:
-- ✅ App serves requests correctly
-- ✅ All routes register successfully
-- ✅ ONNX optimization available (faster inference)
-- ✅ Graceful fallback if ONNX fails
-- ✅ ~0% failure rate (assuming proper deployment)
-
----
-
-## 🔧 Additional Recommendations
-
-### 1. Test Locally Before Deploying
-```bash
-# Set HF Spaces environment
-export HF_SPACES=true
-
-# Install dependencies
-pip install -r requirements.txt
-
-# Run with uvicorn
-uvicorn app:app --host 0.0.0.0 --port 7860
-
-# Test in another terminal
-curl http://localhost:7860/health
-```
-
-### 2. Monitor First Requests
-After deploying, monitor the logs for:
-- Startup messages
-- First request handling
-- Any error patterns
-
-### 3. Consider Adding Health Check Timeout
-In your HF Spaces settings, ensure health check timeout is at least 60 seconds for model loading.
-
----
-
-## 🎓 Lessons Learned
-
-1. **Always use ASGI servers for FastAPI**
-   - Uvicorn (recommended)
-   - Hypercorn
-   - Daphne
-   - Gunicorn with uvicorn.workers.UvicornWorker
-
-2. **Test with production-like environment**
-   - Use containers locally
-   - Match the deployment server type
-   - Test with same Python version
-
-3. **Handle optional dependencies gracefully**
-   - Add try-except for optional imports
-   - Provide fallbacks
-   - Log warnings, not errors
-
-4. **Check requirements carefully**
-   - `optimum` ≠ `optimum[onnxruntime]`
-   - Read package documentation
-   - Test installation in clean environment
-
----
-
-## ✅ Verification Checklist
-
-- [ ] Dockerfile uses uvicorn (or gunicorn with UvicornWorker)
-- [ ] onnxruntime in requirements.txt
-- [ ] inference_service.py has try-except for ONNX import
-- [ ] Local test with uvicorn succeeds
-- [ ] Health endpoint returns 200
-- [ ] No ASGI/WSGI errors in logs
-- [ ] Routes register successfully
-
----
-
-## 📞 If Issues Persist
-
-If you still see errors after applying these fixes:
-
-1. **Check the logs** for new error messages
-2. **Verify the changes** were actually deployed (check build logs)
-3. **Clear cache** in HF Spaces settings
-4. **Restart the Space** manually if needed
-5. **Check dependencies** - ensure all installed correctly
-
----
-
-**Priority:** 🔴 CRITICAL - MUST APPLY IMMEDIATELY
-**Estimated Fix Time:** 5 minutes
-**Deployment Time:** 5-10 minutes (rebuild)
-**Success Probability:** 95%+ with these fixes
-
----
-
-*This document supersedes previous deployment guidance*
-*Apply these fixes before attempting any other changes*
-

DEPLOYMENT.md

@@ -1,106 +0,0 @@
-# Deployment Instructions
-
-This document provides deployment instructions for the Medical AI Service in various environments.
-
-## Local Development
-
-### Prerequisites
-- Python 3.10+
-- Docker (optional, for containerized testing)
-
-### Setup
-1. Clone the repository
-2. Install dependencies: `pip install -r requirements.txt`
-3. Set environment variables (see Configuration section)
-4. Run the application: `python -m uvicorn ai_med_extract.app:create_app --host 0.0.0.0 --port 7860`
-
-### Testing
-- Health check: `curl http://localhost:7860/health/live`
-- API docs: `http://localhost:7860/docs` (FastAPI Swagger UI)
-
-## Docker Deployment
-
-### Build and Run
-```bash
-docker build -t medical-ai-service .
-docker run -p 7860:7860 -e SECRET_KEY=your-secret -e DATABASE_URL=your-db medical-ai-service
-```
-
-### Configuration
-- Exposes port 7860
-- Runs FastAPI app with uvicorn
-- Includes model caching optimizations
-
-## Kubernetes Deployment
-
-### Prerequisites
-- Kubernetes cluster
-- kubectl configured
-- Secrets created for database, Redis, and JWT keys
-
-### Deploy
-```bash
-kubectl apply -f infra/k8s/secure_deployment.yaml
-```
-
-### Features
-- Horizontal Pod Autoscaler (2-10 replicas based on CPU/memory)
-- Resource limits: 1-4 CPU, 4-8Gi memory
-- Prometheus monitoring annotations
-- Security contexts and network policies
-
-### Scaling
-The HPA automatically scales based on:
-- CPU utilization > 70%
-- Memory utilization > 80%
-
-## Hugging Face Spaces Deployment
-
-### Prerequisites
-- Hugging Face account
-- Space created with Docker runtime
-
-### Configuration
-1. Dockerfile exposes port 7860
-2. FastAPI app listens on 0.0.0.0:7860
-3. requirements.txt includes all dependencies
-4. .huggingface.yaml with `runtime: docker`
-5. .dockerignore and .gitignore present
-
-### Deploy
-```bash
-# Test locally
-docker build -t hntai-app .
-docker run -p 7860:7860 hntai-app
-
-# Push to HF Spaces
-# App available at your-space-name.hf.space
-```
-
-## Configuration
-
-### Required Environment Variables
-- `SECRET_KEY`: Application secret key
-- `JWT_SECRET_KEY`: JWT signing key
-- `DATABASE_URL`: PostgreSQL connection string
-- `REDIS_URL`: Redis connection string
-
-### Optional
-- `ENVIRONMENT`: prod/dev (default: prod)
-- `PORT`: Service port (default: 7860)
-- `CORS_ORIGINS`: Allowed CORS origins (default: *)
-- Model cache directories and other settings in config_settings.py
-
-## Monitoring
-
-### Health Checks
-- `/health/live`: Liveness probe
-- `/health/ready`: Readiness probe
-
-### Metrics
-- `/metrics`: Prometheus metrics endpoint
-- Includes performance metrics, model loading status
-
-### Logging
-- Structured JSON logs for production
-- Configurable log levels

DEPLOYMENT_FIX_SUMMARY.md

@@ -1,132 +0,0 @@
-# Hugging Face Spaces Deployment Fix Summary
-
-## Root Cause Analysis
-
-The deployment error `ModuleNotFoundError: No module named 'app'` was caused by the `.dockerignore` file excluding the root `app.py` file from the Docker build context.
-
-### Error Details
-```
-[2025-10-07 12:40:17 +0000] [10] [ERROR] Exception in worker process
-...
-ModuleNotFoundError: No module named 'app'
-```
-
-## Issues Identified and Fixed
-
-### 1. **Critical Issue: .dockerignore Configuration** ✓ FIXED
-- **Problem**: The `.dockerignore` file was excluding everything (`*`) and then only including specific files, but it was missing the root `app.py` file.
-- **Impact**: The `app.py` file was not being copied to the Docker container, causing gunicorn to fail with `ModuleNotFoundError: No module named 'app'`.
-- **Fix**: Added `!app.py` and `!__init__.py` to the `.dockerignore` include list.
-
-### 2. **Missing Import in ai_med_extract/app.py** ✓ FIXED
-- **Problem**: The `model_manager` was being used in the `lifespan` function but was not imported at the module level.
-- **Impact**: Could cause runtime errors when scalable components try to initialize.
-- **Fix**: Added `from .utils.model_manager import model_manager` to the imports.
-
-### 3. **Improved Logging in Root app.py** ✓ FIXED
-- **Problem**: Limited debugging information when imports fail.
-- **Impact**: Made troubleshooting difficult.
-- **Fix**: Added comprehensive logging including:
-  - Python path information
-  - Current working directory
-  - Files in current directory
-  - Full exception tracebacks
-
-### 4. **Enhanced .huggingface.yaml Configuration** ✓ FIXED
-- **Problem**: Missing explicit app entrypoint configuration.
-- **Impact**: Hugging Face Spaces might not know which app to run.
-- **Fix**: Added app configuration section with explicit entrypoint and port.
-
-### 5. **Simplified Root app.py Import Logic** ✓ FIXED
-- **Problem**: Overly complex import logic with importlib that could fail.
-- **Impact**: Made debugging more difficult.
-- **Fix**: Simplified to direct imports with proper error handling.
-
-## Files Modified
-
-1. **`.dockerignore`** - Added root `app.py` and `__init__.py` to include list
-2. **`app.py`** (root) - Enhanced logging and simplified import logic
-3. **`services/ai-service/src/ai_med_extract/app.py`** - Added missing model_manager import
-4. **`.huggingface.yaml`** - Added explicit app configuration
-5. **`Dockerfile`** - Added clarification comment about Hugging Face Spaces usage
-
-## Verification
-
-### Local Testing
-✓ App imports successfully: `import app` works
-✓ App instance created: `app.app.title == "Medical AI Service"`
-✓ All agents initialize correctly
-✓ No import errors or missing dependencies
-
-### Expected Behavior on Hugging Face Spaces
-1. Dockerfile builds successfully with all necessary files
-2. Gunicorn can find and import the `app` module
-3. FastAPI app initializes with minimal preloading (FAST_MODE=true)
-4. App responds to health checks and API requests
-
-## Deployment Steps
-
-1. Commit all changes to Git
-2. Push to Hugging Face Spaces repository
-3. Hugging Face Spaces will automatically:
-   - Build the Docker container with the fixed `.dockerignore`
-   - Install dependencies from `requirements.txt`
-   - Run gunicorn with `app:app` entrypoint
-   - App should start successfully on port 7860
-
-## Key Configuration
-
-### Environment Variables (set in app.py)
-- `FAST_MODE=true` - Enables fast startup mode
-- `PRELOAD_SMALL_MODELS=false` - Disables model preloading
-- `HF_HOME=/tmp/huggingface` - Sets Hugging Face cache directory
-- `TORCH_HOME=/tmp/torch` - Sets PyTorch cache directory
-
-### Gunicorn Configuration (in Dockerfile CMD)
-- Workers: 1
-- Threads: 2
-- Timeout: 0 (unlimited)
-- Bind: 0.0.0.0:7860
-
-## Troubleshooting
-
-If the deployment still fails:
-
-1. **Check the build logs** - Verify that `app.py` is being copied to the container
-2. **Check the runtime logs** - Look for import errors or missing dependencies
-3. **Verify file structure** - Ensure all files are in the correct locations
-4. **Check cache** - Set `cache: false` in `.huggingface.yaml` to force rebuild
-5. **Test locally** - Build the Docker image locally to verify it works
-
-### Local Docker Test
-```bash
-# Build the Docker image
-docker build -t hntai-test .
-
-# Run the container
-docker run -p 7860:7860 hntai-test
-
-# Test the endpoint
-curl http://localhost:7860/health
-```
-
-## Additional Notes
-
-- The app uses a multi-strategy import approach with fallbacks
-- All heavy model loading is deferred to runtime (not import time)
-- Redis and database features are optional and will be skipped if not available
-- The app will start in degraded mode if necessary rather than failing completely
-
-## Success Criteria
-
-✓ No `ModuleNotFoundError` during gunicorn startup
-✓ App responds to health check requests
-✓ API endpoints are accessible
-✓ Agents initialize (with or without models depending on FAST_MODE)
-✓ No critical errors in container logs
-
----
-
-**Status**: Ready for deployment to Hugging Face Spaces
-**Last Updated**: 2025-10-07
-**Priority**: Critical - Blocks deployment

DEVELOPMENT.md

@@ -1,377 +0,0 @@
-# HNTAI - Scalable Medical Data Extraction API - Development Guide
-
-## Overview
-
-This FastAPI-based application provides scalable medical data extraction services, fully aligned with the "ChatGPT Version 3 - Scalable" architecture. It features async processing, Redis caching, PostgreSQL persistence, and enterprise-grade security.
-
-## Architecture
-
-### Core Components
-
-1. **FastAPI Application** (`app.py`)
-   - Main application factory with lifespan events
-   - CORS middleware for cross-origin requests
-   - Centralized agent initialization
-   - Route registration from APIRouter
-
-2. **Configuration** (`config_settings.py`)
-   - Pydantic-based settings with validation
-   - Environment variable loading
-   - Database and Redis URL configuration
-
-3. **Inference Service** (`inference_service.py`)
-   - Async text summarization using thread pools
-   - Model caching for performance
-   - Chunking for long text processing
-
-4. **PHI Scrubber Service** (`phi_scrubber_service.py`)
-   - Regex-based PHI detection and redaction
-   - Audit logging to PostgreSQL
-   - Redis-based statistics tracking
-
-5. **API Routes** (`api/routes_fastapi.py`)
-   - FastAPI APIRouter with async endpoints
-   - Health checks (/live, /ready)
-   - Placeholder routes for full migration
-
-### Data Flow
-
-```
-Client Request → FastAPI → Route Handler → Agent/Service → Redis Cache → PostgreSQL → Response
-```
-
-## Development Setup
-
-### Prerequisites
-
-- Python 3.10+
-- PostgreSQL 13+
-- Redis 6+
-- Docker (optional)
-
-### Local Development
-
-1. **Clone and Setup Virtual Environment**
-   ```bash
-   git clone <repository>
-   cd hntai
-   python -m venv venv
-   source venv/bin/activate  # On Windows: venv\Scripts\activate
-   ```
-
-2. **Install Dependencies**
-   ```bash
-   pip install -r requirements.txt
-   ```
-
-3. **Setup Database and Redis**
-   ```bash
-   # Start PostgreSQL (using Docker)
-   docker run -d --name postgres -e POSTGRES_PASSWORD=password -p 5432:5432 postgres:13
-
-   # Start Redis (using Docker)
-   docker run -d --name redis -p 6379:6379 redis:6
-
-   # Create database
-   createdb medical_ai
-   ```
-
-4. **Environment Variables**
-   Create `.env` file:
-   ```bash
-   DATABASE_URL=postgresql://postgres:password@localhost:5432/medical_ai
-   REDIS_URL=redis://localhost:6379/0
-   SECRET_KEY=your-secret-key-here
-   JWT_SECRET_KEY=your-jwt-secret-key-here
-   ```
-
-5. **Run Database Migrations**
-   ```bash
-   # Apply schema
-   psql -d medical_ai -f database/postgresql/001_schema.sql
-   ```
-
-6. **Run the Application**
-   ```bash
-   # Development mode
-   python -m ai_med_extract.main
-
-   # Or directly
-   uvicorn ai_med_extract.app:create_app --reload --host 0.0.0.0 --port 7860
-   ```
-
-7. **Access the Application**
-   - API: http://localhost:7860
-   - Docs: http://localhost:7860/docs (FastAPI auto-generated)
-   - Health: http://localhost:7860/live
-
-### Debugging
-
-1. **Enable Debug Logging**
-   ```python
-   import logging
-   logging.basicConfig(level=logging.DEBUG)
-   ```
-
-2. **Use FastAPI Debug Mode**
-   ```bash
-   uvicorn ai_med_extract.app:create_app --reload --debug --host 0.0.0.0 --port 7860
-   ```
-
-3. **Test Endpoints**
-   ```bash
-   # Health check
-   curl http://localhost:7860/live
-
-   # API docs
-   curl http://localhost:7860/openapi.json
-   ```
-
-4. **Database Debugging**
-   ```bash
-   # Connect to PostgreSQL
-   psql -d medical_ai
-
-   # Check PHI audit logs
-   SELECT * FROM phi_audit_log LIMIT 10;
-   ```
-
-5. **Redis Debugging**
-   ```bash
-   # Connect to Redis CLI
-   redis-cli
-
-   # Check keys
-   KEYS *
-   ```
-
-## Production Deployment
-
-### Option 1: Docker Deployment
-
-1. **Build Docker Image**
-   ```bash
-   docker build -t hntai-api .
-   ```
-
-2. **Run Container**
-   ```bash
-   docker run -d \
-     --name hntai-api \
-     -p 7860:7860 \
-     -e DATABASE_URL=postgresql://... \
-     -e REDIS_URL=redis://... \
-     -e SECRET_KEY=... \
-     -e JWT_SECRET_KEY=... \
-     hntai-api
-   ```
-
-### Option 2: Kubernetes Deployment
-
-1. **Prerequisites**
-   - Kubernetes cluster
-   - kubectl configured
-   - PostgreSQL and Redis services running
-
-2. **Create Secrets**
-   ```bash
-   kubectl create secret generic medical-ai-secrets \
-     --from-literal=DATABASE_URL=postgresql://... \
-     --from-literal=REDIS_URL=redis://... \
-     --from-literal=SECRET_KEY=... \
-     --from-literal=JWT_SECRET_KEY=...
-   ```
-
-3. **Deploy to Kubernetes**
-   ```bash
-   kubectl apply -f infra/k8s/secure_deployment.yaml
-   ```
-
-4. **Verify Deployment**
-   ```bash
-   kubectl get pods -n medical-ai
-   kubectl logs -n medical-ai deployment/medical-ai-service
-   ```
-
-### Option 3: Hugging Face Spaces (Legacy)
-
-The application still supports HF Spaces deployment for lightweight use cases.
-
-1. **Update app.py** for HF Spaces compatibility
-2. **Deploy via HF Spaces** with Docker SDK
-
-## Monitoring and Observability
-
-### Prometheus Metrics
-
-The application exposes metrics at `/metrics` endpoint.
-
-1. **Setup Prometheus**
-   ```bash
-   kubectl apply -f monitoring/prometheus.yml
-   ```
-
-2. **Access Metrics**
-   ```bash
-   curl http://ai-service.medical-ai.svc.cluster.local:80/metrics
-   ```
-
-### Health Checks
-
-- **Liveness** (`/live`): Basic health check
-- **Readiness** (`/ready`): Checks if agents are initialized
-
-### Logging
-
-- Structured JSON logging
-- PHI operations logged to database
-- Error tracking with stack traces
-
-## Security Features
-
-### HIPAA Compliance
-
-- PHI scrubbing with audit trails
-- Non-root container execution
-- Secrets management via Kubernetes
-- Network policies restricting traffic
-
-### Authentication
-
-- JWT-based authentication (framework ready)
-- API key support (configurable)
-
-## API Usage
-
-### Health Endpoints
-
-```bash
-GET /live
-GET /ready
-```
-
-### PHI Scrubbing
-
-```bash
-POST /phi/scrub
-Content-Type: application/json
-
-{
-  "text": "Patient John Doe, SSN 123-45-6789, diagnosed with diabetes."
-}
-```
-
-Response:
-```json
-{
-  "scrubbed_text": "Patient [REDACTED], SSN [REDACTED], diagnosed with diabetes.",
-  "phi_found": ["NAME", "SSN"],
-  "redaction_count": 2
-}
-```
-
-### Text Summarization
-
-```bash
-POST /api/generate_summary
-Content-Type: application/json
-
-{
-  "text": "Long medical text...",
-  "max_length": 150,
-  "min_length": 50
-}
-```
-
-### Generate Patient Summary
-
-The `generate_patient_summary` endpoint has been migrated from the original Flask implementation to FastAPI. It generates a comprehensive 4-section patient summary from EHR data, with support for streaming (SSE) to handle long-running tasks and prevent timeouts.
-
-**Endpoint**: `POST /generate_patient_summary`
-
-**Query Parameters**:
-- `stream` (optional, default: `false`): Set to `true` for Server-Sent Events (SSE) streaming updates.
-
-**Request Body** (JSON):
-```json
-{
-  "patientid": "12345",
-  "token": "your-auth-token",
-  "key": "your-api-key",
-  "patient_summarizer_model_name": "microsoft/Phi-3-mini-4k-instruct-gguf/Phi-3-mini-4k-instruct-q4.gguf",
-  "patient_summarizer_model_type": "gguf",
-  "generation_mode": "hq",  // Options: "hq" (high-quality), "fast", "rule" (deterministic)
-  "timeout_mode": "fast"    // Options: "fast" (8s EHR timeout), "extended" (30s)
-}
-```
-
-**Synchronous Response** (when `stream=false`):
-```json
-{
-  "summary": "## Clinical Assessment\n- Patient details...\n\n## Key Trends & Changes\n- Changes detected...\n\n## Plan & Suggested Actions\n- Recommendations...\n\n## Direct Guidance for Physician\n- Clinical insights...",
-  "baseline": "Patient baseline data...",
-  "delta": "Changes from previous visits...",
-  "timing": {"ehr_api": 2.5, "generation": 15.3, "total": 17.8},
-  "model_used": "microsoft/Phi-3-mini-4k-instruct (gguf)",
-  "timeout_mode_used": "fast"
-}
-```
-
-**Streaming Response** (when `stream=true`):
-- Returns a `text/event-stream` response with SSE events:
-  - `type: progress` - Progress updates (e.g., 10%, 50%)
-  - `type: complete` - Final result with full summary
-  - `type: error` - Error details if failed
-  - `type: heartbeat` - Keep-alive signals
-
-**Notes**:
-- The endpoint integrates with an external EHR API to fetch patient data.
-- Supports multiple model types: GGUF, text-generation, summarization, seq2seq.
-- Includes fallbacks for timeouts, API errors, and model failures.
-- PHI scrubbing is applied automatically.
-- Full implementation includes delta computation, baseline building, and 4-section markdown output.
-
-### Other Endpoints (Migration in Progress)
-- `POST /upload` - File upload and text extraction
-- `POST /transcribe` - Audio transcription
-- `POST /extract_medical_data` - Structured medical data extraction
-- `POST /api/extract_medical_data_from_audio` - Audio-based medical extraction
-
-## Troubleshooting
-
-### Common Issues
-
-1. **Model Loading Failures**
-   - Check HF_HOME and cache directories
-   - Ensure sufficient memory
-   - Verify internet connectivity for model downloads
-
-2. **Database Connection Errors**
-   - Verify DATABASE_URL format
-   - Check PostgreSQL service status
-   - Ensure database exists and schema applied
-
-3. **Redis Connection Issues**
-   - Verify REDIS_URL format
-   - Check Redis service availability
-   - Monitor Redis memory usage
-
-4. **PHI Scrubbing Not Working**
-   - Check regex patterns in phi_scrubber_service.py
-   - Verify Redis connection for stats
-   - Check database audit logs
-
-### Performance Tuning
-
-- Adjust thread pools in inference_service.py
-- Configure Redis connection pooling
-- Set appropriate resource limits in K8s
-- Monitor memory usage for model caching
-
-## Contributing
-
-1. Follow async/await patterns for new endpoints
-2. Add proper error handling and logging
-3. Update tests for new functionality
-4. Ensure HIPAA compliance for PHI handling
-5. Document API changes in this guide

DEVICE_PARAMETER_FIX_SUMMARY.md

@@ -1,136 +0,0 @@
-# Device Parameter Fix for Accelerate Models
-
-## Issue Description
-
-The patient summarizer was failing with the error:
-```
-The model has been loaded with `accelerate` and therefore cannot be moved to a specific device. Please discard the `device` argument when creating your pipeline object.
-```
-
-This error occurs when a model is loaded with the `accelerate` library and the code tries to specify a `device` parameter in the pipeline creation.
-
-## Root Cause
-
-In `services/ai-service/src/ai_med_extract/api/routes_fastapi.py`, the `get_summarizer_pipeline` function was passing both `device` and `device_map` parameters to the pipeline:
-
-```python
-# PROBLEMATIC CODE (before fix)
-pipeline(
-    task=summarizer_model_type,
-    model=summarizer_model_name,
-    trust_remote_code=True,
-    device=device,           # ❌ Conflicts with accelerate
-    torch_dtype=dtype,
-    **({"device_map": device_map} if device_map else {})  # ❌ Also conflicts
-)
-```
-
-When `device_map="auto"` is used (for GPU), the model is loaded with `accelerate`, which then conflicts with the `device` parameter.
-
-## Fix Applied
-
-### 1. Separated GPU and CPU Pipeline Creation
-
-**For GPU (CUDA available):**
-- Use `device_map="auto"` for automatic device mapping
-- **Do NOT** pass `device` parameter
-- Use `torch.float16` for efficiency
-
-**For CPU:**
-- Use `device=-1` for CPU
-- Use `torch.float32` for compatibility
-
-### 2. Added Fallback Error Handling
-
-If the initial pipeline creation fails due to device conflicts:
-1. Detect accelerate/device-related errors
-2. Retry without any device parameters
-3. Log the fallback process for debugging
-
-### 3. Enhanced Logging
-
-Added detailed logging to track:
-- Pipeline creation parameters
-- Success/failure of initial creation
-- Fallback process when needed
-- Final pipeline status
-
-## Code Changes
-
-### Before (Problematic):
-```python
-get_summarizer_pipeline.cache[key] = pipeline(
-    task=summarizer_model_type,
-    model=summarizer_model_name,
-    trust_remote_code=True,
-    device=device,
-    torch_dtype=dtype,
-    **({"device_map": device_map} if device_map else {})
-)
-```
-
-### After (Fixed):
-```python
-# Separate GPU and CPU handling
-if torch.cuda.is_available():
-    pipeline_kwargs = {
-        "task": summarizer_model_type,
-        "model": summarizer_model_name,
-        "trust_remote_code": True,
-        "device_map": "auto",  # ✅ Only device_map for GPU
-        "torch_dtype": torch.float16
-    }
-else:
-    pipeline_kwargs = {
-        "task": summarizer_model_type,
-        "model": summarizer_model_name,
-        "trust_remote_code": True,
-        "device": -1,  # ✅ Only device for CPU
-        "torch_dtype": torch.float32
-    }
-
-# Try with device parameters first
-try:
-    get_summarizer_pipeline.cache[key] = pipeline(**pipeline_kwargs)
-except Exception as e:
-    # Fallback without device parameters if accelerate conflicts
-    if "accelerate" in str(e).lower() or "device" in str(e).lower():
-        fallback_kwargs = {
-            "task": summarizer_model_type,
-            "model": summarizer_model_name,
-            "trust_remote_code": True,
-            "torch_dtype": dtype
-        }
-        get_summarizer_pipeline.cache[key] = pipeline(**fallback_kwargs)
-```
-
-## Testing
-
-Created `test_device_fix.py` to verify:
-1. ✅ Pipeline creation works without device conflicts
-2. ✅ Fallback behavior works when device parameters fail
-3. ✅ Both GPU and CPU scenarios are handled correctly
-
-## Expected Results
-
-After this fix:
-- ✅ Patient summarizer should work without accelerate device errors
-- ✅ Models load correctly on both GPU and CPU
-- ✅ Fallback ensures compatibility with various model configurations
-- ✅ Detailed logging helps debug any future issues
-
-## Files Modified
-
-1. **`services/ai-service/src/ai_med_extract/api/routes_fastapi.py`**
-   - Fixed `get_summarizer_pipeline` function
-   - Added proper GPU/CPU separation
-   - Added fallback error handling
-   - Enhanced logging
-
-2. **`test_device_fix.py`** (new file)
-   - Test script to verify the fix works
-   - Tests both normal and fallback scenarios
-
-## Deployment
-
-This fix should resolve the patient summarizer error you encountered. The changes are backward compatible and include fallback mechanisms to ensure the service continues working even if there are other device-related issues.

FIX_404_SUMMARY.md

@@ -1,170 +0,0 @@
-# Fix for 404 Error on `/generate_patient_summary` Endpoint
-
-## Problem
-The `/generate_patient_summary` endpoint was returning a 404 Not Found error when accessed on Hugging Face Spaces at:
-```
-https://salvinjose-hntai.hf.space/generate_patient_summary?stream=true
-```
-
-## Root Cause
-1. **Route Registration Issue**: The `/generate_patient_summary` endpoint was defined INSIDE the `register_routes()` function, which meant it was being added to the router AFTER the router was already included in the app. While this should work in FastAPI, it's not best practice and can cause timing issues.
-
-2. **Double Initialization**: The app was being initialized twice:
-   - Once in `create_app()` (which calls `initialize_agents` by default)
-   - Once again in the root `app.py` file
-   
-   This double initialization could cause routes to be registered incorrectly or timing issues.
-
-## Changes Made
-
-### 1. Fixed Route Registration (`services/ai-service/src/ai_med_extract/api/routes_fastapi.py`)
-
-**Before:**
-```python
-def register_routes(app, agents):
-    app.include_router(router)
-    
-    # Routes defined INSIDE the function
-    @router.post("/generate_patient_summary")
-    async def generate_patient_summary(...):
-        ...
-```
-
-**After:**
-```python
-# Define routes at MODULE LEVEL (outside register_routes)
-@router.post("/generate_patient_summary")
-async def generate_patient_summary(
-    request: Request,
-    background_tasks: BackgroundTasks,
-    stream: bool = False
-):
-    """Generate patient summary with optional streaming support."""
-    ...
-
-def register_routes(app, agents):
-    # Just include the router with already-defined routes
-    app.include_router(router)
-    ...
-```
-
-### 2. Fixed Double Initialization (`app.py`)
-
-**Before:**
-```python
-app = create_app()  # This calls initialize_agents internally
-initialize_agents(app, preload_small_models=False)  # Called again!
-```
-
-**After:**
-```python
-app = create_app(initialize=False)  # Don't initialize yet
-initialize_agents(app, preload_small_models=False)  # Initialize once
-```
-
-### 3. Added Comprehensive Logging
-
-Added logging to show all registered routes on startup in both:
-- `services/ai-service/src/ai_med_extract/app.py` (lines 781-786)
-- `app.py` (lines 95-103)
-
-This will help debug any remaining routing issues on HF Spaces.
-
-### 4. Added Diagnostic Endpoint
-
-Added a new `/api/info` endpoint that returns:
-```json
-{
-  "status": "ok",
-  "message": "Medical AI Service API",
-  "version": "1.0.0",
-  "endpoints": {
-    "generate_patient_summary": "/generate_patient_summary (POST)",
-    "upload": "/upload (POST)",
-    "transcribe": "/transcribe (POST)",
-    "health": "/health/* (GET)"
-  }
-}
-```
-
-## Testing
-
-### 1. Verify Routes are Registered
-After deploying to HF Spaces, check the logs for:
-```
-============================================================
-REGISTERED ROUTES:
-  ['POST'] /generate_patient_summary
-  ...
-Total routes registered: X
-============================================================
-```
-
-### 2. Test the Diagnostic Endpoint
-Access: `https://salvinjose-hntai.hf.space/api/info`
-
-Should return:
-```json
-{
-  "status": "ok",
-  "message": "Medical AI Service API",
-  ...
-}
-```
-
-### 3. Test the Debug Endpoint
-Access: `https://salvinjose-hntai.hf.space/debug/routes`
-
-Should return a list of all registered routes:
-```json
-{
-  "routes": [
-    {"path": "/generate_patient_summary", "methods": ["POST"], "name": "generate_patient_summary"},
-    ...
-  ],
-  "total": X
-}
-```
-
-### 4. Test the Target Endpoint
-```bash
-curl -X POST "https://salvinjose-hntai.hf.space/generate_patient_summary?stream=true" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "patientid": "your-patient-id",
-    "token": "your-auth-token",
-    "key": "your-api-key"
-  }'
-```
-
-## Expected Outcome
-
-The `/generate_patient_summary` endpoint should now:
-1. Return a proper response instead of 404
-2. Support both streaming (`stream=true`) and non-streaming modes
-3. Be visible in the route listing at `/debug/routes`
-
-## If Issues Persist
-
-If the 404 error persists after these changes:
-
-1. **Check the logs** - Look for the "REGISTERED ROUTES" section to verify the endpoint is registered
-2. **Test the diagnostic endpoint** - Access `/api/info` to verify the API is accessible
-3. **Check the debug endpoint** - Access `/debug/routes` to see all registered routes
-4. **Verify the URL** - Ensure you're using the correct URL without double slashes
-5. **Check for errors** - Look for any exceptions during route registration in the logs
-
-## Next Steps
-
-1. Commit these changes
-2. Push to HF Spaces
-3. Check the logs for route registration
-4. Test the endpoints as described above
-5. If issues persist, share the logs from HF Spaces
-
-## Files Modified
-
-- `app.py` (root level)
-- `services/ai-service/src/ai_med_extract/app.py`
-- `services/ai-service/src/ai_med_extract/api/routes_fastapi.py`
-

GPU_CONFIGURATION_GUIDE.md

@@ -1,169 +0,0 @@
-# GPU Configuration Guide for Hugging Face Spaces
-
-## Overview
-The GGUF model loader has been updated to automatically detect and use GPU when available in upgraded Hugging Face Spaces.
-
-## How It Works
-
-### Automatic GPU Detection
-The system now automatically detects GPU availability and configures the model accordingly:
-
-1. **GPU Available**: Uses all GPU layers (`n_gpu_layers=-1`) for maximum performance
-2. **CPU Only**: Falls back to CPU-only mode (`n_gpu_layers=0`) when GPU is not available
-3. **Error Handling**: Gracefully falls back to CPU if GPU detection fails
-
-### Configuration Options
-
-#### Environment Variables
-You can control GPU usage through environment variables:
-
-```bash
-# Use all GPU layers (default when GPU is available)
-GGUF_GPU_LAYERS=-1
-
-# Use specific number of GPU layers (e.g., 20 layers)
-GGUF_GPU_LAYERS=20
-
-# Force CPU-only mode even if GPU is available
-GGUF_GPU_LAYERS=0
-```
-
-#### Batch Size Configuration
-```bash
-# Adjust batch size for GPU memory
-GGUF_N_BATCH=32  # Default
-GGUF_N_BATCH=64  # For more GPU memory
-```
-
-## Performance Expectations
-
-### CPU-Only Mode (Current Free Tier)
-- **Speed**: ~2-5 tokens/second
-- **Memory**: ~2-4GB RAM usage
-- **Latency**: 30-60 seconds for patient summaries
-
-### GPU Mode (Upgraded Space)
-- **Speed**: ~10-50 tokens/second (5-10x faster)
-- **Memory**: ~4-8GB GPU memory + 2-4GB RAM
-- **Latency**: 5-15 seconds for patient summaries
-
-## Upgrade Benefits
-
-### 1. **Significantly Faster Generation**
-- 5-10x speed improvement for GGUF models
-- Reduced streaming latency
-- Better user experience
-
-### 2. **Better Resource Utilization**
-- GPU acceleration for model inference
-- More efficient memory usage
-- Parallel processing capabilities
-
-### 3. **Scalability**
-- Handle more concurrent requests
-- Support larger models
-- Better performance under load
-
-## Implementation Details
-
-### Code Changes Made
-```python
-# GPU detection and configuration
-n_gpu_layers = 0  # Default to CPU-only
-gpu_available = False
-
-# Check for CUDA availability
-try:
-    import torch
-    if torch.cuda.is_available():
-        gpu_available = True
-        # Use all GPU layers if available
-        n_gpu_layers = int(os.environ.get("GGUF_GPU_LAYERS", "-1"))
-        logger.info(f"CUDA available, using {n_gpu_layers} GPU layers")
-    else:
-        logger.info("CUDA not available, using CPU only")
-except ImportError:
-    logger.info("PyTorch not available, using CPU only")
-except Exception as e:
-    logger.warning(f"GPU detection failed: {e}, falling back to CPU")
-```
-
-### Logging Output
-The system now provides clear logging about GPU usage:
-
-```
-[GGUF] CUDA available, using -1 GPU layers
-[GGUF] Model initialized in 2.34s from /path/to/model.gguf (threads=4, batch=32, GPU layers=-1)
-```
-
-Or for CPU-only:
-```
-[GGUF] CUDA not available, using CPU only
-[GGUF] Model initialized in 1.23s from /path/to/model.gguf (threads=4, batch=32, CPU-only)
-```
-
-## Testing GPU Usage
-
-### 1. Check GPU Availability
-```python
-import torch
-print(f"CUDA available: {torch.cuda.is_available()}")
-if torch.cuda.is_available():
-    print(f"GPU count: {torch.cuda.device_count()}")
-    print(f"GPU name: {torch.cuda.get_device_name(0)}")
-```
-
-### 2. Monitor GPU Usage
-```bash
-# Check GPU memory usage
-nvidia-smi
-
-# Monitor GPU utilization
-watch -n 1 nvidia-smi
-```
-
-### 3. Test Performance
-The streaming API will show improved performance with GPU:
-- Faster progress updates
-- Reduced generation time
-- Better throughput
-
-## Troubleshooting
-
-### Common Issues
-
-1. **GPU Not Detected**
-   - Ensure PyTorch with CUDA support is installed
-   - Check CUDA_VISIBLE_DEVICES environment variable
-   - Verify GPU is available in the Space
-
-2. **Out of Memory Errors**
-   - Reduce `GGUF_GPU_LAYERS` to use fewer layers
-   - Decrease `GGUF_N_BATCH` for smaller batch size
-   - Use smaller models
-
-3. **Performance Issues**
-   - Check GPU utilization with `nvidia-smi`
-   - Monitor memory usage
-   - Adjust batch size and layer count
-
-### Fallback Behavior
-The system is designed to gracefully fall back to CPU if GPU is not available or fails, ensuring the service remains functional.
-
-## Migration Notes
-
-- **Backward Compatible**: Works on both CPU and GPU Spaces
-- **No Breaking Changes**: Existing functionality preserved
-- **Automatic Detection**: No manual configuration required
-- **Environment Variables**: Optional fine-tuning available
-
-## Expected Results After Upgrade
-
-With GPU acceleration, you should see:
-- **5-10x faster** patient summary generation
-- **Reduced streaming latency** from 30-60s to 5-15s
-- **Better concurrent request handling**
-- **More responsive user interface**
-
-The streaming API will provide the same events but with much faster progression through the processing stages.
-

HF_SPACES_FIXES_APPLIED.md

@@ -1,416 +0,0 @@
-# Hugging Face Spaces - Issues Fixed
-
-## Summary
-This document summarizes all the fixes applied to resolve potential internal server errors when deploying to Hugging Face Spaces.
-
-**Date:** $(date)
-**Total Issues Fixed:** 18 (4 critical, 6 high, 8 medium/minor)
-
----
-
-## ✅ CRITICAL FIXES APPLIED
-
-### 1. ✅ Fixed Redis Connection Blocking Startup
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/app.py` (Lines 61-88)
-- `services/ai-service/src/ai_med_extract/api_endpoints.py` (Lines 18-34)
-- `app.py` (Lines 35-50)
-
-**Changes:**
-- Added HF_SPACES environment detection
-- Redis connections are now skipped entirely on HF Spaces
-- Added proper error handling for Redis initialization failures
-- Empty Redis URL defaults prevent connection attempts
-- Module-level Redis initialization now has try-except wrapper
-
-**Testing:**
-```bash
-# Test app starts without Redis
-export HF_SPACES=true
-export REDIS_URL=""
-python app.py
-# Should see: "Skipping Redis initialization on HF Spaces"
-```
-
-### 2. ✅ Fixed Read-Only Filesystem Issues
-**Files Modified:**
-- `services/ai-service/src/config_settings.py` (Lines 13-14, 28-29, 40-50)
-- `services/ai-service/src/ai_med_extract/utils/file_utils.py` (Lines 33-64)
-
-**Changes:**
-- Changed default UPLOAD_PATH from `/app/uploads` to `/tmp/uploads`
-- Changed default MODEL_CACHE_DIR from `/app/models` to `/tmp/models`
-- DATABASE_URL now defaults to empty string instead of postgres connection
-- REDIS_URL now defaults to empty string
-- Added error handling for directory creation failures
-- HF_SPACES boolean flag now properly read from environment
-
-**Testing:**
-```bash
-# Verify paths
-python -c "from config_settings import get_settings; s = get_settings(); print(f'Upload: {s.UPLOAD_PATH}, Models: {s.MODEL_CACHE_DIR}')"
-# Should output: Upload: /tmp/uploads, Models: /tmp/models
-```
-
-### 3. ✅ Fixed Gradio App Localhost Requests
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/gradio_app.py` (Complete rewrite)
-
-**Changes:**
-- Removed all localhost HTTP requests
-- Functions now call agents directly via imports
-- Added proper async handling for inference service
-- Added comprehensive error handling
-- PHI scrubber agent called directly instead of via API
-- Fallback handling if agents fail to initialize
-
-**Testing:**
-```python
-# Test functions work without HTTP
-from services.ai_service.src.ai_med_extract.gradio_app import summarize_text, scrub_phi
-result = summarize_text("Test medical text here")
-print(result)
-```
-
-### 4. ✅ Fixed API Endpoints Service Redis Initialization
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/api_endpoints.py` (Lines 18-34, 67-79)
-
-**Changes:**
-- Wrapped Redis initialization in try-except at module level
-- Added fallback for PHI scrubbing when Redis unavailable
-- `/phi/scrub` endpoint returns graceful error when Redis not available
-- Proper logging of initialization failures
-
-**Testing:**
-```bash
-# Test API starts without Redis
-curl http://localhost:7860/phi/scrub -X POST -H "Content-Type: application/json" -d '{"text":"test"}'
-# Should return JSON with warning about Redis not available
-```
-
----
-
-## ✅ HIGH SEVERITY FIXES APPLIED
-
-### 5. ✅ Fixed Database Connection Attempts on HF Spaces
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/app.py` (Lines 89-104)
-
-**Changes:**
-- Database connections now skipped on HF Spaces
-- Empty DATABASE_URL prevents connection attempts
-- Added explicit logging for HF Spaces environment detection
-
-### 6. ✅ Fixed Environment Variable Defaults
-**Files Modified:**
-- `services/ai-service/src/config_settings.py`
-- `app.py`
-
-**Changes:**
-- DATABASE_URL defaults to empty string (not postgres URL)
-- REDIS_URL defaults to empty string (not redis URL)
-- HF_SPACES detection via SPACE_ID or SPACE_AUTHOR_NAME environment variables
-- Automatic setting of HF_SPACES=true when detected
-
-### 7. ✅ Improved Model Loading Memory Management
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/app.py` (Lines 558-636)
-
-**Changes:**
-- HF_SPACES detection ensures FAST_MODE is enabled
-- PRELOAD_SMALL_MODELS disabled on HF Spaces
-- Models loaded lazily to reduce memory footprint
-- Better fallback handling for model loading failures
-
-### 8. ✅ Fixed Upload Path Consistency
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/app.py` (Lines 215-244)
-- `services/ai-service/src/ai_med_extract/utils/file_utils.py` (Lines 33-64)
-
-**Changes:**
-- Upload directory resolution now HF_SPACES-aware
-- All file operations consistently use /tmp on HF Spaces
-- Improved error handling for directory creation
-- Fallback chain: /tmp/uploads → /tmp (if all else fails)
-
----
-
-## ✅ MEDIUM SEVERITY FIXES APPLIED
-
-### 9. ✅ Improved External API Error Handling
-**Files Modified:**
-- `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Lines 375-413, 880-928)
-
-**Changes:**
-- Added specific exception handling for timeout, connection, and request errors
-- User-friendly error messages with categories (TIMEOUT, CONNECTION, EHR_API, MEMORY, GENERAL)
-- Better error response format with error_category field
-- Proper logging of error details
-- Graceful degradation with fallback summaries
-
-**Error Categories Implemented:**
-- `TIMEOUT`: Operation took too long
-- `CONNECTION`: Network/connectivity issues
-- `EHR_API`: External EHR system errors
-- `MEMORY`: Insufficient memory errors
-- `GENERAL`: Other errors with truncated message
-
-### 10-18. ✅ Additional Improvements
-- Added comprehensive logging throughout
-- Improved fallback strategies
-- Better async exception handling
-- Cache directory management
-- Thread pool size consideration
-- Model download progress logging
-
----
-
-## 📋 TESTING CHECKLIST
-
-### Basic Functionality Tests
-
-#### 1. App Startup Test
-```bash
-export HF_SPACES=true
-export REDIS_URL=""
-export DATABASE_URL=""
-python app.py
-# Should start without errors
-# Check logs for: "Detected Hugging Face Spaces environment"
-```
-
-#### 2. Health Endpoints Test
-```bash
-curl http://localhost:7860/health
-curl http://localhost:7860/live
-curl http://localhost:7860/ready
-# All should return 200 OK
-```
-
-#### 3. File Operations Test
-```bash
-# Verify /tmp/uploads directory is created
-ls -la /tmp/uploads
-# Should exist and be writable
-```
-
-#### 4. Redis Disabled Test
-```bash
-# Start app without Redis
-export REDIS_URL=""
-python app.py
-# Check logs: "Redis URL not configured"
-# No Redis connection errors should appear
-```
-
-#### 5. Database Disabled Test
-```bash
-# Start app without Database
-export DATABASE_URL=""
-python app.py
-# Check logs: "Database audit logger not configured"
-# No database connection errors should appear
-```
-
-### API Endpoint Tests
-
-#### 6. Summarization Endpoint Test
-```bash
-curl -X POST http://localhost:7860/summarize \
-  -H "Content-Type: application/json" \
-  -d '{"text":"Patient presents with fever and cough."}'
-# Should return summary or graceful error
-```
-
-#### 7. PHI Scrubbing Endpoint Test
-```bash
-curl -X POST http://localhost:7860/phi/scrub \
-  -H "Content-Type: application/json" \
-  -d '{"text":"John Doe, SSN 123-45-6789"}'
-# Should return with warning if Redis unavailable
-# Should not return 500 error
-```
-
-#### 8. Patient Summary Endpoint Test (with Mock Data)
-```bash
-curl -X POST http://localhost:7860/generate_patient_summary \
-  -H "Content-Type: application/json" \
-  -d '{
-    "patientid": "test123",
-    "token": "mock_token",
-    "key": "http://mock-ehr-system.com",
-    "generation_mode": "rule"
-  }'
-# Should return rule-based summary or connection error
-# Should not return 500 error
-```
-
-### Error Handling Tests
-
-#### 9. Timeout Error Test
-```bash
-# Test with unreachable EHR endpoint
-curl -X POST http://localhost:7860/generate_patient_summary \
-  -H "Content-Type: application/json" \
-  -d '{
-    "patientid": "test",
-    "token": "test",
-    "key": "http://1.2.3.4:9999",
-    "generation_mode": "rule",
-    "timeout_mode": "fast"
-  }'
-# Should return error with category: "TIMEOUT" or "CONNECTION"
-```
-
-#### 10. Invalid Input Test
-```bash
-curl -X POST http://localhost:7860/summarize \
-  -H "Content-Type: application/json" \
-  -d '{"text":""}'
-# Should return 400 error (not 500)
-```
-
-### Memory and Resource Tests
-
-#### 11. Model Loading Test
-```python
-# Test lazy model loading
-import os
-os.environ['HF_SPACES'] = 'true'
-os.environ['FAST_MODE'] = 'true'
-from ai_med_extract.app import create_app, initialize_agents
-app = create_app(initialize=False)
-initialize_agents(app, preload_small_models=False)
-# Should complete without loading heavy models
-```
-
-#### 12. Memory Cleanup Test
-```bash
-# Monitor memory usage during operation
-# Start app with monitoring
-python -m memory_profiler app.py &
-# Make several requests
-# Memory should be released after each request
-```
-
----
-
-## 🔧 ENVIRONMENT VARIABLES FOR HF SPACES
-
-Add these to your Hugging Face Space settings or `.env` file:
-
-```bash
-# Required for HF Spaces
-HF_SPACES=true
-FAST_MODE=true
-PRELOAD_SMALL_MODELS=false
-
-# Disable external services
-REDIS_URL=
-DATABASE_URL=
-
-# Configure paths
-UPLOAD_PATH=/tmp/uploads
-MODEL_CACHE_DIR=/tmp/models
-
-# Memory optimization
-PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128
-TOKENIZERS_PARALLELISM=false
-OMP_NUM_THREADS=1
-MKL_NUM_THREADS=1
-
-# Cache directories
-HF_HOME=/tmp/huggingface
-XDG_CACHE_HOME=/tmp
-TORCH_HOME=/tmp/torch
-WHISPER_CACHE=/tmp/whisper
-```
-
----
-
-## 📊 VERIFICATION SUMMARY
-
-### Files Modified: 7
-1. `services/ai-service/src/config_settings.py`
-2. `services/ai-service/src/ai_med_extract/app.py`
-3. `services/ai-service/src/ai_med_extract/api_endpoints.py`
-4. `services/ai-service/src/ai_med_extract/gradio_app.py`
-5. `services/ai-service/src/ai_med_extract/utils/file_utils.py`
-6. `services/ai-service/src/ai_med_extract/api/routes_fastapi.py`
-7. `app.py`
-
-### Lines Changed: ~500+
-### New Features Added:
-- HF_SPACES environment detection
-- Graceful degradation when services unavailable
-- Better error categorization
-- Improved logging
-
-### Backward Compatibility:
-✅ All changes maintain backward compatibility with non-HF Spaces deployments
-
----
-
-## 🚀 DEPLOYMENT READY
-
-The application is now ready for Hugging Face Spaces deployment with:
-
-1. ✅ No Redis dependency
-2. ✅ No Database dependency
-3. ✅ All file operations in /tmp
-4. ✅ Memory-optimized model loading
-5. ✅ Graceful error handling
-6. ✅ Proper async/await patterns
-7. ✅ Comprehensive logging
-8. ✅ Fallback strategies for all critical paths
-
----
-
-## 📝 NEXT STEPS
-
-1. **Test locally with HF_SPACES=true**
-   ```bash
-   export HF_SPACES=true
-   python app.py
-   ```
-
-2. **Deploy to HF Spaces**
-   - Push code to your Hugging Face Space
-   - Set environment variables in Space settings
-   - Monitor startup logs
-
-3. **Verify endpoints**
-   - Test `/health`, `/ready`, `/live`
-   - Test main API endpoints
-   - Check error responses are proper (not 500)
-
-4. **Monitor performance**
-   - Check memory usage
-   - Verify model loading times
-   - Test with realistic workloads
-
----
-
-## ⚠️ KNOWN LIMITATIONS ON HF SPACES
-
-1. **No Redis**: Caching and rate limiting features disabled
-2. **No Database**: Audit logging and persistence disabled
-3. **Memory Limits**: Large models may not load on free tier
-4. **Storage Limits**: /tmp has size restrictions
-5. **No External Services**: EHR API calls may timeout on slow networks
-
-These limitations are handled gracefully with fallbacks and proper error messages.
-
----
-
-## 📧 SUPPORT
-
-If you encounter issues:
-1. Check the logs for specific error messages
-2. Verify environment variables are set correctly
-3. Ensure HF_SPACES=true is set
-4. Check the error category in API responses
-5. Review this document for relevant fixes
-
-

HF_SPACES_ISSUES_REPORT.md

@@ -1,209 +0,0 @@
-# Hugging Face Spaces - Potential Internal Server Error Issues
-
-## 🔴 CRITICAL ISSUES (Will cause 500 errors)
-
-### 1. Redis Connection Will Block Startup
-**File:** `services/ai-service/src/ai_med_extract/app.py` (Lines 63-75)
-**Issue:** The app tries to connect to Redis at startup with a 5-second timeout, but the error handling may not be sufficient.
-```python
-redis_client = redis.from_url(redis_url, decode_responses=True, socket_timeout=5, socket_connect_timeout=5)
-await asyncio.wait_for(redis_client.ping(), timeout=5.0)
-```
-**Impact:** If Redis URL is invalid or connection hangs, it could delay startup significantly.
-**Fix:** Ensure Redis connection is truly optional and doesn't block any critical paths.
-
-### 2. Config Settings Tries to Create Directories in Read-Only Filesystem
-**File:** `services/ai-service/src/config_settings.py` (Lines 41-43)
-```python
-os.makedirs(s.UPLOAD_PATH, exist_ok=True)  # /app/uploads - may be read-only on HF Spaces
-os.makedirs(s.MODEL_CACHE_DIR, exist_ok=True)  # /app/models - may be read-only on HF Spaces
-```
-**Impact:** HF Spaces has a read-only filesystem except for `/tmp`. This will fail.
-**Fix:** Default paths should be in `/tmp/` directory.
-
-### 3. Gradio App Makes Localhost Requests
-**File:** `services/ai-service/src/ai_med_extract/gradio_app.py` (Lines 11, 21)
-```python
-response = requests.post(f"http://localhost:{settings.PORT}/summarize", json={"text": text})
-response = requests.post(f"http://localhost:{settings.PORT}/phi/scrub", json={"text": text})
-```
-**Impact:** On HF Spaces, the Gradio interface can't make requests to localhost. This will cause connection errors.
-**Fix:** Gradio functions should call agents directly, not via HTTP requests.
-
-### 4. API Endpoints Service Tries to Connect to Redis Without Proper Fallback
-**File:** `services/ai-service/src/ai_med_extract/api_endpoints.py` (Lines 17-18)
-```python
-_inf = InferenceService()
-_redis = redis.from_url(settings.REDIS_URL, decode_responses=True)
-_phi = PHIScrubberService(_redis)
-```
-**Impact:** This creates Redis connection at module import time without error handling. Will crash if Redis is not available.
-**Fix:** Wrap in try-except and use lazy initialization.
-
-## 🟠 HIGH SEVERITY ISSUES (Likely to cause errors)
-
-### 5. Database URL May Cause Connection Attempts
-**File:** `services/ai-service/src/ai_med_extract/app.py` (Lines 78-88)
-```python
-database_url = os.getenv('DATABASE_URL')
-if database_url:
-    try:
-        db_audit_logger = await initialize_db_audit_logger(database_url)
-```
-**Impact:** If DATABASE_URL is set but PostgreSQL is not available, this could hang or fail.
-**Fix:** Add connection timeout and better error handling.
-
-### 6. Missing Environment Variable Handling
-**File:** `services/ai-service/src/config_settings.py` (Line 13)
-```python
-DATABASE_URL: str = os.getenv("DATABASE_URL", "postgresql+asyncpg://user:password@postgres:5432/db")
-```
-**Impact:** Default database URL points to `postgres:5432` which won't exist on HF Spaces.
-**Fix:** Should default to `None` or check for HF Spaces environment.
-
-### 7. Model Loading May Exhaust Memory
-**File:** `services/ai-service/src/ai_med_extract/app.py` (Lines 558-636)
-**Issue:** Multiple models are preloaded even in non-fast mode, which could exhaust available memory on free HF Spaces tier.
-```python
-if preload_small_models and not fast_mode:
-    # Loads summarizer_agent, medical_data_extractor_agent, patient_summarizer_agent
-```
-**Impact:** Out of memory errors causing crashes.
-**Fix:** Ensure HF_SPACES environment variable is checked and models are loaded lazily.
-
-### 8. File Upload Paths May Be Incorrect
-**File:** `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Line 1076)
-```python
-upload_dir = '/tmp/uploads'
-os.makedirs(upload_dir, exist_ok=True)
-```
-**Impact:** While this uses /tmp, other parts of the code may use different paths.
-**Fix:** Ensure all upload/temp file operations use /tmp consistently.
-
-## 🟡 MEDIUM SEVERITY ISSUES (May cause errors under certain conditions)
-
-### 9. Model Manager Import May Fail
-**File:** `services/ai-service/src/ai_med_extract/app.py` (Lines 20-35)
-```python
-try:
-    from .utils.model_manager import model_manager
-    logging.info("Model manager imported successfully")
-except ImportError as e:
-    logging.warning(f"Failed to import model_manager: {e}")
-```
-**Impact:** If model_manager import fails, fallback is used but may not work properly for all operations.
-**Fix:** Ensure fallback is comprehensive.
-
-### 10. OpenVINO Model Loading May Not Work on HF Spaces
-**File:** `services/ai-service/src/ai_med_extract/utils/model_loader_spaces.py` (Lines 17-21)
-```python
-model = OVModelForCausalLM.from_pretrained(model_name, compile=True, device="CPU", cache_dir=...)
-```
-**Impact:** OpenVINO may have compatibility issues on HF Spaces infrastructure.
-**Fix:** Add fallback to regular transformers if OpenVINO fails.
-
-### 11. External API Calls May Timeout
-**File:** `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Lines 372-392)
-```python
-response = requests.post(ehr_url, json={"patientid": patientid}, headers=headers, timeout=EHR_TIMEOUT)
-```
-**Impact:** External EHR API calls may fail or timeout, causing endpoint failures.
-**Fix:** Better error messages and graceful degradation.
-
-### 12. Transformers Model Loading May Fail with Device Mapping
-**File:** `services/ai-service/src/ai_med_extract/utils/model_manager.py` (Lines 74-86)
-```python
-self._model = AutoModelForCausalLM.from_pretrained(
-    self.model_name,
-    device_map="auto" if self.device == "cuda" and torch.cuda.is_available() else None,
-```
-**Impact:** `device_map="auto"` may cause issues on HF Spaces with limited resources.
-**Fix:** Force CPU mode on HF Spaces.
-
-### 13. GGUF Model Loading May Download Large Files
-**File:** `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Lines 40-68)
-```python
-GGUF_MODEL_CACHE[key] = GGUFModelPipeline(model_name, filename, timeout=timeout)
-```
-**Impact:** GGUF models can be very large (several GB), exhausting disk space or taking too long to download.
-**Fix:** Add size checks and better error handling.
-
-### 14. Thread Pool Executor May Cause Resource Issues
-**File:** `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Line 1424)
-```python
-with ThreadPoolExecutor(max_workers=4) as executor:
-```
-**Impact:** Multiple threads may compete for limited CPU resources on free tier.
-**Fix:** Reduce max_workers on HF Spaces or use sequential processing.
-
-## 🔵 MINOR ISSUES (Edge cases)
-
-### 15. Werkzeug Import for secure_filename
-**File:** `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` (Line 1215)
-```python
-from werkzeug.utils import secure_filename
-```
-**Impact:** Werkzeug is listed in requirements but only used in one place.
-**Fix:** Could use a simpler alternative to reduce dependencies.
-
-### 16. Missing Error Handlers for Specific Exceptions
-**File:** `services/ai-service/src/ai_med_extract/app.py` (Lines 246-262)
-**Issue:** Global exception handler catches all exceptions but may not properly handle async exceptions.
-**Fix:** Add specific handlers for common async exceptions.
-
-### 17. Cache Directory Cleanup May Not Work
-**File:** Multiple files use cache directories in `/tmp/`
-**Impact:** On HF Spaces, /tmp may persist between requests but have size limits.
-**Fix:** Implement proper cache cleanup strategies.
-
-### 18. Model Download Progress May Block
-**Issue:** Large model downloads without progress indicators may appear as hangs.
-**Fix:** Add progress logging for model downloads.
-
-## 📋 RECOMMENDATIONS
-
-### Immediate Fixes Required:
-
-1. **Fix config_settings.py paths** - Change default UPLOAD_PATH and MODEL_CACHE_DIR to `/tmp/uploads` and `/tmp/models`
-2. **Fix api_endpoints.py Redis initialization** - Wrap in try-except block
-3. **Fix gradio_app.py** - Make it call agents directly instead of HTTP requests
-4. **Add HF_SPACES environment check** - Disable Redis/DB connections when on HF Spaces
-5. **Ensure all file operations use /tmp** - Audit all file write operations
-
-### Testing Checklist:
-
-- [ ] Test app startup without Redis
-- [ ] Test app startup without Database
-- [ ] Test all API endpoints return proper error messages (not 500) when services unavailable
-- [ ] Test model loading with memory constraints
-- [ ] Test file uploads work with /tmp directory
-- [ ] Test that no operations try to write to read-only filesystem
-- [ ] Verify all external API calls have proper timeouts
-- [ ] Check that lazy loading works for all models
-
-### Environment Variables for HF Spaces:
-
-```bash
-FAST_MODE=true
-PRELOAD_SMALL_MODELS=false
-HF_SPACES=true
-REDIS_URL=  # Empty - don't use Redis
-DATABASE_URL=  # Empty - don't use Database
-UPLOAD_PATH=/tmp/uploads
-MODEL_CACHE_DIR=/tmp/models
-```
-
-## 🔧 Priority Fix Order:
-
-1. **Critical Path Issues** - Redis/DB connections, filesystem paths
-2. **Model Loading** - Memory optimization, lazy loading
-3. **API Endpoints** - Error handling, timeouts
-4. **Gradio Integration** - Direct agent calls
-5. **Monitoring** - Better logging and error messages
-
----
-
-**Generated:** $(date)
-**Scan Coverage:** 15 files, 6000+ lines of code
-**Issues Found:** 18 total (4 critical, 6 high, 8 medium/minor)
-

HF_SPACES_RUNTIME_FIX_SUMMARY.md

@@ -1,81 +0,0 @@
-# Hugging Face Spaces Runtime Error Fix Summary
-
-## Issues Identified and Fixed
-
-### 1. Invalid uvicorn option `--no-reload`
-**Problem**: The Dockerfile was using `--no-reload` which is not a valid uvicorn option.
-**Error**: `Error: No such option: --no-reload (Possible options: --reload, --reload-delay, --reload-dir)`
-
-**Fix Applied**:
-- Updated `Dockerfile` line 226: Removed `--no-reload` option
-- Changed from: `CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860", "--no-reload"]`
-- Changed to: `CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]`
-
-### 2. Permission issues with /tmp directory
-**Problem**: The entrypoint script was trying to `chmod -R 777 /tmp` which fails on Hugging Face Spaces due to permission restrictions.
-**Error**: `chmod: changing permissions of '/tmp': Operation not permitted`
-
-**Fix Applied**:
-- Updated `Dockerfile` entrypoint script to only chmod specific subdirectories
-- Changed from: `chmod -R 777 /tmp`
-- Changed to: `chmod -R 777 /tmp/uploads /tmp/huggingface /tmp/torch /tmp/whisper || true`
-- Updated `entrypoint_optimized.sh` with similar fixes
-
-### 3. Entrypoint configuration
-**Problem**: The `.huggingface.yaml` was pointing to the wrong entrypoint path.
-
-**Fix Applied**:
-- Updated `.huggingface.yaml` to use the correct path: `services/ai-service/src/ai_med_extract/app:app`
-- Created `start_hf_spaces.py` as an alternative startup script
-- Both approaches are now available for deployment
-
-## Files Modified
-
-1. **Dockerfile**:
-   - Fixed uvicorn command (removed `--no-reload`)
-   - Updated entrypoint script to avoid chmod on entire `/tmp`
-
-2. **entrypoint_optimized.sh**:
-   - Updated to only chmod specific directories
-   - Added `|| true` to prevent script failure on permission errors
-
-3. **.huggingface.yaml**:
-   - Updated entrypoint path to correct location
-
-4. **start_hf_spaces.py** (new file):
-   - Alternative startup script for HF Spaces
-   - Handles environment setup and app initialization
-
-5. **test_hf_spaces_fix.py** (new file):
-   - Test script to verify fixes work correctly
-
-## Testing
-
-The fixes have been tested locally and should resolve the runtime errors on Hugging Face Spaces:
-
-1. ✅ uvicorn command now uses valid options
-2. ✅ Permission handling avoids chmod on entire `/tmp`
-3. ✅ App import and initialization should work correctly
-
-## Deployment Instructions
-
-1. Commit these changes to your repository
-2. Push to the branch that your Hugging Face Space is monitoring
-3. The Space should automatically rebuild with the fixes
-4. The runtime errors should be resolved
-
-## Alternative Deployment Options
-
-If the main fixes don't work, you can try:
-
-1. **Use the startup script**: Change `.huggingface.yaml` entrypoint to `python start_hf_spaces.py`
-2. **Use Dockerfile directly**: Ensure the Dockerfile is used instead of `.huggingface.yaml`
-3. **Manual deployment**: Use the `deploy_fix.sh` script if available
-
-## Expected Results
-
-After applying these fixes, the Hugging Face Space should:
-- Start without the `--no-reload` error
-- Avoid permission errors with `/tmp`
-- Successfully initialize the FastAPI application
-- Be accessible on port 7860

HUGGINGFACE_DEPLOYMENT_FIX.md

@@ -1,168 +0,0 @@
-# Hugging Face Spaces Deployment Fix
-
-## Problem Summary
-
-The deployment to Hugging Face Spaces was failing with the error:
-```
-ModuleNotFoundError: No module named 'app'
-[2025-10-07 12:40:17 +0000] [10] [ERROR] Exception in worker process
-[2025-10-07 12:40:17 +0000] [1] [ERROR] Worker (pid:10) exited with code 3
-[2025-10-07 12:40:17 +0000] [1] [ERROR] Reason: Worker failed to boot.
-```
-
-## Root Cause
-
-The `.dockerignore` file was configured to exclude everything (`*`) by default, then selectively include specific files. However, the **root `app.py` file was NOT in the include list**, causing it to be excluded from the Docker build context.
-
-When Hugging Face Spaces built the container and gunicorn tried to run `app:app`, the module couldn't be found because the file didn't exist in the container.
-
-## Fixes Applied
-
-### 1. **CRITICAL: Fixed .dockerignore** ✅
-**File**: `.dockerignore`
-
-Added the missing root files to the include list:
-```diff
-!requirements.txt
-!README.md
-!ai_med_extract.py
-+!app.py
-+!__init__.py
-
-# Include source code (but not cache files)
-+!services/
-+!services/ai-service/
-+!services/ai-service/src/
-!services/ai-service/src/ai_med_extract/
-```
-
-This ensures that:
-- Root `app.py` is copied to the container
-- Root `__init__.py` is included for package support
-- Complete `services/` directory structure is preserved
-
-### 2. **Fixed Missing Import** ✅
-**File**: `services/ai-service/src/ai_med_extract/app.py`
-
-Added missing import at module level:
-```python
-from .utils.model_manager import model_manager
-```
-
-This was being used in the `lifespan` function but wasn't imported, which could cause runtime errors.
-
-### 3. **Enhanced Logging** ✅
-**File**: `app.py` (root)
-
-Added comprehensive debug logging:
-```python
-logging.info(f"Python path: {sys.path[:3]}")
-logging.info(f"Current working directory: {os.getcwd()}")
-logging.info(f"Files in current directory: {os.listdir('.')}")
-```
-
-This provides better visibility for troubleshooting import issues.
-
-### 4. **Updated Hugging Face Config** ✅
-**File**: `.huggingface.yaml`
-
-Added explicit app configuration:
-```yaml
-app:
-  entrypoint: app:app
-  port: 7860
-```
-
-### 5. **Documentation Updates** ✅
-- Added `DEPLOYMENT_FIX_SUMMARY.md` with detailed analysis
-- Updated Dockerfile with clarification comments
-
-## Verification
-
-### Local Testing ✅
-```bash
-python -c "import app; print(app.app.title)"
-# Output: Medical AI Service
-```
-
-All local tests passed:
-- ✅ App imports successfully
-- ✅ App instance created
-- ✅ Agents initialized
-- ✅ No module errors
-
-## Deployment Instructions
-
-1. **Commit the changes**:
-   ```bash
-   git add .
-   git commit -m "Fix HF Spaces deployment - resolve ModuleNotFoundError"
-   ```
-
-2. **Push to Hugging Face Spaces**:
-   ```bash
-   git push origin main
-   ```
-
-3. **Monitor the deployment**:
-   - Check build logs to verify `app.py` is being copied
-   - Check runtime logs for successful import
-   - Verify gunicorn starts without errors
-
-## Expected Behavior
-
-### Before Fix ❌
-```
-ModuleNotFoundError: No module named 'app'
-Worker failed to boot
-Exit code: 3
-```
-
-### After Fix ✅
-```
-[INFO] Starting gunicorn 21.2.0
-[INFO] Listening at: http://0.0.0.0:7860
-[INFO] Booting worker with pid: 10
-[INFO] Attempting to import from ai_med_extract package...
-[INFO] Successfully imported create_app and initialize_agents
-[INFO] App instance created successfully
-[INFO] Agents initialized successfully
-```
-
-## Files Modified
-
-| File | Change | Priority |
-|------|--------|----------|
-| `.dockerignore` | Added `!app.py` and `!__init__.py` | **CRITICAL** |
-| `app.py` | Enhanced logging | High |
-| `services/ai-service/src/ai_med_extract/app.py` | Fixed missing import | High |
-| `.huggingface.yaml` | Added app config | Medium |
-| `Dockerfile` | Added clarification comment | Low |
-
-## Confidence Level
-
-**HIGH** - The root cause has been definitively identified and fixed. Local testing confirms the fix works correctly. The changes are minimal and targeted.
-
-## Rollback Plan
-
-If deployment still fails:
-```bash
-git revert HEAD
-git push origin main
-```
-
-Then investigate additional issues in the Hugging Face Spaces build/runtime logs.
-
-## Success Criteria
-
-- ✅ No `ModuleNotFoundError` during startup
-- ✅ Gunicorn worker boots successfully
-- ✅ App responds to health checks: `/health`
-- ✅ API documentation accessible: `/docs`
-- ✅ No exit code 3 errors
-
----
-
-**Status**: Ready for deployment  
-**Date**: 2025-10-07  
-**Priority**: Critical - Unblocks production deployment

QUICK_REFERENCE.md

@@ -1,157 +0,0 @@
-# HF Spaces Deployment - Quick Reference Card
-
-## 🚀 Quick Deploy Checklist
-
-### 1. Environment Variables (Set in HF Spaces Settings)
-```bash
-HF_SPACES=true
-FAST_MODE=true
-PRELOAD_SMALL_MODELS=false
-REDIS_URL=
-DATABASE_URL=
-```
-
-### 2. Verify Before Pushing
-```bash
-✓ All changes committed
-✓ requirements.txt up to date
-✓ No hardcoded localhost URLs
-✓ No hardcoded /app/ or /data/ paths
-```
-
-### 3. After Deployment
-```bash
-✓ Check startup logs
-✓ Test /health endpoint
-✓ Test main API endpoints
-✓ Monitor memory usage
-```
-
----
-
-## 🔧 Files Modified (7 Total)
-
-| File | Change |
-|------|--------|
-| `config_settings.py` | Paths → /tmp, Redis/DB defaults → empty |
-| `app.py` (root) | HF_SPACES detection |
-| `app.py` (ai_med_extract) | Redis/DB optional |
-| `api_endpoints.py` | Redis init with error handling |
-| `gradio_app.py` | Direct agent calls (no HTTP) |
-| `file_utils.py` | HF_SPACES-aware paths |
-| `routes_fastapi.py` | Better error handling |
-
----
-
-## ⚠️ Common Issues & Solutions
-
-| Issue | Solution |
-|-------|----------|
-| **500 Error on Startup** | Check logs for missing env vars |
-| **Redis Connection Error** | Set `REDIS_URL=` (empty) |
-| **Database Connection Error** | Set `DATABASE_URL=` (empty) |
-| **File Write Error** | Paths should use /tmp |
-| **Memory Error** | Set `FAST_MODE=true`, `PRELOAD_SMALL_MODELS=false` |
-| **Timeout Error** | External API calls - expected behavior |
-
----
-
-## 🧪 Quick Test Commands
-
-### Test Health
-```bash
-curl https://your-space.hf.space/health
-curl https://your-space.hf.space/ready
-```
-
-### Test Summarization
-```bash
-curl -X POST https://your-space.hf.space/summarize \
-  -H "Content-Type: application/json" \
-  -d '{"text":"Medical text here"}'
-```
-
-### Test PHI Scrubbing
-```bash
-curl -X POST https://your-space.hf.space/phi/scrub \
-  -H "Content-Type: application/json" \
-  -d '{"text":"Patient name: John Doe"}'
-```
-
----
-
-## 📊 What's Fixed
-
-| Category | Status |
-|----------|--------|
-| Redis Dependency | ✅ Optional |
-| Database Dependency | ✅ Optional |
-| File Operations | ✅ Use /tmp |
-| Gradio Localhost | ✅ Direct calls |
-| Error Handling | ✅ User-friendly |
-| Memory Usage | ✅ Optimized |
-
----
-
-## 🔍 Monitoring Checklist
-
-- [ ] Startup logs show "Detected Hugging Face Spaces environment"
-- [ ] No Redis connection errors
-- [ ] No Database connection errors
-- [ ] All endpoints return proper status codes
-- [ ] Error messages are user-friendly
-- [ ] Memory usage < 16GB (Basic tier)
-- [ ] Response times < 30s
-
----
-
-## 📞 Emergency Debug
-
-If app crashes on HF Spaces:
-
-1. **Check Startup Logs** - Look for first error
-2. **Verify Env Vars** - HF_SPACES=true set?
-3. **Test Locally** - `export HF_SPACES=true && python app.py`
-4. **Check Memory** - Model too large?
-5. **Review Fixes** - See `HF_SPACES_FIXES_APPLIED.md`
-
----
-
-## 📚 Full Documentation
-
-- **Issues Found:** `HF_SPACES_ISSUES_REPORT.md`
-- **Fixes Applied:** `HF_SPACES_FIXES_APPLIED.md`
-- **Summary:** `SCAN_SUMMARY.md`
-- **This Card:** `QUICK_REFERENCE.md`
-
----
-
-## ✅ Success Indicators
-
-Your deployment is successful if:
-
-✓ App starts without crashes
-✓ /health returns 200
-✓ API endpoints respond (even if with errors)
-✓ No 500 errors in logs
-✓ Memory usage stable
-✓ Error messages are informative
-
----
-
-## 🎯 Expected Behavior on HF Spaces
-
-| Feature | Behavior |
-|---------|----------|
-| Redis | Disabled, features degraded gracefully |
-| Database | Disabled, no audit logging |
-| File Uploads | Work in /tmp |
-| Model Loading | Lazy, optimized for memory |
-| External APIs | May timeout, handled gracefully |
-| Caching | Limited to /tmp (ephemeral) |
-
----
-
-*Quick Reference for HF Spaces Deployment*
-*For detailed information, see full documentation*
-

README.md

@@ -1,83 +1,361 @@
----
-title: HNTAI - Medical Data Extraction API
-emoji: 📉
-colorFrom: blue
-colorTo: green
-sdk: docker
-app_port: 7860
-pinned: false
----
+# HNTAI - Medical Data Extraction & AI Processing Platform
 
-# HNTAI - Scalable Medical Data Extraction API
-
-This is a FastAPI-based scalable API for extracting and processing medical data from various document formats, aligned with "ChatGPT Version 3 - Scalable" architecture.
-
-## Features
-- Document text extraction (PDF, DOCX, Images)
-- Audio transcription
-- Medical data extraction
-- PHI (Protected Health Information) scrubbing with audit logging
-- Text summarization with Redis caching
-- PostgreSQL database integration for persistence
-- Async processing for scalability
-- Health endpoints (/live, /ready)
-- Security features (non-root containers, secrets management, HIPAA compliance)
-
-## Architecture Alignment
-Fully aligned with "ChatGPT Version 3 - Scalable":
-- FastAPI for async API handling
-- Redis for caching and PHI stats
-- PostgreSQL for audit logs and data persistence
-- Kubernetes deployment with security contexts
-- Network policies and HIPAA compliance
-- Prometheus monitoring
-- Proper resource limits and health probes
-
-## Deployment Options
-- **Hugging Face Spaces**: Lightweight Docker deployment (legacy)
-- **Kubernetes**: Scalable production deployment with security features
-
-## Environment Variables
-- `DATABASE_URL`: PostgreSQL connection string
-- `REDIS_URL`: Redis connection string
-- `SECRET_KEY`: Application secret key
-- `JWT_SECRET_KEY`: JWT signing key
-
-## API Endpoints
-- GET /health/live - Liveness health check
-- GET /health/ready - Readiness health check
-- GET /metrics - Prometheus metrics
-- POST /generate_patient_summary - Generate comprehensive patient summaries (with streaming support)
-- POST /upload - Upload and process medical documents
-- GET /get_updated_medical_data - Retrieve processed medical data
-- PUT /update_medical_data - Update medical data fields
-- POST /transcribe - Transcribe audio files
-- POST /extract_medical_data - Extract structured medical data
-- POST /api/generate_summary - Generate text summaries
-- POST /api/extract_medical_data_from_audio - Process audio recordings
-- POST /api/patient_summary_openvino - Generate patient summaries using OpenVINO
-
-## Development
+A comprehensive, scalable AI platform for medical data extraction, processing, and analysis. Built with FastAPI, supporting multiple AI model backends including Transformers, OpenVINO, and GGUF models with automatic GPU/CPU optimization.
 
-### Code Quality
-This project uses the following tools for code quality:
-- **Black**: Code formatting
-- **isort**: Import sorting
-- **flake8**: Linting
-- **mypy**: Type checking
+## 🏥 Overview
+
+HNTAI is a production-ready medical AI platform that provides:
+- **Medical Document Processing**: PDF, DOCX, image, and audio transcription
+- **Protected Health Information (PHI) Scrubbing**: HIPAA-compliant data anonymization
+- **AI-Powered Summarization**: Multi-model support with automatic device optimization
+- **Patient Summary Generation**: Comprehensive clinical assessments
+- **Scalable Architecture**: Kubernetes-ready with monitoring and security features
+
+## 🚀 Key Features
+
+### 🤖 Multi-Model AI Support
+- **Transformers Models**: Hugging Face models with automatic GPU/CPU detection
+- **OpenVINO Optimization**: Intel-optimized models for production performance
+- **GGUF Models**: Quantized models for efficient inference
+- **Automatic Device Selection**: GPU when available, CPU fallback
+- **Model Caching**: Intelligent model management and caching
+
+### 📄 Document Processing
+- **Multi-format Support**: PDF, DOCX, images, audio files
+- **OCR Integration**: Tesseract-based text extraction
+- **Audio Transcription**: Whisper-based speech-to-text
+- **Batch Processing**: Async processing for scalability
+
+### 🔒 Security & Compliance
+- **HIPAA Compliance**: PHI scrubbing with audit logging
+- **Data Encryption**: Secure data handling and storage
+- **Audit Trails**: Comprehensive logging for compliance
+- **Non-root Containers**: Security-hardened deployments
+
+### 📊 Monitoring & Observability
+- **Health Endpoints**: `/health/live`, `/health/ready`
+- **Prometheus Metrics**: `/metrics` endpoint
+- **Structured Logging**: Comprehensive application monitoring
+- **Performance Tracking**: Model inference metrics
+
+## 🏗️ Architecture
+
+```
+┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
+│   FastAPI       │    │   AI Models     │    │   PostgreSQL    │
+│   Web Server    │◄──►│   (Multi-backend)│    │   Database      │
+└─────────────────┘    └─────────────────┘    └─────────────────┘
+         │                       │                       │
+         ▼                       ▼                       ▼
+┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
+│   Redis Cache   │    │   File Storage  │    │   Audit Logs    │
+│   (PHI Stats)   │    │   (Documents)   │    │   (Compliance)   │
+└─────────────────┘    └─────────────────┘    └─────────────────┘
+```
+
+## 🛠️ Installation
+
+### Prerequisites
+- Python 3.11+
+- CUDA 11.8+ (for GPU support)
+- Docker (for containerized deployment)
+- PostgreSQL 13+
+- Redis 6+
+
+### Local Development
+
+1. **Clone the repository**:
+```bash
+git clone <repository-url>
+cd HNTAI
+```
+
+2. **Create virtual environment**:
+```bash
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+```
+
+3. **Install dependencies**:
+```bash
+pip install -r requirements.txt
+```
+
+4. **Set up environment variables**:
+```bash
+export DATABASE_URL="postgresql://user:password@localhost:5432/hntai"
+export REDIS_URL="redis://localhost:6379"
+export SECRET_KEY="your-secret-key"
+export JWT_SECRET_KEY="your-jwt-secret"
+export HF_HOME="/tmp/huggingface"
+```
+
+5. **Run the application**:
+```bash
+# Development server
+python -m uvicorn services.ai-service.src.ai_med_extract.main:app --reload --host 0.0.0.0 --port 7860
+
+# Or using the service directly
+cd services/ai-service
+python src/ai_med_extract/main.py
+```
+
+### Docker Deployment
+
+1. **Build the image**:
+```bash
+docker build -t hntai:latest .
+```
+
+2. **Run with Docker Compose**:
+```bash
+docker-compose up -d
+```
+
+### Kubernetes Deployment
+
+1. **Apply Kubernetes manifests**:
+```bash
+kubectl apply -f infra/k8s/secure_deployment.yaml
+```
+
+2. **Check deployment status**:
+```bash
+kubectl get pods -l app=hntai
+```
+
+## 📚 API Documentation
+
+### Core Endpoints
+
+#### Health & Monitoring
+- `GET /health/live` - Liveness probe
+- `GET /health/ready` - Readiness probe  
+- `GET /metrics` - Prometheus metrics
+
+#### Document Processing
+- `POST /upload` - Upload and process documents
+- `POST /transcribe` - Transcribe audio files
+- `GET /get_updated_medical_data` - Retrieve processed data
+- `PUT /update_medical_data` - Update medical data
+
+#### AI Processing
+- `POST /generate_patient_summary` - Generate comprehensive patient summaries
+- `POST /api/generate_summary` - Generate text summaries
+- `POST /api/patient_summary_openvino` - OpenVINO-optimized summaries
+- `POST /extract_medical_data` - Extract structured medical data
+
+### Model Management
+- `POST /api/load_model` - Load specific AI models
+- `GET /api/model_info` - Get model information
+- `POST /api/switch_model` - Switch between models
+
+## 🤖 AI Model Configuration
+
+### Supported Model Types
 
-Run quality checks:
+#### 1. Transformers Models
+```python
+{
+    "model_name": "microsoft/Phi-3-mini-4k-instruct",
+    "model_type": "text-generation"
+}
+```
+
+#### 2. OpenVINO Models
+```python
+{
+    "model_name": "OpenVINO/Phi-3-mini-4k-instruct-fp16-ov",
+    "model_type": "openvino"
+}
+```
+
+#### 3. GGUF Models
+```python
+{
+    "model_name": "microsoft/Phi-3-mini-4k-instruct-gguf",
+    "model_type": "gguf"
+}
+```
+
+### Automatic Device Detection
+The system automatically detects and uses:
+- **GPU**: When CUDA is available
+- **CPU**: Fallback when GPU is not available
+- **Optimization**: Intel OpenVINO for production performance
+
+## 🔧 Configuration
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `DATABASE_URL` | PostgreSQL connection string | Required |
+| `REDIS_URL` | Redis connection string | Required |
+| `SECRET_KEY` | Application secret key | Required |
+| `JWT_SECRET_KEY` | JWT signing key | Required |
+| `HF_HOME` | Hugging Face cache directory | `/tmp/huggingface` |
+| `TORCH_HOME` | PyTorch cache directory | `/tmp/torch` |
+| `WHISPER_CACHE` | Whisper model cache | `/tmp/whisper` |
+| `HF_SPACES` | Hugging Face Spaces mode | `false` |
+| `PRELOAD_GGUF` | Preload GGUF models | `false` |
+
+### Model Configuration
+
+The system supports flexible model configuration through `model_config.py`:
+
+```python
+# Default models for different tasks
+DEFAULT_MODELS = {
+    "text-generation": {
+        "primary": "microsoft/Phi-3-mini-4k-instruct",
+        "fallback": "facebook/bart-base"
+    },
+    "openvino": {
+        "primary": "OpenVINO/Phi-3-mini-4k-instruct-fp16-ov",
+        "fallback": "microsoft/Phi-3-mini-4k-instruct"
+    },
+    "gguf": {
+        "primary": "microsoft/Phi-3-mini-4k-instruct-gguf",
+        "fallback": "microsoft/Phi-3-mini-4k-instruct-gguf"
+    }
+}
+```
+
+## 🧪 Testing
+
+### Run Tests
+```bash
+# Unit tests
+python -m pytest tests/
+
+# Smoke test (no model loading)
+cd services/ai-service
+python run_smoke_test.py
+
+# Integration tests
+python -m pytest tests/integration/
+```
+
+### Code Quality
 ```bash
+# Format code
 black .
 isort .
+
+# Lint code
 flake8 .
 mypy .
+
+# Type checking
+mypy services/ai-service/src/ai_med_extract/
+```
+
+## 📊 Monitoring
+
+### Health Checks
+- **Liveness**: `GET /health/live` - Application is running
+- **Readiness**: `GET /health/ready` - Application is ready to serve requests
+
+### Metrics
+- **Prometheus**: `GET /metrics` - Application and model metrics
+- **Custom Metrics**: Model inference time, success rates, error rates
+
+### Logging
+- **Structured Logging**: JSON-formatted logs
+- **Audit Trails**: PHI access and modification logs
+- **Performance Logs**: Model loading and inference timing
+
+## 🔒 Security Features
+
+### HIPAA Compliance
+- **PHI Scrubbing**: Automatic removal of protected health information
+- **Audit Logging**: Comprehensive access and modification logs
+- **Data Encryption**: Secure data handling and storage
+- **Access Controls**: Role-based access to sensitive data
+
+### Container Security
+- **Non-root Containers**: Security-hardened container images
+- **Resource Limits**: CPU and memory limits
+- **Network Policies**: Secure network communication
+- **Secrets Management**: Secure handling of sensitive configuration
+
+## 🚀 Deployment Options
+
+### 1. Local Development
+```bash
+python -m uvicorn services.ai-service.src.ai_med_extract.main:app --reload
 ```
 
-### Testing
-Run tests with:
+### 2. Docker
 ```bash
-python -m pytest
+docker run -p 7860:7860 hntai:latest
 ```
 
-For more details, check the API documentation at `/docs`, [DEVELOPMENT.md](DEVELOPMENT.md) for development guides, and [DEPLOYMENT.md](DEPLOYMENT.md) for deployment instructions.
+### 3. Kubernetes
+```bash
+kubectl apply -f infra/k8s/secure_deployment.yaml
+```
+
+### 4. Hugging Face Spaces
+```bash
+# Configure for HF Spaces
+export HF_SPACES=true
+python start_hf_spaces.py
+```
+
+## 📁 Project Structure
+
+```
+HNTAI/
+├── services/
+│   └── ai-service/
+│       ├── src/ai_med_extract/
+│       │   ├── agents/           # AI agents and processors
+│       │   ├── api/             # FastAPI routes and management
+│       │   ├── utils/           # Utilities and model management
+│       │   ├── app.py          # Main application
+│       │   └── main.py         # Application entry point
+│       ├── docker-compose.yml  # Docker services
+│       └── Dockerfile          # Container image
+├── infra/
+│   └── k8s/                   # Kubernetes manifests
+├── monitoring/
+│   └── prometheus.yml         # Monitoring configuration
+├── database/
+│   └── postgresql/           # Database schemas
+└── requirements.txt          # Python dependencies
+```
+
+## 🤝 Contributing
+
+1. **Fork the repository**
+2. **Create a feature branch**: `git checkout -b feature/amazing-feature`
+3. **Make your changes**
+4. **Run tests**: `python -m pytest`
+5. **Commit changes**: `git commit -m 'Add amazing feature'`
+6. **Push to branch**: `git push origin feature/amazing-feature`
+7. **Open a Pull Request**
+
+## 📄 License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## 🆘 Support
+
+- **Documentation**: Check the `/docs` endpoint for interactive API documentation
+- **Issues**: Report bugs and feature requests via GitHub Issues
+- **Discussions**: Join community discussions for questions and support
+
+## 🔄 Changelog
+
+### Latest Updates
+- ✅ **Fixed OpenVINO GPU/CPU auto-detection**
+- ✅ **Improved model loading with fallback mechanisms**
+- ✅ **Enhanced security and HIPAA compliance**
+- ✅ **Added comprehensive monitoring and health checks**
+- ✅ **Optimized for production deployment**
+
+---
+
+**Built with ❤️ for the medical AI community**

README_HF_SPACES.md

@@ -1,72 +0,0 @@
-# Hugging Face Spaces Deployment
-
-This document explains the changes made to support Hugging Face Spaces deployment.
-
-## Changes Made
-
-### 1. Root-level Entry Point (`app.py`)
-- Created a root-level `app.py` file that serves as the entry point for Hugging Face Spaces
-- This file imports the FastAPI app from the `ai_med_extract` package
-- Includes multiple fallback strategies for robust error handling
-- Added comprehensive logging for debugging
-
-### 2. Package Structure
-- Added `__init__.py` at the root level to make it a proper Python package
-- The main application code remains in `services/ai-service/src/ai_med_extract/`
-
-### 3. Requirements File
-- Created a root-level `requirements.txt` with all necessary dependencies
-- This is used by Hugging Face Spaces for dependency installation
-
-### 4. Environment Configuration
-- Set `FAST_MODE=true` and `PRELOAD_SMALL_MODELS=false` for Hugging Face Spaces
-- This ensures faster startup and reduced memory usage
-
-### 5. Dockerfile Updates
-- Updated the Dockerfile to use `app:app` instead of `ai_med_extract.app:app`
-- Added cache clearing configuration in `.huggingface.yaml`
-
-## How It Works
-
-1. Hugging Face Spaces looks for `app.py` at the root level
-2. The `app.py` file adds the source directory to the Python path
-3. It tries multiple import strategies:
-   - Primary: Import from `ai_med_extract.app`
-   - Fallback: Direct import from nested structure
-   - Emergency: Create minimal FastAPI app
-4. The app is initialized with minimal preloading for faster startup
-
-## Fallback Strategies
-
-The app includes three levels of fallback:
-
-1. **Primary**: Normal import from `ai_med_extract.app`
-2. **Fallback**: Direct import from nested structure if package import fails
-3. **Emergency**: Minimal FastAPI app if all imports fail
-
-## Testing
-
-To test the import structure locally:
-
-```bash
-python -c "import app; print('App imported successfully:', app.app.title)"
-```
-
-## Deployment
-
-The app should now work correctly when deployed to Hugging Face Spaces. The key changes ensure that:
-
-- The module structure is properly recognized
-- Dependencies are correctly installed
-- The app starts with minimal resource usage
-- Multiple fallback strategies provide robust error handling
-- Comprehensive logging helps with debugging
-
-## Troubleshooting
-
-If you still encounter issues:
-
-1. **Check the logs** - The app now includes comprehensive logging
-2. **Verify file structure** - Ensure all files are in the correct locations
-3. **Clear cache** - The `.huggingface.yaml` includes cache clearing
-4. **Check dependencies** - Ensure all requirements are properly specified

SCAN_SUMMARY.md

@@ -1,294 +0,0 @@
-# Hugging Face Spaces API Internal Server Error Scan - Summary
-
-## 🎯 Scan Complete
-
-**Date:** $(date)
-**Status:** ✅ All Critical Issues Resolved
-**Files Scanned:** 15+ files, 6000+ lines of code
-**Issues Found:** 18 (4 critical, 6 high, 8 medium/minor)
-**Issues Fixed:** 18 (100%)
-
----
-
-## 📊 Executive Summary
-
-The codebase has been thoroughly scanned for issues that could cause internal server errors (HTTP 500) when deployed to Hugging Face Spaces. All critical and high-severity issues have been identified and resolved.
-
-**Main Problems Identified:**
-1. ❌ Redis connections attempted at startup and module import time
-2. ❌ Database connections attempted without proper fallbacks
-3. ❌ File operations using read-only filesystem paths
-4. ❌ Gradio app making localhost HTTP requests
-5. ❌ Poor error handling for external API timeouts
-
-**Status After Fixes:**
-1. ✅ Redis completely optional with graceful degradation
-2. ✅ Database completely optional with proper fallbacks
-3. ✅ All file operations use /tmp directory
-4. ✅ Gradio app uses direct agent calls
-5. ✅ Comprehensive error handling with user-friendly messages
-
----
-
-## 📁 Documents Generated
-
-### 1. `HF_SPACES_ISSUES_REPORT.md`
-- Comprehensive list of all 18 issues found
-- Detailed descriptions of each issue
-- Impact assessment and severity ratings
-- Recommendations for fixes
-
-### 2. `HF_SPACES_FIXES_APPLIED.md`
-- Complete documentation of all fixes applied
-- Before/after code comparisons
-- Testing procedures for each fix
-- Environment variable configuration
-- Deployment checklist
-
-### 3. `SCAN_SUMMARY.md` (this file)
-- High-level overview
-- Quick reference for key changes
-- Next steps for deployment
-
----
-
-## 🔥 Critical Fixes Applied
-
-### 1. Redis Connection Fix
-**Problem:** App tried to connect to Redis at startup, causing hangs or crashes.
-
-**Solution:**
-- Added HF_SPACES environment detection
-- Redis connections skipped entirely on HF Spaces
-- Module-level initialization wrapped in try-except
-- Graceful fallback when Redis unavailable
-
-**Files Changed:**
-- `services/ai-service/src/ai_med_extract/app.py`
-- `services/ai-service/src/ai_med_extract/api_endpoints.py`
-- `app.py`
-
-### 2. Filesystem Path Fix
-**Problem:** App tried to write to `/app/uploads` and `/app/models` (read-only on HF Spaces).
-
-**Solution:**
-- Changed all default paths to `/tmp/uploads` and `/tmp/models`
-- Added error handling for directory creation failures
-- HF_SPACES-aware path resolution
-
-**Files Changed:**
-- `services/ai-service/src/config_settings.py`
-- `services/ai-service/src/ai_med_extract/utils/file_utils.py`
-- `services/ai-service/src/ai_med_extract/app.py`
-
-### 3. Gradio Localhost Fix
-**Problem:** Gradio app made HTTP requests to localhost, which fails on HF Spaces.
-
-**Solution:**
-- Completely rewrote gradio_app.py
-- Functions now call agents directly
-- Proper async/await handling
-- Comprehensive error handling
-
-**Files Changed:**
-- `services/ai-service/src/ai_med_extract/gradio_app.py`
-
-### 4. Database Connection Fix
-**Problem:** App tried to connect to PostgreSQL database.
-
-**Solution:**
-- Database connections skipped on HF Spaces
-- Empty DATABASE_URL prevents connection attempts
-- Audit logging gracefully disabled when DB unavailable
-
-**Files Changed:**
-- `services/ai-service/src/ai_med_extract/app.py`
-- `services/ai-service/src/config_settings.py`
-
-### 5. Error Handling Improvements
-**Problem:** Generic 500 errors with no useful information.
-
-**Solution:**
-- Added specific exception handling for timeouts, connections, etc.
-- Error categorization (TIMEOUT, CONNECTION, EHR_API, MEMORY, GENERAL)
-- User-friendly error messages
-- Proper logging of error details
-
-**Files Changed:**
-- `services/ai-service/src/ai_med_extract/api/routes_fastapi.py`
-
----
-
-## 🎨 Code Changes Summary
-
-### Lines Modified: ~500+
-### Files Modified: 7
-
-1. **config_settings.py** - Paths and defaults
-2. **app.py** (root) - HF_SPACES detection
-3. **app.py** (ai_med_extract) - Redis/DB handling
-4. **api_endpoints.py** - Redis initialization
-5. **gradio_app.py** - Complete rewrite
-6. **file_utils.py** - Path resolution
-7. **routes_fastapi.py** - Error handling
-
----
-
-## 🚀 Deployment Instructions
-
-### Step 1: Set Environment Variables
-
-In your Hugging Face Space settings, add:
-
-```bash
-HF_SPACES=true
-FAST_MODE=true
-PRELOAD_SMALL_MODELS=false
-REDIS_URL=
-DATABASE_URL=
-UPLOAD_PATH=/tmp/uploads
-MODEL_CACHE_DIR=/tmp/models
-```
-
-### Step 2: Push Code
-
-```bash
-git add .
-git commit -m "Fix HF Spaces compatibility issues"
-git push origin main
-```
-
-### Step 3: Verify Deployment
-
-1. Check startup logs for "Detected Hugging Face Spaces environment"
-2. Test health endpoints: `/health`, `/ready`, `/live`
-3. Test main API endpoints
-4. Verify no 500 errors in logs
-
-### Step 4: Monitor
-
-- Watch memory usage
-- Check error rates
-- Verify model loading times
-
----
-
-## ✅ Testing Checklist
-
-- [ ] App starts without Redis
-- [ ] App starts without Database
-- [ ] All file operations use /tmp
-- [ ] Health endpoints return 200
-- [ ] API endpoints return proper errors (not 500)
-- [ ] Gradio interface works
-- [ ] External API timeouts handled gracefully
-- [ ] Memory usage stays within limits
-- [ ] Model loading is lazy and efficient
-- [ ] Error messages are user-friendly
-
----
-
-## 📈 Before vs After
-
-### Before:
-- ❌ App crashes on startup without Redis
-- ❌ App crashes on startup without Database
-- ❌ File operations fail due to read-only filesystem
-- ❌ Gradio interface doesn't work
-- ❌ Generic 500 errors with no information
-- ❌ External API timeouts crash the app
-
-### After:
-- ✅ App starts successfully without Redis
-- ✅ App starts successfully without Database
-- ✅ All file operations work in /tmp
-- ✅ Gradio interface works perfectly
-- ✅ User-friendly error messages with categories
-- ✅ External API timeouts handled gracefully
-- ✅ Proper fallbacks for all critical paths
-- ✅ Comprehensive logging for debugging
-
----
-
-## 🔍 What Was Not Changed
-
-The following were intentionally left unchanged to maintain functionality:
-
-1. **Core business logic** - All medical AI functionality preserved
-2. **API interface contracts** - Endpoints maintain same request/response format
-3. **Model functionality** - Model loading and inference unchanged
-4. **Security features** - All security middleware preserved
-5. **Backward compatibility** - Works on non-HF Spaces environments
-
----
-
-## 🎓 Key Learnings
-
-### HF Spaces Constraints:
-1. Read-only filesystem except /tmp
-2. No Redis available by default
-3. No PostgreSQL available by default
-4. Localhost HTTP requests don't work
-5. Memory limits on free tier
-
-### Best Practices Applied:
-1. Environment detection (HF_SPACES flag)
-2. Graceful degradation
-3. Comprehensive error handling
-4. Proper async/await patterns
-5. Lazy loading for resources
-6. User-friendly error messages
-
----
-
-## 📞 Next Actions
-
-### Immediate:
-1. ✅ Review the fixes applied
-2. ✅ Test locally with HF_SPACES=true
-3. ⏭️ Deploy to Hugging Face Spaces
-4. ⏭️ Monitor for issues
-
-### Short-term:
-1. Add integration tests for HF Spaces mode
-2. Document API behavior when services unavailable
-3. Add monitoring/alerting
-4. Optimize memory usage further
-
-### Long-term:
-1. Consider adding Redis support (if HF Spaces adds it)
-2. Implement persistent storage alternatives
-3. Add rate limiting without Redis
-4. Improve caching strategies
-
----
-
-## 📚 Reference Documents
-
-1. **HF_SPACES_ISSUES_REPORT.md** - Full issue analysis
-2. **HF_SPACES_FIXES_APPLIED.md** - Complete fix documentation
-3. **README_HF_SPACES.md** - Deployment guide
-4. **requirements.txt** - Dependencies (already HF Spaces compatible)
-
----
-
-## 🎉 Conclusion
-
-The application is now **production-ready** for Hugging Face Spaces deployment. All critical issues have been resolved, and the app will:
-
-✅ Start successfully without external dependencies
-✅ Handle errors gracefully
-✅ Provide useful error messages to users
-✅ Use only writable filesystem locations
-✅ Work within HF Spaces memory constraints
-✅ Maintain backward compatibility with other deployment environments
-
-**Risk Level:** Low ✅
-**Deployment Confidence:** High 🚀
-**Estimated Success Rate:** 95%+
-
----
-
-*Scan completed and documented on $(date)*
-*All critical and high-severity issues have been resolved*
-

STREAMING_FIX_SUMMARY.md

@@ -1,175 +0,0 @@
-# Streaming API Fix Summary
-
-## Issue Description
-The `generate_patient_summary` API with `stream=true` was stopping after sending heartbeat events, with no completion or error messages being streamed to the client.
-
-## Root Cause Analysis
-1. **Improper async handling**: The `process_patient_summary_background` function was using `asyncio.run()` inside a thread, which can cause event loop conflicts.
-2. **Insufficient error handling**: Errors during GGUF model generation were not being properly caught and reported through the streaming interface.
-3. **No timeout protection**: The GGUF model generation could hang indefinitely without any timeout mechanism.
-4. **Limited progress feedback**: The streaming interface wasn't providing detailed progress updates during the generation process.
-
-## Fixes Applied
-
-### 1. Improved Background Processing (`routes_fastapi.py`)
-- **Before**: Used `asyncio.run()` which can cause event loop conflicts
-- **After**: Created a new event loop for the thread using `asyncio.new_event_loop()`
-- **Added**: Comprehensive error handling with stack traces
-- **Added**: Proper cleanup of event loops
-
-```python
-def process_patient_summary_background(data, job_id):
-    """Background task for patient summary generation"""
-    print(f"Background task started for job_id: {job_id}")
-    try:
-        # Create a new event loop for this thread
-        loop = asyncio.new_event_loop()
-        asyncio.set_event_loop(loop)
-        
-        try:
-            result = loop.run_until_complete(async_patient_summary(data, job_id))
-            update_job(job_id, 'completed', progress=100, data=result)
-            print(f"Background task completed successfully for job_id: {job_id}")
-        except Exception as e:
-            print(f"Async task error for job_id {job_id}: {str(e)}")
-            import traceback
-            traceback.print_exc()
-            update_job(job_id, 'error', error=str(e))
-        finally:
-            loop.close()
-    except Exception as e:
-        print(f"Background task error for job_id {job_id}: {str(e)}")
-        import traceback
-        traceback.print_exc()
-        update_job(job_id, 'error', error=str(e))
-```
-
-### 2. Enhanced GGUF Generation Error Handling
-- **Added**: Timeout protection (5 minutes) for GGUF model generation
-- **Added**: Specific error handling for GGUF generation failures
-- **Added**: Progress updates during generation process
-
-```python
-try:
-    # Add timeout to prevent hanging
-    if job_id:
-        update_job(job_id, 'processing', progress=75, data={'message': 'Running GGUF model inference...'})
-    
-    raw_summary = await asyncio.wait_for(
-        asyncio.to_thread(pipeline.generate, full_prompt, max_tokens=1500, temperature=0.1, top_p=0.5),
-        timeout=300  # 5 minutes timeout
-    )
-    print(f"GGUF raw summary length: {len(raw_summary)} chars")
-    
-    if job_id:
-        update_job(job_id, 'processing', progress=85, data={'message': 'Processing generated summary...'})
-except asyncio.TimeoutError:
-    error_msg = "GGUF generation timed out after 5 minutes"
-    print(error_msg)
-    if job_id:
-        update_job(job_id, 'error', error=error_msg)
-    raise Exception(error_msg)
-except Exception as e:
-    print(f"GGUF generation failed: {str(e)}")
-    if job_id:
-        update_job(job_id, 'error', error=f"GGUF generation failed: {str(e)}")
-    raise Exception(f"GGUF model generation failed: {str(e)}")
-```
-
-### 3. Improved SSE Generator
-- **Added**: Overall timeout protection (10 minutes max wait time)
-- **Added**: Elapsed time tracking in events
-- **Added**: Better error reporting with status information
-- **Added**: Conditional heartbeat sending (only for active processing states)
-
-```python
-def sse_generator(job_id):
-    import json
-    start_time = time.time()
-    max_wait_time = 600  # 10 minutes max wait time
-    
-    while True:
-        current_time = time.time()
-        elapsed_time = current_time - start_time
-        
-        with job_lock:
-            if job_id not in jobs:
-                yield f"data: {json.dumps({'type': 'error', 'error': 'Job not found'})}\n\n"
-                break
-                
-            job = jobs[job_id]
-            status = job.get('status', 'unknown')
-            progress = job.get('progress', 0)
-            data = job.get('data', {})
-            error = job.get('error')
-            
-            # Check for timeout
-            if elapsed_time > max_wait_time:
-                yield f"data: {json.dumps({'type': 'error', 'error': 'Job timed out after 10 minutes'})}\n\n"
-                cleanup_job(job_id)
-                break
-                
-            if error:
-                yield f"data: {json.dumps({'type': 'error', 'error': error, 'status': status})}\n\n"
-                threading.Timer(5.0, lambda: cleanup_job(job_id)).start()
-                break
-                
-            event_data = {
-                'type': 'progress',
-                'status': status,
-                'progress': progress,
-                'data': data,
-                'elapsed_time': round(elapsed_time, 1)
-            }
-            yield f"data: {json.dumps(event_data)}\n\n"
-            
-            if status == 'completed':
-                yield f"data: {json.dumps({'type': 'complete', 'data': data})}\n\n"
-                threading.Timer(5.0, lambda: cleanup_job(job_id)).start()
-                break
-                
-            # Only send heartbeat if we're still processing
-            if status in ['queued', 'processing', 'started', 'ehr_success', 'processing_data']:
-                yield f"data: {json.dumps({'type': 'heartbeat', 'status': status, 'elapsed_time': round(elapsed_time, 1)})}\n\n"
-                
-        time.sleep(1)
-    yield "data: [DONE]\n\n"
-```
-
-### 4. Enhanced Progress Updates
-- **Added**: More granular progress updates during GGUF generation
-- **Added**: Final progress update before completion
-- **Added**: Better status messages for each processing stage
-
-## Expected Behavior After Fix
-
-### Successful Stream Flow:
-1. `{"type": "progress", "status": "queued", "progress": 0, "data": {"job_id": "...", "message": "Job queued ..."}}`
-2. `{"type": "heartbeat", "status": "queued"}`
-3. `{"type": "progress", "status": "started", "progress": 5, "data": {"message": "Task started"}}`
-4. `{"type": "progress", "status": "ehr_success", "progress": 20, "data": {"message": "EHR data fetched successfully"}}`
-5. `{"type": "progress", "status": "processing_data", "progress": 30, "data": {"message": "Processing patient data"}}`
-6. `{"type": "progress", "status": "processing", "progress": 60, "data": {"message": "Generating summary with gguf model..."}}`
-7. `{"type": "progress", "status": "processing", "progress": 70, "data": {"message": "Generating summary with GGUF model..."}}`
-8. `{"type": "progress", "status": "processing", "progress": 75, "data": {"message": "Running GGUF model inference..."}}`
-9. `{"type": "progress", "status": "processing", "progress": 85, "data": {"message": "Processing generated summary..."}}`
-10. `{"type": "progress", "status": "processing", "progress": 95, "data": {"message": "Finalizing summary..."}}`
-11. `{"type": "complete", "data": {"summary": "...", "baseline": "...", "delta": "...", "timing": {...}}}`
-
-### Error Stream Flow:
-1. Progress events as above until error occurs
-2. `{"type": "error", "error": "GGUF generation failed: [specific error]", "status": "processing"}`
-
-## Testing
-A test script `test_streaming_fix.py` has been created to verify the streaming functionality with the provided payload.
-
-## Files Modified
-- `services/ai-service/src/ai_med_extract/api/routes_fastapi.py` - Main fixes applied
-- `test_streaming_fix.py` - Test script for verification
-- `STREAMING_FIX_SUMMARY.md` - This documentation
-
-## Deployment Notes
-- The fixes are backward compatible
-- No database schema changes required
-- No additional dependencies required
-- The fixes improve error handling and provide better user feedback

TODO.md

@@ -1,12 +0,0 @@
-# TODO: Integrate Sinkhorn-Normalized Quantization
-
-## Steps to Complete
-- [x] Create quantization_utils.py with Sinkhorn-Normalized Quantization implementation
-- [x] Modify model_manager.py to support optional quantization during model loading
-- [x] Add configuration options for quantization in model_config.py
-- [x] Test quantization on a sample model without affecting existing workflows
-- [x] Verify that existing model loading and inference still work
-- [ ] Update documentation if needed
-
-## Current Status
-Basic tests completed successfully. Quantization is disabled by default, so existing workflows are unaffected. API endpoints can be tested by running the FastAPI app.

requirements.txt

@@ -1,7 +1,7 @@
 # Core AI/ML dependencies
-torch==2.3.0
-torchvision==0.18.0
-torchaudio==2.3.0
+torch>=2.3.0
+torchvision>=0.18.0
+torchaudio>=2.3.0
 transformers>=4.42.0
 tokenizers==0.21.4
 accelerate>=0.30.0
@@ -49,8 +49,8 @@ scipy==1.11.4
 joblib==1.5.1
 
 # Model Optimization & Quantization
-optimum==1.27.0
-optimum-intel==1.25.2
+optimum>=1.27.0
+optimum-intel>=1.25.2
 onnxruntime==1.16.3
 nncf==2.17.0
 bitsandbytes==0.47.0
@@ -58,10 +58,10 @@ ctransformers==0.2.27
 llama_cpp_python==0.2.72
 
 # Intel Optimization
-openvino==2025.2.0
-openvino-tokenizers==2025.2.0.1
-intel-openmp==2021.4.0
-mkl==2021.4.0
+openvino>=2024.4.0
+openvino-tokenizers>=2024.4.0
+intel-openmp>=2024.0.0
+mkl>=2024.0.0
 
 # Utilities & Helpers
 aiofiles==23.2.1
@@ -82,7 +82,13 @@ websockets==11.0.3
 # Database & Caching
 redis==6.4.0
 asyncpg==0.30.0
+sqlalchemy>=2.0.0
 
 # Development & Monitoring (minimal)
 rich==13.9.4
-typer==0.9.4
+typer==0.9.4
+
+# Additional dependencies for medical AI platform
+python-multipart>=0.0.6
+python-jose[cryptography]>=3.3.0
+passlib[bcrypt]>=1.7.4

services/ai-service/src/ai_med_extract/api/routes_fastapi.py

@@ -407,9 +407,9 @@ async def async_patient_summary(data, job_id=None):
         except Exception as e:
             print(f"Cache read failed: {e}")
 
-    # Set timeouts based on mode
-    EHR_TIMEOUT = 20 if timeout_mode == "fast" else 20
-    GEN_TIMEOUT = 20 if timeout_mode == "fast" else 60
+    # Set timeouts based on mode - Fixed inconsistent timeout values
+    EHR_TIMEOUT = 10 if timeout_mode == "fast" else 30
+    GEN_TIMEOUT = 30 if timeout_mode == "fast" else 120
 
     try:
         # Step 1: Fetch EHR data
@@ -575,26 +575,10 @@ async def async_patient_summary(data, job_id=None):
                 print(f"🔄 Loading new GGUF pipeline for {cache_key}")
                 pipeline = await asyncio.to_thread(get_cached_gguf_pipeline, repo_id, filename)
 
+            from ..utils.openvino_summarizer_utils import build_full_prompt
+            base_prompt = build_full_prompt(all_visits)
             full_prompt = f"""<|system|>
-You are a clinical AI assistant. Generate a COMPLETE patient summary with EXACTLY 4 sections in markdown format. Ensure ALL sections are fully generated and detailed with bullet points. Do not skip or abbreviate any section.
-do not halucinate or invent any information. Base ONLY on provided data.
-DATA:
-visits: {all_visits}
-
-REQUIRED OUTPUT FORMAT (must include all, each with at least 3-5 bullet points):
-## Clinical Assessment
-- Bullet points analyzing current state, diagnoses, vitals, labs, medications.
-
-## Key Trends & Changes
-- Bullet points on trends, deltas, new developments, changes in vitals/labs over time.
-
-## Plan & Suggested Actions
-- Bullet points with recommended next steps, monitoring, treatments, follow-ups.
-
-## Direct Guidance for Physician
-- Bullet points with key clinical insights, warnings, considerations, potential risks.
-
-Use bullet points with "- ". Base ONLY on provided data. No preamble, explanations, or extra text. Start immediately with "## Clinical Assessment" and ensure all 4 sections are complete and detailed:</s>
+{base_prompt}</s>
 <|user|>
 Generate the full 4-section summary based on the data.</s>
 <|assistant|>"""
@@ -722,8 +706,8 @@ Generate the full 4-section summary based on the data.</s>
             if not pipeline:
                 raise ValueError("Pipeline not available")
 
-            from ..utils.openvino_summarizer_utils import build_main_prompt
-            prompt = build_main_prompt(baseline, delta_text)
+            from ..utils.openvino_summarizer_utils import build_full_prompt
+            prompt = build_full_prompt(all_visits)
             inputs = pipeline.tokenizer([prompt], return_tensors="pt")
             outputs = await asyncio.to_thread(pipeline.model.generate, **inputs, max_new_tokens=800, do_sample=False, pad_token_id=pipeline.tokenizer.pad_token_id or pipeline.tokenizer.eos_token_id or 0)
             text = pipeline.tokenizer.decode(outputs[0], skip_special_tokens=True)
@@ -884,33 +868,78 @@ Generic summary — verify details clinically.
                 return result
 
         else:
-            print(f"Unsupported model_type: {model_type}")
-            generic_fallback = f"""
-## Clinical Assessment
-- Unsupported model type: {model_type}
-
-## Key Trends & Changes
-- Please use model_type: gguf, text-generation, causal-openvino, summarization, or seq2seq
-
-## Plan & Suggested Actions
-- Update API request with supported model type.
-
-## Direct Guidance for Physician
-- System configuration error — contact administrator.
-"""
-            total_time = time.perf_counter() - start_time
-            result = {
-                "summary": ensure_four_sections(generic_fallback),
-                "baseline": baseline,
-                "delta": delta_text,
-                "warning": f"Unsupported model_type: {model_type}",
-                "supported_types": ["gguf", "text-generation", "causal-openvino", "summarization", "seq2seq"],
-                "timing": {"total": round(total_time, 1)},
-                "timeout_mode_used": timeout_mode
-            }
+            # Universal model handling - try to use any model type
+            print(f"Universal model handling for type: {model_type}")
             if job_id:
-                update_job(job_id, 'completed', progress=100, data=result)
-            return result
+                update_job(job_id, 'processing', progress=70, data={'message': f'Loading universal model: {model_name} ({model_type})'})
+            
+            try:
+                # Use the unified model manager for any model type
+                from ..utils.model_manager import model_manager as _unified_manager
+                loader_obj = _unified_manager.get_model_loader(
+                    model_name=model_name,
+                    model_type=model_type,
+                    quantize=True
+                )
+                pipeline = loader_obj.load()
+                
+                if job_id:
+                    update_job(job_id, 'processing', progress=80, data={'message': f'Generating summary with {model_type} model...'})
+                
+                # Generate summary using the universal pipeline
+                if hasattr(pipeline, 'generate'):
+                    # For GGUF and custom models
+                    raw_summary = await asyncio.wait_for(
+                        asyncio.to_thread(pipeline.generate, prompt, max_tokens=1500, temperature=0.1, top_p=0.5),
+                        timeout=300  # 5 minutes timeout
+                    )
+                elif hasattr(pipeline, '__call__'):
+                    # For transformers pipelines
+                    result = await asyncio.to_thread(pipeline, prompt, max_length=400, min_length=100, do_sample=False)
+                    if isinstance(result, list) and result and "summary_text" in result[0]:
+                        raw_summary = result[0]["summary_text"]
+                    else:
+                        raw_summary = str(result)
+                else:
+                    raise ValueError("Pipeline does not support generation")
+                
+                # Process the summary
+                markdown_summary = summary_to_markdown(raw_summary)
+                markdown_summary = ensure_four_sections(markdown_summary)
+                
+                total_time = time.perf_counter() - start_time
+                print(f"[✅ SUCCESS] Universal {model_type} | TIMEOUT_MODE: {timeout_mode} | TOTAL: {total_time:.1f}s")
+                
+                result = {
+                    "summary": markdown_summary,
+                    "baseline": baseline,
+                    "delta": delta_text,
+                    "prompt": prompt,
+                    "timing": {"total": round(total_time, 1)},
+                    "model_used": f"{model_name} ({model_type})",
+                    "timeout_mode_used": timeout_mode
+                }
+                if job_id:
+                    update_job(job_id, 'completed', progress=100, data=result)
+                return result
+                
+            except Exception as e:
+                print(f"Universal model handling failed: {e}")
+                # Fallback to rule-based generation
+                markdown_summary = generate_rule_based_summary(baseline, delta_text, all_visits, patientid)
+                total_time = time.perf_counter() - start_time
+                result = {
+                    "summary": markdown_summary,
+                    "baseline": baseline,
+                    "delta": delta_text,
+                    "warning": f"Model {model_name} ({model_type}) failed, used rule-based fallback: {str(e)}",
+                    "timing": {"total": round(total_time, 1)},
+                    "model_used": f"{model_name} ({model_type}) - fallback",
+                    "timeout_mode_used": timeout_mode
+                }
+                if job_id:
+                    update_job(job_id, 'completed', progress=100, data=result)
+                return result
 
         # Step 5: Finalize (safety net)
         if job_id:
@@ -1937,13 +1966,13 @@ def register_routes(app, agents):
                 patient_info = f"Patient: {patient_name} (ID: {patient_id}, Age: {age}, Gender: {gender})\nPast Medical History: {past_medical_history}\nSocial History: {social_history}\n"
 
             # Use utils for processing
-            from ..utils.openvino_summarizer_utils import parse_ehr_chartsummarydtl, compute_deltas, visits_sorted, build_compact_baseline, delta_to_text, build_main_prompt
+            from ..utils.openvino_summarizer_utils import parse_ehr_chartsummarydtl, compute_deltas, visits_sorted, build_compact_baseline, delta_to_text, build_full_prompt
             visits = parse_ehr_chartsummarydtl(chartsummarydtl)
             delta = compute_deltas([], visits)
             all_visits = visits_sorted(visits)
             baseline = build_compact_baseline(all_visits)
             delta_text = delta_to_text(delta)
-            prompt = build_main_prompt(baseline, delta_text, patient_info)
+            prompt = build_full_prompt(all_visits, patient_info)
 
             # Model selection
             from ..utils import model_config as _mc

services/ai-service/src/ai_med_extract/utils/model_config.py

@@ -47,7 +47,36 @@ MODEL_TYPE_MAPPINGS = {
     "summarization": "summarization",
     "ner": "ner",
     "question-answering": "text-generation",
-    "translation": "text-generation"
+    "translation": "text-generation",
+    "causal": "text-generation",
+    "causal-lm": "text-generation",
+    "gpt": "text-generation",
+    "llama": "text-generation",
+    "mistral": "text-generation",
+    "phi": "text-generation",
+    "gemma": "text-generation",
+    "qwen": "text-generation",
+    "chat": "text-generation",
+    "instruct": "text-generation",
+    "conversational": "text-generation",
+    "dialogue": "text-generation",
+    "seq2seq": "summarization",
+    "t5": "summarization",
+    "bart": "summarization",
+    "pegasus": "summarization",
+    "led": "summarization",
+    "encoder-decoder": "summarization",
+    "bert": "ner",
+    "roberta": "ner",
+    "xlm": "ner",
+    "deberta": "ner",
+    "electra": "ner",
+    "distilbert": "ner",
+    "albert": "ner",
+    "medical": "text-generation",
+    "clinical": "text-generation",
+    "healthcare": "text-generation",
+    "biomedical": "text-generation"
 }
 
 # Memory-optimized models for Hugging Face Spaces
@@ -128,10 +157,71 @@ def detect_model_type(model_name: str) -> str:
     # Check file extensions
     if model_name.endswith('.gguf'):
         return "gguf"
+    if model_name.endswith('.onnx'):
+        return "openvino"
+    
+    # Try to detect from HuggingFace model info (if available)
+    try:
+        from huggingface_hub import model_info
+        info = model_info(model_name)
+        if hasattr(info, 'pipeline_tag') and info.pipeline_tag:
+            pipeline_tag = info.pipeline_tag.lower()
+            # Map HuggingFace pipeline tags to our types
+            if pipeline_tag in ['text-generation', 'text2text-generation']:
+                return "text-generation"
+            elif pipeline_tag in ['summarization', 'text-summarization']:
+                return "summarization"
+            elif pipeline_tag in ['ner', 'token-classification']:
+                return "ner"
+            elif pipeline_tag in ['conversational', 'chat']:
+                return "text-generation"
+            else:
+                # For unknown pipeline tags, try to infer from model name
+                return detect_model_type_from_name(model_name)
+    except Exception:
+        # If HuggingFace detection fails, fall back to name-based detection
+        pass
     
     # Default to text-generation for unknown types
     return "text-generation"
 
+def detect_model_type_from_name(model_name: str) -> str:
+    """Detect model type from model name patterns"""
+    model_name_lower = model_name.lower()
+    
+    # Check for specific model families
+    if any(family in model_name_lower for family in ['gpt', 'gpt2', 'gpt3', 'gpt4']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['llama', 'llama2', 'llama3']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['mistral', 'mixtral']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['phi', 'phi2', 'phi3']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['gemma', 'gemma2']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['qwen', 'qwen2']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['t5', 't5-']):
+        return "summarization"
+    elif any(family in model_name_lower for family in ['bart', 'bart-']):
+        return "summarization"
+    elif any(family in model_name_lower for family in ['pegasus', 'pegasus-']):
+        return "summarization"
+    elif any(family in model_name_lower for family in ['bert', 'roberta', 'deberta', 'electra', 'distilbert', 'albert']):
+        return "ner"
+    elif any(family in model_name_lower for family in ['medical', 'clinical', 'healthcare', 'biomedical', 'bio']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['chat', 'instruct', 'conversational', 'dialogue']):
+        return "text-generation"
+    elif any(family in model_name_lower for family in ['summar', 'summary']):
+        return "summarization"
+    elif any(family in model_name_lower for family in ['ner', 'entity', 'named-entity']):
+        return "ner"
+    
+    # Default fallback
+    return "text-generation"
+
 def validate_model_config(model_name: str, model_type: str) -> dict:
     """Validate model configuration and return validation result"""
     result = {
@@ -141,11 +231,12 @@ def validate_model_config(model_name: str, model_type: str) -> dict:
         "recommendations": []
     }
     
-    # Check if model type is supported
+    # Check if model type is supported - now more flexible
     if model_type not in MODEL_VALIDATION_RULES:
-        result["valid"] = False
-        result["errors"].append(f"Unsupported model type: {model_type}")
-        return result
+        # For unknown types, create default validation rules
+        result["warnings"].append(f"Model type '{model_type}' not in predefined rules, using default settings")
+        result["recommendations"].append("Consider using a known model type for optimal performance")
+        # Don't mark as invalid, just warn
     
     # Check model name format
     if model_type == "gguf":
@@ -153,11 +244,15 @@ def validate_model_config(model_name: str, model_type: str) -> dict:
             result["warnings"].append("GGUF model should have .gguf extension or be in repo/filename format")
     
     # Check for memory optimization recommendations
-    if model_type in ["text-generation", "summarization"]:
-        if "large" in model_name.lower() or "xl" in model_name.lower():
+    if model_type in ["text-generation", "summarization"] or "large" in model_name.lower() or "xl" in model_name.lower():
+        if "large" in model_name.lower() or "xl" in model_name.lower() or "7b" in model_name.lower() or "13b" in model_name.lower():
             result["warnings"].append("Large models may cause memory issues on limited resources")
             result["recommendations"].append("Consider using a smaller model for better performance")
     
+    # Check for medical/clinical models
+    if any(keyword in model_name.lower() for keyword in ['medical', 'clinical', 'healthcare', 'biomedical', 'bio']):
+        result["recommendations"].append("Medical model detected - ensure appropriate medical data handling")
+    
     return result
 
 def get_model_info(model_name: str, model_type: str) -> dict:

services/ai-service/src/ai_med_extract/utils/model_loader_spaces.py

@@ -7,16 +7,23 @@ class OpenVinoPipeline:
 		self.model = model
 		self.tokenizer = tokenizer
 
-def get_openvino_pipeline(model_name: str):
+def get_openvino_pipeline(model_name: str, device: str = None):
 	"""
 	Loads an OpenVINO CausalLM pipeline for the given model name or IR directory.
+	Automatically detects GPU/CPU and uses appropriate device.
 	"""
-	# If model_name is a directory, try to load IR from there; else, download and export
 	import os
+	import torch
+	
+	# Auto-detect device if not provided
+	if device is None:
+		device = "GPU" if torch.cuda.is_available() else "CPU"
+	
+	# If model_name is a directory, try to load IR from there; else, download and export
 	if os.path.isdir(model_name):
-		model = OVModelForCausalLM.from_pretrained(model_name, compile=True, device="CPU", cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
+		model = OVModelForCausalLM.from_pretrained(model_name, compile=True, device=device, cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
 		tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True, cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
 	else:
-		model = OVModelForCausalLM.from_pretrained(model_name, export=False, compile=False, device="CPU", cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
+		model = OVModelForCausalLM.from_pretrained(model_name, export=False, compile=False, device=device, cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
 		tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True, cache_dir=os.environ.get('HF_HOME', '/tmp/huggingface'))
 	return OpenVinoPipeline(model, tokenizer)

services/ai-service/src/ai_med_extract/utils/model_manager.py

@@ -356,9 +356,9 @@ class OpenVINOModelLoader(BaseModelLoader):
             try:
                 from .model_loader_spaces import get_openvino_pipeline
 
-                logger.info(f"Loading OpenVINO model: {self.model_name}")
-                self._pipeline = get_openvino_pipeline(self.model_name)
-                logger.info(f"OpenVINO model loaded successfully: {self.model_name}")
+                logger.info(f"Loading OpenVINO model: {self.model_name} on device: {self.device}")
+                self._pipeline = get_openvino_pipeline(self.model_name, self.device)
+                logger.info(f"OpenVINO model loaded successfully: {self.model_name} on {self.device}")
 
             except ImportError as import_error:
                 logger.warning(f"OpenVINO model loader not available: {import_error}")
@@ -440,24 +440,78 @@ class UnifiedModelManager:
     ) -> BaseModelLoader:
         """
         Get a model loader for the specified model and type
+        Now supports ANY model type with intelligent fallback
         """
         cache_key = f"{model_name}:{model_type}:{filename or ''}:{quantize}"
         
         if not force_reload and cache_key in self._model_cache:
             return self._model_cache[cache_key]
         
+        # Try multiple loader strategies for maximum compatibility
+        loader = None
+        last_error = None
+        
+        # Strategy 1: Try the specified model type first
         try:
-            # Determine loader type and create appropriate loader
             if model_type == "gguf":
                 loader = GGUFModelLoader(model_name, filename)
             elif model_type == "openvino":
                 loader = OpenVINOModelLoader(model_name)
             else:
-                # Default to transformers for text-generation, summarization, ner, etc.
+                # Default to transformers for any other type
                 loader = TransformersModelLoader(model_name, model_type)
             
             # Test load the model
             pipeline = loader.load()
+            logger.info(f"Successfully loaded {model_name} with {model_type} loader")
+            
+        except Exception as e:
+            logger.warning(f"Failed to load {model_name} with {model_type} loader: {e}")
+            last_error = e
+            loader = None
+            
+            # Strategy 2: Try alternative loaders based on model name patterns
+            alternative_strategies = []
+            
+            # Check if it's a GGUF model by extension or name
+            if model_name.endswith('.gguf') or 'gguf' in model_name.lower():
+                alternative_strategies.append(("gguf", lambda: GGUFModelLoader(model_name, filename)))
+            
+            # Check if it's an OpenVINO model
+            if model_name.endswith('.onnx') or 'openvino' in model_name.lower() or 'ov' in model_name.lower():
+                alternative_strategies.append(("openvino", lambda: OpenVINOModelLoader(model_name)))
+            
+            # Try transformers with different task types
+            if any(keyword in model_name.lower() for keyword in ['summar', 'summary', 't5', 'bart', 'pegasus']):
+                alternative_strategies.append(("summarization", lambda: TransformersModelLoader(model_name, "summarization")))
+            elif any(keyword in model_name.lower() for keyword in ['ner', 'bert', 'roberta', 'entity']):
+                alternative_strategies.append(("ner", lambda: TransformersModelLoader(model_name, "ner")))
+            else:
+                # Try as text-generation
+                alternative_strategies.append(("text-generation", lambda: TransformersModelLoader(model_name, "text-generation")))
+            
+            # Try each alternative strategy
+            for alt_type, alt_loader_func in alternative_strategies:
+                try:
+                    logger.info(f"Trying alternative loader: {alt_type} for {model_name}")
+                    loader = alt_loader_func()
+                    pipeline = loader.load()
+                    logger.info(f"Successfully loaded {model_name} with alternative {alt_type} loader")
+                    break
+                except Exception as alt_error:
+                    logger.warning(f"Alternative {alt_type} loader failed: {alt_error}")
+                    last_error = alt_error
+                    loader = None
+                    continue
+        
+        # If all strategies failed, create a fallback loader
+        if loader is None:
+            logger.error(f"All loading strategies failed for {model_name}. Creating fallback loader.")
+            loader = self._create_fallback_loader(model_name, model_type, last_error)
+        
+        # Test load the model
+        try:
+            pipeline = loader.load()
             
             # Apply quantization if enabled and applicable
             if quantize and quantization_config is None and model_config:
@@ -527,6 +581,49 @@ class UnifiedModelManager:
             cache_key: loader.get_model_info() 
             for cache_key, loader in self._model_cache.items()
         }
+    
+    def _create_fallback_loader(self, model_name: str, model_type: str, error: Exception = None) -> BaseModelLoader:
+        """Create a fallback loader when all other strategies fail"""
+        class FallbackModelLoader(BaseModelLoader):
+            def __init__(self, model_name: str, model_type: str, error: Exception = None):
+                self.model_name = model_name
+                self.model_type = model_type
+                self.error = error
+                self._pipeline = None
+                
+            def load(self):
+                if self._pipeline is None:
+                    # Create a simple fallback pipeline
+                    class FallbackPipeline:
+                        def __init__(self, model_name, model_type, error):
+                            self.model_name = model_name
+                            self.model_type = model_type
+                            self.error = error
+                            
+                        def generate(self, prompt, **kwargs):
+                            return f"Model '{self.model_name}' ({self.model_type}) not available. Error: {str(self.error)[:100]}..."
+                        
+                        def __call__(self, prompt, **kwargs):
+                            return [{"generated_text": self.generate(prompt, **kwargs)}]
+                    
+                    self._pipeline = FallbackPipeline(self.model_name, self.model_type, self.error)
+                return self._pipeline
+            
+            def generate(self, prompt: str, **kwargs) -> str:
+                pipeline = self.load()
+                return pipeline.generate(prompt, **kwargs)
+            
+            def get_model_info(self) -> Dict[str, Any]:
+                return {
+                    "type": "fallback",
+                    "model_name": self.model_name,
+                    "model_type": self.model_type,
+                    "loaded": True,
+                    "fallback": True,
+                    "error": str(self.error) if self.error else None
+                }
+        
+        return FallbackModelLoader(model_name, model_type, error)
 
 # Global instance
 model_manager = UnifiedModelManager()

services/ai-service/src/ai_med_extract/utils/openvino_summarizer_utils.py

@@ -252,6 +252,47 @@ def build_main_prompt(baseline, delta_text, patient_info="", section=None):
         "Now generate the complete clinical summary with all four sections in markdown format:"
     )
 
+def build_full_prompt(all_visits, patient_info="", section=None):
+    """
+    Build the full prompt using the enhanced format that was previously only used for GGUF models.
+    This provides more detailed instructions and better formatting for all model types.
+    """
+    base_instruction = (
+        "You are a clinical AI assistant. Generate a COMPLETE patient summary with EXACTLY 4 sections in markdown format. "
+        "Ensure ALL sections are fully generated and detailed with bullet points. Do not skip or abbreviate any section. "
+        "Do not hallucinate or invent any information. Base ONLY on provided data."
+    )
+    
+    if section:
+        section_instructions = {
+            "Clinical Assessment": "Generate ONLY the 'Clinical Assessment' section. Be concise, accurate, and evidence-based with bullet points.",
+            "Key Trends & Changes": "Generate ONLY the 'Key Trends & Changes' section. Focus on deltas, trends, vitals, labs, and med changes with bullet points.",
+            "Plan & Suggested Actions": "Generate ONLY the 'Plan & Suggested Actions' section. Suggest next steps, monitoring, treatments, follow-ups with bullet points.",
+            "Direct Guidance for Physician": "Generate ONLY the 'Direct Guidance for Physician' section. Give clear, actionable advice for the clinician with bullet points."
+        }
+        instruction = section_instructions.get(section, f"Generate the '{section}' section.")
+        return f"{base_instruction}\n\nDATA:\nvisits: {all_visits}\n\n{instruction}\n\nUse bullet points with '- '. Base ONLY on provided data. No preamble, explanations, or extra text. Start immediately with the section content:"
+    
+    # Default: generate full 4-section summary
+    return f"""{base_instruction}
+DATA:
+visits: {all_visits}
+
+REQUIRED OUTPUT FORMAT (must include all, each with at least 3-5 bullet points):
+## Clinical Assessment
+- Bullet points analyzing current state, diagnoses, vitals, labs, medications.
+
+## Key Trends & Changes
+- Bullet points on trends, deltas, new developments, changes in vitals/labs over time.
+
+## Plan & Suggested Actions
+- Bullet points with recommended next steps, monitoring, treatments, follow-ups.
+
+## Direct Guidance for Physician
+- Bullet points with key clinical insights, warnings, considerations, potential risks.
+
+Use bullet points with "- ". Base ONLY on provided data. No preamble, explanations, or extra text. Start immediately with "## Clinical Assessment" and ensure all 4 sections are complete and detailed:"""
+
 def validate_and_compare_summaries(old_summary, new_summary, update_name=""):
     report = f"### Validation Report for {update_name}\n"
     report += "This report validates that the updated summary incorporates new information correctly.\n"

test_device_fix.py

@@ -14,6 +14,9 @@ current_dir = Path(__file__).parent
 ai_med_extract_path = current_dir / "services" / "ai-service" / "src"
 sys.path.insert(0, str(ai_med_extract_path))
 
+# Also add the parent directory for proper module resolution
+sys.path.insert(0, str(ai_med_extract_path.parent))
+
 # Set environment variables
 os.environ.setdefault('HF_SPACES', 'true')
 os.environ.setdefault('PYTHONUNBUFFERED', '1')

test_hf_spaces_fix.py

@@ -67,9 +67,12 @@ def test_app_import():
     try:
         # Add the ai_med_extract module to Python path
         current_dir = Path(__file__).parent
-        ai_med_extract_path = current_dir / "services" / "ai-service" / "src" / "ai_med_extract"
+        ai_med_extract_path = current_dir / "services" / "ai-service" / "src"
         sys.path.insert(0, str(ai_med_extract_path))
         
+        # Also add the parent directory for proper module resolution
+        sys.path.insert(0, str(ai_med_extract_path.parent))
+        
         # Set HF Spaces environment
         os.environ['HF_SPACES'] = 'true'