cvpfus Codex commited on
Commit
35f2e3b
·
1 Parent(s): e163e74

Implement missing model runtime polish

Browse files

Co-authored-by: Codex <codex@openai.com>

.env.example CHANGED
@@ -1,5 +1,6 @@
1
  # Tiny Narrator local configuration.
2
  # Copy this file to .env and fill in the values for the services you want to run live.
 
3
 
4
  # App server
5
  GRADIO_SERVER_NAME=0.0.0.0
@@ -21,4 +22,5 @@ MINICPM_VISION_TIMEOUT_SECONDS=45
21
 
22
  # Image generation: Modal-hosted FLUX.2 klein worker
23
  KLEIN_MODAL_ENDPOINT=
 
24
  KLEIN_MODAL_TIMEOUT_SECONDS=120
 
1
  # Tiny Narrator local configuration.
2
  # Copy this file to .env and fill in the values for the services you want to run live.
3
+ # The app loads .env automatically at startup. Process environment variables still override .env values.
4
 
5
  # App server
6
  GRADIO_SERVER_NAME=0.0.0.0
 
22
 
23
  # Image generation: Modal-hosted FLUX.2 klein worker
24
  KLEIN_MODAL_ENDPOINT=
25
+ KLEIN_MODAL_TOKEN=
26
  KLEIN_MODAL_TIMEOUT_SECONDS=120
FIELD_NOTES.md CHANGED
@@ -106,3 +106,7 @@ Deploy the worker with `modal deploy modal_workers/klein_image.py`, then set `KL
106
  `KLEIN_MODAL_TIMEOUT_SECONDS` defaults to 120 seconds, which accommodates warm inference but avoids blocking local demos indefinitely during cold starts.
107
 
108
  This design keeps verification stable without requiring Modal credentials or GPU access: the verifier checks fallback behavior, and live Modal deployment is a manual step documented in the README.
 
 
 
 
 
106
  `KLEIN_MODAL_TIMEOUT_SECONDS` defaults to 120 seconds, which accommodates warm inference but avoids blocking local demos indefinitely during cold starts.
107
 
108
  This design keeps verification stable without requiring Modal credentials or GPU access: the verifier checks fallback behavior, and live Modal deployment is a manual step documented in the README.
109
+
110
+ ## Pre-Demo Validation
111
+
112
+ `scripts/live_smoke.py` is an opt-in script for confirming live model inference before a demo. It calls `/api/runtime-status`, `/api/describe-image`, and `/api/generate-image`, then reports pass/skip/fail per role. Unconfigured runtimes are skipped, and the script exits non-zero only when a configured live check fails. Secrets are never printed, so the output is safe to include in demo logs or field notes.
README.md CHANGED
@@ -40,7 +40,7 @@ Install dependencies:
40
  python -m pip install -r requirements.txt
41
  ```
42
 
43
- Copy the example environment file if you want a local `.env` starting point:
44
 
45
  ```powershell
46
  Copy-Item .env.example .env
@@ -68,7 +68,7 @@ The image generation path uses a Modal-hosted `black-forest-labs/FLUX.2-klein-4B
68
  modal deploy modal_workers/klein_image.py
69
  ```
70
 
71
- Set `KLEIN_MODAL_ENDPOINT` to the deployed worker URL. When configured, `/api/generate-image` and `/api/generate-article` call the live Modal endpoint for real Klein inference. When the endpoint is not set, unreachable, or returns invalid data, the app falls back to bundled SVG assets with explicit fallback metadata.
72
 
73
  `/api/runtime-status` reports whether the Modal Klein worker is online or fallback-ready. `/api/runtime-setup` includes the deploy command and required environment variables.
74
 
@@ -88,6 +88,7 @@ Useful environment variables:
88
  | `GRADIO_SERVER_PORT` / `PORT` | `7860` | App port |
89
  | `PUBLIC_BASE_URL` | `http://localhost:7860` | Base URL used in generated judge API commands |
90
  | `KLEIN_MODAL_ENDPOINT` | *(empty)* | Base URL for the Modal Klein worker, without trailing slash. When set, `/api/generate-image` calls the live worker. |
 
91
  | `KLEIN_MODAL_TIMEOUT_SECONDS` | `120` | Request timeout for Modal Klein image generation. |
92
  | `MINICPM_VISION_BASE_URL` | *(empty)* | OpenAI-compatible MiniCPM-V-4.6 base URL; root or `/v1` both work. |
93
  | `MINICPM_VISION_API_KEY` | *(empty)* | Bearer token for the MiniCPM vision endpoint. |
@@ -102,6 +103,16 @@ python scripts/verify.py
102
 
103
  The verifier checks syntax, static assets, Space metadata consistency, deterministic fallback model paths, and generated speech file behavior.
104
 
 
 
 
 
 
 
 
 
 
 
105
  `/api/model-budget` exposes numeric parameter counts for every model role and reports whether the full stack stays within the 4B Tiny Titan limit.
106
 
107
  `/api/runtime-setup` exposes the commands, environment values, and fallback paths used for the model stack so the demo can be reproduced from the same data the UI displays.
 
40
  python -m pip install -r requirements.txt
41
  ```
42
 
43
+ Copy the example environment file to `.env` and fill in the values. The app loads `.env` automatically at startup, so no shell exports are required. Process environment variables still override `.env` values.
44
 
45
  ```powershell
46
  Copy-Item .env.example .env
 
68
  modal deploy modal_workers/klein_image.py
69
  ```
70
 
71
+ Set `KLEIN_MODAL_ENDPOINT` to the deployed worker URL. Set `KLEIN_MODAL_TOKEN` to a shared secret and configure the same token in the Modal worker environment (e.g. via `modal secret create`). When the token is configured, the worker rejects unauthenticated `POST /generate` requests with HTTP 401. When no token is set on either side, requests are allowed for local compatibility.
72
 
73
  `/api/runtime-status` reports whether the Modal Klein worker is online or fallback-ready. `/api/runtime-setup` includes the deploy command and required environment variables.
74
 
 
88
  | `GRADIO_SERVER_PORT` / `PORT` | `7860` | App port |
89
  | `PUBLIC_BASE_URL` | `http://localhost:7860` | Base URL used in generated judge API commands |
90
  | `KLEIN_MODAL_ENDPOINT` | *(empty)* | Base URL for the Modal Klein worker, without trailing slash. When set, `/api/generate-image` calls the live worker. |
91
+ | `KLEIN_MODAL_TOKEN` | *(empty)* | Shared bearer token for Modal worker auth. When set, the app sends `Authorization: Bearer <token>` and the worker rejects unauthenticated requests. |
92
  | `KLEIN_MODAL_TIMEOUT_SECONDS` | `120` | Request timeout for Modal Klein image generation. |
93
  | `MINICPM_VISION_BASE_URL` | *(empty)* | OpenAI-compatible MiniCPM-V-4.6 base URL; root or `/v1` both work. |
94
  | `MINICPM_VISION_API_KEY` | *(empty)* | Bearer token for the MiniCPM vision endpoint. |
 
103
 
104
  The verifier checks syntax, static assets, Space metadata consistency, deterministic fallback model paths, and generated speech file behavior.
105
 
106
+ ## Live Model Smoke Tests
107
+
108
+ After starting the app with configured model endpoints, run the opt-in smoke script to verify live inference:
109
+
110
+ ```powershell
111
+ python scripts/live_smoke.py --base-url http://127.0.0.1:7860
112
+ ```
113
+
114
+ The smoke script checks `/api/runtime-status`, `/api/describe-image`, and `/api/generate-image`. It skips checks for unconfigured runtimes and exits non-zero only when a configured live check fails. Secrets are never printed.
115
+
116
  `/api/model-budget` exposes numeric parameter counts for every model role and reports whether the full stack stays within the 4B Tiny Titan limit.
117
 
118
  `/api/runtime-setup` exposes the commands, environment values, and fallback paths used for the model stack so the demo can be reproduced from the same data the UI displays.
app.py CHANGED
@@ -12,12 +12,14 @@ from pathlib import Path
12
  from typing import Any
13
  from uuid import uuid4
14
 
 
15
  from fastapi import Request
16
  from fastapi.responses import HTMLResponse, JSONResponse
17
  from fastapi.staticfiles import StaticFiles
18
  from gradio import Server
19
  from pydantic import BaseModel
20
 
 
21
 
22
  ROOT = Path(__file__).parent
23
  STATIC_DIR = ROOT / "static"
@@ -38,6 +40,7 @@ GRADIO_SERVER_NAME = os.getenv("GRADIO_SERVER_NAME", "0.0.0.0")
38
  GRADIO_SERVER_PORT = int(os.getenv("PORT", os.getenv("GRADIO_SERVER_PORT", "7860")))
39
  PUBLIC_BASE_URL = os.getenv("PUBLIC_BASE_URL", f"http://localhost:{GRADIO_SERVER_PORT}").rstrip("/")
40
  KLEIN_MODAL_ENDPOINT = os.getenv("KLEIN_MODAL_ENDPOINT", "").rstrip("/")
 
41
  KLEIN_MODAL_TIMEOUT_SECONDS = _int_env("KLEIN_MODAL_TIMEOUT_SECONDS", 120)
42
  MINICPM_VISION_BASE_URL = os.getenv("MINICPM_VISION_BASE_URL", "").rstrip("/")
43
  MINICPM_VISION_API_KEY = os.getenv("MINICPM_VISION_API_KEY", "")
@@ -220,6 +223,7 @@ def runtime_setup_core() -> dict[str, Any]:
220
  "command": "modal deploy modal_workers/klein_image.py",
221
  "env": {
222
  "KLEIN_MODAL_ENDPOINT": KLEIN_MODAL_ENDPOINT or "(set to Modal worker URL)",
 
223
  "KLEIN_MODAL_TIMEOUT_SECONDS": str(KLEIN_MODAL_TIMEOUT_SECONDS),
224
  },
225
  "fallback": "bundled generated article assets",
@@ -619,8 +623,12 @@ def _runtime_status_core() -> dict[str, Any]:
619
  klein_status: dict[str, Any]
620
  if KLEIN_MODAL_ENDPOINT:
621
  try:
 
 
 
622
  request = urllib.request.Request(
623
  f"{KLEIN_MODAL_ENDPOINT}/health", method="GET",
 
624
  )
625
  with urllib.request.urlopen(request, timeout=5) as response:
626
  payload = json.loads(response.read().decode("utf-8"))
@@ -1056,10 +1064,13 @@ def _call_modal_klein(prompt: str, seed: int | None) -> dict[str, Any]:
1056
  """Call the Modal Klein worker for live image generation."""
1057
  start = time.perf_counter()
1058
  body = json.dumps({"prompt": prompt, "seed": seed}).encode("utf-8")
 
 
 
1059
  request = urllib.request.Request(
1060
  f"{KLEIN_MODAL_ENDPOINT}/generate",
1061
  data=body,
1062
- headers={"Content-Type": "application/json"},
1063
  method="POST",
1064
  )
1065
  with urllib.request.urlopen(request, timeout=KLEIN_MODAL_TIMEOUT_SECONDS) as response:
 
12
  from typing import Any
13
  from uuid import uuid4
14
 
15
+ from dotenv import load_dotenv
16
  from fastapi import Request
17
  from fastapi.responses import HTMLResponse, JSONResponse
18
  from fastapi.staticfiles import StaticFiles
19
  from gradio import Server
20
  from pydantic import BaseModel
21
 
22
+ load_dotenv()
23
 
24
  ROOT = Path(__file__).parent
25
  STATIC_DIR = ROOT / "static"
 
40
  GRADIO_SERVER_PORT = int(os.getenv("PORT", os.getenv("GRADIO_SERVER_PORT", "7860")))
41
  PUBLIC_BASE_URL = os.getenv("PUBLIC_BASE_URL", f"http://localhost:{GRADIO_SERVER_PORT}").rstrip("/")
42
  KLEIN_MODAL_ENDPOINT = os.getenv("KLEIN_MODAL_ENDPOINT", "").rstrip("/")
43
+ KLEIN_MODAL_TOKEN = os.getenv("KLEIN_MODAL_TOKEN", "")
44
  KLEIN_MODAL_TIMEOUT_SECONDS = _int_env("KLEIN_MODAL_TIMEOUT_SECONDS", 120)
45
  MINICPM_VISION_BASE_URL = os.getenv("MINICPM_VISION_BASE_URL", "").rstrip("/")
46
  MINICPM_VISION_API_KEY = os.getenv("MINICPM_VISION_API_KEY", "")
 
223
  "command": "modal deploy modal_workers/klein_image.py",
224
  "env": {
225
  "KLEIN_MODAL_ENDPOINT": KLEIN_MODAL_ENDPOINT or "(set to Modal worker URL)",
226
+ "KLEIN_MODAL_TOKEN": "(configured)" if KLEIN_MODAL_TOKEN else "(not set)",
227
  "KLEIN_MODAL_TIMEOUT_SECONDS": str(KLEIN_MODAL_TIMEOUT_SECONDS),
228
  },
229
  "fallback": "bundled generated article assets",
 
623
  klein_status: dict[str, Any]
624
  if KLEIN_MODAL_ENDPOINT:
625
  try:
626
+ health_headers: dict[str, str] = {}
627
+ if KLEIN_MODAL_TOKEN:
628
+ health_headers["Authorization"] = f"Bearer {KLEIN_MODAL_TOKEN}"
629
  request = urllib.request.Request(
630
  f"{KLEIN_MODAL_ENDPOINT}/health", method="GET",
631
+ headers=health_headers,
632
  )
633
  with urllib.request.urlopen(request, timeout=5) as response:
634
  payload = json.loads(response.read().decode("utf-8"))
 
1064
  """Call the Modal Klein worker for live image generation."""
1065
  start = time.perf_counter()
1066
  body = json.dumps({"prompt": prompt, "seed": seed}).encode("utf-8")
1067
+ headers: dict[str, str] = {"Content-Type": "application/json"}
1068
+ if KLEIN_MODAL_TOKEN:
1069
+ headers["Authorization"] = f"Bearer {KLEIN_MODAL_TOKEN}"
1070
  request = urllib.request.Request(
1071
  f"{KLEIN_MODAL_ENDPOINT}/generate",
1072
  data=body,
1073
+ headers=headers,
1074
  method="POST",
1075
  )
1076
  with urllib.request.urlopen(request, timeout=KLEIN_MODAL_TIMEOUT_SECONDS) as response:
modal_workers/klein_image.py CHANGED
@@ -14,6 +14,7 @@ Tiny Narrator's app.py calls these routes through KLEIN_MODAL_ENDPOINT.
14
  from __future__ import annotations
15
 
16
  import io
 
17
  import time
18
  from pathlib import Path
19
  from uuid import uuid4
@@ -70,14 +71,30 @@ def _get_pipeline():
70
  )
71
  @modal.asgi_app()
72
  def klein_api():
73
- from fastapi import FastAPI, HTTPException
74
  from fastapi.responses import Response
75
  import torch
76
 
77
  api = FastAPI(title="Tiny Narrator Klein Worker")
78
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  @api.get("/health")
80
- async def health() -> dict:
 
81
  return {
82
  "ok": True,
83
  "model": IMAGE_MODEL_ID,
@@ -85,13 +102,14 @@ def klein_api():
85
  }
86
 
87
  @api.post("/generate")
88
- async def generate(request: dict) -> dict:
 
89
  start = time.perf_counter()
90
- prompt = str(request.get("prompt") or "").strip()
91
  if not prompt:
92
  raise HTTPException(status_code=400, detail="prompt is required")
93
 
94
- seed = request.get("seed")
95
  generator = None
96
  if seed is not None:
97
  try:
 
14
  from __future__ import annotations
15
 
16
  import io
17
+ import os
18
  import time
19
  from pathlib import Path
20
  from uuid import uuid4
 
71
  )
72
  @modal.asgi_app()
73
  def klein_api():
74
+ from fastapi import FastAPI, HTTPException, Request
75
  from fastapi.responses import Response
76
  import torch
77
 
78
  api = FastAPI(title="Tiny Narrator Klein Worker")
79
 
80
+ def _check_token(request: Request) -> None:
81
+ """Reject the request if a token is configured but not provided or mismatched."""
82
+ expected = os.getenv("KLEIN_MODAL_TOKEN", "")
83
+ if not expected:
84
+ return
85
+ auth_header = request.headers.get("authorization", "")
86
+ token_header = request.headers.get("x-tiny-narrator-token", "")
87
+ provided = ""
88
+ if auth_header.startswith("Bearer "):
89
+ provided = auth_header[len("Bearer "):]
90
+ elif token_header:
91
+ provided = token_header
92
+ if provided != expected:
93
+ raise HTTPException(status_code=401, detail="Unauthorized")
94
+
95
  @api.get("/health")
96
+ async def health(request: Request) -> dict:
97
+ _check_token(request)
98
  return {
99
  "ok": True,
100
  "model": IMAGE_MODEL_ID,
 
102
  }
103
 
104
  @api.post("/generate")
105
+ async def generate(request: Request, body: dict) -> dict:
106
+ _check_token(request)
107
  start = time.perf_counter()
108
+ prompt = str(body.get("prompt") or "").strip()
109
  if not prompt:
110
  raise HTTPException(status_code=400, detail="prompt is required")
111
 
112
+ seed = body.get("seed")
113
  generator = None
114
  if seed is not None:
115
  try:
requirements.txt CHANGED
@@ -2,3 +2,4 @@ gradio==6.16.0
2
  pydantic>=2.7.0
3
  kokoro>=0.9.4
4
  soundfile>=0.12.1
 
 
2
  pydantic>=2.7.0
3
  kokoro>=0.9.4
4
  soundfile>=0.12.1
5
+ python-dotenv
scripts/live_smoke.py ADDED
@@ -0,0 +1,215 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Live model smoke tests for Tiny Narrator.
2
+
3
+ Run against a running local app to verify configured model endpoints:
4
+
5
+ python scripts/live_smoke.py --base-url http://127.0.0.1:7860
6
+
7
+ Checks:
8
+ - /api/runtime-status: confirms reader brain, speech, vision, image generation status
9
+ - /api/describe-image: confirms MiniCPM-V-4.6 image description path
10
+ - /api/generate-image: confirms Modal Klein image generation path
11
+
12
+ Exit codes:
13
+ 0 - all configured checks passed or skipped
14
+ 1 - one or more configured checks failed
15
+
16
+ Secrets (API keys, tokens) are never printed.
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ import argparse
22
+ import json
23
+ import sys
24
+ import time
25
+ import urllib.error
26
+ import urllib.request
27
+
28
+
29
+ def _get_json(url: str, timeout: int = 30) -> dict:
30
+ """Fetch a JSON GET endpoint and return the parsed payload."""
31
+ request = urllib.request.Request(url, method="GET")
32
+ with urllib.request.urlopen(request, timeout=timeout) as response:
33
+ return json.loads(response.read().decode("utf-8"))
34
+
35
+
36
+ def _post_json(url: str, payload: dict, timeout: int = 60) -> dict:
37
+ """POST JSON to an endpoint and return the parsed response."""
38
+ body = json.dumps(payload).encode("utf-8")
39
+ request = urllib.request.Request(
40
+ url,
41
+ data=body,
42
+ headers={"Content-Type": "application/json"},
43
+ method="POST",
44
+ )
45
+ with urllib.request.urlopen(request, timeout=timeout) as response:
46
+ return json.loads(response.read().decode("utf-8"))
47
+
48
+
49
+ def _format_elapsed(ms: float) -> str:
50
+ if ms < 1000:
51
+ return f"{round(ms)} ms"
52
+ return f"{ms / 1000:.2f} s"
53
+
54
+
55
+ def _status_label(status: str) -> str:
56
+ return {
57
+ "online": "PASS",
58
+ "fallback-ready": "SKIP",
59
+ "fallback": "SKIP",
60
+ "placeholder-ready": "SKIP",
61
+ "offline": "FAIL",
62
+ }.get(status, "SKIP")
63
+
64
+
65
+ def _role_status(runtime_status: dict[str, dict], role: str) -> str:
66
+ return str(runtime_status.get(role, {}).get("status", "unknown"))
67
+
68
+
69
+ def _role_is_configured(runtime_status: dict[str, dict], role: str) -> bool:
70
+ details = runtime_status.get(role, {})
71
+ if details.get("configured") is True:
72
+ return True
73
+ return bool(details.get("endpoint") or details.get("base_url"))
74
+
75
+
76
+ def check_runtime_status(base_url: str) -> dict[str, dict]:
77
+ """Fetch runtime status and return per-role metadata."""
78
+ start = time.perf_counter()
79
+ try:
80
+ payload = _get_json(f"{base_url}/api/runtime-status")
81
+ elapsed = (time.perf_counter() - start) * 1000
82
+ roles = {
83
+ "reader_brain": payload.get("reader_brain", {}),
84
+ "vision": payload.get("vision", {}),
85
+ "speech": payload.get("speech", {}),
86
+ "image_generation": payload.get("image_generation", {}),
87
+ }
88
+ labels = ", ".join(f"{role}={_role_status(roles, role)}" for role in roles)
89
+ print(f" runtime-status {_format_elapsed(elapsed):>10} {labels}")
90
+ return roles
91
+ except (OSError, urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:
92
+ elapsed = (time.perf_counter() - start) * 1000
93
+ print(f" runtime-status {_format_elapsed(elapsed):>10} FAIL ({exc.__class__.__name__})")
94
+ return {}
95
+
96
+
97
+ def check_describe_image(base_url: str, runtime_status: dict[str, dict]) -> bool:
98
+ """Smoke test /api/describe-image for a known article image."""
99
+ vision_status = _role_status(runtime_status, "vision")
100
+ vision_configured = _role_is_configured(runtime_status, "vision")
101
+ if not vision_configured and vision_status in {"fallback-ready", "unknown"}:
102
+ print(f" describe-image {'SKIP':>10} vision not configured")
103
+ return True
104
+ if vision_configured and vision_status != "online":
105
+ print(f" describe-image {'FAIL':>10} vision configured but status={vision_status}")
106
+ return False
107
+
108
+ start = time.perf_counter()
109
+ try:
110
+ payload = _post_json(
111
+ f"{base_url}/api/describe-image",
112
+ {"image_id": "desk-reader", "caption": "article reader"},
113
+ timeout=90,
114
+ )
115
+ elapsed = (time.perf_counter() - start) * 1000
116
+ runtime = payload.get("runtime", "unknown")
117
+ alt_text = payload.get("alt_text", "")
118
+ truncated = alt_text[:80] + ("..." if len(alt_text) > 80 else "") if alt_text else "(empty)"
119
+ if runtime == "minicpm-v4.6":
120
+ print(f" describe-image {_format_elapsed(elapsed):>10} PASS runtime={runtime} alt={truncated}")
121
+ return True
122
+ elif runtime == "fallback":
123
+ print(f" describe-image {_format_elapsed(elapsed):>10} FAIL runtime={runtime} (expected live)")
124
+ return False
125
+ else:
126
+ print(f" describe-image {_format_elapsed(elapsed):>10} FAIL runtime={runtime} (unexpected)")
127
+ return False
128
+ except (OSError, urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:
129
+ elapsed = (time.perf_counter() - start) * 1000
130
+ print(f" describe-image {_format_elapsed(elapsed):>10} FAIL ({exc.__class__.__name__})")
131
+ return False
132
+
133
+
134
+ def check_generate_image(base_url: str, runtime_status: dict[str, dict]) -> bool:
135
+ """Smoke test /api/generate-image with a deterministic prompt."""
136
+ image_status = _role_status(runtime_status, "image_generation")
137
+ image_configured = _role_is_configured(runtime_status, "image_generation")
138
+ if not image_configured and image_status in {"fallback-ready", "unknown"}:
139
+ print(f" generate-image {'SKIP':>10} Klein not configured")
140
+ return True
141
+ if image_configured and image_status != "online":
142
+ print(f" generate-image {'FAIL':>10} Klein configured but status={image_status}")
143
+ return False
144
+
145
+ start = time.perf_counter()
146
+ try:
147
+ payload = _post_json(
148
+ f"{base_url}/api/generate-image",
149
+ {"prompt": "smoke test accessible article thumbnail", "seed": 42},
150
+ timeout=180,
151
+ )
152
+ elapsed = (time.perf_counter() - start) * 1000
153
+ runtime = payload.get("runtime", "unknown")
154
+ image_url = payload.get("image_url", "")
155
+ if runtime == "modal-klein":
156
+ print(f" generate-image {_format_elapsed(elapsed):>10} PASS runtime={runtime} url={image_url[:60]}")
157
+ return True
158
+ elif runtime == "fallback":
159
+ print(f" generate-image {_format_elapsed(elapsed):>10} FAIL runtime={runtime} (expected live Klein)")
160
+ return False
161
+ else:
162
+ print(f" generate-image {_format_elapsed(elapsed):>10} FAIL runtime={runtime} (unexpected)")
163
+ return False
164
+ except (OSError, urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:
165
+ elapsed = (time.perf_counter() - start) * 1000
166
+ print(f" generate-image {_format_elapsed(elapsed):>10} FAIL ({exc.__class__.__name__})")
167
+ return False
168
+
169
+
170
+ def main() -> None:
171
+ parser = argparse.ArgumentParser(description="Tiny Narrator live model smoke tests")
172
+ parser.add_argument("--base-url", default="http://127.0.0.1:7860", help="App base URL (default: http://127.0.0.1:7860)")
173
+ args = parser.parse_args()
174
+
175
+ base_url = args.base_url.rstrip("/")
176
+ overall_start = time.perf_counter()
177
+
178
+ print(f"Tiny Narrator live smoke tests — {base_url}")
179
+ print()
180
+
181
+ print("Checks:")
182
+ runtime_status = check_runtime_status(base_url)
183
+
184
+ if not runtime_status:
185
+ print()
186
+ print("Could not reach the app. Is it running?")
187
+ sys.exit(1)
188
+
189
+ vision_configured = _role_is_configured(runtime_status, "vision")
190
+ klein_configured = _role_is_configured(runtime_status, "image_generation")
191
+
192
+ describe_ok = check_describe_image(base_url, runtime_status)
193
+ generate_ok = check_generate_image(base_url, runtime_status)
194
+
195
+ overall_elapsed = (time.perf_counter() - overall_start) * 1000
196
+ print()
197
+
198
+ failures = []
199
+ if vision_configured and not describe_ok:
200
+ failures.append("describe-image")
201
+ if klein_configured and not generate_ok:
202
+ failures.append("generate-image")
203
+
204
+ if failures:
205
+ print(f"FAILED: {', '.join(failures)} ({_format_elapsed(overall_elapsed)} total)")
206
+ sys.exit(1)
207
+ else:
208
+ configured_count = sum(1 for v in [vision_configured, klein_configured] if v)
209
+ skipped_count = 2 - configured_count
210
+ print(f"DONE: {configured_count} passed, {skipped_count} skipped ({_format_elapsed(overall_elapsed)} total)")
211
+ sys.exit(0)
212
+
213
+
214
+ if __name__ == "__main__":
215
+ main()
scripts/verify.py CHANGED
@@ -90,6 +90,8 @@ def verify_static_assets() -> None:
90
  assert_true("loadDemoScript" not in app_js, "Article frontend should not render the structured demo script")
91
  assert_true("loadModelBudget" in app_js, "Article frontend should render the model stack panel")
92
  assert_true("/api/model-budget" in app_js, "Article frontend should fetch model budget data")
 
 
93
  assert_true("modelStackList.innerHTML" in app_js, "Article frontend should render model stack items")
94
  assert_true("Tiny Titan pass" in app_js, "Article frontend should render per-model Tiny Titan pass labels")
95
  assert_true("/evidence" not in index_html, "Article page should not link to a removed evidence page")
@@ -118,6 +120,12 @@ def verify_static_assets() -> None:
118
  assert_true("readerQueueList" not in app_js, "Frontend should not bind a visible reader queue")
119
  assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
120
 
 
 
 
 
 
 
121
  submission = (ROOT / "SUBMISSION.md").read_text(encoding="utf-8")
122
  for target in ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"]:
123
  assert_true(target in submission, f"Submission packet should mention {target}")
@@ -174,13 +182,62 @@ def verify_space_metadata() -> None:
174
  assert_true("Apache License" in license_text, "Repository should include the Apache license text")
175
  assert_true("Version 2.0" in license_text, "Repository license should match Space metadata")
176
  assert_true(f"gradio=={sdk_version}" in requirements, "requirements.txt should pin Gradio to sdk_version")
177
- for package in ["pydantic", "kokoro", "soundfile"]:
178
  assert_true(
179
- any(line.startswith(f"{package}>=") or line.startswith(f"{package}==") for line in requirements),
 
 
 
180
  f"requirements.txt should include {package}",
181
  )
182
 
183
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
184
  def verify_core_fallbacks() -> None:
185
  narration = app.reader_brain_core(
186
  node_type="heading",
@@ -337,6 +394,58 @@ def verify_modal_klein_integration() -> None:
337
  assert_true("404" in worker_source or "not found" in worker_source.lower(), "Modal worker should handle missing media with 404")
338
  assert_true("reload" in worker_source, "Modal worker should reload volume before serving")
339
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
340
  # Test: runtime status includes Modal Klein path
341
  with patch.object(app, "KLEIN_MODAL_ENDPOINT", ""):
342
  status = app._runtime_status_core()
@@ -821,6 +930,8 @@ def main() -> None:
821
  py_compile.compile(str(ROOT / "modal_workers" / "klein_image.py"), doraise=True)
822
  verify_static_assets()
823
  verify_space_metadata()
 
 
824
  verify_core_fallbacks()
825
  verify_modal_klein_integration()
826
  verify_minicpm_vision_integration()
 
90
  assert_true("loadDemoScript" not in app_js, "Article frontend should not render the structured demo script")
91
  assert_true("loadModelBudget" in app_js, "Article frontend should render the model stack panel")
92
  assert_true("/api/model-budget" in app_js, "Article frontend should fetch model budget data")
93
+ assert_true("/api/runtime-status" in app_js, "Article frontend should fetch runtime status")
94
+ assert_true("status-pill" in app_js or "statusClass" in app_js, "Article frontend should render per-role status labels")
95
  assert_true("modelStackList.innerHTML" in app_js, "Article frontend should render model stack items")
96
  assert_true("Tiny Titan pass" in app_js, "Article frontend should render per-model Tiny Titan pass labels")
97
  assert_true("/evidence" not in index_html, "Article page should not link to a removed evidence page")
 
120
  assert_true("readerQueueList" not in app_js, "Frontend should not bind a visible reader queue")
121
  assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
122
 
123
+ assert_true("describeGeneratedThumbnail" in generate_js, "Generate frontend should describe generated thumbnails")
124
+ assert_true("/api/describe-image" in generate_js, "Generate frontend should call describe-image for thumbnails")
125
+ assert_true("image_url" in generate_js, "Generate frontend should send image_url to describe-image")
126
+ assert_true("generatedThumbnail.alt" in generate_js, "Generate frontend should update thumbnail alt from descriptor")
127
+ assert_true("fallbackAlt" in generate_js, "Generate frontend should preserve fallback alt on descriptor failure")
128
+
129
  submission = (ROOT / "SUBMISSION.md").read_text(encoding="utf-8")
130
  for target in ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"]:
131
  assert_true(target in submission, f"Submission packet should mention {target}")
 
182
  assert_true("Apache License" in license_text, "Repository should include the Apache license text")
183
  assert_true("Version 2.0" in license_text, "Repository license should match Space metadata")
184
  assert_true(f"gradio=={sdk_version}" in requirements, "requirements.txt should pin Gradio to sdk_version")
185
+ for package in ["pydantic", "kokoro", "soundfile", "python-dotenv"]:
186
  assert_true(
187
+ any(
188
+ line.startswith(f"{package}>=") or line.startswith(f"{package}==") or line.strip() == package
189
+ for line in requirements
190
+ ),
191
  f"requirements.txt should include {package}",
192
  )
193
 
194
 
195
+ def verify_dotenv_wiring() -> None:
196
+ """Verify that python-dotenv is wired in before core env constants are read."""
197
+ requirements = (ROOT / "requirements.txt").read_text(encoding="utf-8").splitlines()
198
+ assert_true(
199
+ any(line.strip() == "python-dotenv" or line.startswith("python-dotenv>=") for line in requirements),
200
+ "requirements.txt should include python-dotenv",
201
+ )
202
+
203
+ app_source = (ROOT / "app.py").read_text(encoding="utf-8")
204
+ assert_true("from dotenv import load_dotenv" in app_source, "app.py should import load_dotenv")
205
+ assert_true("load_dotenv()" in app_source, "app.py should call load_dotenv()")
206
+
207
+ import_pos = app_source.index("from dotenv import load_dotenv")
208
+ call_pos = app_source.index("load_dotenv()")
209
+ llama_const = app_source.index("LLAMA_CPP_BASE_URL = os.getenv")
210
+ assert_true(
211
+ import_pos < call_pos < llama_const,
212
+ "load_dotenv() should be called before LLAMA_CPP_BASE_URL is assigned",
213
+ )
214
+
215
+ env_example = ROOT / ".env.example"
216
+ assert_true(env_example.exists(), ".env.example should exist")
217
+ env_content = env_example.read_text(encoding="utf-8")
218
+ for secret_pattern in ["sk-", "AKIA", "ghp_", "xoxb-"]:
219
+ assert_true(
220
+ secret_pattern not in env_content,
221
+ f".env.example should not contain real-looking secrets ({secret_pattern})",
222
+ )
223
+
224
+
225
+ def verify_live_smoke_script() -> None:
226
+ """Verify that the live smoke test script exists, is syntactically valid, and documents redaction."""
227
+ smoke_path = ROOT / "scripts" / "live_smoke.py"
228
+ assert_true(smoke_path.exists(), "scripts/live_smoke.py should exist")
229
+ py_compile.compile(str(smoke_path), doraise=True)
230
+ smoke_source = smoke_path.read_text(encoding="utf-8")
231
+ assert_true("--base-url" in smoke_source, "Smoke script should support --base-url argument")
232
+ assert_true("/api/runtime-status" in smoke_source, "Smoke script should check runtime status")
233
+ assert_true("/api/describe-image" in smoke_source, "Smoke script should check describe-image")
234
+ assert_true("/api/generate-image" in smoke_source, "Smoke script should check generate-image")
235
+ assert_true("_role_is_configured" in smoke_source, "Smoke script should fail configured live paths instead of skipping them")
236
+ assert_true('details.get("configured") is True' in smoke_source, "Smoke script should preserve configured metadata from runtime status")
237
+ assert_true("SKIP" in smoke_source or "skip" in smoke_source, "Smoke script should document skipping behavior")
238
+ assert_true("Secrets" in smoke_source or "secrets" in smoke_source or "never printed" in smoke_source.lower(), "Smoke script should document secret redaction")
239
+
240
+
241
  def verify_core_fallbacks() -> None:
242
  narration = app.reader_brain_core(
243
  node_type="heading",
 
394
  assert_true("404" in worker_source or "not found" in worker_source.lower(), "Modal worker should handle missing media with 404")
395
  assert_true("reload" in worker_source, "Modal worker should reload volume before serving")
396
 
397
+ # Test: auth wiring — app sends bearer token when configured
398
+ with patch.object(app, "KLEIN_MODAL_ENDPOINT", "https://example-modal.modal.run"), patch.object(app, "KLEIN_MODAL_TOKEN", "test-secret-token"):
399
+ with patch("urllib.request.urlopen", return_value=mock_response) as urlopen_mock:
400
+ app.generate_image_core("auth test", seed=1)
401
+ sent_request = urlopen_mock.call_args.args[0]
402
+ assert_true(
403
+ sent_request.headers.get("Authorization") == "Bearer test-secret-token",
404
+ "App should send Bearer token when KLEIN_MODAL_TOKEN is configured",
405
+ )
406
+
407
+ # Test: auth wiring — health check sends bearer token when configured
408
+ good_health = MagicMock()
409
+ good_health.read.return_value = json.dumps({"ok": True, "model": app.MODEL_MANIFEST["image_generation"]["id"], "runtime": "modal-klein"}).encode("utf-8")
410
+ good_health.__enter__ = lambda s: s
411
+ good_health.__exit__ = MagicMock(return_value=False)
412
+ with patch.object(app, "KLEIN_MODAL_ENDPOINT", "https://example-modal.modal.run"), patch.object(app, "KLEIN_MODAL_TOKEN", "test-secret-token"):
413
+ with patch("urllib.request.urlopen", return_value=good_health) as urlopen_mock:
414
+ app._runtime_status_core()
415
+ health_request = urlopen_mock.call_args.args[0]
416
+ assert_true(
417
+ health_request.headers.get("Authorization") == "Bearer test-secret-token",
418
+ "App should send Bearer token for health checks when KLEIN_MODAL_TOKEN is configured",
419
+ )
420
+
421
+ # Test: .env.example includes KLEIN_MODAL_TOKEN
422
+ env_example = (ROOT / ".env.example").read_text(encoding="utf-8")
423
+ assert_true("KLEIN_MODAL_TOKEN" in env_example, ".env.example should document KLEIN_MODAL_TOKEN")
424
+
425
+ # Test: worker validates token
426
+ assert_true("KLEIN_MODAL_TOKEN" in worker_source, "Modal worker should read KLEIN_MODAL_TOKEN")
427
+ assert_true("401" in worker_source, "Modal worker should reject unauthorized requests with 401")
428
+ assert_true("_check_token" in worker_source, "Modal worker should have a token validation helper")
429
+ assert_true(
430
+ "async def health(request: Request)" in worker_source,
431
+ "Modal worker health route should accept the request for token validation",
432
+ )
433
+
434
+ # Test: runtime setup does not expose the token value
435
+ with patch.object(app, "KLEIN_MODAL_TOKEN", "super-secret-value"):
436
+ setup = app.runtime_setup_core()
437
+ setup_json = json.dumps(setup)
438
+ assert_true("super-secret-value" not in setup_json, "Runtime setup must never expose the token value")
439
+ image_step = next(step for step in setup["steps"] if step["role"] == "image_generation")
440
+ assert_true(
441
+ "KLEIN_MODAL_TOKEN" in image_step["env"],
442
+ "Runtime setup should document KLEIN_MODAL_TOKEN",
443
+ )
444
+ assert_true(
445
+ image_step["env"]["KLEIN_MODAL_TOKEN"] == "(configured)",
446
+ "Runtime setup should show token as configured without exposing the value",
447
+ )
448
+
449
  # Test: runtime status includes Modal Klein path
450
  with patch.object(app, "KLEIN_MODAL_ENDPOINT", ""):
451
  status = app._runtime_status_core()
 
930
  py_compile.compile(str(ROOT / "modal_workers" / "klein_image.py"), doraise=True)
931
  verify_static_assets()
932
  verify_space_metadata()
933
+ verify_dotenv_wiring()
934
+ verify_live_smoke_script()
935
  verify_core_fallbacks()
936
  verify_modal_klein_integration()
937
  verify_minicpm_vision_integration()
static/app.css CHANGED
@@ -589,6 +589,33 @@ dd {
589
  border: 1px solid rgba(42, 122, 110, 0.2);
590
  }
591
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
592
  .image-receipt-pill {
593
  background: rgba(180, 150, 60, 0.1);
594
  color: #7a6520;
 
589
  border: 1px solid rgba(42, 122, 110, 0.2);
590
  }
591
 
592
+ .status-pill {
593
+ font-family: var(--mono);
594
+ font-size: 0.66rem;
595
+ font-weight: 700;
596
+ padding: 1px 6px;
597
+ letter-spacing: 0.04em;
598
+ text-transform: uppercase;
599
+ }
600
+
601
+ .status-online {
602
+ background: rgba(34, 140, 60, 0.12);
603
+ color: #1d7a34;
604
+ border: 1px solid rgba(34, 140, 60, 0.25);
605
+ }
606
+
607
+ .status-fallback {
608
+ background: rgba(160, 140, 60, 0.1);
609
+ color: #7a6a20;
610
+ border: 1px solid rgba(160, 140, 60, 0.25);
611
+ }
612
+
613
+ .status-offline {
614
+ background: rgba(180, 50, 50, 0.1);
615
+ color: #a03030;
616
+ border: 1px solid rgba(180, 50, 50, 0.25);
617
+ }
618
+
619
  .image-receipt-pill {
620
  background: rgba(180, 150, 60, 0.1);
621
  color: #7a6520;
static/app.js CHANGED
@@ -213,24 +213,12 @@ function roleLabel(role) {
213
  }
214
 
215
  async function loadModelBudget() {
216
- try {
217
- const payload = await postJson("/api/model-budget");
218
- modelBudgetStatus.textContent = payload.all_models_within_limit
219
- ? `All <= ${payload.limit_billion}B`
220
- : "Review needed";
221
- modelStackList.innerHTML = payload.models
222
- .map((model) => `
223
- <li>
224
- <div class="budget-row">
225
- <span>${escapeHtml(roleLabel(model.role))}</span>
226
- <span class="budget-pill">${escapeHtml(model.params)}</span>
227
- </div>
228
- <p>${escapeHtml(model.runtime)} | ${model.within_limit ? "Tiny Titan pass" : "Review parameter budget"}</p>
229
- <p>${escapeHtml(model.id)}</p>
230
- </li>
231
- `)
232
- .join("");
233
- } catch {
234
  modelBudgetStatus.textContent = "Unavailable";
235
  modelStackList.innerHTML = `
236
  <li>
@@ -241,7 +229,35 @@ async function loadModelBudget() {
241
  <p>Model budget is unavailable.</p>
242
  </li>
243
  `;
 
244
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
245
  }
246
 
247
  async function loadImageDescriptions() {
 
213
  }
214
 
215
  async function loadModelBudget() {
216
+ const [budget, runtimeStatus] = await Promise.allSettled([
217
+ postJson("/api/model-budget"),
218
+ postJson("/api/runtime-status"),
219
+ ]);
220
+
221
+ if (budget.status === "rejected") {
 
 
 
 
 
 
 
 
 
 
 
 
222
  modelBudgetStatus.textContent = "Unavailable";
223
  modelStackList.innerHTML = `
224
  <li>
 
229
  <p>Model budget is unavailable.</p>
230
  </li>
231
  `;
232
+ return;
233
  }
234
+
235
+ const payload = budget.value;
236
+ const statusMap = runtimeStatus.status === "fulfilled" ? runtimeStatus.value : null;
237
+
238
+ modelBudgetStatus.textContent = payload.all_models_within_limit
239
+ ? `All <= ${payload.limit_billion}B`
240
+ : "Review needed";
241
+
242
+ modelStackList.innerHTML = payload.models
243
+ .map((model) => {
244
+ const status = statusMap?.[model.role];
245
+ const statusLabel = status?.status || "unknown";
246
+ const statusClass = statusLabel === "online" ? "status-online"
247
+ : statusLabel === "offline" ? "status-offline"
248
+ : "status-fallback";
249
+ return `
250
+ <li>
251
+ <div class="budget-row">
252
+ <span>${escapeHtml(roleLabel(model.role))}</span>
253
+ <span class="budget-pill">${escapeHtml(model.params)}</span>
254
+ </div>
255
+ <p>${escapeHtml(model.runtime)} | ${model.within_limit ? "Tiny Titan pass" : "Review parameter budget"} | <span class="status-pill ${statusClass}">${escapeHtml(statusLabel)}</span></p>
256
+ <p>${escapeHtml(model.id)}</p>
257
+ </li>
258
+ `;
259
+ })
260
+ .join("");
261
  }
262
 
263
  async function loadImageDescriptions() {
static/generate.js CHANGED
@@ -427,6 +427,27 @@ function renderArticle(payload) {
427
  }
428
  }
429
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
430
  async function loadManifest() {
431
  try {
432
  const manifest = await postJson("/api/article-manifest");
@@ -464,6 +485,7 @@ form.addEventListener("submit", async (event) => {
464
  try {
465
  const payload = await postJson("/api/generate-article", { topic });
466
  renderArticle(payload);
 
467
  } catch (error) {
468
  generatorStatus.textContent = `Generation failed: ${error.message}`;
469
  } finally {
 
427
  }
428
  }
429
 
430
+ async function describeGeneratedThumbnail(payload) {
431
+ const thumbnail = payload.thumbnail;
432
+ const fallbackAlt = generatedThumbnail.alt;
433
+ try {
434
+ const result = await postJson("/api/describe-image", {
435
+ image_id: "generated-thumbnail",
436
+ caption: payload.article.title || payload.topic,
437
+ prompt: thumbnail.prompt || payload.topic,
438
+ image_url: thumbnail.image_url,
439
+ });
440
+ if (result.ok && result.alt_text) {
441
+ generatedThumbnail.alt = result.alt_text;
442
+ refreshReaderNodes();
443
+ thumbnailReceipt.textContent =
444
+ `${thumbnail.generation_model} | seed ${thumbnail.seed} | ${thumbnail.runtime} | described by ${result.runtime}`;
445
+ }
446
+ } catch {
447
+ generatedThumbnail.alt = fallbackAlt;
448
+ }
449
+ }
450
+
451
  async function loadManifest() {
452
  try {
453
  const manifest = await postJson("/api/article-manifest");
 
485
  try {
486
  const payload = await postJson("/api/generate-article", { topic });
487
  renderArticle(payload);
488
+ describeGeneratedThumbnail(payload);
489
  } catch (error) {
490
  generatorStatus.textContent = `Generation failed: ${error.message}`;
491
  } finally {