cvpfus Codex commited on
Commit
5b2e87d
·
1 Parent(s): 5aad2d1

Add runtime status to readiness checks

Browse files

Co-authored-by: Codex <codex@openai.com>

Files changed (5) hide show
  1. FIELD_NOTES.md +1 -1
  2. README.md +1 -1
  3. SUBMISSION.md +1 -1
  4. app.py +14 -0
  5. scripts/verify.py +3 -0
FIELD_NOTES.md CHANGED
@@ -30,7 +30,7 @@ Runtime setup is also data-backed. `/api/runtime-setup` lists the app command, l
30
 
31
  The judge runbook lives at `/api/demo-script` and is rendered in the session panel with its API evidence checks. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech. Each API check includes copyable curl and PowerShell-friendly `curl.exe` commands for quick reproduction. Deployed demos can set `PUBLIC_BASE_URL` so those commands point at the public Space instead of localhost.
32
 
33
- Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, accessibility audit, image receipts, executable demo API checks, and command base URL checks. POST checks must include sample bodies before readiness passes.
34
 
35
  The copyable judge evidence bundle lives at `/api/evidence-bundle`. The session panel's Copy Evidence button uses that endpoint to put the core receipts, including `PUBLIC_BASE_URL`, runtime status, and readiness, on the clipboard as formatted JSON.
36
 
 
30
 
31
  The judge runbook lives at `/api/demo-script` and is rendered in the session panel with its API evidence checks. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech. Each API check includes copyable curl and PowerShell-friendly `curl.exe` commands for quick reproduction. Deployed demos can set `PUBLIC_BASE_URL` so those commands point at the public Space instead of localhost.
32
 
33
+ Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, runtime status, accessibility audit, image receipts, executable demo API checks, and command base URL checks. POST checks must include sample bodies before readiness passes.
34
 
35
  The copyable judge evidence bundle lives at `/api/evidence-bundle`. The session panel's Copy Evidence button uses that endpoint to put the core receipts, including `PUBLIC_BASE_URL`, runtime status, and readiness, on the clipboard as formatted JSON.
36
 
README.md CHANGED
@@ -82,7 +82,7 @@ The verifier checks syntax, static assets, Space metadata consistency, determini
82
 
83
  `/api/image-descriptions` includes image-generation provenance for every article illustration: the planned FLUX.2 klein model, prompt, seed, bundled asset URL, and fallback-ready status.
84
 
85
- `/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, accessibility, image receipts, demo API checks, and command base URL checks.
86
 
87
  The readiness rollup only passes the demo API check when POST entries include executable sample bodies.
88
 
 
82
 
83
  `/api/image-descriptions` includes image-generation provenance for every article illustration: the planned FLUX.2 klein model, prompt, seed, bundled asset URL, and fallback-ready status.
84
 
85
+ `/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, runtime status, accessibility, image receipts, demo API checks, and command base URL checks.
86
 
87
  The readiness rollup only passes the demo API check when POST entries include executable sample bodies.
88
 
SUBMISSION.md CHANGED
@@ -43,7 +43,7 @@ The prototype is designed for a live hackathon demo: every model-facing path has
43
  - `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
44
  - `/api/demo-script`: repeatable judge runbook and API checks.
45
  - `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
46
- - `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, accessibility, image receipts, and demo API checks.
47
  - `/api/evidence-bundle`: copyable JSON bundle containing the core judge evidence receipts, runtime status, and readiness rollup.
48
 
49
  The checks in `/api/demo-script` include copyable curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
 
43
  - `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
44
  - `/api/demo-script`: repeatable judge runbook and API checks.
45
  - `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
46
+ - `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, runtime status, accessibility, image receipts, and demo API checks.
47
  - `/api/evidence-bundle`: copyable JSON bundle containing the core judge evidence receipts, runtime status, and readiness rollup.
48
 
49
  The checks in `/api/demo-script` include copyable curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
app.py CHANGED
@@ -304,6 +304,7 @@ def demo_script_core() -> dict[str, Any]:
304
  def submission_readiness_core() -> dict[str, Any]:
305
  model_budget = model_budget_core()
306
  runtime_setup = runtime_setup_core()
 
307
  accessibility = accessibility_audit_core()
308
  demo_script = demo_script_core()
309
  image_descriptions = describe_article_images_core()
@@ -316,6 +317,13 @@ def submission_readiness_core() -> dict[str, Any]:
316
  PUBLIC_BASE_URL in item["curl"] and PUBLIC_BASE_URL in item["powershell"]
317
  for item in demo_script["api_checks"]
318
  )
 
 
 
 
 
 
 
319
  checks = [
320
  {
321
  "id": "tiny_titan_budget",
@@ -341,6 +349,12 @@ def submission_readiness_core() -> dict[str, Any]:
341
  "status": "pass" if runtime_roles == expected_roles else "fail",
342
  "evidence": "Runtime setup covers reader brain, speech, vision, and image generation paths.",
343
  },
 
 
 
 
 
 
344
  {
345
  "id": "reader_accessibility",
346
  "label": "Reader accessibility",
 
304
  def submission_readiness_core() -> dict[str, Any]:
305
  model_budget = model_budget_core()
306
  runtime_setup = runtime_setup_core()
307
+ runtime_status = _runtime_status_core()
308
  accessibility = accessibility_audit_core()
309
  demo_script = demo_script_core()
310
  image_descriptions = describe_article_images_core()
 
317
  PUBLIC_BASE_URL in item["curl"] and PUBLIC_BASE_URL in item["powershell"]
318
  for item in demo_script["api_checks"]
319
  )
320
+ runtime_statuses = {
321
+ runtime_status["reader_brain"]["status"],
322
+ runtime_status["vision"]["status"],
323
+ runtime_status["speech"]["status"],
324
+ runtime_status["image_generation"]["status"],
325
+ }
326
+ runtime_status_ready = runtime_statuses <= {"online", "fallback-ready", "placeholder-ready"}
327
  checks = [
328
  {
329
  "id": "tiny_titan_budget",
 
349
  "status": "pass" if runtime_roles == expected_roles else "fail",
350
  "evidence": "Runtime setup covers reader brain, speech, vision, and image generation paths.",
351
  },
352
+ {
353
+ "id": "runtime_status",
354
+ "label": "Runtime status",
355
+ "status": "pass" if runtime_status_ready else "fail",
356
+ "evidence": "Runtime status labels every model path as online, fallback-ready, or placeholder-ready.",
357
+ },
358
  {
359
  "id": "reader_accessibility",
360
  "label": "Reader accessibility",
scripts/verify.py CHANGED
@@ -466,6 +466,7 @@ def verify_routes() -> None:
466
  "award_targets",
467
  "custom_frontend",
468
  "runtime_setup",
 
469
  "reader_accessibility",
470
  "image_receipts",
471
  "demo_api_checks",
@@ -473,6 +474,8 @@ def verify_routes() -> None:
473
  },
474
  "Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
475
  )
 
 
476
  demo_api_check = next(item for item in readiness_payload["checks"] if item["id"] == "demo_api_checks")
477
  assert_true(demo_api_check["status"] == "pass", "Submission readiness should pass executable demo API checks")
478
  command_base_check = next(item for item in readiness_payload["checks"] if item["id"] == "command_base_url")
 
466
  "award_targets",
467
  "custom_frontend",
468
  "runtime_setup",
469
+ "runtime_status",
470
  "reader_accessibility",
471
  "image_receipts",
472
  "demo_api_checks",
 
474
  },
475
  "Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
476
  )
477
+ runtime_status_check = next(item for item in readiness_payload["checks"] if item["id"] == "runtime_status")
478
+ assert_true(runtime_status_check["status"] == "pass", "Submission readiness should pass runtime status checks")
479
  demo_api_check = next(item for item in readiness_payload["checks"] if item["id"] == "demo_api_checks")
480
  assert_true(demo_api_check["status"] == "pass", "Submission readiness should pass executable demo API checks")
481
  command_base_check = next(item for item in readiness_payload["checks"] if item["id"] == "command_base_url")