cvpfus Codex commited on
Commit ·
5b2e87d
1
Parent(s): 5aad2d1
Add runtime status to readiness checks
Browse filesCo-authored-by: Codex <codex@openai.com>
- FIELD_NOTES.md +1 -1
- README.md +1 -1
- SUBMISSION.md +1 -1
- app.py +14 -0
- scripts/verify.py +3 -0
FIELD_NOTES.md
CHANGED
|
@@ -30,7 +30,7 @@ Runtime setup is also data-backed. `/api/runtime-setup` lists the app command, l
|
|
| 30 |
|
| 31 |
The judge runbook lives at `/api/demo-script` and is rendered in the session panel with its API evidence checks. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech. Each API check includes copyable curl and PowerShell-friendly `curl.exe` commands for quick reproduction. Deployed demos can set `PUBLIC_BASE_URL` so those commands point at the public Space instead of localhost.
|
| 32 |
|
| 33 |
-
Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, accessibility audit, image receipts, executable demo API checks, and command base URL checks. POST checks must include sample bodies before readiness passes.
|
| 34 |
|
| 35 |
The copyable judge evidence bundle lives at `/api/evidence-bundle`. The session panel's Copy Evidence button uses that endpoint to put the core receipts, including `PUBLIC_BASE_URL`, runtime status, and readiness, on the clipboard as formatted JSON.
|
| 36 |
|
|
|
|
| 30 |
|
| 31 |
The judge runbook lives at `/api/demo-script` and is rendered in the session panel with its API evidence checks. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech. Each API check includes copyable curl and PowerShell-friendly `curl.exe` commands for quick reproduction. Deployed demos can set `PUBLIC_BASE_URL` so those commands point at the public Space instead of localhost.
|
| 32 |
|
| 33 |
+
Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, runtime status, accessibility audit, image receipts, executable demo API checks, and command base URL checks. POST checks must include sample bodies before readiness passes.
|
| 34 |
|
| 35 |
The copyable judge evidence bundle lives at `/api/evidence-bundle`. The session panel's Copy Evidence button uses that endpoint to put the core receipts, including `PUBLIC_BASE_URL`, runtime status, and readiness, on the clipboard as formatted JSON.
|
| 36 |
|
README.md
CHANGED
|
@@ -82,7 +82,7 @@ The verifier checks syntax, static assets, Space metadata consistency, determini
|
|
| 82 |
|
| 83 |
`/api/image-descriptions` includes image-generation provenance for every article illustration: the planned FLUX.2 klein model, prompt, seed, bundled asset URL, and fallback-ready status.
|
| 84 |
|
| 85 |
-
`/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, accessibility, image receipts, demo API checks, and command base URL checks.
|
| 86 |
|
| 87 |
The readiness rollup only passes the demo API check when POST entries include executable sample bodies.
|
| 88 |
|
|
|
|
| 82 |
|
| 83 |
`/api/image-descriptions` includes image-generation provenance for every article illustration: the planned FLUX.2 klein model, prompt, seed, bundled asset URL, and fallback-ready status.
|
| 84 |
|
| 85 |
+
`/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, runtime status, accessibility, image receipts, demo API checks, and command base URL checks.
|
| 86 |
|
| 87 |
The readiness rollup only passes the demo API check when POST entries include executable sample bodies.
|
| 88 |
|
SUBMISSION.md
CHANGED
|
@@ -43,7 +43,7 @@ The prototype is designed for a live hackathon demo: every model-facing path has
|
|
| 43 |
- `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
|
| 44 |
- `/api/demo-script`: repeatable judge runbook and API checks.
|
| 45 |
- `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
|
| 46 |
-
- `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, accessibility, image receipts, and demo API checks.
|
| 47 |
- `/api/evidence-bundle`: copyable JSON bundle containing the core judge evidence receipts, runtime status, and readiness rollup.
|
| 48 |
|
| 49 |
The checks in `/api/demo-script` include copyable curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
|
|
|
| 43 |
- `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
|
| 44 |
- `/api/demo-script`: repeatable judge runbook and API checks.
|
| 45 |
- `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
|
| 46 |
+
- `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, runtime status, accessibility, image receipts, and demo API checks.
|
| 47 |
- `/api/evidence-bundle`: copyable JSON bundle containing the core judge evidence receipts, runtime status, and readiness rollup.
|
| 48 |
|
| 49 |
The checks in `/api/demo-script` include copyable curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
app.py
CHANGED
|
@@ -304,6 +304,7 @@ def demo_script_core() -> dict[str, Any]:
|
|
| 304 |
def submission_readiness_core() -> dict[str, Any]:
|
| 305 |
model_budget = model_budget_core()
|
| 306 |
runtime_setup = runtime_setup_core()
|
|
|
|
| 307 |
accessibility = accessibility_audit_core()
|
| 308 |
demo_script = demo_script_core()
|
| 309 |
image_descriptions = describe_article_images_core()
|
|
@@ -316,6 +317,13 @@ def submission_readiness_core() -> dict[str, Any]:
|
|
| 316 |
PUBLIC_BASE_URL in item["curl"] and PUBLIC_BASE_URL in item["powershell"]
|
| 317 |
for item in demo_script["api_checks"]
|
| 318 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
checks = [
|
| 320 |
{
|
| 321 |
"id": "tiny_titan_budget",
|
|
@@ -341,6 +349,12 @@ def submission_readiness_core() -> dict[str, Any]:
|
|
| 341 |
"status": "pass" if runtime_roles == expected_roles else "fail",
|
| 342 |
"evidence": "Runtime setup covers reader brain, speech, vision, and image generation paths.",
|
| 343 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 344 |
{
|
| 345 |
"id": "reader_accessibility",
|
| 346 |
"label": "Reader accessibility",
|
|
|
|
| 304 |
def submission_readiness_core() -> dict[str, Any]:
|
| 305 |
model_budget = model_budget_core()
|
| 306 |
runtime_setup = runtime_setup_core()
|
| 307 |
+
runtime_status = _runtime_status_core()
|
| 308 |
accessibility = accessibility_audit_core()
|
| 309 |
demo_script = demo_script_core()
|
| 310 |
image_descriptions = describe_article_images_core()
|
|
|
|
| 317 |
PUBLIC_BASE_URL in item["curl"] and PUBLIC_BASE_URL in item["powershell"]
|
| 318 |
for item in demo_script["api_checks"]
|
| 319 |
)
|
| 320 |
+
runtime_statuses = {
|
| 321 |
+
runtime_status["reader_brain"]["status"],
|
| 322 |
+
runtime_status["vision"]["status"],
|
| 323 |
+
runtime_status["speech"]["status"],
|
| 324 |
+
runtime_status["image_generation"]["status"],
|
| 325 |
+
}
|
| 326 |
+
runtime_status_ready = runtime_statuses <= {"online", "fallback-ready", "placeholder-ready"}
|
| 327 |
checks = [
|
| 328 |
{
|
| 329 |
"id": "tiny_titan_budget",
|
|
|
|
| 349 |
"status": "pass" if runtime_roles == expected_roles else "fail",
|
| 350 |
"evidence": "Runtime setup covers reader brain, speech, vision, and image generation paths.",
|
| 351 |
},
|
| 352 |
+
{
|
| 353 |
+
"id": "runtime_status",
|
| 354 |
+
"label": "Runtime status",
|
| 355 |
+
"status": "pass" if runtime_status_ready else "fail",
|
| 356 |
+
"evidence": "Runtime status labels every model path as online, fallback-ready, or placeholder-ready.",
|
| 357 |
+
},
|
| 358 |
{
|
| 359 |
"id": "reader_accessibility",
|
| 360 |
"label": "Reader accessibility",
|
scripts/verify.py
CHANGED
|
@@ -466,6 +466,7 @@ def verify_routes() -> None:
|
|
| 466 |
"award_targets",
|
| 467 |
"custom_frontend",
|
| 468 |
"runtime_setup",
|
|
|
|
| 469 |
"reader_accessibility",
|
| 470 |
"image_receipts",
|
| 471 |
"demo_api_checks",
|
|
@@ -473,6 +474,8 @@ def verify_routes() -> None:
|
|
| 473 |
},
|
| 474 |
"Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
|
| 475 |
)
|
|
|
|
|
|
|
| 476 |
demo_api_check = next(item for item in readiness_payload["checks"] if item["id"] == "demo_api_checks")
|
| 477 |
assert_true(demo_api_check["status"] == "pass", "Submission readiness should pass executable demo API checks")
|
| 478 |
command_base_check = next(item for item in readiness_payload["checks"] if item["id"] == "command_base_url")
|
|
|
|
| 466 |
"award_targets",
|
| 467 |
"custom_frontend",
|
| 468 |
"runtime_setup",
|
| 469 |
+
"runtime_status",
|
| 470 |
"reader_accessibility",
|
| 471 |
"image_receipts",
|
| 472 |
"demo_api_checks",
|
|
|
|
| 474 |
},
|
| 475 |
"Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
|
| 476 |
)
|
| 477 |
+
runtime_status_check = next(item for item in readiness_payload["checks"] if item["id"] == "runtime_status")
|
| 478 |
+
assert_true(runtime_status_check["status"] == "pass", "Submission readiness should pass runtime status checks")
|
| 479 |
demo_api_check = next(item for item in readiness_payload["checks"] if item["id"] == "demo_api_checks")
|
| 480 |
assert_true(demo_api_check["status"] == "pass", "Submission readiness should pass executable demo API checks")
|
| 481 |
command_base_check = next(item for item in readiness_payload["checks"] if item["id"] == "command_base_url")
|