cvpfus Codex commited on
Commit ·
8b872c3
1
Parent(s): 0347c74
Add copyable judge evidence bundle
Browse filesCo-authored-by: Codex <codex@openai.com>
- FIELD_NOTES.md +2 -0
- README.md +5 -1
- SUBMISSION.md +3 -0
- app.py +27 -0
- scripts/verify.py +15 -0
- static/app.css +2 -1
- static/app.js +15 -0
- static/index.html +1 -0
FIELD_NOTES.md
CHANGED
|
@@ -32,6 +32,8 @@ The judge runbook lives at `/api/demo-script` and is rendered in the session pan
|
|
| 32 |
|
| 33 |
Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, accessibility audit, image receipts, and executable demo API checks.
|
| 34 |
|
|
|
|
|
|
|
| 35 |
Image provenance lives in `/api/image-descriptions`. Each article illustration carries the planned FLUX.2 klein model id, prompt, seed, asset URL, and fallback status, and the session panel renders those receipts so the generated-image claim is inspectable.
|
| 36 |
|
| 37 |
Accessibility evidence lives at `/api/accessibility-audit`. It records the reader-mode choices that matter most for this prototype: semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
|
|
|
|
| 32 |
|
| 33 |
Submission readiness lives at `/api/submission-readiness`. It aggregates the demo-critical checks into one payload: model budget, award evidence, custom frontend assets, runtime setup, accessibility audit, image receipts, and executable demo API checks.
|
| 34 |
|
| 35 |
+
The copyable judge evidence bundle lives at `/api/evidence-bundle`. The session panel's Copy Evidence button uses that endpoint to put the core receipts on the clipboard as formatted JSON.
|
| 36 |
+
|
| 37 |
Image provenance lives in `/api/image-descriptions`. Each article illustration carries the planned FLUX.2 klein model id, prompt, seed, asset URL, and fallback status, and the session panel renders those receipts so the generated-image claim is inspectable.
|
| 38 |
|
| 39 |
Accessibility evidence lives at `/api/accessibility-audit`. It records the reader-mode choices that matter most for this prototype: semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
|
README.md
CHANGED
|
@@ -52,7 +52,7 @@ Start the app:
|
|
| 52 |
python app.py
|
| 53 |
```
|
| 54 |
|
| 55 |
-
Open the local URL printed by Gradio. The custom frontend calls `/api/reader-brain`, `/api/image-descriptions`, `/api/describe-image`, `/api/speak`, `/api/generate-image`, `/api/model-budget`, `/api/runtime-setup`, `/api/demo-script`, `/api/accessibility-audit`,
|
| 56 |
|
| 57 |
Useful environment variables:
|
| 58 |
|
|
@@ -83,6 +83,8 @@ The verifier checks syntax, static assets, Space metadata consistency, determini
|
|
| 83 |
|
| 84 |
`/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, accessibility, image receipts, and demo API checks.
|
| 85 |
|
|
|
|
|
|
|
| 86 |
## Screen Reader Mode
|
| 87 |
|
| 88 |
The frontend builds a reading queue from semantic article nodes. When screen-reader mode is on:
|
|
@@ -124,6 +126,8 @@ The session panel renders the judge runbook and API evidence checks from `/api/d
|
|
| 124 |
|
| 125 |
A submission-readiness panel reads `/api/submission-readiness`, giving judges one compact rollup of the claims the live app can prove.
|
| 126 |
|
|
|
|
|
|
|
| 127 |
A runtime-plan panel reads `/api/runtime-setup` and summarizes each model path's runtime plus fallback, keeping the live demo honest about what is online and what is deterministic.
|
| 128 |
|
| 129 |
`/api/runtime-status` performs a short readiness check for llama.cpp and local speech dependencies, then reports which fallback paths are ready for a live demo.
|
|
|
|
| 52 |
python app.py
|
| 53 |
```
|
| 54 |
|
| 55 |
+
Open the local URL printed by Gradio. The custom frontend calls `/api/reader-brain`, `/api/image-descriptions`, `/api/describe-image`, `/api/speak`, `/api/generate-image`, `/api/model-budget`, `/api/runtime-setup`, `/api/demo-script`, `/api/accessibility-audit`, `/api/submission-readiness`, and `/api/evidence-bundle`.
|
| 56 |
|
| 57 |
Useful environment variables:
|
| 58 |
|
|
|
|
| 83 |
|
| 84 |
`/api/submission-readiness` aggregates the judging receipts into one pass/fail payload covering model budget, award evidence, custom frontend assets, runtime setup, accessibility, image receipts, and demo API checks.
|
| 85 |
|
| 86 |
+
`/api/evidence-bundle` returns a copyable JSON bundle of the main judging receipts. The Submission Readiness panel includes a Copy Evidence button that writes this bundle to the clipboard.
|
| 87 |
+
|
| 88 |
## Screen Reader Mode
|
| 89 |
|
| 90 |
The frontend builds a reading queue from semantic article nodes. When screen-reader mode is on:
|
|
|
|
| 126 |
|
| 127 |
A submission-readiness panel reads `/api/submission-readiness`, giving judges one compact rollup of the claims the live app can prove.
|
| 128 |
|
| 129 |
+
The Copy Evidence button reads `/api/evidence-bundle` and copies the core judge receipts as formatted JSON for quick review or submission notes.
|
| 130 |
+
|
| 131 |
A runtime-plan panel reads `/api/runtime-setup` and summarizes each model path's runtime plus fallback, keeping the live demo honest about what is online and what is deterministic.
|
| 132 |
|
| 133 |
`/api/runtime-status` performs a short readiness check for llama.cpp and local speech dependencies, then reports which fallback paths are ready for a live demo.
|
SUBMISSION.md
CHANGED
|
@@ -44,6 +44,7 @@ The prototype is designed for a live hackathon demo: every model-facing path has
|
|
| 44 |
- `/api/demo-script`: repeatable judge runbook and API checks.
|
| 45 |
- `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
|
| 46 |
- `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, accessibility, image receipts, and demo API checks.
|
|
|
|
| 47 |
|
| 48 |
The POST checks in `/api/demo-script` include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
| 49 |
|
|
@@ -53,6 +54,8 @@ The image receipts keep the generated-asset claim inspectable: each article illu
|
|
| 53 |
|
| 54 |
The submission-readiness panel and API give judges a compact checklist for the whole build, so the live demo can move from individual receipts to an overall readiness view.
|
| 55 |
|
|
|
|
|
|
|
| 56 |
## Reliability notes
|
| 57 |
|
| 58 |
The demo remains navigable when local models are unavailable. The reader brain falls back to deterministic narration, image descriptions fall back to cached alt text, speech falls back to browser speech plus transcript, and generated images fall back to bundled article assets. Fallback states are labeled instead of hidden.
|
|
|
|
| 44 |
- `/api/demo-script`: repeatable judge runbook and API checks.
|
| 45 |
- `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
|
| 46 |
- `/api/submission-readiness`: one pass/fail rollup for model budget, award evidence, custom frontend, runtime setup, accessibility, image receipts, and demo API checks.
|
| 47 |
+
- `/api/evidence-bundle`: copyable JSON bundle containing the core judge evidence receipts.
|
| 48 |
|
| 49 |
The POST checks in `/api/demo-script` include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
| 50 |
|
|
|
|
| 54 |
|
| 55 |
The submission-readiness panel and API give judges a compact checklist for the whole build, so the live demo can move from individual receipts to an overall readiness view.
|
| 56 |
|
| 57 |
+
The Copy Evidence button pulls `/api/evidence-bundle` and writes the formatted JSON bundle to the clipboard for quick judging notes.
|
| 58 |
+
|
| 59 |
## Reliability notes
|
| 60 |
|
| 61 |
The demo remains navigable when local models are unavailable. The reader brain falls back to deterministic narration, image descriptions fall back to cached alt text, speech falls back to browser speech plus transcript, and generated images fall back to bundled article assets. Fallback states are labeled instead of hidden.
|
app.py
CHANGED
|
@@ -354,6 +354,23 @@ def submission_readiness_core() -> dict[str, Any]:
|
|
| 354 |
}
|
| 355 |
|
| 356 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 357 |
def accessibility_audit_core() -> dict[str, Any]:
|
| 358 |
checks = [
|
| 359 |
{
|
|
@@ -792,6 +809,11 @@ async def submission_readiness() -> JSONResponse:
|
|
| 792 |
return _json(submission_readiness_core())
|
| 793 |
|
| 794 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 795 |
@app.get("/api/runtime-status")
|
| 796 |
async def runtime_status() -> JSONResponse:
|
| 797 |
return _json(_runtime_status_core())
|
|
@@ -862,6 +884,11 @@ def submission_readiness_api() -> str:
|
|
| 862 |
return json.dumps(submission_readiness_core())
|
| 863 |
|
| 864 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 865 |
@app.api(name="speak")
|
| 866 |
def speak_api(text: str, voice: str = "af_heart", speed: float = 1.0) -> str:
|
| 867 |
return json.dumps(speak_core(text, voice, speed))
|
|
|
|
| 354 |
}
|
| 355 |
|
| 356 |
|
| 357 |
+
def evidence_bundle_core() -> dict[str, Any]:
|
| 358 |
+
return {
|
| 359 |
+
"ok": True,
|
| 360 |
+
"title": "Tiny Narrator judge evidence bundle",
|
| 361 |
+
"frontend": "custom Gradio Server HTML/CSS/JS",
|
| 362 |
+
"bonus_targets": ARTICLE_MANIFEST["bonus_targets"],
|
| 363 |
+
"models": MODEL_MANIFEST,
|
| 364 |
+
"award_evidence": {"ok": True, "items": AWARD_EVIDENCE},
|
| 365 |
+
"model_budget": model_budget_core(),
|
| 366 |
+
"runtime_setup": runtime_setup_core(),
|
| 367 |
+
"demo_script": demo_script_core(),
|
| 368 |
+
"accessibility_audit": accessibility_audit_core(),
|
| 369 |
+
"image_descriptions": describe_article_images_core(),
|
| 370 |
+
"submission_readiness": submission_readiness_core(),
|
| 371 |
+
}
|
| 372 |
+
|
| 373 |
+
|
| 374 |
def accessibility_audit_core() -> dict[str, Any]:
|
| 375 |
checks = [
|
| 376 |
{
|
|
|
|
| 809 |
return _json(submission_readiness_core())
|
| 810 |
|
| 811 |
|
| 812 |
+
@app.get("/api/evidence-bundle")
|
| 813 |
+
async def evidence_bundle() -> JSONResponse:
|
| 814 |
+
return _json(evidence_bundle_core())
|
| 815 |
+
|
| 816 |
+
|
| 817 |
@app.get("/api/runtime-status")
|
| 818 |
async def runtime_status() -> JSONResponse:
|
| 819 |
return _json(_runtime_status_core())
|
|
|
|
| 884 |
return json.dumps(submission_readiness_core())
|
| 885 |
|
| 886 |
|
| 887 |
+
@app.api(name="evidence_bundle")
|
| 888 |
+
def evidence_bundle_api() -> str:
|
| 889 |
+
return json.dumps(evidence_bundle_core())
|
| 890 |
+
|
| 891 |
+
|
| 892 |
@app.api(name="speak")
|
| 893 |
def speak_api(text: str, voice: str = "af_heart", speed: float = 1.0) -> str:
|
| 894 |
return json.dumps(speak_core(text, voice, speed))
|
scripts/verify.py
CHANGED
|
@@ -42,8 +42,10 @@ def verify_static_assets() -> None:
|
|
| 42 |
assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
|
| 43 |
assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
|
| 44 |
assert_true("submissionReadinessList" in index_html, "Article should expose submission readiness checks")
|
|
|
|
| 45 |
assert_true("loadDemoScript" in app_js, "Frontend should render the structured demo script")
|
| 46 |
assert_true("payload.api_checks" in app_js, "Frontend should render structured demo API checks")
|
|
|
|
| 47 |
assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
|
| 48 |
assert_true(
|
| 49 |
"haltPlayback({ clearAutoAdvance: false });" in app_js,
|
|
@@ -63,6 +65,7 @@ def verify_static_assets() -> None:
|
|
| 63 |
"/api/accessibility-audit",
|
| 64 |
"/api/demo-script",
|
| 65 |
"/api/submission-readiness",
|
|
|
|
| 66 |
]:
|
| 67 |
assert_true(endpoint in submission, f"Submission packet should mention {endpoint}")
|
| 68 |
assert_true(
|
|
@@ -210,6 +213,7 @@ def verify_routes() -> None:
|
|
| 210 |
assert_true("awardEvidenceList" in home.text, "Home route should include award evidence list")
|
| 211 |
assert_true("submissionReadinessStatus" in home.text, "Home route should include submission readiness status")
|
| 212 |
assert_true("submissionReadinessList" in home.text, "Home route should include submission readiness list")
|
|
|
|
| 213 |
assert_true("budgetStatus" in home.text, "Home route should include model budget status")
|
| 214 |
assert_true("modelBudgetList" in home.text, "Home route should include model budget list")
|
| 215 |
assert_true("runtimeSetupStatus" in home.text, "Home route should include runtime setup status")
|
|
@@ -390,6 +394,17 @@ def verify_routes() -> None:
|
|
| 390 |
"Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
|
| 391 |
)
|
| 392 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 393 |
runtime = client.get("/api/runtime-status")
|
| 394 |
assert_true(runtime.status_code == 200, "Runtime status route should return 200")
|
| 395 |
runtime_payload = runtime.json()
|
|
|
|
| 42 |
assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
|
| 43 |
assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
|
| 44 |
assert_true("submissionReadinessList" in index_html, "Article should expose submission readiness checks")
|
| 45 |
+
assert_true("copyEvidenceButton" in index_html, "Article should expose a copyable evidence bundle button")
|
| 46 |
assert_true("loadDemoScript" in app_js, "Frontend should render the structured demo script")
|
| 47 |
assert_true("payload.api_checks" in app_js, "Frontend should render structured demo API checks")
|
| 48 |
+
assert_true("/api/evidence-bundle" in app_js, "Frontend should fetch the evidence bundle for copying")
|
| 49 |
assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
|
| 50 |
assert_true(
|
| 51 |
"haltPlayback({ clearAutoAdvance: false });" in app_js,
|
|
|
|
| 65 |
"/api/accessibility-audit",
|
| 66 |
"/api/demo-script",
|
| 67 |
"/api/submission-readiness",
|
| 68 |
+
"/api/evidence-bundle",
|
| 69 |
]:
|
| 70 |
assert_true(endpoint in submission, f"Submission packet should mention {endpoint}")
|
| 71 |
assert_true(
|
|
|
|
| 213 |
assert_true("awardEvidenceList" in home.text, "Home route should include award evidence list")
|
| 214 |
assert_true("submissionReadinessStatus" in home.text, "Home route should include submission readiness status")
|
| 215 |
assert_true("submissionReadinessList" in home.text, "Home route should include submission readiness list")
|
| 216 |
+
assert_true("copyEvidenceButton" in home.text, "Home route should include copy evidence button")
|
| 217 |
assert_true("budgetStatus" in home.text, "Home route should include model budget status")
|
| 218 |
assert_true("modelBudgetList" in home.text, "Home route should include model budget list")
|
| 219 |
assert_true("runtimeSetupStatus" in home.text, "Home route should include runtime setup status")
|
|
|
|
| 394 |
"Submission readiness should aggregate model, award, frontend, runtime, accessibility, image, and demo evidence",
|
| 395 |
)
|
| 396 |
|
| 397 |
+
evidence = client.get("/api/evidence-bundle")
|
| 398 |
+
assert_true(evidence.status_code == 200, "Evidence bundle route should return 200")
|
| 399 |
+
evidence_payload = evidence.json()
|
| 400 |
+
assert_true(evidence_payload["ok"], "Evidence bundle payload should be ok")
|
| 401 |
+
assert_true(evidence_payload["submission_readiness"]["all_passed"], "Evidence bundle should include passing readiness")
|
| 402 |
+
assert_true(evidence_payload["model_budget"]["all_models_within_limit"], "Evidence bundle should include Tiny Titan proof")
|
| 403 |
+
assert_true(
|
| 404 |
+
{"demo_script", "accessibility_audit", "image_descriptions"} <= set(evidence_payload),
|
| 405 |
+
"Evidence bundle should include demo script, accessibility audit, and image descriptions",
|
| 406 |
+
)
|
| 407 |
+
|
| 408 |
runtime = client.get("/api/runtime-status")
|
| 409 |
assert_true(runtime.status_code == 200, "Runtime status route should return 200")
|
| 410 |
runtime_payload = runtime.json()
|
static/app.css
CHANGED
|
@@ -334,7 +334,8 @@ dd {
|
|
| 334 |
text-transform: uppercase;
|
| 335 |
}
|
| 336 |
|
| 337 |
-
.history-header button
|
|
|
|
| 338 |
min-height: 34px;
|
| 339 |
border-radius: 6px;
|
| 340 |
padding: 0 10px;
|
|
|
|
| 334 |
text-transform: uppercase;
|
| 335 |
}
|
| 336 |
|
| 337 |
+
.history-header button,
|
| 338 |
+
.submission-header button {
|
| 339 |
min-height: 34px;
|
| 340 |
border-radius: 6px;
|
| 341 |
padding: 0 10px;
|
static/app.js
CHANGED
|
@@ -23,6 +23,7 @@ const demoApiCheckList = document.querySelector("#demoApiCheckList");
|
|
| 23 |
const awardEvidenceList = document.querySelector("#awardEvidenceList");
|
| 24 |
const submissionReadinessStatus = document.querySelector("#submissionReadinessStatus");
|
| 25 |
const submissionReadinessList = document.querySelector("#submissionReadinessList");
|
|
|
|
| 26 |
const budgetStatus = document.querySelector("#budgetStatus");
|
| 27 |
const modelBudgetList = document.querySelector("#modelBudgetList");
|
| 28 |
const runtimeSetupStatus = document.querySelector("#runtimeSetupStatus");
|
|
@@ -698,6 +699,20 @@ clearTranscriptButton.addEventListener("click", () => {
|
|
| 698 |
renderTranscript();
|
| 699 |
liveNarration.textContent = "Transcript cleared.";
|
| 700 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 701 |
controls.play.addEventListener("click", () => {
|
| 702 |
if (!enabled) return;
|
| 703 |
if (currentIndex < 0) {
|
|
|
|
| 23 |
const awardEvidenceList = document.querySelector("#awardEvidenceList");
|
| 24 |
const submissionReadinessStatus = document.querySelector("#submissionReadinessStatus");
|
| 25 |
const submissionReadinessList = document.querySelector("#submissionReadinessList");
|
| 26 |
+
const copyEvidenceButton = document.querySelector("#copyEvidenceButton");
|
| 27 |
const budgetStatus = document.querySelector("#budgetStatus");
|
| 28 |
const modelBudgetList = document.querySelector("#modelBudgetList");
|
| 29 |
const runtimeSetupStatus = document.querySelector("#runtimeSetupStatus");
|
|
|
|
| 699 |
renderTranscript();
|
| 700 |
liveNarration.textContent = "Transcript cleared.";
|
| 701 |
});
|
| 702 |
+
copyEvidenceButton.addEventListener("click", async () => {
|
| 703 |
+
try {
|
| 704 |
+
const payload = await postJson("/api/evidence-bundle");
|
| 705 |
+
const text = JSON.stringify(payload, null, 2);
|
| 706 |
+
if (!navigator.clipboard) {
|
| 707 |
+
liveNarration.textContent = "Evidence bundle loaded, but clipboard access is unavailable.";
|
| 708 |
+
return;
|
| 709 |
+
}
|
| 710 |
+
await navigator.clipboard.writeText(text);
|
| 711 |
+
liveNarration.textContent = "Evidence bundle copied.";
|
| 712 |
+
} catch (error) {
|
| 713 |
+
liveNarration.textContent = `Evidence bundle failed: ${error.message}`;
|
| 714 |
+
}
|
| 715 |
+
});
|
| 716 |
controls.play.addEventListener("click", () => {
|
| 717 |
if (!enabled) return;
|
| 718 |
if (currentIndex < 0) {
|
static/index.html
CHANGED
|
@@ -135,6 +135,7 @@
|
|
| 135 |
<div class="submission-header">
|
| 136 |
<h3>Submission Readiness</h3>
|
| 137 |
<span id="submissionReadinessStatus">Checking</span>
|
|
|
|
| 138 |
</div>
|
| 139 |
<ul id="submissionReadinessList" class="award-list" aria-label="Submission readiness checks"></ul>
|
| 140 |
<div class="budget-header">
|
|
|
|
| 135 |
<div class="submission-header">
|
| 136 |
<h3>Submission Readiness</h3>
|
| 137 |
<span id="submissionReadinessStatus">Checking</span>
|
| 138 |
+
<button id="copyEvidenceButton" type="button">Copy Evidence</button>
|
| 139 |
</div>
|
| 140 |
<ul id="submissionReadinessList" class="award-list" aria-label="Submission readiness checks"></ul>
|
| 141 |
<div class="budget-header">
|