cvpfus Codex commited on
Commit
f32d42c
·
1 Parent(s): e0f1840

Expose image generation provenance

Browse files

Co-authored-by: Codex <codex@openai.com>

Files changed (8) hide show
  1. FIELD_NOTES.md +4 -0
  2. README.md +4 -0
  3. SUBMISSION.md +3 -1
  4. app.py +13 -1
  5. scripts/verify.py +21 -0
  6. static/app.css +25 -6
  7. static/app.js +25 -0
  8. static/index.html +5 -0
FIELD_NOTES.md CHANGED
@@ -30,6 +30,8 @@ Runtime setup is also data-backed. `/api/runtime-setup` lists the app command, l
30
 
31
  The judge runbook lives at `/api/demo-script`. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech.
32
 
 
 
33
  Accessibility evidence lives at `/api/accessibility-audit`. It records the reader-mode choices that matter most for this prototype: semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
34
 
35
  ## Reader Mode Behavior
@@ -59,6 +61,8 @@ Live accessibility demos need dependable behavior. If llama.cpp is not available
59
 
60
  The app preloads image descriptions through `/api/image-descriptions`, caches them in the frontend, and writes the descriptions back into each image's `alt` attribute. That keeps the visible article, transcript, and spoken image narration aligned.
61
 
 
 
62
  Kokoro is still the intended model-backed TTS path. For local resilience, the frontend can use the browser speech engine when Kokoro is unavailable, and the session panel labels that as a browser fallback instead of model audio.
63
 
64
  Generated speech outputs are capped by a small retention cleanup. That keeps repeated demo sessions from turning the Space output directory into hidden state.
 
30
 
31
  The judge runbook lives at `/api/demo-script`. It keeps the live presentation repeatable by pairing visible actions with API checks for health, model budget, runtime setup, runtime status, image descriptions, reader narration, and speech.
32
 
33
+ Image provenance lives in `/api/image-descriptions`. Each article illustration carries the planned FLUX.2 klein model id, prompt, seed, asset URL, and fallback status, and the session panel renders those receipts so the generated-image claim is inspectable.
34
+
35
  Accessibility evidence lives at `/api/accessibility-audit`. It records the reader-mode choices that matter most for this prototype: semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
36
 
37
  ## Reader Mode Behavior
 
61
 
62
  The app preloads image descriptions through `/api/image-descriptions`, caches them in the frontend, and writes the descriptions back into each image's `alt` attribute. That keeps the visible article, transcript, and spoken image narration aligned.
63
 
64
+ The same payload carries image-generation receipts. In fallback mode, the app is explicit that bundled assets stand in for the FLUX.2 klein runtime instead of presenting static files as live model output.
65
+
66
  Kokoro is still the intended model-backed TTS path. For local resilience, the frontend can use the browser speech engine when Kokoro is unavailable, and the session panel labels that as a browser fallback instead of model audio.
67
 
68
  Generated speech outputs are capped by a small retention cleanup. That keeps repeated demo sessions from turning the Space output directory into hidden state.
README.md CHANGED
@@ -79,6 +79,8 @@ The verifier checks syntax, static assets, Space metadata consistency, determini
79
 
80
  `/api/accessibility-audit` exposes structured evidence for semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
81
 
 
 
82
  ## Screen Reader Mode
83
 
84
  The frontend builds a reading queue from semantic article nodes. When screen-reader mode is on:
@@ -102,6 +104,8 @@ The session panel keeps a transcript of recent narration with copy and clear con
102
 
103
  Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
104
 
 
 
105
  Kokoro remains the planned tiny-model TTS path. During local demos, if the server-side Kokoro call falls back, the browser speech engine can read the same transcript so screen-reader mode still produces audible feedback.
106
 
107
  Generated speech files are pruned automatically so repeated demos do not grow the Space's `outputs` directory without bound.
 
79
 
80
  `/api/accessibility-audit` exposes structured evidence for semantic reading order, keyboard navigation, reader cursor state, shortcut safety, live narration, image alt text, transcript review, user-controlled playback, and fallback resilience.
81
 
82
+ `/api/image-descriptions` includes image-generation provenance for every article illustration: the planned FLUX.2 klein model, prompt, seed, bundled asset URL, and fallback-ready status.
83
+
84
  ## Screen Reader Mode
85
 
86
  The frontend builds a reading queue from semantic article nodes. When screen-reader mode is on:
 
104
 
105
  Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
106
 
107
+ The session panel also renders image receipts from `/api/image-descriptions`, so judges can inspect the prompt, seed, planned <=4B image model, and fallback status behind each bundled article illustration.
108
+
109
  Kokoro remains the planned tiny-model TTS path. During local demos, if the server-side Kokoro call falls back, the browser speech engine can read the same transcript so screen-reader mode still produces audible feedback.
110
 
111
  Generated speech files are pruned automatically so repeated demos do not grow the Space's `outputs` directory without bound.
SUBMISSION.md CHANGED
@@ -42,12 +42,14 @@ The prototype is designed for a live hackathon demo: every model-facing path has
42
  - `/api/runtime-status`: online or fallback-ready status for model paths.
43
  - `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
44
  - `/api/demo-script`: repeatable judge runbook and API checks.
45
- - `/api/image-descriptions`: generated article image descriptions for reader mode.
46
 
47
  The POST checks in `/api/demo-script` include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
48
 
49
  The accessibility audit also documents two reader-mode details that matter during judging: the active item is exposed as a reader cursor with focus, visible outline, stable id, and `aria-current`; global shortcuts ignore form controls so voice, speed, and auto-advance settings remain usable while reader mode is active.
50
 
 
 
51
  ## Reliability notes
52
 
53
  The demo remains navigable when local models are unavailable. The reader brain falls back to deterministic narration, image descriptions fall back to cached alt text, speech falls back to browser speech plus transcript, and generated images fall back to bundled article assets. Fallback states are labeled instead of hidden.
 
42
  - `/api/runtime-status`: online or fallback-ready status for model paths.
43
  - `/api/accessibility-audit`: semantic queue, keyboard navigation, reader cursor state, shortcut safety, live narration, alt text, transcript, user control, and fallback evidence.
44
  - `/api/demo-script`: repeatable judge runbook and API checks.
45
+ - `/api/image-descriptions`: generated article image descriptions plus prompt, seed, model, asset URL, and fallback status receipts.
46
 
47
  The POST checks in `/api/demo-script` include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
48
 
49
  The accessibility audit also documents two reader-mode details that matter during judging: the active item is exposed as a reader cursor with focus, visible outline, stable id, and `aria-current`; global shortcuts ignore form controls so voice, speed, and auto-advance settings remain usable while reader mode is active.
50
 
51
+ The image receipts keep the generated-asset claim inspectable: each article illustration names the planned `black-forest-labs/FLUX.2-klein-4B` path, prompt, seed, and bundled fallback asset.
52
+
53
  ## Reliability notes
54
 
55
  The demo remains navigable when local models are unavailable. The reader brain falls back to deterministic narration, image descriptions fall back to cached alt text, speech falls back to browser speech plus transcript, and generated images fall back to bundled article assets. Fallback states are labeled instead of hidden.
app.py CHANGED
@@ -60,24 +60,36 @@ MODEL_MANIFEST: dict[str, dict[str, Any]] = {
60
  },
61
  }
62
 
63
- ARTICLE_IMAGES: list[dict[str, str]] = [
64
  {
65
  "id": "desk-reader",
66
  "asset_url": "/static/generated/desk-reader.svg",
67
  "caption": "The article view doubles as the demo surface, so every feature has a real reading task.",
68
  "prompt": "Accessibility article reader with highlighted paragraph and narration controls.",
 
 
 
 
69
  },
70
  {
71
  "id": "model-map",
72
  "asset_url": "/static/generated/model-map.svg",
73
  "caption": "Each model stays at or below four billion parameters for Tiny Titan eligibility.",
74
  "prompt": "Diagram of four small AI models working together in an accessibility reader.",
 
 
 
 
75
  },
76
  {
77
  "id": "field-notes",
78
  "asset_url": "/static/generated/field-notes.svg",
79
  "caption": "Field notes document the choices behind the screen-reader behavior.",
80
  "prompt": "Notebook page with model sizes, keyboard controls, and accessibility checks.",
 
 
 
 
81
  },
82
  ]
83
 
 
60
  },
61
  }
62
 
63
+ ARTICLE_IMAGES: list[dict[str, Any]] = [
64
  {
65
  "id": "desk-reader",
66
  "asset_url": "/static/generated/desk-reader.svg",
67
  "caption": "The article view doubles as the demo surface, so every feature has a real reading task.",
68
  "prompt": "Accessibility article reader with highlighted paragraph and narration controls.",
69
+ "seed": 11,
70
+ "generation_model": "black-forest-labs/FLUX.2-klein-4B",
71
+ "generation_runtime": "bundled fallback asset",
72
+ "generation_status": "fallback-ready",
73
  },
74
  {
75
  "id": "model-map",
76
  "asset_url": "/static/generated/model-map.svg",
77
  "caption": "Each model stays at or below four billion parameters for Tiny Titan eligibility.",
78
  "prompt": "Diagram of four small AI models working together in an accessibility reader.",
79
+ "seed": 22,
80
+ "generation_model": "black-forest-labs/FLUX.2-klein-4B",
81
+ "generation_runtime": "bundled fallback asset",
82
+ "generation_status": "fallback-ready",
83
  },
84
  {
85
  "id": "field-notes",
86
  "asset_url": "/static/generated/field-notes.svg",
87
  "caption": "Field notes document the choices behind the screen-reader behavior.",
88
  "prompt": "Notebook page with model sizes, keyboard controls, and accessibility checks.",
89
+ "seed": 33,
90
+ "generation_model": "black-forest-labs/FLUX.2-klein-4B",
91
+ "generation_runtime": "bundled fallback asset",
92
+ "generation_status": "fallback-ready",
93
  },
94
  ]
95
 
scripts/verify.py CHANGED
@@ -38,6 +38,7 @@ def verify_static_assets() -> None:
38
  assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
39
  assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
40
  assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
 
41
  assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
42
  assert_true(
43
  "haltPlayback({ clearAutoAdvance: false });" in app_js,
@@ -59,6 +60,8 @@ def verify_static_assets() -> None:
59
  )
60
  for evidence in ["reader cursor", "shortcut safety", "aria-current"]:
61
  assert_true(evidence in submission, f"Submission packet should mention {evidence}")
 
 
62
 
63
 
64
  def front_matter_value(readme: str, key: str) -> str:
@@ -129,6 +132,14 @@ def verify_core_fallbacks() -> None:
129
  article_descriptions = app.describe_article_images_core()
130
  assert_true(article_descriptions["ok"], "Article image descriptions did not return ok")
131
  assert_true(len(article_descriptions["descriptions"]) == 3, "Article should expose three image descriptions")
 
 
 
 
 
 
 
 
132
  assert_true(isinstance(article_descriptions["elapsed_ms"], int), "Article image descriptions should include elapsed_ms")
133
 
134
  speech = app.speak_core("Tiny Narrator verification.", voice="af_heart", speed=1.0)
@@ -187,6 +198,8 @@ def verify_routes() -> None:
187
  assert_true("modelBudgetList" in home.text, "Home route should include model budget list")
188
  assert_true("runtimeSetupStatus" in home.text, "Home route should include runtime setup status")
189
  assert_true("runtimeSetupList" in home.text, "Home route should include runtime setup list")
 
 
190
 
191
  health = client.get("/api/health")
192
  assert_true(health.status_code == 200, "Health route should return 200")
@@ -357,6 +370,14 @@ def verify_routes() -> None:
357
  {item["id"] for item in image_payload["descriptions"]} == {"desk-reader", "model-map", "field-notes"},
358
  "Image route should describe all article images",
359
  )
 
 
 
 
 
 
 
 
360
 
361
 
362
  def main() -> None:
 
38
  assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
39
  assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
40
  assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
41
+ assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
42
  assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
43
  assert_true(
44
  "haltPlayback({ clearAutoAdvance: false });" in app_js,
 
60
  )
61
  for evidence in ["reader cursor", "shortcut safety", "aria-current"]:
62
  assert_true(evidence in submission, f"Submission packet should mention {evidence}")
63
+ for evidence in ["prompt", "seed", "FLUX.2-klein-4B", "fallback asset"]:
64
+ assert_true(evidence in submission, f"Submission packet should mention image receipt evidence: {evidence}")
65
 
66
 
67
  def front_matter_value(readme: str, key: str) -> str:
 
132
  article_descriptions = app.describe_article_images_core()
133
  assert_true(article_descriptions["ok"], "Article image descriptions did not return ok")
134
  assert_true(len(article_descriptions["descriptions"]) == 3, "Article should expose three image descriptions")
135
+ assert_true(
136
+ all(item["generation_model"] == app.MODEL_MANIFEST["image_generation"]["id"] for item in article_descriptions["descriptions"]),
137
+ "Article image descriptions should include the planned image-generation model",
138
+ )
139
+ assert_true(
140
+ all(isinstance(item["seed"], int) for item in article_descriptions["descriptions"]),
141
+ "Article image descriptions should include generation seeds",
142
+ )
143
  assert_true(isinstance(article_descriptions["elapsed_ms"], int), "Article image descriptions should include elapsed_ms")
144
 
145
  speech = app.speak_core("Tiny Narrator verification.", voice="af_heart", speed=1.0)
 
198
  assert_true("modelBudgetList" in home.text, "Home route should include model budget list")
199
  assert_true("runtimeSetupStatus" in home.text, "Home route should include runtime setup status")
200
  assert_true("runtimeSetupList" in home.text, "Home route should include runtime setup list")
201
+ assert_true("imageReceiptStatus" in home.text, "Home route should include image receipt status")
202
+ assert_true("imageReceiptList" in home.text, "Home route should include image receipt list")
203
 
204
  health = client.get("/api/health")
205
  assert_true(health.status_code == 200, "Health route should return 200")
 
370
  {item["id"] for item in image_payload["descriptions"]} == {"desk-reader", "model-map", "field-notes"},
371
  "Image route should describe all article images",
372
  )
373
+ assert_true(
374
+ all(item["generation_model"] == app.MODEL_MANIFEST["image_generation"]["id"] for item in image_payload["descriptions"]),
375
+ "Image route should expose image-generation provenance",
376
+ )
377
+ assert_true(
378
+ all(item["generation_status"] == "fallback-ready" for item in image_payload["descriptions"]),
379
+ "Image route should label bundled image fallback status",
380
+ )
381
 
382
 
383
  def main() -> None:
static/app.css CHANGED
@@ -290,7 +290,8 @@ dd {
290
  margin-top: 22px;
291
  }
292
 
293
- .budget-header {
 
294
  display: flex;
295
  align-items: center;
296
  justify-content: space-between;
@@ -307,7 +308,8 @@ dd {
307
  }
308
 
309
  .budget-header span,
310
- .runtime-header span {
 
311
  color: var(--accent-strong);
312
  font-size: 0.76rem;
313
  font-weight: 850;
@@ -386,8 +388,17 @@ dd {
386
  list-style: none;
387
  }
388
 
 
 
 
 
 
 
 
 
389
  .budget-list li,
390
- .runtime-list li {
 
391
  border: 1px solid var(--line);
392
  border-radius: 8px;
393
  padding: 10px 12px;
@@ -395,7 +406,8 @@ dd {
395
  }
396
 
397
  .budget-row,
398
- .runtime-row {
 
399
  display: flex;
400
  align-items: center;
401
  justify-content: space-between;
@@ -406,7 +418,8 @@ dd {
406
  }
407
 
408
  .budget-pill,
409
- .runtime-pill {
 
410
  border-radius: 999px;
411
  background: rgba(197, 69, 44, 0.12);
412
  color: var(--focus);
@@ -419,8 +432,14 @@ dd {
419
  color: var(--accent-strong);
420
  }
421
 
 
 
 
 
 
422
  .budget-list p,
423
- .runtime-list p {
 
424
  margin: 6px 0 0;
425
  color: var(--muted);
426
  font-size: 0.78rem;
 
290
  margin-top: 22px;
291
  }
292
 
293
+ .budget-header,
294
+ .image-receipt-header {
295
  display: flex;
296
  align-items: center;
297
  justify-content: space-between;
 
308
  }
309
 
310
  .budget-header span,
311
+ .runtime-header span,
312
+ .image-receipt-header span {
313
  color: var(--accent-strong);
314
  font-size: 0.76rem;
315
  font-weight: 850;
 
388
  list-style: none;
389
  }
390
 
391
+ .image-receipt-list {
392
+ display: grid;
393
+ gap: 8px;
394
+ margin: 12px 0 0;
395
+ padding: 0;
396
+ list-style: none;
397
+ }
398
+
399
  .budget-list li,
400
+ .runtime-list li,
401
+ .image-receipt-list li {
402
  border: 1px solid var(--line);
403
  border-radius: 8px;
404
  padding: 10px 12px;
 
406
  }
407
 
408
  .budget-row,
409
+ .runtime-row,
410
+ .image-receipt-row {
411
  display: flex;
412
  align-items: center;
413
  justify-content: space-between;
 
418
  }
419
 
420
  .budget-pill,
421
+ .runtime-pill,
422
+ .image-receipt-pill {
423
  border-radius: 999px;
424
  background: rgba(197, 69, 44, 0.12);
425
  color: var(--focus);
 
432
  color: var(--accent-strong);
433
  }
434
 
435
+ .image-receipt-pill {
436
+ background: rgba(244, 211, 94, 0.22);
437
+ color: #6f4b00;
438
+ }
439
+
440
  .budget-list p,
441
+ .runtime-list p,
442
+ .image-receipt-list p {
443
  margin: 6px 0 0;
444
  color: var(--muted);
445
  font-size: 0.78rem;
static/app.js CHANGED
@@ -22,6 +22,8 @@ const budgetStatus = document.querySelector("#budgetStatus");
22
  const modelBudgetList = document.querySelector("#modelBudgetList");
23
  const runtimeSetupStatus = document.querySelector("#runtimeSetupStatus");
24
  const runtimeSetupList = document.querySelector("#runtimeSetupList");
 
 
25
 
26
  const controls = {
27
  prev: document.querySelector("#prevButton"),
@@ -318,8 +320,31 @@ async function loadImageDescriptions() {
318
  }
319
  }
320
  imageStatus.textContent = `${imageDescriptions.size} ready`;
 
 
 
 
 
 
 
 
 
 
 
 
 
321
  } catch {
322
  imageStatus.textContent = "Fallback on demand";
 
 
 
 
 
 
 
 
 
 
323
  }
324
  }
325
 
 
22
  const modelBudgetList = document.querySelector("#modelBudgetList");
23
  const runtimeSetupStatus = document.querySelector("#runtimeSetupStatus");
24
  const runtimeSetupList = document.querySelector("#runtimeSetupList");
25
+ const imageReceiptStatus = document.querySelector("#imageReceiptStatus");
26
+ const imageReceiptList = document.querySelector("#imageReceiptList");
27
 
28
  const controls = {
29
  prev: document.querySelector("#prevButton"),
 
320
  }
321
  }
322
  imageStatus.textContent = `${imageDescriptions.size} ready`;
323
+ imageReceiptStatus.textContent = `${payload.descriptions.length} receipts`;
324
+ imageReceiptList.innerHTML = payload.descriptions
325
+ .map((description) => `
326
+ <li>
327
+ <div class="image-receipt-row">
328
+ <span>${escapeHtml(description.id)}</span>
329
+ <span class="image-receipt-pill">${escapeHtml(description.generation_status)}</span>
330
+ </div>
331
+ <p>${escapeHtml(description.generation_model)} | seed ${escapeHtml(String(description.seed))}</p>
332
+ <p>${escapeHtml(description.prompt)}</p>
333
+ </li>
334
+ `)
335
+ .join("");
336
  } catch {
337
  imageStatus.textContent = "Fallback on demand";
338
+ imageReceiptStatus.textContent = "Unavailable";
339
+ imageReceiptList.innerHTML = `
340
+ <li>
341
+ <div class="image-receipt-row">
342
+ <span>Image receipts</span>
343
+ <span class="image-receipt-pill">offline</span>
344
+ </div>
345
+ <p>Generated image provenance is unavailable.</p>
346
+ </li>
347
+ `;
348
  }
349
  }
350
 
static/index.html CHANGED
@@ -136,6 +136,11 @@
136
  <span id="runtimeSetupStatus">Loading</span>
137
  </div>
138
  <ul id="runtimeSetupList" class="runtime-list" aria-label="Model runtime setup plan"></ul>
 
 
 
 
 
139
  </aside>
140
  </main>
141
 
 
136
  <span id="runtimeSetupStatus">Loading</span>
137
  </div>
138
  <ul id="runtimeSetupList" class="runtime-list" aria-label="Model runtime setup plan"></ul>
139
+ <div class="image-receipt-header">
140
+ <h3>Image Receipts</h3>
141
+ <span id="imageReceiptStatus">Loading</span>
142
+ </div>
143
+ <ul id="imageReceiptList" class="image-receipt-list" aria-label="Generated image provenance"></ul>
144
  </aside>
145
  </main>
146