cvpfus Codex commited on
Commit
d1cf993
·
1 Parent(s): d7543f6

Expose reader shortcuts to assistive tech

Browse files

Co-authored-by: Codex <codex@openai.com>

Files changed (7) hide show
  1. FIELD_NOTES.md +2 -0
  2. README.md +2 -0
  3. SUBMISSION.md +1 -1
  4. app.py +9 -9
  5. scripts/verify.py +14 -2
  6. static/app.js +6 -4
  7. static/index.html +7 -6
FIELD_NOTES.md CHANGED
@@ -46,6 +46,8 @@ When screen reader mode turns on, the reader cursor starts at the focused articl
46
 
47
  Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro voice, speed slider, and auto-advance checkbox without the page treating those keystrokes as navigation commands.
48
 
 
 
49
  Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article.
50
 
51
  Keyboard map:
 
46
 
47
  Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro voice, speed slider, and auto-advance checkbox without the page treating those keystrokes as navigation commands.
48
 
49
+ The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Escape also has a visible Stop button, which keeps the assistive command model available to pointer users and easy for judges to inspect.
50
+
51
  Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article.
52
 
53
  Keyboard map:
README.md CHANGED
@@ -107,6 +107,8 @@ When screen-reader mode turns on, it selects the focused or most visible article
107
 
108
  Global reader shortcuts ignore buttons, links, selects, and inputs so the voice, speed, and auto-advance controls remain usable while reader mode is active.
109
 
 
 
110
  The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up.
111
 
112
  Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
 
107
 
108
  Global reader shortcuts ignore buttons, links, selects, and inputs so the voice, speed, and auto-advance controls remain usable while reader mode is active.
109
 
110
+ Reader buttons expose `aria-keyshortcuts`, and the Escape shortcut is mirrored by a visible Stop control so pointer and keyboard users share the same command surface.
111
+
112
  The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up.
113
 
114
  Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
SUBMISSION.md CHANGED
@@ -48,7 +48,7 @@ The prototype is designed for a live hackathon demo: every model-facing path has
48
 
49
  The checks in `/api/demo-script` include curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
50
 
51
- The accessibility audit also documents two reader-mode details that matter during judging: the active item is exposed as a reader cursor with focus, visible outline, stable id, and `aria-current`; global shortcuts ignore form controls so voice, speed, and auto-advance settings remain usable while reader mode is active.
52
 
53
  The image receipts keep the generated-asset claim inspectable: each article illustration names the planned `black-forest-labs/FLUX.2-klein-4B` path, prompt, seed, and bundled fallback asset.
54
 
 
48
 
49
  The checks in `/api/demo-script` include curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
50
 
51
+ The accessibility audit also documents reader-mode details that matter during judging: the active item is exposed as a reader cursor with focus, visible outline, stable id, and `aria-current`; global shortcuts ignore form controls so voice, speed, and auto-advance settings remain usable while reader mode is active; reader controls expose `aria-keyshortcuts` and a visible Stop command for Escape.
52
 
53
  The image receipts keep the generated-asset claim inspectable: each article illustration names the planned `black-forest-labs/FLUX.2-klein-4B` path, prompt, seed, and bundled fallback asset.
54
 
app.py CHANGED
@@ -421,7 +421,7 @@ def accessibility_audit_core() -> dict[str, Any]:
421
  "id": "keyboard_navigation",
422
  "label": "Keyboard navigation",
423
  "status": "pass",
424
- "evidence": "Reader controls expose Space, N, P, H, I, S, R, and Esc shortcuts through the manifest.",
425
  },
426
  {
427
  "id": "reader_cursor",
@@ -478,14 +478,14 @@ def accessibility_audit_core() -> dict[str, Any]:
478
  ARTICLE_MANIFEST: dict[str, Any] = {
479
  "title": "A tiny model reader that turns articles into guided narration",
480
  "reader_controls": [
481
- {"key": "Space", "action": "Play or pause"},
482
- {"key": "N", "action": "Next item"},
483
- {"key": "P", "action": "Previous item"},
484
- {"key": "H", "action": "Next heading"},
485
- {"key": "I", "action": "Next image"},
486
- {"key": "S", "action": "Summarize current section"},
487
- {"key": "R", "action": "Repeat current item"},
488
- {"key": "Esc", "action": "Stop audio"},
489
  ],
490
  "bonus_targets": ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"],
491
  "images": ARTICLE_IMAGES,
 
421
  "id": "keyboard_navigation",
422
  "label": "Keyboard navigation",
423
  "status": "pass",
424
+ "evidence": "Reader controls expose Space, N, P, H, I, S, R, and Esc shortcuts through the manifest and aria-keyshortcuts.",
425
  },
426
  {
427
  "id": "reader_cursor",
 
478
  ARTICLE_MANIFEST: dict[str, Any] = {
479
  "title": "A tiny model reader that turns articles into guided narration",
480
  "reader_controls": [
481
+ {"key": "Space", "aria_keyshortcuts": "Space", "action": "Play or pause"},
482
+ {"key": "N", "aria_keyshortcuts": "N", "action": "Next item"},
483
+ {"key": "P", "aria_keyshortcuts": "P", "action": "Previous item"},
484
+ {"key": "H", "aria_keyshortcuts": "H", "action": "Next heading"},
485
+ {"key": "I", "aria_keyshortcuts": "I", "action": "Next image"},
486
+ {"key": "S", "aria_keyshortcuts": "S", "action": "Summarize current section"},
487
+ {"key": "R", "aria_keyshortcuts": "R", "action": "Repeat current item"},
488
+ {"key": "Esc", "aria_keyshortcuts": "Escape", "action": "Stop audio"},
489
  ],
490
  "bonus_targets": ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"],
491
  "images": ARTICLE_IMAGES,
scripts/verify.py CHANGED
@@ -37,7 +37,10 @@ def verify_static_assets() -> None:
37
  assert_true('data-reader-type="heading"' in index_html, "Article should mark heading reader nodes")
38
  assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
39
  assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
 
 
40
  assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
 
41
  assert_true("demoScriptList" in index_html, "Article should expose the judge demo runbook")
42
  assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
43
  assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
@@ -55,6 +58,7 @@ def verify_static_assets() -> None:
55
  )
56
  assert_true("aria-current" in app_js, "Reader mode should expose the active item as current")
57
  assert_true("shouldHandleReaderShortcut" in app_js, "Reader shortcuts should not hijack form controls")
 
58
  assert_true("reader-node-" in app_js, "Reader nodes should receive stable ids for control context")
59
  assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
60
 
@@ -237,9 +241,17 @@ def verify_routes() -> None:
237
  assert_true(manifest.status_code == 200, "Article manifest route should return 200")
238
  manifest_payload = manifest.json()
239
  assert_true("Tiny Titan" in manifest_payload["bonus_targets"], "Manifest should include Tiny Titan target")
 
 
 
240
  assert_true(
241
- {"key": "S", "action": "Summarize current section"} in manifest_payload["reader_controls"],
242
- "Manifest should include summary shortcut",
 
 
 
 
 
243
  )
244
  assert_true(
245
  manifest_payload["models"]["speech"]["id"] == "hexgrad/Kokoro-82M",
 
37
  assert_true('data-reader-type="heading"' in index_html, "Article should mark heading reader nodes")
38
  assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
39
  assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
40
+ for shortcut in ['aria-keyshortcuts="Space"', 'aria-keyshortcuts="N"', 'aria-keyshortcuts="S"']:
41
+ assert_true(shortcut in index_html, f"Reader controls should expose {shortcut}")
42
  assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
43
+ assert_true("stopButton" in index_html, "Reader controls should expose a visible stop command")
44
  assert_true("demoScriptList" in index_html, "Article should expose the judge demo runbook")
45
  assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
46
  assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
 
58
  )
59
  assert_true("aria-current" in app_js, "Reader mode should expose the active item as current")
60
  assert_true("shouldHandleReaderShortcut" in app_js, "Reader shortcuts should not hijack form controls")
61
+ assert_true("controls.stop.click()" in app_js, "Escape should route through the visible stop command")
62
  assert_true("reader-node-" in app_js, "Reader nodes should receive stable ids for control context")
63
  assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
64
 
 
241
  assert_true(manifest.status_code == 200, "Article manifest route should return 200")
242
  manifest_payload = manifest.json()
243
  assert_true("Tiny Titan" in manifest_payload["bonus_targets"], "Manifest should include Tiny Titan target")
244
+ reader_controls = manifest_payload["reader_controls"]
245
+ summary_shortcut = next((item for item in reader_controls if item["key"] == "S"), None)
246
+ assert_true(summary_shortcut is not None, "Manifest should include summary shortcut")
247
  assert_true(
248
+ summary_shortcut["action"] == "Summarize current section"
249
+ and summary_shortcut["aria_keyshortcuts"] == "S",
250
+ "Manifest summary shortcut should include action and aria-keyshortcuts value",
251
+ )
252
+ assert_true(
253
+ {item["aria_keyshortcuts"] for item in reader_controls} >= {"Space", "N", "P", "H", "I", "S", "R", "Escape"},
254
+ "Manifest should expose aria-keyshortcuts values for reader controls",
255
  )
256
  assert_true(
257
  manifest_payload["models"]["speech"]["id"] == "hexgrad/Kokoro-82M",
static/app.js CHANGED
@@ -38,6 +38,7 @@ const controls = {
38
  heading: document.querySelector("#headingButton"),
39
  image: document.querySelector("#imageButton"),
40
  summary: document.querySelector("#summaryButton"),
 
41
  };
42
 
43
  const nodes = [...document.querySelectorAll(".speakable")].map((element, index) => ({
@@ -675,6 +676,10 @@ controls.prev.addEventListener("click", () => narrate(Math.max(currentIndex - 1,
675
  controls.heading.addEventListener("click", () => nextByType("heading"));
676
  controls.image.addEventListener("click", () => nextByType("image"));
677
  controls.summary.addEventListener("click", () => summarizeCurrentSection());
 
 
 
 
678
  nodes.forEach((node) => {
679
  node.element.addEventListener("click", (event) => {
680
  if (!enabled || isControlTarget(event.target)) return;
@@ -772,8 +777,5 @@ document.addEventListener("keydown", (event) => {
772
  if (key === "i") controls.image.click();
773
  if (key === "s") controls.summary.click();
774
  if (key === "r" && currentIndex >= 0) narrate(currentIndex);
775
- if (key === "escape") {
776
- stopAudio();
777
- liveNarration.textContent = "Reading stopped.";
778
- }
779
  });
 
38
  heading: document.querySelector("#headingButton"),
39
  image: document.querySelector("#imageButton"),
40
  summary: document.querySelector("#summaryButton"),
41
+ stop: document.querySelector("#stopButton"),
42
  };
43
 
44
  const nodes = [...document.querySelectorAll(".speakable")].map((element, index) => ({
 
676
  controls.heading.addEventListener("click", () => nextByType("heading"));
677
  controls.image.addEventListener("click", () => nextByType("image"));
678
  controls.summary.addEventListener("click", () => summarizeCurrentSection());
679
+ controls.stop.addEventListener("click", () => {
680
+ stopAudio();
681
+ liveNarration.textContent = "Reading stopped.";
682
+ });
683
  nodes.forEach((node) => {
684
  node.element.addEventListener("click", (event) => {
685
  if (!enabled || isControlTarget(event.target)) return;
 
777
  if (key === "i") controls.image.click();
778
  if (key === "s") controls.summary.click();
779
  if (key === "r" && currentIndex >= 0) narrate(currentIndex);
780
+ if (key === "escape") controls.stop.click();
 
 
 
781
  });
static/index.html CHANGED
@@ -157,12 +157,13 @@
157
  </main>
158
 
159
  <section class="reader-bar" aria-label="Screen reader controls" hidden>
160
- <button id="prevButton" type="button" title="Previous item">Prev</button>
161
- <button id="playButton" type="button" title="Play or pause">Play</button>
162
- <button id="nextButton" type="button" title="Next item">Next</button>
163
- <button id="headingButton" type="button" title="Next heading">Heading</button>
164
- <button id="imageButton" type="button" title="Next image">Image</button>
165
- <button id="summaryButton" type="button" title="Summarize current section">Summary</button>
 
166
  <label>
167
  Voice
168
  <select id="voiceControl" aria-label="Reader voice"></select>
 
157
  </main>
158
 
159
  <section class="reader-bar" aria-label="Screen reader controls" hidden>
160
+ <button id="prevButton" type="button" title="Previous item" aria-keyshortcuts="P">Prev</button>
161
+ <button id="playButton" type="button" title="Play or pause" aria-keyshortcuts="Space">Play</button>
162
+ <button id="nextButton" type="button" title="Next item" aria-keyshortcuts="N">Next</button>
163
+ <button id="headingButton" type="button" title="Next heading" aria-keyshortcuts="H">Heading</button>
164
+ <button id="imageButton" type="button" title="Next image" aria-keyshortcuts="I">Image</button>
165
+ <button id="summaryButton" type="button" title="Summarize current section" aria-keyshortcuts="S">Summary</button>
166
+ <button id="stopButton" type="button" title="Stop audio" aria-keyshortcuts="Escape">Stop</button>
167
  <label>
168
  Voice
169
  <select id="voiceControl" aria-label="Reader voice"></select>