cvpfus Codex commited on
Commit ·
d1cf993
1
Parent(s): d7543f6
Expose reader shortcuts to assistive tech
Browse filesCo-authored-by: Codex <codex@openai.com>
- FIELD_NOTES.md +2 -0
- README.md +2 -0
- SUBMISSION.md +1 -1
- app.py +9 -9
- scripts/verify.py +14 -2
- static/app.js +6 -4
- static/index.html +7 -6
FIELD_NOTES.md
CHANGED
|
@@ -46,6 +46,8 @@ When screen reader mode turns on, the reader cursor starts at the focused articl
|
|
| 46 |
|
| 47 |
Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro voice, speed slider, and auto-advance checkbox without the page treating those keystrokes as navigation commands.
|
| 48 |
|
|
|
|
|
|
|
| 49 |
Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article.
|
| 50 |
|
| 51 |
Keyboard map:
|
|
|
|
| 46 |
|
| 47 |
Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro voice, speed slider, and auto-advance checkbox without the page treating those keystrokes as navigation commands.
|
| 48 |
|
| 49 |
+
The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Escape also has a visible Stop button, which keeps the assistive command model available to pointer users and easy for judges to inspect.
|
| 50 |
+
|
| 51 |
Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article.
|
| 52 |
|
| 53 |
Keyboard map:
|
README.md
CHANGED
|
@@ -107,6 +107,8 @@ When screen-reader mode turns on, it selects the focused or most visible article
|
|
| 107 |
|
| 108 |
Global reader shortcuts ignore buttons, links, selects, and inputs so the voice, speed, and auto-advance controls remain usable while reader mode is active.
|
| 109 |
|
|
|
|
|
|
|
| 110 |
The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up.
|
| 111 |
|
| 112 |
Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
|
|
|
|
| 107 |
|
| 108 |
Global reader shortcuts ignore buttons, links, selects, and inputs so the voice, speed, and auto-advance controls remain usable while reader mode is active.
|
| 109 |
|
| 110 |
+
Reader buttons expose `aria-keyshortcuts`, and the Escape shortcut is mirrored by a visible Stop control so pointer and keyboard users share the same command surface.
|
| 111 |
+
|
| 112 |
The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up.
|
| 113 |
|
| 114 |
Image descriptions are preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
|
SUBMISSION.md
CHANGED
|
@@ -48,7 +48,7 @@ The prototype is designed for a live hackathon demo: every model-facing path has
|
|
| 48 |
|
| 49 |
The checks in `/api/demo-script` include curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
| 50 |
|
| 51 |
-
The accessibility audit also documents
|
| 52 |
|
| 53 |
The image receipts keep the generated-asset claim inspectable: each article illustration names the planned `black-forest-labs/FLUX.2-klein-4B` path, prompt, seed, and bundled fallback asset.
|
| 54 |
|
|
|
|
| 48 |
|
| 49 |
The checks in `/api/demo-script` include curl and PowerShell-friendly `curl.exe` commands generated from `PUBLIC_BASE_URL`, and the POST checks include sample JSON bodies for `/api/reader-brain` and `/api/speak`.
|
| 50 |
|
| 51 |
+
The accessibility audit also documents reader-mode details that matter during judging: the active item is exposed as a reader cursor with focus, visible outline, stable id, and `aria-current`; global shortcuts ignore form controls so voice, speed, and auto-advance settings remain usable while reader mode is active; reader controls expose `aria-keyshortcuts` and a visible Stop command for Escape.
|
| 52 |
|
| 53 |
The image receipts keep the generated-asset claim inspectable: each article illustration names the planned `black-forest-labs/FLUX.2-klein-4B` path, prompt, seed, and bundled fallback asset.
|
| 54 |
|
app.py
CHANGED
|
@@ -421,7 +421,7 @@ def accessibility_audit_core() -> dict[str, Any]:
|
|
| 421 |
"id": "keyboard_navigation",
|
| 422 |
"label": "Keyboard navigation",
|
| 423 |
"status": "pass",
|
| 424 |
-
"evidence": "Reader controls expose Space, N, P, H, I, S, R, and Esc shortcuts through the manifest.",
|
| 425 |
},
|
| 426 |
{
|
| 427 |
"id": "reader_cursor",
|
|
@@ -478,14 +478,14 @@ def accessibility_audit_core() -> dict[str, Any]:
|
|
| 478 |
ARTICLE_MANIFEST: dict[str, Any] = {
|
| 479 |
"title": "A tiny model reader that turns articles into guided narration",
|
| 480 |
"reader_controls": [
|
| 481 |
-
{"key": "Space", "action": "Play or pause"},
|
| 482 |
-
{"key": "N", "action": "Next item"},
|
| 483 |
-
{"key": "P", "action": "Previous item"},
|
| 484 |
-
{"key": "H", "action": "Next heading"},
|
| 485 |
-
{"key": "I", "action": "Next image"},
|
| 486 |
-
{"key": "S", "action": "Summarize current section"},
|
| 487 |
-
{"key": "R", "action": "Repeat current item"},
|
| 488 |
-
{"key": "Esc", "action": "Stop audio"},
|
| 489 |
],
|
| 490 |
"bonus_targets": ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"],
|
| 491 |
"images": ARTICLE_IMAGES,
|
|
|
|
| 421 |
"id": "keyboard_navigation",
|
| 422 |
"label": "Keyboard navigation",
|
| 423 |
"status": "pass",
|
| 424 |
+
"evidence": "Reader controls expose Space, N, P, H, I, S, R, and Esc shortcuts through the manifest and aria-keyshortcuts.",
|
| 425 |
},
|
| 426 |
{
|
| 427 |
"id": "reader_cursor",
|
|
|
|
| 478 |
ARTICLE_MANIFEST: dict[str, Any] = {
|
| 479 |
"title": "A tiny model reader that turns articles into guided narration",
|
| 480 |
"reader_controls": [
|
| 481 |
+
{"key": "Space", "aria_keyshortcuts": "Space", "action": "Play or pause"},
|
| 482 |
+
{"key": "N", "aria_keyshortcuts": "N", "action": "Next item"},
|
| 483 |
+
{"key": "P", "aria_keyshortcuts": "P", "action": "Previous item"},
|
| 484 |
+
{"key": "H", "aria_keyshortcuts": "H", "action": "Next heading"},
|
| 485 |
+
{"key": "I", "aria_keyshortcuts": "I", "action": "Next image"},
|
| 486 |
+
{"key": "S", "aria_keyshortcuts": "S", "action": "Summarize current section"},
|
| 487 |
+
{"key": "R", "aria_keyshortcuts": "R", "action": "Repeat current item"},
|
| 488 |
+
{"key": "Esc", "aria_keyshortcuts": "Escape", "action": "Stop audio"},
|
| 489 |
],
|
| 490 |
"bonus_targets": ["Tiny Titan", "Llama Champion", "Off-Brand", "Field Notes"],
|
| 491 |
"images": ARTICLE_IMAGES,
|
scripts/verify.py
CHANGED
|
@@ -37,7 +37,10 @@ def verify_static_assets() -> None:
|
|
| 37 |
assert_true('data-reader-type="heading"' in index_html, "Article should mark heading reader nodes")
|
| 38 |
assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
|
| 39 |
assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
|
|
|
|
|
|
|
| 40 |
assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
|
|
|
|
| 41 |
assert_true("demoScriptList" in index_html, "Article should expose the judge demo runbook")
|
| 42 |
assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
|
| 43 |
assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
|
|
@@ -55,6 +58,7 @@ def verify_static_assets() -> None:
|
|
| 55 |
)
|
| 56 |
assert_true("aria-current" in app_js, "Reader mode should expose the active item as current")
|
| 57 |
assert_true("shouldHandleReaderShortcut" in app_js, "Reader shortcuts should not hijack form controls")
|
|
|
|
| 58 |
assert_true("reader-node-" in app_js, "Reader nodes should receive stable ids for control context")
|
| 59 |
assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
|
| 60 |
|
|
@@ -237,9 +241,17 @@ def verify_routes() -> None:
|
|
| 237 |
assert_true(manifest.status_code == 200, "Article manifest route should return 200")
|
| 238 |
manifest_payload = manifest.json()
|
| 239 |
assert_true("Tiny Titan" in manifest_payload["bonus_targets"], "Manifest should include Tiny Titan target")
|
|
|
|
|
|
|
|
|
|
| 240 |
assert_true(
|
| 241 |
-
|
| 242 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
)
|
| 244 |
assert_true(
|
| 245 |
manifest_payload["models"]["speech"]["id"] == "hexgrad/Kokoro-82M",
|
|
|
|
| 37 |
assert_true('data-reader-type="heading"' in index_html, "Article should mark heading reader nodes")
|
| 38 |
assert_true('data-reader-type="image"' in index_html, "Article should mark image reader nodes")
|
| 39 |
assert_true('aria-live="polite"' in index_html, "Article should expose an aria-live narration region")
|
| 40 |
+
for shortcut in ['aria-keyshortcuts="Space"', 'aria-keyshortcuts="N"', 'aria-keyshortcuts="S"']:
|
| 41 |
+
assert_true(shortcut in index_html, f"Reader controls should expose {shortcut}")
|
| 42 |
assert_true("transcriptLog" in index_html, "Article should expose a visible transcript log")
|
| 43 |
+
assert_true("stopButton" in index_html, "Reader controls should expose a visible stop command")
|
| 44 |
assert_true("demoScriptList" in index_html, "Article should expose the judge demo runbook")
|
| 45 |
assert_true("demoApiCheckList" in index_html, "Article should expose judge API evidence checks")
|
| 46 |
assert_true("imageReceiptList" in index_html, "Article should expose generated image receipts")
|
|
|
|
| 58 |
)
|
| 59 |
assert_true("aria-current" in app_js, "Reader mode should expose the active item as current")
|
| 60 |
assert_true("shouldHandleReaderShortcut" in app_js, "Reader shortcuts should not hijack form controls")
|
| 61 |
+
assert_true("controls.stop.click()" in app_js, "Escape should route through the visible stop command")
|
| 62 |
assert_true("reader-node-" in app_js, "Reader nodes should receive stable ids for control context")
|
| 63 |
assert_true("narrate(node.index)" in app_js, "Reader mode should support click-to-read article items")
|
| 64 |
|
|
|
|
| 241 |
assert_true(manifest.status_code == 200, "Article manifest route should return 200")
|
| 242 |
manifest_payload = manifest.json()
|
| 243 |
assert_true("Tiny Titan" in manifest_payload["bonus_targets"], "Manifest should include Tiny Titan target")
|
| 244 |
+
reader_controls = manifest_payload["reader_controls"]
|
| 245 |
+
summary_shortcut = next((item for item in reader_controls if item["key"] == "S"), None)
|
| 246 |
+
assert_true(summary_shortcut is not None, "Manifest should include summary shortcut")
|
| 247 |
assert_true(
|
| 248 |
+
summary_shortcut["action"] == "Summarize current section"
|
| 249 |
+
and summary_shortcut["aria_keyshortcuts"] == "S",
|
| 250 |
+
"Manifest summary shortcut should include action and aria-keyshortcuts value",
|
| 251 |
+
)
|
| 252 |
+
assert_true(
|
| 253 |
+
{item["aria_keyshortcuts"] for item in reader_controls} >= {"Space", "N", "P", "H", "I", "S", "R", "Escape"},
|
| 254 |
+
"Manifest should expose aria-keyshortcuts values for reader controls",
|
| 255 |
)
|
| 256 |
assert_true(
|
| 257 |
manifest_payload["models"]["speech"]["id"] == "hexgrad/Kokoro-82M",
|
static/app.js
CHANGED
|
@@ -38,6 +38,7 @@ const controls = {
|
|
| 38 |
heading: document.querySelector("#headingButton"),
|
| 39 |
image: document.querySelector("#imageButton"),
|
| 40 |
summary: document.querySelector("#summaryButton"),
|
|
|
|
| 41 |
};
|
| 42 |
|
| 43 |
const nodes = [...document.querySelectorAll(".speakable")].map((element, index) => ({
|
|
@@ -675,6 +676,10 @@ controls.prev.addEventListener("click", () => narrate(Math.max(currentIndex - 1,
|
|
| 675 |
controls.heading.addEventListener("click", () => nextByType("heading"));
|
| 676 |
controls.image.addEventListener("click", () => nextByType("image"));
|
| 677 |
controls.summary.addEventListener("click", () => summarizeCurrentSection());
|
|
|
|
|
|
|
|
|
|
|
|
|
| 678 |
nodes.forEach((node) => {
|
| 679 |
node.element.addEventListener("click", (event) => {
|
| 680 |
if (!enabled || isControlTarget(event.target)) return;
|
|
@@ -772,8 +777,5 @@ document.addEventListener("keydown", (event) => {
|
|
| 772 |
if (key === "i") controls.image.click();
|
| 773 |
if (key === "s") controls.summary.click();
|
| 774 |
if (key === "r" && currentIndex >= 0) narrate(currentIndex);
|
| 775 |
-
if (key === "escape")
|
| 776 |
-
stopAudio();
|
| 777 |
-
liveNarration.textContent = "Reading stopped.";
|
| 778 |
-
}
|
| 779 |
});
|
|
|
|
| 38 |
heading: document.querySelector("#headingButton"),
|
| 39 |
image: document.querySelector("#imageButton"),
|
| 40 |
summary: document.querySelector("#summaryButton"),
|
| 41 |
+
stop: document.querySelector("#stopButton"),
|
| 42 |
};
|
| 43 |
|
| 44 |
const nodes = [...document.querySelectorAll(".speakable")].map((element, index) => ({
|
|
|
|
| 676 |
controls.heading.addEventListener("click", () => nextByType("heading"));
|
| 677 |
controls.image.addEventListener("click", () => nextByType("image"));
|
| 678 |
controls.summary.addEventListener("click", () => summarizeCurrentSection());
|
| 679 |
+
controls.stop.addEventListener("click", () => {
|
| 680 |
+
stopAudio();
|
| 681 |
+
liveNarration.textContent = "Reading stopped.";
|
| 682 |
+
});
|
| 683 |
nodes.forEach((node) => {
|
| 684 |
node.element.addEventListener("click", (event) => {
|
| 685 |
if (!enabled || isControlTarget(event.target)) return;
|
|
|
|
| 777 |
if (key === "i") controls.image.click();
|
| 778 |
if (key === "s") controls.summary.click();
|
| 779 |
if (key === "r" && currentIndex >= 0) narrate(currentIndex);
|
| 780 |
+
if (key === "escape") controls.stop.click();
|
|
|
|
|
|
|
|
|
|
| 781 |
});
|
static/index.html
CHANGED
|
@@ -157,12 +157,13 @@
|
|
| 157 |
</main>
|
| 158 |
|
| 159 |
<section class="reader-bar" aria-label="Screen reader controls" hidden>
|
| 160 |
-
<button id="prevButton" type="button" title="Previous item">Prev</button>
|
| 161 |
-
<button id="playButton" type="button" title="Play or pause">Play</button>
|
| 162 |
-
<button id="nextButton" type="button" title="Next item">Next</button>
|
| 163 |
-
<button id="headingButton" type="button" title="Next heading">Heading</button>
|
| 164 |
-
<button id="imageButton" type="button" title="Next image">Image</button>
|
| 165 |
-
<button id="summaryButton" type="button" title="Summarize current section">Summary</button>
|
|
|
|
| 166 |
<label>
|
| 167 |
Voice
|
| 168 |
<select id="voiceControl" aria-label="Reader voice"></select>
|
|
|
|
| 157 |
</main>
|
| 158 |
|
| 159 |
<section class="reader-bar" aria-label="Screen reader controls" hidden>
|
| 160 |
+
<button id="prevButton" type="button" title="Previous item" aria-keyshortcuts="P">Prev</button>
|
| 161 |
+
<button id="playButton" type="button" title="Play or pause" aria-keyshortcuts="Space">Play</button>
|
| 162 |
+
<button id="nextButton" type="button" title="Next item" aria-keyshortcuts="N">Next</button>
|
| 163 |
+
<button id="headingButton" type="button" title="Next heading" aria-keyshortcuts="H">Heading</button>
|
| 164 |
+
<button id="imageButton" type="button" title="Next image" aria-keyshortcuts="I">Image</button>
|
| 165 |
+
<button id="summaryButton" type="button" title="Summarize current section" aria-keyshortcuts="S">Summary</button>
|
| 166 |
+
<button id="stopButton" type="button" title="Stop audio" aria-keyshortcuts="Escape">Stop</button>
|
| 167 |
<label>
|
| 168 |
Voice
|
| 169 |
<select id="voiceControl" aria-label="Reader voice"></select>
|