cvpfus Codex commited on
Commit ·
3b1f921
1
Parent(s): a13e743
Include reader position in transcript
Browse filesCo-authored-by: Codex <codex@openai.com>
- FIELD_NOTES.md +1 -1
- README.md +1 -1
- SUBMISSION.md +1 -1
- app.py +1 -1
- scripts/verify.py +6 -0
- static/app.js +4 -1
FIELD_NOTES.md
CHANGED
|
@@ -48,7 +48,7 @@ Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro
|
|
| 48 |
|
| 49 |
The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Repeat and Stop also have visible buttons, which keeps the assistive command model available to pointer users and easy for judges to inspect.
|
| 50 |
|
| 51 |
-
Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article. Clipboard failures are labeled in the live narration region instead of being treated as successful copies.
|
| 52 |
|
| 53 |
Keyboard map:
|
| 54 |
|
|
|
|
| 48 |
|
| 49 |
The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Repeat and Stop also have visible buttons, which keeps the assistive command model available to pointer users and easy for judges to inspect.
|
| 50 |
|
| 51 |
+
Each narration is also added to a visible transcript log with reader position, runtime, and latency. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article. Clipboard failures are labeled in the live narration region instead of being treated as successful copies.
|
| 52 |
|
| 53 |
Keyboard map:
|
| 54 |
|
README.md
CHANGED
|
@@ -109,7 +109,7 @@ Global reader shortcuts ignore buttons, links, selects, and inputs so the voice,
|
|
| 109 |
|
| 110 |
Reader buttons expose `aria-keyshortcuts`, and the Repeat and Stop shortcuts are mirrored by visible controls so pointer and keyboard users share the same command surface.
|
| 111 |
|
| 112 |
-
The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up. Copy actions report when browser clipboard access is unavailable, so the visible transcript and commands remain inspectable.
|
| 113 |
|
| 114 |
Images start with meaningful fallback `alt` text in the HTML. Image descriptions are then preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
|
| 115 |
|
|
|
|
| 109 |
|
| 110 |
Reader buttons expose `aria-keyshortcuts`, and the Repeat and Stop shortcuts are mirrored by visible controls so pointer and keyboard users share the same command surface.
|
| 111 |
|
| 112 |
+
The session panel keeps a transcript of recent narration with reader position, runtime, latency, copy, and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up. Copy actions report when browser clipboard access is unavailable, so the visible transcript and commands remain inspectable.
|
| 113 |
|
| 114 |
Images start with meaningful fallback `alt` text in the HTML. Image descriptions are then preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
|
| 115 |
|
SUBMISSION.md
CHANGED
|
@@ -6,7 +6,7 @@ Tiny Narrator turns an article into a guided screen-reader experience using smal
|
|
| 6 |
|
| 7 |
## Short description
|
| 8 |
|
| 9 |
-
Tiny Narrator is a custom Gradio Server app that looks like an article/blog reader until screen-reader mode is switched on. In reader mode, the app builds a semantic reading queue, narrates headings and paragraphs, describes generated images, summarizes the current section, speaks with Kokoro, and keeps a visible transcript of the spoken path.
|
| 10 |
|
| 11 |
The prototype is designed for a live hackathon demo: every model-facing path has a deterministic fallback, runtime readiness is labeled in the UI, and the repo exposes machine-readable evidence for model size, setup, demo flow, and accessibility behavior.
|
| 12 |
|
|
|
|
| 6 |
|
| 7 |
## Short description
|
| 8 |
|
| 9 |
+
Tiny Narrator is a custom Gradio Server app that looks like an article/blog reader until screen-reader mode is switched on. In reader mode, the app builds a semantic reading queue, narrates headings and paragraphs, describes generated images, summarizes the current section, speaks with Kokoro, and keeps a visible transcript of the spoken path with reader position, runtime, and latency.
|
| 10 |
|
| 11 |
The prototype is designed for a live hackathon demo: every model-facing path has a deterministic fallback, runtime readiness is labeled in the UI, and the repo exposes machine-readable evidence for model size, setup, demo flow, and accessibility behavior.
|
| 12 |
|
app.py
CHANGED
|
@@ -451,7 +451,7 @@ def accessibility_audit_core() -> dict[str, Any]:
|
|
| 451 |
"id": "inspectable_transcript",
|
| 452 |
"label": "Inspectable transcript",
|
| 453 |
"status": "pass",
|
| 454 |
-
"evidence": "Narration entries are stored in a visible transcript with copy and clear controls.",
|
| 455 |
},
|
| 456 |
{
|
| 457 |
"id": "user_control",
|
|
|
|
| 451 |
"id": "inspectable_transcript",
|
| 452 |
"label": "Inspectable transcript",
|
| 453 |
"status": "pass",
|
| 454 |
+
"evidence": "Narration entries are stored in a visible transcript with reader position, runtime, latency, copy, and clear controls.",
|
| 455 |
},
|
| 456 |
{
|
| 457 |
"id": "user_control",
|
scripts/verify.py
CHANGED
|
@@ -65,6 +65,12 @@ def verify_static_assets() -> None:
|
|
| 65 |
"Transcript is visible, but clipboard access is unavailable." in app_js,
|
| 66 |
"Transcript copy should report unavailable clipboard access",
|
| 67 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
|
| 69 |
assert_true(
|
| 70 |
"haltPlayback({ clearAutoAdvance: false });" in app_js,
|
|
|
|
| 65 |
"Transcript is visible, but clipboard access is unavailable." in app_js,
|
| 66 |
"Transcript copy should report unavailable clipboard access",
|
| 67 |
)
|
| 68 |
+
assert_true("transcript-position" in app_js, "Transcript should render the narrated reader position")
|
| 69 |
+
assert_true("readerItemStatus(node)" in app_js, "Narration transcript entries should include reader item position")
|
| 70 |
+
assert_true(
|
| 71 |
+
"[${entry.type} / ${entry.position} / ${entry.runtime}]" in app_js,
|
| 72 |
+
"Copied transcript should include type, position, and runtime",
|
| 73 |
+
)
|
| 74 |
assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
|
| 75 |
assert_true(
|
| 76 |
"haltPlayback({ clearAutoAdvance: false });" in app_js,
|
static/app.js
CHANGED
|
@@ -100,6 +100,7 @@ function renderTranscript() {
|
|
| 100 |
<span>${escapeHtml(entry.type)} (${escapeHtml(formatElapsed(entry.elapsedMs))})</span>
|
| 101 |
<span>${escapeHtml(entry.runtime)}</span>
|
| 102 |
</div>
|
|
|
|
| 103 |
<p class="transcript-text">${escapeHtml(entry.text)}</p>
|
| 104 |
</li>
|
| 105 |
`)
|
|
@@ -646,6 +647,7 @@ async function narrate(index) {
|
|
| 646 |
latencyStatus.textContent = formatElapsed(elapsedMs);
|
| 647 |
addTranscriptEntry({
|
| 648 |
type: node.type,
|
|
|
|
| 649 |
runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
|
| 650 |
text: result.narration,
|
| 651 |
elapsedMs,
|
|
@@ -684,6 +686,7 @@ async function summarizeCurrentSection() {
|
|
| 684 |
latencyStatus.textContent = formatElapsed(elapsedMs);
|
| 685 |
addTranscriptEntry({
|
| 686 |
type: "summary",
|
|
|
|
| 687 |
runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
|
| 688 |
text: result.narration,
|
| 689 |
elapsedMs,
|
|
@@ -749,7 +752,7 @@ copyTranscriptButton.addEventListener("click", async () => {
|
|
| 749 |
const text = transcriptEntries
|
| 750 |
.slice()
|
| 751 |
.reverse()
|
| 752 |
-
.map((entry) => `[${entry.type} / ${entry.runtime}] ${entry.text}`)
|
| 753 |
.join("\n");
|
| 754 |
await copyTextToClipboard(
|
| 755 |
text,
|
|
|
|
| 100 |
<span>${escapeHtml(entry.type)} (${escapeHtml(formatElapsed(entry.elapsedMs))})</span>
|
| 101 |
<span>${escapeHtml(entry.runtime)}</span>
|
| 102 |
</div>
|
| 103 |
+
<p class="transcript-position">${escapeHtml(entry.position)}</p>
|
| 104 |
<p class="transcript-text">${escapeHtml(entry.text)}</p>
|
| 105 |
</li>
|
| 106 |
`)
|
|
|
|
| 647 |
latencyStatus.textContent = formatElapsed(elapsedMs);
|
| 648 |
addTranscriptEntry({
|
| 649 |
type: node.type,
|
| 650 |
+
position: readerItemStatus(node),
|
| 651 |
runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
|
| 652 |
text: result.narration,
|
| 653 |
elapsedMs,
|
|
|
|
| 686 |
latencyStatus.textContent = formatElapsed(elapsedMs);
|
| 687 |
addTranscriptEntry({
|
| 688 |
type: "summary",
|
| 689 |
+
position: section.title,
|
| 690 |
runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
|
| 691 |
text: result.narration,
|
| 692 |
elapsedMs,
|
|
|
|
| 752 |
const text = transcriptEntries
|
| 753 |
.slice()
|
| 754 |
.reverse()
|
| 755 |
+
.map((entry) => `[${entry.type} / ${entry.position} / ${entry.runtime}] ${entry.text}`)
|
| 756 |
.join("\n");
|
| 757 |
await copyTextToClipboard(
|
| 758 |
text,
|