cvpfus Codex commited on
Commit
3b1f921
·
1 Parent(s): a13e743

Include reader position in transcript

Browse files

Co-authored-by: Codex <codex@openai.com>

Files changed (6) hide show
  1. FIELD_NOTES.md +1 -1
  2. README.md +1 -1
  3. SUBMISSION.md +1 -1
  4. app.py +1 -1
  5. scripts/verify.py +6 -0
  6. static/app.js +4 -1
FIELD_NOTES.md CHANGED
@@ -48,7 +48,7 @@ Reader shortcuts deliberately ignore form controls. A user can adjust the Kokoro
48
 
49
  The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Repeat and Stop also have visible buttons, which keeps the assistive command model available to pointer users and easy for judges to inspect.
50
 
51
- Each narration is also added to a visible transcript log. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article. Clipboard failures are labeled in the live narration region instead of being treated as successful copies.
52
 
53
  Keyboard map:
54
 
 
48
 
49
  The reader bar exposes the same shortcut contract through `aria-keyshortcuts`. Repeat and Stop also have visible buttons, which keeps the assistive command model available to pointer users and easy for judges to inspect.
50
 
51
+ Each narration is also added to a visible transcript log with reader position, runtime, and latency. This gives the demo a second accessible surface: users can review what was spoken, copy the session transcript, or clear it before exploring another part of the article. Clipboard failures are labeled in the live narration region instead of being treated as successful copies.
52
 
53
  Keyboard map:
54
 
README.md CHANGED
@@ -109,7 +109,7 @@ Global reader shortcuts ignore buttons, links, selects, and inputs so the voice,
109
 
110
  Reader buttons expose `aria-keyshortcuts`, and the Repeat and Stop shortcuts are mirrored by visible controls so pointer and keyboard users share the same command surface.
111
 
112
- The session panel keeps a transcript of recent narration with copy and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up. Copy actions report when browser clipboard access is unavailable, so the visible transcript and commands remain inspectable.
113
 
114
  Images start with meaningful fallback `alt` text in the HTML. Image descriptions are then preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
115
 
 
109
 
110
  Reader buttons expose `aria-keyshortcuts`, and the Repeat and Stop shortcuts are mirrored by visible controls so pointer and keyboard users share the same command surface.
111
 
112
+ The session panel keeps a transcript of recent narration with reader position, runtime, latency, copy, and clear controls, making the spoken path inspectable during demos and useful for the Field Notes write-up. Copy actions report when browser clipboard access is unavailable, so the visible transcript and commands remain inspectable.
113
 
114
  Images start with meaningful fallback `alt` text in the HTML. Image descriptions are then preloaded into a local cache and written into the page's real `img alt` attributes. When the planned MiniCPM-V runtime is unavailable, deterministic alt-text fallbacks keep the screen-reader path usable.
115
 
SUBMISSION.md CHANGED
@@ -6,7 +6,7 @@ Tiny Narrator turns an article into a guided screen-reader experience using smal
6
 
7
  ## Short description
8
 
9
- Tiny Narrator is a custom Gradio Server app that looks like an article/blog reader until screen-reader mode is switched on. In reader mode, the app builds a semantic reading queue, narrates headings and paragraphs, describes generated images, summarizes the current section, speaks with Kokoro, and keeps a visible transcript of the spoken path.
10
 
11
  The prototype is designed for a live hackathon demo: every model-facing path has a deterministic fallback, runtime readiness is labeled in the UI, and the repo exposes machine-readable evidence for model size, setup, demo flow, and accessibility behavior.
12
 
 
6
 
7
  ## Short description
8
 
9
+ Tiny Narrator is a custom Gradio Server app that looks like an article/blog reader until screen-reader mode is switched on. In reader mode, the app builds a semantic reading queue, narrates headings and paragraphs, describes generated images, summarizes the current section, speaks with Kokoro, and keeps a visible transcript of the spoken path with reader position, runtime, and latency.
10
 
11
  The prototype is designed for a live hackathon demo: every model-facing path has a deterministic fallback, runtime readiness is labeled in the UI, and the repo exposes machine-readable evidence for model size, setup, demo flow, and accessibility behavior.
12
 
app.py CHANGED
@@ -451,7 +451,7 @@ def accessibility_audit_core() -> dict[str, Any]:
451
  "id": "inspectable_transcript",
452
  "label": "Inspectable transcript",
453
  "status": "pass",
454
- "evidence": "Narration entries are stored in a visible transcript with copy and clear controls.",
455
  },
456
  {
457
  "id": "user_control",
 
451
  "id": "inspectable_transcript",
452
  "label": "Inspectable transcript",
453
  "status": "pass",
454
+ "evidence": "Narration entries are stored in a visible transcript with reader position, runtime, latency, copy, and clear controls.",
455
  },
456
  {
457
  "id": "user_control",
scripts/verify.py CHANGED
@@ -65,6 +65,12 @@ def verify_static_assets() -> None:
65
  "Transcript is visible, but clipboard access is unavailable." in app_js,
66
  "Transcript copy should report unavailable clipboard access",
67
  )
 
 
 
 
 
 
68
  assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
69
  assert_true(
70
  "haltPlayback({ clearAutoAdvance: false });" in app_js,
 
65
  "Transcript is visible, but clipboard access is unavailable." in app_js,
66
  "Transcript copy should report unavailable clipboard access",
67
  )
68
+ assert_true("transcript-position" in app_js, "Transcript should render the narrated reader position")
69
+ assert_true("readerItemStatus(node)" in app_js, "Narration transcript entries should include reader item position")
70
+ assert_true(
71
+ "[${entry.type} / ${entry.position} / ${entry.runtime}]" in app_js,
72
+ "Copied transcript should include type, position, and runtime",
73
+ )
74
  assert_true("function haltPlayback" in app_js, "Reader controls should expose a shared playback halt helper")
75
  assert_true(
76
  "haltPlayback({ clearAutoAdvance: false });" in app_js,
static/app.js CHANGED
@@ -100,6 +100,7 @@ function renderTranscript() {
100
  <span>${escapeHtml(entry.type)} (${escapeHtml(formatElapsed(entry.elapsedMs))})</span>
101
  <span>${escapeHtml(entry.runtime)}</span>
102
  </div>
 
103
  <p class="transcript-text">${escapeHtml(entry.text)}</p>
104
  </li>
105
  `)
@@ -646,6 +647,7 @@ async function narrate(index) {
646
  latencyStatus.textContent = formatElapsed(elapsedMs);
647
  addTranscriptEntry({
648
  type: node.type,
 
649
  runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
650
  text: result.narration,
651
  elapsedMs,
@@ -684,6 +686,7 @@ async function summarizeCurrentSection() {
684
  latencyStatus.textContent = formatElapsed(elapsedMs);
685
  addTranscriptEntry({
686
  type: "summary",
 
687
  runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
688
  text: result.narration,
689
  elapsedMs,
@@ -749,7 +752,7 @@ copyTranscriptButton.addEventListener("click", async () => {
749
  const text = transcriptEntries
750
  .slice()
751
  .reverse()
752
- .map((entry) => `[${entry.type} / ${entry.runtime}] ${entry.text}`)
753
  .join("\n");
754
  await copyTextToClipboard(
755
  text,
 
100
  <span>${escapeHtml(entry.type)} (${escapeHtml(formatElapsed(entry.elapsedMs))})</span>
101
  <span>${escapeHtml(entry.runtime)}</span>
102
  </div>
103
+ <p class="transcript-position">${escapeHtml(entry.position)}</p>
104
  <p class="transcript-text">${escapeHtml(entry.text)}</p>
105
  </li>
106
  `)
 
647
  latencyStatus.textContent = formatElapsed(elapsedMs);
648
  addTranscriptEntry({
649
  type: node.type,
650
+ position: readerItemStatus(node),
651
  runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
652
  text: result.narration,
653
  elapsedMs,
 
686
  latencyStatus.textContent = formatElapsed(elapsedMs);
687
  addTranscriptEntry({
688
  type: "summary",
689
+ position: section.title,
690
  runtime: `${result.runtime}/${speech?.runtime || "voice"}`,
691
  text: result.narration,
692
  elapsedMs,
 
752
  const text = transcriptEntries
753
  .slice()
754
  .reverse()
755
+ .map((entry) => `[${entry.type} / ${entry.position} / ${entry.runtime}] ${entry.text}`)
756
  .join("\n");
757
  await copyTextToClipboard(
758
  text,