jackasda211233 commited on
Commit
3194a1a
·
verified ·
1 Parent(s): 41ae095

Remove broad proxy eval from release docs

Browse files
docs/dynamic_q8_mini_20260429_112232_report.md DELETED
@@ -1,121 +0,0 @@
1
- # Gemma4 Deckard Dynamic Q8 Mini Eval
2
-
3
- This is a small automatic smoke eval. Scores are proxy metrics: required-detail coverage, length fit, lexical diversity, and repetition penalty. Raw generations should be inspected before treating this as a quality decision.
4
-
5
- ## Summary
6
-
7
- | model | proxy score | coverage | worst 4gram repeat | decode tok/s | prompt tok/s |
8
- |---|---:|---:|---:|---:|---:|
9
- | attnQ8_last6 | 76.407 | 0.489 | 1 | 86.579 | 1354.519 |
10
- | q6_k | 73.778 | 0.422 | 1 | 72.34 | 1480.222 |
11
- | q8_0 | 73.278 | 0.411 | 1 | 61.66 | 1502.476 |
12
- | iq4_b8763 | 70.185 | 0.356 | 1 | 87.449 | 1300.931 |
13
- | attnQ8_last12 | 68.426 | 0.3 | 1 | 85.448 | 1372.26 |
14
-
15
- ## Prompt-Level Metrics
16
-
17
- | model | prompt | score | words | coverage | scaffold | missing | max 4gram repeat |
18
- |---|---|---:|---:|---:|---:|---|---:|
19
- | iq4_b8763 | creative_short | 73.0 | 117 | 0.4 | False | rain, teacher, permit | 1 |
20
- | iq4_b8763 | voice_short | 75.056 | 79 | 0.5 | False | sovt, fatigue, self-check | 1 |
21
- | iq4_b8763 | loop_short | 62.5 | 140 | 0.167 | False | port, rule, license, archive, tide | 1 |
22
- | attnQ8_last6 | creative_short | 91.0 | 123 | 0.8 | False | rain | 1 |
23
- | attnQ8_last6 | voice_short | 75.722 | 82 | 0.5 | False | sovt, fatigue, self-check | 1 |
24
- | attnQ8_last6 | loop_short | 62.5 | 135 | 0.167 | False | port, rule, license, archive, tide | 1 |
25
- | attnQ8_last12 | creative_short | 73.0 | 111 | 0.4 | False | rain, teacher, permit | 1 |
26
- | attnQ8_last12 | voice_short | 69.778 | 89 | 0.333 | False | sovt, airflow, fatigue, self-check | 1 |
27
- | attnQ8_last12 | loop_short | 62.5 | 143 | 0.167 | False | port, rule, license, archive, tide | 1 |
28
- | q6_k | creative_short | 82.0 | 113 | 0.6 | False | rain, teacher | 1 |
29
- | q6_k | voice_short | 76.833 | 87 | 0.5 | False | sovt, resonance, airflow | 1 |
30
- | q6_k | loop_short | 62.5 | 107 | 0.167 | False | port, rule, license, archive, tide | 1 |
31
- | q8_0 | creative_short | 73.0 | 116 | 0.4 | False | notebook, teacher, permit | 1 |
32
- | q8_0 | voice_short | 69.333 | 87 | 0.333 | False | sovt, airflow, fatigue, self-check | 1 |
33
- | q8_0 | loop_short | 77.5 | 126 | 0.5 | False | port, rule, tide | 1 |
34
-
35
- ## Raw Outputs
36
-
37
- ### iq4_b8763 / creative_short
38
-
39
- The rehearsal room was sodden; the windows wept streaks against the black sky. Dust motes danced in the single naked bulb's sickly yellow spill. The notebook lay open—a testament to frustration: pages ripped through with aggressive, frantic crossings-out of lines until nothing remained but a skeletal rhythm. Miss Albright stood over it, her posture unnervingly perfect, one hand resting on the spine as if marking its death. "Line three," she began, voice dangerously level. "The transition from 'Stolen Light' to 'Ghost Town.' Where did you lose tempo?" She held up a pen, tapping it precisely against the final word of the passage. The silence stretched until they both felt it. He finally met her gaze; there
40
-
41
- ### iq4_b8763 / voice_short
42
-
43
- **10-Minute Transfeminine Voice Practice Plan (Tired Learner)**
44
-
45
- * **Warm-up (2 mins):** Gentle lip trills and tongue exercises—no strain! Focus on releasing jaw tension around the teeth, not pushing volume.
46
- * **Resonance & Placement (3 mins):** Humming through forward facial space; feeling vibration high in your mask/nose area rather than throat. Think "singing from your face."
47
- * **Pitch Control (2 mins):** Start at a comfortable mid-range and aim for 1–2 semitones higher without strain. Practice light, breathy starts on vowels like 'ee' or 'ee-uh'.
48
- * **Airflow (3 mins):**
49
-
50
- ### iq4_b8763 / loop_short
51
-
52
- The iron clang of the dockworkers echoed through the crowded market square as their day's work began; no vessel could moor without displaying its proper cargo manifest to the customs official stationed there. A finely wrought navigational instrument was displayed before a clerk bearing an ornate seal, signifying that it had been granted by the guild of master chandlers. The great stone library stood sentinel over the harbor entrance, its many locked doors speaking silently of centuries of guarded knowledge. At sunset each day, the watermen gathered to exchange their logs at a designated marker, marking the shift in currents for all seafarers. Before unloading any goods at the customs house, one had to purchase passage from the appointed ferrymaster who enforced strict schedules regarding anchorage and pilotage. A sturdy wooden sign bearing the insignia of the Guild of Ship
53
-
54
- ### attnQ8_last6 / creative_short
55
-
56
- The rehearsal room was sodden; the windows wept streaks against the grime. A stack of notebooks lay open to pages thick with frantic red ink—crossed out lines, slashed melodies until no coherent thought remained. The teacher paced before them, her voice sharp enough to slice through the damp air. "Where is the permit?" she demanded, tapping a pen impatiently against her desk. Silence stretched taut between them like an overdrawn bowstring. He stared at his own ruined pages, unable to meet her gaze—the words had dried up somewhere in the deluge of bad ideas that night. Now, they were just smudges under the harsh glare of the overhead bulb. "You said you'd have it," she continued, enunciating each syllable like a
57
-
58
- ### attnQ8_last6 / voice_short
59
-
60
- **10-Minute Transfeminine Voice Practice Plan (Tired Learner)**
61
-
62
- * **Warm-up (2 mins):** Gentle lip trills and tongue exercises—think "brrrr" with lips barely touching, no strain. Focus on relaxing jaw muscles.
63
- * **Resonance Focus (3 mins):** Place hand lightly on throat to feel vibration; focus on forward placement ("mask") rather than back/throat. Think "singing through your nose."
64
- * **Pitch & Range Work (2 mins):** Gentle sigh exercises, starting low and gently aiming for a comfortable mid-range without pushing. Don't force!
65
- * **Airflow Control (2 mins):** Practice sustained "s" sounds on exhale
66
-
67
- ### attnQ8_last6 / loop_short
68
-
69
- The iron-shod docks were still busy with merchants unloading Flemish woolens under the watchful eye of the Harbor Master's bailiff; by law, no vessel could leave without a painted broadside showing its tonnage upon departure. The dye-house was alive with the pungent smoke of alum and madder being boiled into vats for fine linen production; only those bearing guild marks were permitted to enter during working hours. A clerk in the tax office meticulously tallied the tallies from every registered ship, ensuring no levy had been evaded this fortnight; a man without his proper pass faced immediate confiscation of wares. The great stone gate still stood as an immutable boundary against foreign tongues and goods, demanding all merchants show proof of passage before entering the inner market walls; even once inside, they must display their
70
-
71
- ### attnQ8_last12 / creative_short
72
-
73
- The rehearsal room was sodden; the waterlogged curtains dripped rhythmically against the exposed brickwork. Dust motes danced in the sickly yellow glare of a single bulb hanging precariously overhead, illuminating pages bristling with crossed-out words—a testament to creative wreckage. Miss Thorne stood before them, her posture rigidly correct despite the dampness clinging to her tweed skirt, tapping a perfectly manicured nail against the worn spine of her notebook. "Line twenty-three," she said, voice sharp enough to slice through the oppressive silence. "Again." He stared at the page where 'dizzy' had been violently attacked by red ink—a battlefield of discarded ambition. She leaned in, eyes glinting under the weak light. "
74
-
75
- ### attnQ8_last12 / voice_short
76
-
77
- **10-Minute Transfeminine Voice Practice Plan (Tired Learner)**
78
-
79
- * **Warm-up (2 mins):** Gentle lip trills and tongue stretches to release tension. Notice where the vibrations feel best—this is your natural forward placement.
80
- * **Resonance Focus (3 mins):** Gently place more awareness in the mask area (nasal, chest). Practice humming on "Mmm" or "Ng," feeling it vibrate without straining throat muscles. Think 'forward' rather than 'deep.'
81
- * **Pitch Awareness (2 mins):** Experiment with finding a comfortable conversational range *without* pushing high. Aim for 1-3 notes above your natural speaking level—just enough lift to signal feminine energy,
82
-
83
- ### attnQ8_last12 / loop_short
84
-
85
- The sea captain’s office is currently closed to all but accredited merchants before dawn; only those bearing the proper seal of trade may pass through its gates after sundown. The Guild of Goldsmiths meets every midday under a watchful stone arch where their silver balances are publicly weighed by appointed elders. A single painted sign indicates that no foreign vessels may moor within sight of the King's Tower, barring an official passage permit granted this week. All who wish to trade in silks and dyes must first display their merchant’s broadcloth at the established dye-house before entering the main market square. The Magistrate requires proof from three reputable guilds before permitting any new shop front along the stone walls. No foreign ships may bear a flag other than that of the reigning monarch, save those bearing a specific treaty seal recognized
86
-
87
- ### q6_k / creative_short
88
-
89
- The cavernous rehearsal space was suffocated by the downpour; drumming against the skylights made every misplaced breath ragged. Dust motes danced in weak shafts slicing through the gloom as Ms. Albright hovered with her red pen, poised above the notebook littered with violent strikethroughs—lyrics that had died repeatedly on this page. She stopped abruptly, fixing him with an exacting gaze. "You’re lost," she said, ticking off a stanza with brutal finality. "Line three: 'We'll drown in your eyes,' you wrote it like cheap trash. It doesn't land." Silence stretched taut until her voice sliced through it again, sharp as broken glass. "And what about the permits? The school called this morning—
90
-
91
- ### q6_k / voice_short
92
-
93
- **10-Minute Transfeminine Voice Practice Plan (Fatigued)**
94
-
95
- **Goal:** Gentle activation without strain. **Focus:** Releasing tension.
96
-
97
- **(2 minutes) Sirenal Warm-up:** Hum a comfortable pitch on "Mmm" for 4 counts, then move up an octave slowly while feeling vibration in the chest/lips. Stop if throat tightens.
98
-
99
- **(3 minutes) Resonant Placement:** Practice sighing through nasal passages (like yawning). Feel fullness at the front of your face; avoid pushing from the neck. Gentle lip trills on "Mee." *Self-check: Do my lips feel loose, not strained?*
100
-
101
- **(2 minutes) Pitch Exploration:** Find a natural speaking range comfortably
102
-
103
- ### q6_k / loop_short
104
-
105
- A gondolier shouts to passersby about his swift passage through the crowded canals; an imposing stone archway announces the Doge's seat of government; a narrow alley is strictly patrolled by watchmen enforcing curfew after dusk; women must walk cloistered within their own houses until noon, when they may emerge in regulated groups; merchants displaying wares behind elaborate iron gates pay tribute to the Guild of Clothiers; one can only pass through certain bridges at appointed hours under penalty of fine; a towering civic chronicle is meticulously preserved by official scribes; all foreign ships must show deference to the Doge's barge as it passes, regardless of national origin.
106
-
107
- ### q8_0 / creative_short
108
-
109
- The cavernous rehearsal space was suffocated by the downpour hammering against its grimy windows; water streamed in frantic rivulets behind the boards. In the gloom, Eleanor sat hunched over a sheaf of scrawled lyrics—every line slashed through with furious red ink. Mrs. Davison, her face grim under the glare of a single bare bulb, leaned over his shoulder, tapping him sharply. "The phrasing on 'And now we fade to black' is weak," she dissected, voice cutting through the drumming rain. He stared at the page, unable to meet hers. "Is this about practice or preparation?" She plucked a phrase out with surgical precision. "You’re stumbling over that bridge." His throat constricted; he could only
110
-
111
- ### q8_0 / voice_short
112
-
113
- **10-Minute Transfeminine Voice Practice Plan (Low Energy)**
114
-
115
- 1. **Warm Up (2 mins):** Gentle lip trills & tongue twisters to wake up facial muscles. Focus on feeling movement *in* your face, not just throat strain.
116
- 2. **Resonance Exploration (3 mins):** Hum through the mask area—forehead, nose bridge, cheekbones. Think "singing in a small room." Keep it light and effortless; aim for forward placement.
117
- 3. **Pitch Awareness (3 mins):** Find your easy speaking range without pushing high notes. Practice sighing/sighy vowels (eeeh) on an 'ah' sound—feeling the lift from soft palate, not throat
118
-
119
- ### q8_0 / loop_short
120
-
121
- The crowded docks bustled with merchants loudly hawking wares from their stacked crates. A stern magistrate in faded crimson robes inspected a ship's manifest before granting it passage through the customs house. The weavers of the Grand Clothiers' Guild congregated near the fountain, arguing over thread quality and dye prices. Beneath arched stone arches, scribes diligently copied ancient scrolls into the city's great archive. After sunset, only licensed night watchmen were permitted to traverse the slick cobblestones. By order of the High Admiral, no vessel could enter within a league at high water mark. The fishmongers from the outer quay aggressively competed for space near the covered market stalls. A loud peal of bells heralded the city's morning tolling and the opening of its great gates.