| --- |
| title: DoodleBook |
| emoji: 📚 |
| colorFrom: red |
| colorTo: yellow |
| sdk: gradio |
| sdk_version: "6.18.0" |
| app_file: app.py |
| pinned: false |
| license: apache-2.0 |
| tags: |
| - hackathon |
| - build-small |
| - adventure-in-thousand-token-wood |
| - gradio |
| - flux |
| - minicpm |
| - voxcpm |
| - storybook |
| - coloring-book |
| - voice-cloning |
| - children |
| - track:wood |
| - sponsor:openbmb |
| - sponsor:openai |
| - sponsor:modal |
| - achievement:offgrid |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:sharing |
| - achievement:fieldnotes |
| models: |
| - black-forest-labs/FLUX.2-klein-4B |
| - openbmb/MiniCPM5-1B |
| - openbmb/VoxCPM2 |
| --- |
| |
| # DoodleBook 📚🖍️ |
|
|
| **A child draws a character → DoodleBook turns it into a narrated, illustrated crayon storybook *and* a matching printable coloring book — with their own voice if they want.** |
|
|
| Built for the **Build Small Hackathon 2026 · Adventure in Thousand Token Wood**. Every model is under 32B; the reasoning stack (story + voice) is just ~3B. |
|
|
| > Open the Space and a real sample book loads instantly. Then draw your own character, upload it, and watch your hero come to life across six pages. |
|
|
| --- |
|
|
| ## ✅ Pre-flight Checklist |
|
|
| | Requirement | Status | Notes | |
| |---|---|---| |
| | **Stay under 32B** | ✅ | MiniCPM5-1B (1B) + VoxCPM2 (2B) + FLUX.2-klein-4B (4B) = **7B total** | |
| | **Ship a Gradio app** | ✅ | Gradio 6 Space in the Build Small org | |
| | **Record a demo** | ✅ | [MP4 demo](docs/demo-doodlebook.mp4) + [Supademo walkthrough](https://app.supademo.com/demo/cmqfkwlro4f4wqmgj218kxnqp?utm_source=link) | |
| | **Post on social media** | ✅ | [X/Twitter post](https://x.com/sushruthsgowda/status/2066639063168225452?s=46) | |
| | **GPU limit (≤10 ZeroGPU)** | ✅ | 1 ZeroGPU Space | |
|
|
| --- |
|
|
| ## 🔗 Links |
|
|
| | | | |
| |---|---| |
| | 🚀 **Live Space** | [huggingface.co/spaces/build-small-hackathon/DoodleBook](https://huggingface.co/spaces/build-small-hackathon/DoodleBook) | |
| | ▶️ **Demo video** | [MP4 demo](docs/demo-doodlebook.mp4) · [Supademo walkthrough](https://app.supademo.com/demo/cmqfkwlro4f4wqmgj218kxnqp?utm_source=link) | |
| | 📣 **Social post** | [X/Twitter announcement](https://x.com/sushruthsgowda/status/2066639063168225452?s=46) | |
| | 📝 **Project article** (complete technical paper) | [docs/article.md](docs/article.md) | |
| | 🔬 **Field Notes** (technical deep-dive) | [docs/blog.md](docs/blog.md) | |
| | 💻 **Source code** | [github.com/Sushruths04/Doodle-book](https://github.com/Sushruths04/Doodle-book) | |
| | 🤖 **MiniCPM5-1B** | [huggingface.co/openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B) | |
| | 🔊 **VoxCPM2** | [huggingface.co/openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) | |
| | 🎨 **FLUX.2-klein-4B** | [huggingface.co/black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) | |
| | 🗣️ **Kannada TTS fine-tune** | [huggingface.co/sush0401/IndicF5-Kannada-Bedtime-v2](https://huggingface.co/sush0401/IndicF5-Kannada-Bedtime-v2) | |
|
|
| --- |
|
|
| ## ✨ What it does |
|
|
| 1. **Draw & upload** a character (upload or webcam snap). |
| 2. **Story** — MiniCPM5-1B writes a complete six-page story with a consistent hero and a real emotional arc. |
| 3. **Illustrations** — FLUX.2-klein renders each page where **your character stays consistent** across all pages (built from *your* drawing via img2img). |
| 4. **Narration** — VoxCPM2 reads the whole book aloud in the child's choice of voice — kid, storyteller, grandpa, or **their own cloned voice** from an uploaded reference recording. |
| 5. **Coloring book** — a matching black-and-white line-art version to print and color. |
| 6. **Downloads** — one-tap **Story PDF** and **Coloring PDF** (works on mobile too). |
|
|
| ### What makes a story good |
|
|
| DoodleBook uses a structured prompting system to guarantee quality: |
|
|
| - **Few-shot exemplar**: A full 6-page story is shown as a gold example so MiniCPM learns the exact format and richness required. |
| - **Story arc rules**: Pages 1–2 introduce the hero and challenge; middle pages build it; final pages resolve and teach a clear lesson. |
| - **2–3 rich sentences per page**: Every page uses sensory detail — colours, sounds, textures, feelings — not just plot. |
| - **10 diverse themes** covering kindness, imagination, friendship, courage, identity, and adventure. |
| - **Continuity rules**: Each page flows from the one before — the story reads as one coherent arc, not isolated scenes. |
| - **Six-page pacing**: Every story follows a compact picture-book arc designed for the live generation budget. |
|
|
| --- |
|
|
| ## 🧠 Models & the "Tiny Titan" argument |
|
|
| | Role | Model | Params | Sponsor | What it does | |
| |---|---|---|---|---| |
| | 📖 Story writer | `openbmb/MiniCPM5-1B` | **1B** | **OpenBMB** | Writes the full story as structured JSON — title, character description, per-page text and scene descriptions | |
| | 🔊 Voice narrator | `openbmb/VoxCPM2` | **2B** | **OpenBMB** | Reads the story aloud with voice design prefixes; optionally clones the user's own voice via reference audio | |
| | 🎨 Illustrator | `black-forest-labs/FLUX.2-klein-4B` | **4B** | **Black Forest Labs** | Renders each page + the coloring line-art as FLUX img2img; keeps the child's drawn character consistent | |
|
|
| **Total parameter count: 7B.** Each individual model is well below the 32B cap. |
|
|
| **The product's reasoning is a ~3B small-model stack.** MiniCPM5-1B writes the narrative and scene plans; VoxCPM2 performs it. FLUX is not the "intelligence" — it's the renderer. That's the Tiny Titan story: the small models drive the experience. |
|
|
| --- |
|
|
| ## 🎙️ Voice Cloning — the "My Voice" option |
|
|
| When a parent selects **"🎙️ My Voice"**, they upload 5–60 seconds of clear speech. VoxCPM2 uses this as a `reference_wav_path` to clone the voice and narrate the book in it — so children hear their parent's actual voice reading their story, even on demand. |
|
|
| The reference audio is processed locally (never stored). Sentences are capped at 15 for the ZeroGPU budget. |
|
|
| --- |
|
|
| ## 🌍 Community uploads and fine-tuning work |
|
|
| DoodleBook's live Space is the English six-page storybook flow powered by MiniCPM5-1B, |
| VoxCPM2, and FLUX.2-klein-4B. Alongside the deployed app, we also uploaded and documented |
| community-facing assets so the work is inspectable: |
|
|
| - **Fine-tuned Kannada bedtime narration checkpoint:** [`sush0401/IndicF5-Kannada-Bedtime-v2`](https://huggingface.co/sush0401/IndicF5-Kannada-Bedtime-v2). This experimental voice model work targets warm Kannada bedtime narration and is referenced in [`indic_tts.py`](indic_tts.py) with fallback tiers to MMS-TTS and gTTS. It is not loaded by the current English single-tab Space, but it documents the multilingual narration path we built and tested toward. |
| - **Demo video uploaded to the Space:** [`docs/demo-doodlebook.mp4`](docs/demo-doodlebook.mp4), stored through Git LFS so judges can view the end-to-end generation process directly from the Hugging Face repository. |
| - **Interactive walkthrough:** [Supademo demo](https://app.supademo.com/demo/cmqfkwlro4f4wqmgj218kxnqp?utm_source=link), showing the product flow step by step. |
| - **Social sharing:** [X/Twitter launch post](https://x.com/sushruthsgowda/status/2066639063168225452?s=46), satisfying the hackathon sharing requirement. |
| - **Technical article:** [`docs/article.md`](docs/article.md), explaining the objective, model stack, character-consistency method, narration path, ZeroGPU engineering, and limitations. |
| - **Field Notes:** [`docs/blog.md`](docs/blog.md), a focused write-up on keeping one child-drawn hero consistent across all story pages without per-user training. |
|
|
| We keep the distinction explicit: the deployed app uses **zero-shot VoxCPM2 voice cloning** |
| for English custom voices, while the Kannada IndicF5 checkpoint is a separate fine-tuned |
| community asset and future multilingual direction. |
|
|
| --- |
|
|
| ## 🎨 Cross-page character consistency |
|
|
| Generate pages independently and you get six different characters. DoodleBook keeps **one** hero — *the one the child drew* — with **no per-user training**: |
|
|
| 1. **Canonical-character pass** — the doodle goes through FLUX **img2img once** to produce a clean "model-sheet" hero. |
| 2. **Every page is conditioned on that canonical image**, so the same creature appears in every scene. |
| 3. **Seed-locking** (deterministic per page) + a **fixed character description** anchor identity. |
|
|
| A cat doodle becomes a cat hero across all pages. Full write-up: **[Field Notes →](docs/blog.md)**. |
|
|
| --- |
|
|
| ## 🖍️ Coloring book: redraw, don't trace |
|
|
| Tracing finished crayon pages turned textures into speckle. Instead, DoodleBook passes each color page back to FLUX as img2img with a *"clean coloring-book line art"* prompt — it **redraws** clean outlines, then a local pass crisps them to pure black-on-white. |
|
|
| --- |
|
|
| ## 🔍 Open Trace — every book is reproducible |
|
|
| Open **"Behind the magic → Trace"** on any generated book: |
|
|
| - locked **seed** (same inputs → same book) |
| - per-page prompts and scene plan |
| - exact **model IDs** and stage timings |
| - fallback reasons (surfaced, never silent) |
|
|
| --- |
|
|
| ## 🏗️ Architecture |
|
|
| | Layer | What | File | |
| |---|---|---| |
| | Product UI | Custom scrapbook Gradio 6 Blocks | [`ui/layout.py`](ui/layout.py) | |
| | Book / PDF | HTML book + printable PDFs | [`book_builder.py`](book_builder.py) | |
| | Story | MiniCPM5-1B + deterministic arc fallback | [`app.py`](app.py) | |
| | Images | FLUX.2-klein canonical + per-page img2img | [`app.py`](app.py) | |
| | Voice | VoxCPM2 narration (parallel) + voice cloning | [`app.py`](app.py) | |
| | Coloring | FLUX line art + crisp/threshold cleanup | [`services/coloring.py`](services/coloring.py) | |
| | Config | Models, seeds, voices, palette | [`config.py`](config.py) | |
|
|
| ### How it runs (ZeroGPU) |
|
|
| - **Gradio 6** Space, **ZeroGPU** hardware — free T4 GPU per request. |
| - All three models load **on CUDA at module scope** (ZeroGPU pattern); each stage is a `@spaces.GPU` call. |
| - Narration runs **in parallel** with illustration and surfaces the moment it's ready. |
| - A real, pre-generated **sample book loads instantly** on open — no GPU required. |
|
|
| --- |
|
|
| ## 🏅 Hackathon badges |
|
|
| | Badge | Status | Evidence | |
| |---|---|---| |
| | **Off-Brand** | ✅ | Fully custom scrapbook UI — Gaegu/Caveat fonts, paper textures, hand-drawn SVG frames, light-locked. Zero Gradio defaults. See [`ui/layout.py`](ui/layout.py). | |
| | **Open Trace** | ✅ | Every book exposes seed, per-page prompts, model IDs, and timings in the in-app Trace panel. | |
| | **Field Notes** | ✅ | Engineering write-up on cross-page consistency: **[docs/blog.md](docs/blog.md)**. | |
| | **Tiny Titan** | ✅ | MiniCPM5-1B + VoxCPM2 ≈ **3B reasoning stack**; FLUX is the renderer. | |
| | **Sponsor — OpenBMB** | ✅ | `MiniCPM5-1B` writes every story; `VoxCPM2` narrates and clones voices. | |
| | **Sponsor — Black Forest Labs** | ✅ | `FLUX.2-klein-4B` renders every page illustration and coloring line art. | |
|
|
| --- |
|
|
| ## 🤖 End-to-end pipeline completed by Codex |
|
|
| Codex completed the final shipped DoodleBook pipeline end to end: app implementation fixes, |
| ZeroGPU debugging, output ordering, PDF export behavior, voice-flow fixes, story prompt |
| refinement, image/coloring prompt refinement, README updates, technical article writing, |
| demo/video publishing, hackathon metadata, sponsor documentation, and Hugging Face Space |
| deployment. |
|
|
| The final shipped implementation and documentation commit series is authored as |
| `Codex <codex@openai.com>` in Git, so reviewers can see the work was completed through |
| Codex-authored commits. |
|
|
| Recent Codex-authored contributions include: |
|
|
| - `40d2703` — documented the Kannada fine-tuned TTS checkpoint and community assets. |
| - `be27d4c` — added Build Small hackathon metadata tags. |
| - `be8ab4a` — uploaded the MP4 demo and added sponsor, social, and Codex publication links. |
| - `bc7f386` — wrote the detailed project article and demo-video script. |
| - `67c02d3` — improved story, narration, illustration, and coloring prompts. |
| - `d61efd6` through `3c0a4d2` — restored the original output-card layout while preserving the |
| audio -> PDF -> pages reveal order. |
| - `ef904db` through `786a8ec` — fixed voice handling, cover design, output ordering, and PDF |
| export behavior. |
|
|
| --- |
|
|
| ## ▶️ Run locally |
|
|
| ```bash |
| pip install -r requirements.txt |
| python app.py |
| ``` |
|
|
| The example doodle (`assets/sample_doodle.jpg`) and the instant sample book are included. |
|
|
| --- |
|
|
| ## 🤝 Sponsor & tool stack |
|
|
| - **OpenBMB** — powers the 3B reasoning-and-voice core. `MiniCPM5-1B` writes the structured |
| story JSON, title, character description, page text, and visual scene plan. `VoxCPM2` |
| narrates the book, supports expressive narrator presets, and enables the My Voice option |
| through zero-shot voice cloning from an uploaded reference clip. |
| - **Black Forest Labs** — powers all visual generation through `FLUX.2-klein-4B`. It creates |
| the canonical hero from the child's doodle, renders all six crayon-style story pages, and |
| redraws finished pages into clean coloring-book line art. |
| - **Hugging Face Spaces / ZeroGPU** — hosts the public Gradio app and provides the GPU runtime |
| that makes the full pipeline publicly accessible without running private infrastructure. |
| - **Gradio 6** — provides the app framework; DoodleBook customizes it into a scrapbook-style, |
| mobile-friendly product interface instead of a default demo layout. |
| - **Codex** — used as the coding and documentation agent for repository changes, prompt |
| engineering, debugging, commit preparation, and deployment updates. |
|
|
| --- |
|
|
| ## License |
|
|
| Apache-2.0. See [LICENSE](LICENSE). |
|
|
