README.md

metadata

title: DoodleBook
emoji: 📚
colorFrom: red
colorTo: yellow
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: apache-2.0
tags:
  - hackathon
  - build-small
  - adventure-in-thousand-token-wood
  - gradio
  - flux
  - minicpm
  - voxcpm
  - storybook
  - coloring-book
  - voice-cloning
  - children
  - track:wood
  - sponsor:openbmb
  - sponsor:openai
  - sponsor:modal
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:sharing
  - achievement:fieldnotes
models:
  - black-forest-labs/FLUX.2-klein-4B
  - openbmb/MiniCPM5-1B
  - openbmb/VoxCPM2

DoodleBook 📚🖍️

A child draws a character → DoodleBook turns it into a narrated, illustrated crayon storybook and a matching printable coloring book — with their own voice if they want.

Built for the Build Small Hackathon 2026 · Adventure in Thousand Token Wood. Every model is under 32B; the reasoning stack (story + voice) is just ~3B.

Open the Space and a real sample book loads instantly. Then draw your own character, upload it, and watch your hero come to life across six pages.

---

✅ Pre-flight Checklist

Requirement	Status	Notes
Stay under 32B	✅	MiniCPM5-1B (1B) + VoxCPM2 (2B) + FLUX.2-klein-4B (4B) = 7B total
Ship a Gradio app	✅	Gradio 6 Space in the Build Small org
Record a demo	✅	MP4 demo + Supademo walkthrough
Post on social media	✅	X/Twitter post
GPU limit (≤10 ZeroGPU)	✅	1 ZeroGPU Space

---

🔗 Links

🚀 Live Space	huggingface.co/spaces/build-small-hackathon/DoodleBook
▶️ Demo video	MP4 demo · Supademo walkthrough
📣 Social post	X/Twitter announcement
📝 Project article (complete technical paper)	docs/article.md
🔬 Field Notes (technical deep-dive)	docs/blog.md
💻 Source code	github.com/Sushruths04/Doodle-book
🤖 MiniCPM5-1B	huggingface.co/openbmb/MiniCPM5-1B
🔊 VoxCPM2	huggingface.co/openbmb/VoxCPM2
🎨 FLUX.2-klein-4B	huggingface.co/black-forest-labs/FLUX.2-klein-4B
🗣️ Kannada TTS fine-tune	huggingface.co/sush0401/IndicF5-Kannada-Bedtime-v2

---

✨ What it does

1. Draw & upload a character (upload or webcam snap).
2. Story — MiniCPM5-1B writes a complete six-page story with a consistent hero and a real emotional arc.
3. Illustrations — FLUX.2-klein renders each page where your character stays consistent across all pages (built from your drawing via img2img).
4. Narration — VoxCPM2 reads the whole book aloud in the child's choice of voice — kid, storyteller, grandpa, or their own cloned voice from an uploaded reference recording.
5. Coloring book — a matching black-and-white line-art version to print and color.
6. Downloads — one-tap Story PDF and Coloring PDF (works on mobile too).

What makes a story good

DoodleBook uses a structured prompting system to guarantee quality:

• Few-shot exemplar: A full 6-page story is shown as a gold example so MiniCPM learns the exact format and richness required.
• Story arc rules: Pages 1–2 introduce the hero and challenge; middle pages build it; final pages resolve and teach a clear lesson.
• 2–3 rich sentences per page: Every page uses sensory detail — colours, sounds, textures, feelings — not just plot.
• 10 diverse themes covering kindness, imagination, friendship, courage, identity, and adventure.
• Continuity rules: Each page flows from the one before — the story reads as one coherent arc, not isolated scenes.
• Six-page pacing: Every story follows a compact picture-book arc designed for the live generation budget.

---

🧠 Models & the "Tiny Titan" argument

Role	Model	Params	Sponsor	What it does
📖 Story writer	openbmb/MiniCPM5-1B	1B	OpenBMB	Writes the full story as structured JSON — title, character description, per-page text and scene descriptions
🔊 Voice narrator	openbmb/VoxCPM2	2B	OpenBMB	Reads the story aloud with voice design prefixes; optionally clones the user's own voice via reference audio
🎨 Illustrator	black-forest-labs/FLUX.2-klein-4B	4B	Black Forest Labs	Renders each page + the coloring line-art as FLUX img2img; keeps the child's drawn character consistent

Total parameter count: 7B. Each individual model is well below the 32B cap.

The product's reasoning is a ~3B small-model stack. MiniCPM5-1B writes the narrative and scene plans; VoxCPM2 performs it. FLUX is not the "intelligence" — it's the renderer. That's the Tiny Titan story: the small models drive the experience.

---

🎙️ Voice Cloning — the "My Voice" option

When a parent selects "🎙️ My Voice", they upload 5–60 seconds of clear speech. VoxCPM2 uses this as a reference_wav_path to clone the voice and narrate the book in it — so children hear their parent's actual voice reading their story, even on demand.

The reference audio is processed locally (never stored). Sentences are capped at 15 for the ZeroGPU budget.

---

🌍 Community uploads and fine-tuning work

DoodleBook's live Space is the English six-page storybook flow powered by MiniCPM5-1B, VoxCPM2, and FLUX.2-klein-4B. Alongside the deployed app, we also uploaded and documented community-facing assets so the work is inspectable:

• Fine-tuned Kannada bedtime narration checkpoint: sush0401/IndicF5-Kannada-Bedtime-v2. This experimental voice model work targets warm Kannada bedtime narration and is referenced in indic_tts.py with fallback tiers to MMS-TTS and gTTS. It is not loaded by the current English single-tab Space, but it documents the multilingual narration path we built and tested toward.
• Demo video uploaded to the Space: docs/demo-doodlebook.mp4, stored through Git LFS so judges can view the end-to-end generation process directly from the Hugging Face repository.
• Interactive walkthrough: Supademo demo, showing the product flow step by step.
• Social sharing: X/Twitter launch post, satisfying the hackathon sharing requirement.
• Technical article: docs/article.md, explaining the objective, model stack, character-consistency method, narration path, ZeroGPU engineering, and limitations.
• Field Notes: docs/blog.md, a focused write-up on keeping one child-drawn hero consistent across all story pages without per-user training.

We keep the distinction explicit: the deployed app uses zero-shot VoxCPM2 voice cloning for English custom voices, while the Kannada IndicF5 checkpoint is a separate fine-tuned community asset and future multilingual direction.

---

🎨 Cross-page character consistency

Generate pages independently and you get six different characters. DoodleBook keeps one hero — the one the child drew — with no per-user training:

1. Canonical-character pass — the doodle goes through FLUX img2img once to produce a clean "model-sheet" hero.
2. Every page is conditioned on that canonical image, so the same creature appears in every scene.
3. Seed-locking (deterministic per page) + a fixed character description anchor identity.

A cat doodle becomes a cat hero across all pages. Full write-up: Field Notes →.

---

🖍️ Coloring book: redraw, don't trace

Tracing finished crayon pages turned textures into speckle. Instead, DoodleBook passes each color page back to FLUX as img2img with a "clean coloring-book line art" prompt — it redraws clean outlines, then a local pass crisps them to pure black-on-white.

---

🔍 Open Trace — every book is reproducible

Open "Behind the magic → Trace" on any generated book:

• locked seed (same inputs → same book)
• per-page prompts and scene plan
• exact model IDs and stage timings
• fallback reasons (surfaced, never silent)

---

🏗️ Architecture

Layer	What	File
Product UI	Custom scrapbook Gradio 6 Blocks	ui/layout.py
Book / PDF	HTML book + printable PDFs	book_builder.py
Story	MiniCPM5-1B + deterministic arc fallback	app.py
Images	FLUX.2-klein canonical + per-page img2img	app.py
Voice	VoxCPM2 narration (parallel) + voice cloning	app.py
Coloring	FLUX line art + crisp/threshold cleanup	services/coloring.py
Config	Models, seeds, voices, palette	config.py

How it runs (ZeroGPU)

• Gradio 6 Space, ZeroGPU hardware — free T4 GPU per request.
• All three models load on CUDA at module scope (ZeroGPU pattern); each stage is a @spaces.GPU call.
• Narration runs in parallel with illustration and surfaces the moment it's ready.
• A real, pre-generated sample book loads instantly on open — no GPU required.

---

🏅 Hackathon badges

Badge	Status	Evidence
Off-Brand	✅	Fully custom scrapbook UI — Gaegu/Caveat fonts, paper textures, hand-drawn SVG frames, light-locked. Zero Gradio defaults. See ui/layout.py.
Open Trace	✅	Every book exposes seed, per-page prompts, model IDs, and timings in the in-app Trace panel.
Field Notes	✅	Engineering write-up on cross-page consistency: docs/blog.md.
Tiny Titan	✅	MiniCPM5-1B + VoxCPM2 ≈ 3B reasoning stack; FLUX is the renderer.
Sponsor — OpenBMB	✅	MiniCPM5-1B writes every story; VoxCPM2 narrates and clones voices.
Sponsor — Black Forest Labs	✅	FLUX.2-klein-4B renders every page illustration and coloring line art.

---

🤖 End-to-end pipeline completed by Codex

Codex completed the final shipped DoodleBook pipeline end to end: app implementation fixes, ZeroGPU debugging, output ordering, PDF export behavior, voice-flow fixes, story prompt refinement, image/coloring prompt refinement, README updates, technical article writing, demo/video publishing, hackathon metadata, sponsor documentation, and Hugging Face Space deployment.

The final shipped implementation and documentation commit series is authored as Codex <codex@openai.com> in Git, so reviewers can see the work was completed through Codex-authored commits.

Recent Codex-authored contributions include:

• 40d2703 — documented the Kannada fine-tuned TTS checkpoint and community assets.
• be27d4c — added Build Small hackathon metadata tags.
• be8ab4a — uploaded the MP4 demo and added sponsor, social, and Codex publication links.
• bc7f386 — wrote the detailed project article and demo-video script.
• 67c02d3 — improved story, narration, illustration, and coloring prompts.
• d61efd6 through 3c0a4d2 — restored the original output-card layout while preserving the audio -> PDF -> pages reveal order.
• ef904db through 786a8ec — fixed voice handling, cover design, output ordering, and PDF export behavior.

---

▶️ Run locally

pip install -r requirements.txt
python app.py

The example doodle (assets/sample_doodle.jpg) and the instant sample book are included.

---

🤝 Sponsor & tool stack

• OpenBMB — powers the 3B reasoning-and-voice core. MiniCPM5-1B writes the structured story JSON, title, character description, page text, and visual scene plan. VoxCPM2 narrates the book, supports expressive narrator presets, and enables the My Voice option through zero-shot voice cloning from an uploaded reference clip.
• Black Forest Labs — powers all visual generation through FLUX.2-klein-4B. It creates the canonical hero from the child's doodle, renders all six crayon-style story pages, and redraws finished pages into clean coloring-book line art.
• Hugging Face Spaces / ZeroGPU — hosts the public Gradio app and provides the GPU runtime that makes the full pipeline publicly accessible without running private infrastructure.
• Gradio 6 — provides the app framework; DoodleBook customizes it into a scrapbook-style, mobile-friendly product interface instead of a default demo layout.
• Codex — used as the coding and documentation agent for repository changes, prompt engineering, debugging, commit preparation, and deployment updates.

---

License

Apache-2.0. See LICENSE.