Spaces:
Running
Running
File size: 4,587 Bytes
14d681b e85709c 0aedcbf e85709c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | ---
title: Slapstack Studio
emoji: 🥞🪐
colorFrom: indigo
colorTo: blue
sdk: gradio
app_file: app.py
pinned: false
license: mit
---
# Slapstack Studio (Bet 8 Playroom)
An interactive, layered visual sandbox running multi-object Belief Propagation (BP) over Gabor packet representations. The system supports full 2D depth occlusion matting, spatial grab/drag/scale/rotate mechanics, semantic prompt interventions, and server-side neural layer distillation via Score Distillation Sampling (SDS).
## Architecture & Division of Labor
- **The Server (Python):** Handles visual generation and heavy pixel-to-atom fitting. It uses an SDS pipeline with Stable Diffusion 2.1 or deterministic MSE optimization to turn pictures/prompts into compact, normalized Gabor-atom templates.
- **The Client (Browser JS):** Runs a highly optimized, loopy Belief Propagation factor graph entirely client-side. The frontend manages pose clustering, tracks identities through deep occlusion, calculates real-time alpha masks, and translates user gestures into exact group-algebra conditioning updates.
---
## File Structure
```text
slapstack-bet8/
├── app.py # HuggingFace Space / Gradio server entry point
├── oracle.py # Image-to-atom fit & text-to-atom SDS generation paths
├── bet5_gabor_sds.py # Underlying Gabor Packet model & differentiable renderer
├── requirements.txt # Python dependencies for GPU/CPU runtimes
│
├── studio/ # Static folder mounted by FastAPI
│ ├── studio.html # Single-file production client (fully built)
│ └── builtins.json # Pre-trained compact templates (tractor, star, boat)
│
├── studio_ui.html # Raw source markup for the client layout
├── studio_core.js # Real-time alpha-matting & painter compositor logic
├── build.js # Assembly script (combines UI + core logic into studio.html)
└── tests_studio.js # Headless test suite validating depth occlusion math
```
## Getting Started
### Prerequisites
Make sure your environment has Python 3.10+ installed. For neural text-to-layer distillation, a CUDA-capable GPU is highly recommended.
### Installation
1. Clone or download the repository directory.
2. Install the necessary dependencies directly via pip:
```bash
pip install -r requirements.txt
```
### Running the App Locally
Launch the Gradio server by executing:
```bash
python app.py
```
Open your browser and navigate to http://127.0.0.1:7860. The unified workspace will load instantly without requiring any building or file compilation.
## Feature Overview & Control Guide
### Interactive Canvas
- **Scramble:** Randomizes object poses to throw all atoms into an unassigned "soup."
- **Bind:** Runs loopy BP live in the browser. You can watch the atoms settle, crystallize into boundaries, and snap back to their matching identity profiles.
- **Mouse Controls (Grabbable Posterior):**
- Click + Drag any recognized object to clamp its position. The environment dynamically re-settles around it.
- Mouse Wheel rotates the targeted object about its calculated center.
- Shift + Mouse Wheel scales the targeted object up or down.
### Layer Manipulation & Occlusion Semantics
- Switch the scene blend setting from **Field** (additive summation) to **Painter** to observe full alpha-compositing across individual depths.
- Moving an object entirely behind another activates **Depth Occlusion**. Covered atoms cleanly release their assignments and revert back toward a state of uniform uncertainty (their marginal prior), while the shape's pose boundary coasts seamlessly behind the obstruction. Moving the object away re-establishes evidence and triggers immediate re-binding.
### Spawning Custom Layers
- **Fit (CPU):** Upload any image containing a subject on a neutral, solid gray background. The optimizer will spend its entire atom budget crafting a tight, isolated layer template.
- **Distill (GPU):** Enter a descriptive text prompt to distill an atom map from Stable Diffusion.
- *Note for Local Deployment:* The system is pre-configured to look for `sd2-community/stable-diffusion-2-1-base` to efficiently reuse local caches located under `~/.cache/huggingface/hub/`.
### Prompt Interventions
Enter clean, natural commands in the prompt box (e.g., *"move the boat left, send the star behind the tractor"*) to execute hard algebraic transformations. The graph instantly recalculates the new joint posterior distribution around your action.
|