--- title: Agentic Space Factory sdk: docker hf_oauth: true hf_oauth_expiration_minutes: 480 hf_oauth_scopes: - read-repos - write-repos - manage-repos - gated-repos - inference-api - jobs - read-billing --- ## v190.33 Linked validation terminal summary consistency - v190.33 rewrites linked Space Test validation `summary.json` at the end of success and failure paths, so validation runs no longer remain listed as `running` after `state.json` has terminalized. - Validation list/detail views continue to canonicalize from state and diagnostic artefacts, but the lightweight bucket summary is now terminal-consistent for manual audits and Runs Explorer cards. ## v190.32 Robust linked validation schema parsing - v190.32 hardens linked Space Test payload resolution when Gradio endpoint schemas contain heterogeneous parameter lists such as bare strings mixed with dict objects. The validation worker now normalizes schema parameters before reading metadata, preventing payload-resolution crashes like `'str' object has no attribute 'get'`. - Replay mode now fails fast with a clear launch error if a successful parent automatic-smoke payload cannot be resolved, instead of silently downgrading to endpoint-discovery-first while still presenting itself as replay. ## v190.32 Eval archive run-scoped reset and layout stability - v190.32 makes the Active Run eval archive card strictly run-scoped: launching, deleting, or switching Build Runs clears stale eval archive state before the selected run snapshot arrives. - The eval archive key/value layout now uses a stable single-column grid with safe wrapping so long bucket names and record paths never collapse into letter-by-letter columns. ## v190.32 Runs Explorer linked validation parent metadata - v190.32 makes Build Run cards display manual linked validation state directly from parent metadata (`manual_validation_status.json` / `linked_validations.json`), so the summary is visible even when linked Validation Runs are not listed as standalone rows. Automatic smoke status remains separate from manual linked validation status. ## v190.32 Replay uses parent smoke artefact as source of truth - v190.32 makes linked Space Test replay mode load the parent Build Run's successful automatic smoke artefacts before attempting any schema reconstruction. The validation worker writes `replay_source.json`, `resolved_validation_request.json`, and diagnostics so a replay cannot fail opaquely at the Payload stage. ## v190.32 Linked validation hardening - v190.32 hardens linked Space Test success/failure paths: parent Build Run updates read API/output metadata from the smoke result, failed manual attempts cannot downgrade an already validated parent, and latency/ZeroGPU recommendations expose their linked-test hardware provenance. - Runs Explorer and Run Stats now treat automatic and effective statuses explicitly so manually validated partial builds remain discoverable in both Partial and Validated views. - Space Test is contextualized from the selected Build Run even before pressing Prefill, while remaining linked-only for launches. ## v190.16 Cascade delete linked Space Tests - Build Run deletion now removes linked Space Test validation runs from the run bucket as part of the same cleanup. - Linked validation deletion keeps the parent Build Run but refreshes its manual validation metadata. - Runs Explorer remains Build-first and clears parent/child rows from cache immediately after deletion. ## v190.16 Build-first explorer compatibility - v190.16 keeps Runs Explorer simple: Build Runs are the primary objects, linked Space Tests are summarized on their parent, and legacy/incomplete Build Runs remain visible without requiring validation metadata. - It hardens the run cache against null or partial records and keeps validation runs out of the primary explorer by default. ## v190.16 smoke retry and live validation telemetry - v190.16 keeps Space Test prefill disabled until a build has reached a final Space state that can be tested manually. - v190.16 retries generation smoke once when Gradio reports schema-choice type mismatches, and writes terminal summary.json plus live_status.json for cleaner audits. - The canonical run snapshot now includes a `live_validation` telemetry model so the Live test phase can show Space build/runtime, health, Gradio endpoint discovery and generation smoke state instead of a generic running message. - Active polling runs at a faster visible cadence for smoother event updates. ## v190.10 live convergence validation - Active Run polling, Runs Explorer cards, Run Stats, and Eval Archive must converge to the same terminal snapshot without a browser refresh. - Intermediate fallback/live wait failures are diagnostics only; they must not mark the run failed unless the canonical timeline is terminal. ## v190.6 — Space Test terminal-state lock - v190.6 prevents terminal Space Test validations from oscillating back to Running/Pending when slower index or poll responses arrive later. - The Space Test panel now keeps the terminal detail visible while refreshing, and the Runs Explorer uses the locked terminal status for the same validation id. ## v190.5 — Active Run timeline hotfix - v190.5 fixes the Active Run canonical timeline after the v190.0 fit-to-width pass: desktop/tablet rendering is horizontal again, active steps pulse correctly, and `pi_run` started/running keeps Agent in progress instead of complete. ## v190.1 — Space Test validation UI - v190.1 gives Space Test its own validation timeline, fixes validation runs stuck as Running in Run Explorer, and humanizes Gradio endpoint schema rendering so `[object Object]` never appears. ## v190.0 — Active Run layout consolidation - v190.0 is a UI-only consolidation pass: it reduces Active Run redun­dancy, fits the canonical timeline into the center column, keeps Recovery details compact when Agent recovery is present, and improves recovery text wrapping. ## v189.9 — Timeline polish and Run notes clarity - v189.9 keeps the v189.8 runtime guardrails and visual polish, then tightens the remaining UI details: Recovery phase details stay compact when Agent recovery is present, Run notes is the single place for non-blocking warnings, and opaque +N note counters are removed. ## v189.8 — Failed run UI polish Timeline state hotfix lineage: this release keeps the v189.6 timeline cleanup and adds runtime guardrails. - Adds `run_timeline_model.v1`, a canonical projection of run artifacts/events into product phases. - Separates process completion from final verdict so a completed run can be success, partial, failed, blocked, or manual-action-required. - Defines stable phases for Start, Model, Agent, Hardware, Deploy, Live test, Recovery, Archive, and Done. - Preserves validated v188.9 runtime behavior while preparing the UI for a cleaner timeline/progress redesign. - Keeps the global Eval archive storage panel separate from the per-run Eval archive publication panel. - v189.4 adds Mobile timeline polish: a compact active phase detail panel, vertical canonical timeline on narrow screens, and readable Needs attention summaries. - v189.8 keeps the v189.7 runtime guardrails and polishes failed-run UI: one Run notes banner, compact Recovery details, final build blocker focus, and contextual latency messaging. ## v181 — Backend-published private eval archive - Keeps the operator eval bucket private and mounted only on the ASF Space, not on per-user Jobs. - User Jobs now write only `runs//eval_record.json` and `events_compact.jsonl` into the signed-in user's run bucket. - The ASF backend copies validated, anonymized records from user run buckets into the private operator archive mounted at `/evals`. - This lets all users contribute anonymized eval records without giving their Jobs direct write access to `fffiloni/agentic-space-factory-evals`. ## v175 — Anonymous cross-run evaluation archive - Introduced the optional operator-owned evaluation archive; v181 changes publication to backend-copy instead of Job-mounted writes. - Workers write local `eval_record.json` plus a compact event timeline with hashed run/user/Space identifiers, structured hardware/Pi/recovery/validation metrics, and no raw generated code, prompts, tokens, or private bucket paths. - Controlled by `ASF_EVAL_ENABLED`, `ASF_EVAL_BUCKET_SOURCE`, `ASF_EVAL_BUCKET_PATH`, `ASF_EVAL_BUCKET_MOUNT`, and `ASF_EVAL_SALT`. - The Run storage card now indicates whether anonymous project evaluation is enabled for the instance. ## v174 — Bucket run-prefix mount and robust worker bootstrap writes - Mounts only the `runs/` bucket prefix into Jobs at `/output/runs`, while the worker still writes to `/output/runs//...`. - Keeps API-side writes on canonical `hf://buckets///runs//...` object URIs, with `HfFileSystem` attempts followed by a `sync_bucket` fallback for fresh prefixes. - Removes the v172 parent-prefix `makedirs()` workaround, because Bucket prefixes are virtual object paths rather than git-style folders. - Requires `huggingface_hub[oauth]>=1.11.0` so Buckets, Jobs volumes and sync helpers are available in fresh installs. ## v172 — Fresh bucket writes and pre-scan polish - Keeps the Hugging Face Space metadata at the very top of `README.md`, so the app cannot boot with a configuration error after archive edits. - Makes Bucket writes resilient for fresh run prefixes by creating parent prefixes before writing `_worker.py`, launch metadata, state, summaries, and events. - Refreshes the Model pre-scan card into a calmer status panel with compact metadata chips and less oversized green treatment. ## v170 — Real bucket recursive run deletion - DELETE `/api/runs/{run_id}` now removes concrete bucket objects under `runs//` using recursive discovery instead of relying only on virtual-folder deletion. - The API returns a cleanup report with deleted and remaining object counts. - Frontend cache invalidation remains as a protection layer, but deletion is now a real backend bucket cleanup. ## Anonymous eval archive activation Recommended setup for a private cross-user archive: ```text Bucket source: fffiloni/agentic-space-factory-evals Bucket path: evals Space mount: /evals Mode: read-write ``` Then sign in as the bucket owner and click **Enable eval archive** in the Run Storage card. ASF writes: ```text /evals/.asf_eval_config.json ``` The archive bucket stays private. User Jobs do **not** mount it. Instead: ```text User Job → writes runs//eval_record.json in the user's private run bucket ASF backend → reads that anonymized record → publishes it to /evals/YYYY/MM/DD//eval_record.json ``` Environment variables remain supported as an admin override: ```env ASF_EVAL_ENABLED=true ASF_EVAL_BUCKET_SOURCE=fffiloni/agentic-space-factory-evals ASF_EVAL_BUCKET_PATH=evals ASF_EVAL_BUCKET_MOUNT=/evals ASF_EVAL_SALT= ASF_EVAL_INCLUDE_REDACTED_TAILS=false ASF_EVAL_INCLUDE_MODEL_ID=false ``` The archive does not store raw generated code, prompts, tokens, private bucket IDs, target Spaces in clear text, raw logs, or outputs by default. # Agentic Space Factory - **v170** fixes run deletion so the backend recursively removes all concrete Bucket objects for the run prefix. - **v168** refreshes Pi assistant model choices and surfaces requested/effective provider model routing in Active Run. - **v165** fixes fresh-install recovery failures, README metadata preservation, timeline failure anchoring, premium pre-scan/progress polish, and broader Pi assistant model choices. - **v161** fixes duplicate delete modal triggers and stuck `Deleting…` states with single-flight delete guards. # Agentic Space Factory Agentic Space Factory turns a Hugging Face model card into a **private, testable Gradio Space** using a Hugging Face Job, Pi, a coding model through Hugging Face Inference Providers, and a private Storage Bucket for run state, traces, reports, generated files, and validation artifacts. It is a Hugging Face-native productization of the agentic ZeroGPU Spaces workflow: ```text model card → agentic build job → private Space → hardware if available → live validation → traces/report/artifacts ``` ## What the app does - Creates private generated Spaces under the signed-in user's namespace. - Launches long-running build and validation work as Hugging Face Jobs. - Uses Pi with a coding model served through Hugging Face Inference Providers. - Injects the configured GitHub gist as Pi's operational instruction source. - Stores each run in a per-user private Storage Bucket. - Attempts ZeroGPU first when requested, then optional fixed-GPU fallback. - Supports manual hardware selection when automatic assignment is unavailable. Automatic fallback avoids high/restricted tiers such as A100/H200; use Space Settings manually if an account or organization has access. - Distinguishes bootability from real inference. - Measures live inference latency when a generation smoke test succeeds. - Uses measured latency to recommend `@spaces.GPU(duration=...)` when the generated Space uses ZeroGPU. ## Current UI The product root `/` is a custom Docker/FastAPI dashboard. `/custom` is kept as an alias. The UI is organized into three main areas: ## v122 stability note The v122 line intentionally rolls back the experimental v121 client-side caching/polling optimizations after real UI testing exposed flicker and broken build/run behavior. The app keeps the v120 Agent recovery UI and the v118/v119 core blockage protocol, but returns to the proven polling and run-selection model for production stability. ### Left column - **Run storage** — check or create the signed-in user's private Bucket. - **New Build** — starts from `Tongyi-MAI/Z-Image-Turbo` by default, fast pre-scans the model card, then launches a private Space build only after an acceptable scan verdict. - **Billing & compute** — show payment readiness, link to HF Billing, and centralize the compute-cost note for Jobs, Inference Providers, and safe hardware fallbacks. ### Center workspace Two tabs share the main detail area: - **Active Run** — build progress, run actions, Space/Settings/Job/Artifacts links, manual hardware guidance, inference latency, a vector-icon latest-events feed, and curated trace documents. - **Space Test** — validation run details, actionnable Gradio endpoint picker, endpoint params, validation status, latency, point-based progress, and validation actions. Selecting a build run opens `Active Run`. Selecting a validation run opens `Space Test`. The Runs Explorer is intentionally compact: each row prioritizes the model or Space, status, relative freshness, and a secondary context line while keeping the raw run ID available in the row. ### Right column - **Runs Explorer** — navigation-only list of build and validation runs with filters, search, status badges, and inline delete. - **Run Stats** — aggregate metrics across indexed runs: totals, success/pass rates, validation latency median/p95, and issue counts. ### Agent recovery visibility When a build or runtime blockage appears, the Active Run panel shows an **Agent recovery** card. It summarizes Pi's diagnosis decision, the selected recovery action (`wait_for_logs`, `factory_rebuild_same_code`, `patch_code`, manual hardware, or blocker), key evidence, and the next step. This makes the blockage protocol visible without turning the UI into a raw log viewer. The Run traces dock also links to the unified `agent_trace.jsonl`, the diagnosis `REPAIR_DECISION.json`, repair artifacts, and blockage details when they exist. ## Model pre-scan New builds are gated by a fast model-card pre-scan. The scan is metadata-only and does not download model weights. It checks: - model-card guidance and runnable usage examples; - Diffusers signals such as `library_name`, `model_index.json`, pipeline class, and `from_pretrained(...)` examples; - safetensors versus unsafe or legacy weight formats; - gated/private access and unsupported task signals; - custom-code indicators such as `auto_map`, custom pipeline files, or `trust_remote_code` hints. Verdicts are intentionally product-facing: `safe` and `caution` allow launch, `risky` requires explicit acknowledgement, and `unsupported` blocks launch. `Tongyi-MAI/Z-Image-Turbo` is treated as the known-good default example because it has a clear Diffusers model card and works in the app's default test flow. ## Run storage Each signed-in user writes to their own private Storage Bucket. The default bucket is: ```text https://huggingface.co/buckets//space-factory-runs ``` For local dev or deployment, configure only the bucket name; the namespace is derived from the signed-in OAuth user: ```bash SPACE_FACTORY_BUCKET_NAME=space-factory-runs SPACE_FACTORY_BUCKET_MOUNT=/output ``` Typical run layout: ```text runs//launch.json runs//state.json runs//summary.json runs//events.jsonl runs//report.md runs//generated/ runs//tests/ runs//artifacts/ runs//traces/raw/ runs//traces/raw/agent_trace.jsonl runs//traces/redacted/ runs//traces/redacted/agent_trace.jsonl runs//logs/ ``` ## Main workflow ### 1. Build from model card Paste a Hugging Face model ID or model-card URL. The app launches an HF Job that: - fetches model metadata and repo information; - writes a strict `GOAL.md` for Pi; - asks Pi to follow the configured GitHub gist before coding; - generates a Gradio app; - creates and uploads a private Space; - attempts ZeroGPU/fixed-GPU assignment when requested; - runs health/API checks and a live generation smoke test when possible; - writes events, report, traces, and structured artifacts to the user's Bucket. ### 2. Validate Space Validation is a separate HF Job. Use it after a build succeeds or after manually selecting hardware in Space Settings. The validation job: - waits for the target Space; - discovers Gradio endpoints when possible; - calls the selected endpoint; - checks the returned output type; - stores returned artifacts; - measures latency; - recommends a conservative ZeroGPU duration when relevant; - writes validation state, events, report data, and smoke-test artifacts. ## Result statuses - `full_inference_success`: a live generation smoke test returned the expected output type. - `manual_hardware_required`: the Space was generated, but hardware must be selected manually before validation. - `full_inference_candidate_health_passed`: the Space boots and contains inference signals, but generation was not fully smoke-tested. - `health_only`: the Space boots, but no real inference path was validated. - `technical_blocker`: concrete blockers such as multi-GPU requirements, missing licenses, custom CUDA, or unclear usage. - `failed`: the build, runtime, or validation job failed. ## Guarantees - Generated Spaces are private by default. - Nothing is published automatically. - Runs are persisted in the configured per-user Storage Bucket. - The app distinguishes health checks from full inference validation. - Hardware assignment is best-effort and can fall back to manual user action. - Raw and redacted Pi traces are kept separate. ### Unified agent trace Pi can be invoked more than once during a run: the initial build, a blockage diagnosis pass, and an optional structured repair patch. The Factory now writes a canonical run-level agent journal to `traces/raw/agent_trace.jsonl` and `traces/redacted/agent_trace.jsonl` so those phases read as one auditable agentic loop instead of separate conversations. Specialized stdout logs remain available under `logs/`, but the Active Run `Pi RAW` / `Pi redacted` buttons prefer the unified trace when it exists. ## Non-goals and limits The app does not guarantee that every model card becomes a working Space. Some models require multi-GPU execution, Docker, unavailable CUDA extensions, external services, accepted licenses, very large hardware, or manual fixes. The app does not mirror live spend/quota totals from Hugging Face Billing. Users should consult the HF Billing dashboard for exact usage. ## Local development Install dependencies and run tests: ```bash pip install -r requirements.txt pytest ``` The Docker Space starts the app with one uvicorn process from the Dockerfile. Do not add an extra `uvicorn.run(...)` launcher inside `app.py`. ## Documentation See: - `docs/ARCHITECTURE.md` - `docs/PUBLIC_PREVIEW.md` - `docs/SECURITY.md` - `docs/NEXT_STEPS.md` Default example model: `Tongyi-MAI/Z-Image-Turbo`. It is treated as a known-good Diffusers text-to-image model for the fast pre-scan calibration and public-preview smoke tests. ### Pi diagnosis and structured recovery When the generated Space fails live validation, the factory now enters a diagnosis-first blockage protocol instead of asking Pi to patch immediately. The worker collects HF build/runtime logs, Space status and validation errors, writes `repair/INCIDENT_BRIEF.md`, then asks Pi/Qwen Coder to choose one bounded action in `repair/REPAIR_DECISION.json`: wait for delayed logs, inspect again, force a same-code factory rebuild, patch code, request manual hardware, or declare a technical blocker. A code repair is allowed only when Pi selects `patch_code` and the Factory accepts the decision. Empty/no-reason HF failures are routed through wait/rebuild/blocker paths rather than speculative code edits. Patch repair still writes `REPAIR_BRIEF.md`, `REPAIR_PLAN.md`, and `REPAIR_SUMMARY.md`, uploads once, then revalidates the live Space with the normal API checks. Repair mode is intentionally strict: it must preserve the original model and real-inference contract, avoid fake/static outputs, keep a cheap health endpoint, and report blockers instead of hiding runtime errors. ## v123 — Critical Job launch + failure timeline fix - Fixed a production-critical HF Job startup failure where the large embedded worker script could make Python fail before startup with `argument list too long`. - The worker script is now persisted as `runs//_worker.py` in the mounted run bucket before launch, and the Job receives only `WORKER_SCRIPT_PATH`. - Kept a small `WORKER_SCRIPT_B64` fallback only for compatibility/manual launch paths; normal app launches no longer put the worker source in env/argv. - Added a failure-log fallback so pre-worker Job failures such as `argument list too long` still create an explicit failed timeline event and red failure point. ## v124 — Timeline and Run Explorer stabilization - Fixed the compact Overall Progress timeline so dynamically visible repair/failure groups cannot wrap dots onto a second row. - Removed smooth timeline auto-scroll that could cause visible flicker during polling. - Simplified Run Explorer type/status badges to neutral type labels and short status labels such as Success, Failed, Running and Manual. - Added regression checks for timeline single-row behavior and compact Run Explorer labels. ## v127 — Billing usefulness + UI event/timeline audit - Reworked the Billing & compute card into an actionable compute checklist: payment readiness, Pi Inference Providers, build Job cost, generated Space hardware and high-tier GPU guardrails. - Kept exact spend/quotas delegated to the Hugging Face Billing dashboard instead of inventing usage totals. - Audited JS event bindings for one-shot/delegated listeners across navigation, run rows and timeline scroll. - Reduced timeline auto-scroll churn: identical timeline renders no longer force keep-visible scrolling after user interaction. - Confirmed run selection keeps the immediate cached/optimistic render path before network hydration. ### Active Run source documents Run traces now renders as a horizontal document dock: compact source-file icons for Pi traces, repair decisions, reports, smoke results, and blockers. Agent recovery remains the human-readable recovery summary; the dock is the raw source-file vault. ## v164 timeline The Active Run timeline uses compact product milestones backed by exact worker steps. Grouped milestones expose clockwise partial progress rings and substep counts, while recovery details remain in Agent recovery and source files remain in Run traces. ### v167 note: run artifact manifest The worker writes `runs//artifact_manifest.json` as the source of truth for run artifacts. This lets the API expose Run traces from what the worker actually wrote, including failed runs where the Space was created but later entered build/runtime error. ## v171 notes Run deletion now supports an explicit opt-in option to delete the associated generated Space for build runs. Validation run deletion remains run-artifact-only by default. The Run Storage card also summarizes bucket setup errors so fresh installs remain visually clean. ## v187 note Signed-out users can see the public eval archive status for this ASF instance, while management actions remain restricted to the instance owner/admins. ## v189.4 notes The run view model now exposes `timeline_model`, which summarizes progress, verdict, phase status, and non-blocking warnings without replacing the existing UI yet. Eval archive sidebar restoration from v188.9 remains carried forward. ### v189.4 Semantic progress bar The Active Run progress header now separates pipeline completion from product verdict. A run can be 100% complete while still being partial, failed, or requiring manual action. ## v190.9 — Space Test canonical validation persistence - v190.9 makes terminal Space Test validation status survive page reloads. Failed, success, partial, and manual validation outcomes override stale `running` launch metadata in the Runs Explorer, delete modal, and progress polling. ## v190.32 Eval archive effective outcomes - Eval archive records now distinguish automatic verdicts from effective verdicts after linked Space Test validation. - Archived records include anonymized linked-validation, endpoint-discovery and smoke-retry metrics while preserving privacy constraints. - Instance owners can flush all archived eval records from the mounted eval bucket without deleting the eval archive configuration. ## v190.32 Linked validation replay reliability - Linked Space Test validations now normalize replay/complete/recover payloads against the current Gradio schema before predict(). - Validation runs write explicit preflight, resolved request, schema coercion, and failure diagnosis artifacts for debugging. - Replay from a full-inference-success Build Run should reuse the validated endpoint/payload and coerce Gradio dropdown/radio choices as needed. ## v190.32 validation hardening Linked Space Test validations now write engine/preflight/failure diagnostics before and after Gradio endpoint discovery so payload-resolution failures cannot appear as opaque smoke failures.