fffiloni commited on
Commit
b3c91c5
·
verified ·
1 Parent(s): d63c03b

Upload 6 files

Browse files
Files changed (6) hide show
  1. ARCHITECTURE.md +190 -0
  2. NEXT_STEPS.md +110 -0
  3. PUBLIC_PREVIEW.md +79 -0
  4. RECOVERY_ALGORITHM.md +83 -0
  5. RELEASE_VALIDATION.md +51 -0
  6. SECURITY.md +77 -0
ARCHITECTURE.md ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Architecture
2
+
3
+ Agentic Space Factory is a Hugging Face-native productization of the workflow described in Victor's ZeroGPU Spaces article: an agent receives a model-card build goal, follows a shared Pi instruction gist, creates a private Gradio Space, validates it, and records the full run.
4
+
5
+ ```text
6
+ User
7
+ → Docker/FastAPI orchestrator Space with Hugging Face OAuth
8
+ → per-user private Storage Bucket
9
+ → ephemeral Hugging Face Job
10
+ → Pi coding agent + HF Inference Providers model
11
+ → generated private target Space
12
+ → live build progress, traces, reports, validation results
13
+ ```
14
+
15
+ ## Core components
16
+
17
+ ### Orchestrator Space
18
+
19
+ The public product UI is a custom Docker/FastAPI dashboard served at `/`. It uses Hugging Face OAuth and does not store a global admin token.
20
+
21
+ The UI is organized around three responsibilities:
22
+
23
+ - **Left column** — storage setup, new build launch, and billing/compute readiness links.
24
+ - **Center workspace** — tabbed detail area with `Active Run` for build runs and `Space Test` for validation runs.
25
+ - **Right column** — `Runs Explorer` for compact navigation and `Run Stats` for aggregate history metrics. The Run Explorer uses delegated row events, cached rerender signatures, relative freshness labels, and a compact two-line row layout so polling does not repeatedly attach listeners or redraw unchanged lists.
26
+
27
+ ### Build workflow
28
+
29
+ A build starts from a Hugging Face model ID or model-card URL. Before launching a paid Job, the orchestrator runs a metadata-only model pre-scan. The pre-scan reads Hub metadata and small repo files such as the model card, `config.json`, and `model_index.json`; it does not download weights. It looks for runnable examples, Diffusers compatibility, safetensors, unsupported tasks, gated/private access, and custom-code risk.
30
+
31
+ After an acceptable verdict, the orchestrator launches a Hugging Face Job that:
32
+
33
+ 1. reads the model card and model metadata;
34
+ 2. creates a private target Space under the signed-in user's namespace;
35
+ 3. launches Pi with a generated `GOAL.md`;
36
+ 4. instructs Pi to read the configured GitHub gist before coding;
37
+ 5. uploads generated Space files;
38
+ 6. attempts hardware assignment on a best-effort basis;
39
+ 7. runs health/API checks and, when possible, live generation smoke validation;
40
+ 8. writes events, traces, report, and structured artifacts to the user's Storage Bucket.
41
+
42
+ ### Pi instruction model
43
+
44
+ The worker injects the operational gist into the goal sent to Pi. Pi is expected to follow the gist's workflow for Spaces iteration: inspect logs, patch generated code, upload only relevant files, wait for rebuilds, and verify the generated app through Gradio/API calls.
45
+
46
+ The local worker adds stricter product requirements around that method:
47
+
48
+ - generated Spaces stay private;
49
+ - placeholders must not be reported as full inference;
50
+ - blockers must be written explicitly when full inference is impossible;
51
+ - `/health` and generation endpoints should be added where possible;
52
+ - ZeroGPU duration should be informed by measured inference latency when applicable.
53
+
54
+ ### Storage Bucket
55
+
56
+ Runs are stored in the signed-in user's Bucket, normally:
57
+
58
+ ```text
59
+ https://huggingface.co/buckets/<username>/space-factory-runs
60
+ ```
61
+
62
+ Typical run layout:
63
+
64
+ ```text
65
+ runs/<run_id>/launch.json
66
+ runs/<run_id>/state.json
67
+ runs/<run_id>/summary.json
68
+ runs/<run_id>/events.jsonl
69
+ runs/<run_id>/report.md
70
+ runs/<run_id>/generated/
71
+ runs/<run_id>/tests/
72
+ runs/<run_id>/artifacts/
73
+ runs/<run_id>/traces/raw/
74
+ runs/<run_id>/traces/raw/agent_trace.jsonl
75
+ runs/<run_id>/traces/redacted/
76
+ runs/<run_id>/traces/redacted/agent_trace.jsonl
77
+ runs/<run_id>/logs/
78
+ ```
79
+
80
+ The UI exposes the full run folder through the `Artifacts` button and highlights a small curated trace dock in `Active Run`: raw Pi traces, redacted traces, report, smoke result, and blockers when relevant.
81
+
82
+ ### Validation workflow
83
+
84
+ Validation runs are separate from build runs. Selecting a validation run in the Run Explorer opens the `Space Test` tab, not `Active Run`.
85
+
86
+ A validation job:
87
+
88
+ 1. waits for the target Space to be reachable;
89
+ 2. discovers Gradio API endpoints when possible;
90
+ 3. calls the selected endpoint with configured args/kwargs;
91
+ 4. checks the returned output type;
92
+ 5. measures latency;
93
+ 6. stores returned output artifacts;
94
+ 7. writes a validation timeline and summary.
95
+
96
+ The `Space Test` panel displays validation status, latency, Gradio endpoints, selected endpoint parameters, a point-based progress line, and the validation run actions.
97
+
98
+ ### Run discovery and polling
99
+
100
+ The Run Explorer reads indexed run summaries from the Bucket. It can recover in-progress or partial runs from `launch.json`, `summary.json`, and `state.json`, so a run can appear before all final artifacts exist.
101
+
102
+ The frontend avoids duplicate `/api/runs` refreshes, uses delegated event listeners for dynamic lists, and keeps the active run panel responsive by rendering lightweight summaries before heavier run details finish loading.
103
+
104
+ ## Result statuses
105
+
106
+ - `full_inference_success` �� live generation returned the expected output type.
107
+ - `manual_hardware_required` — the Space exists, but hardware must be configured manually before validation.
108
+ - `full_inference_candidate_health_passed` — the Space boots and appears inference-capable, but live generation was not fully validated.
109
+ - `health_only` — the Space boots, but no real inference path was validated.
110
+ - `technical_blocker` — concrete blockers prevent reliable full inference.
111
+ - `failed` — build, runtime, or validation failed.
112
+
113
+ ## Known limits
114
+
115
+ - Automatic paid hardware assignment is best-effort and may fail because of OAuth, quota, billing, namespace limits, or restricted hardware availability. The app avoids A100/H200 as automatic fallbacks; if those are needed, select them manually in Space Settings only when available for the account or organization.
116
+ - Numeric spend/quota totals are not mirrored in the app; users should use the Hugging Face Billing dashboard for live usage. The Billing & compute card also centralizes the Inference Providers / paid credits / restricted fallback warning so the New Build form stays focused on build configuration.
117
+ - Raw traces can contain sensitive operational details. Redaction is best-effort, not a formal data-loss-prevention guarantee.
118
+ - Some models require Docker, multi-GPU, custom CUDA/FlashAttention, external services, gated licenses, or manual engineering.
119
+
120
+
121
+ ## Model pre-scan
122
+
123
+ The custom API exposes `POST /api/models/pre-scan`, which performs a fast metadata-only scan of the selected Hugging Face model before `/api/build` is enabled in the browser. The scan uses Hub metadata and small files such as `config.json`, `model_index.json`, and `README.md`; it avoids downloading weights. This keeps the feature cheap and prevents accidental compute spend on obviously risky or unsupported model cards.
124
+
125
+
126
+ ## Frontend stability pass
127
+
128
+ The production-preview UI avoids listener accumulation by binding navigation, action buttons, run rows, pagination, and endpoint chips with one-time or delegated event handlers. Run list refreshes are deduped through a shared in-flight promise and skip DOM rerenders when the visible signature is unchanged.
129
+
130
+ CSS is organized around the current three-column layout and keeps narrow panels safe through explicit `min-width: 0`, truncation, and responsive fallbacks for run cards, endpoint chips, activity rows, stats, and pre-scan content.
131
+
132
+
133
+ ### v114 activity feed iconography
134
+
135
+ The main `Latest events` feed now renders event-type icons as inline SVG, not emoji and not two-letter text badges. The icon selection is deterministic and local to the frontend: steps such as `upload_files`, `hardware`, `create_space`, `generation_smoke`, `traces`, `report_write`, `auth`, and `model_analysis` map to compact vector symbols. Status classes still control color and motion, so running, success, warning, hardware fallback, and error events stay visually distinct without changing the event payload contract.
136
+
137
+ ### v113 UI and caching notes
138
+
139
+ Run selection now uses a small client-side detail cache. Previously loaded run details are rendered immediately when the user switches between runs, and the app only fetches missing or stale light/heavy details. This keeps the central `Active Run` / `Space Test` panels responsive without changing the backend run bundle API.
140
+
141
+ The `New Build` form no longer requires a target Space slug. The backend's `normalize_target_space()` remains the source of truth and generates a `space-factory-<run_id>` slug when the user leaves the field empty.
142
+
143
+
144
+ ## Navigation note
145
+
146
+ The top bar intentionally does not include a resume-latest action. Run recovery and historical navigation are handled by the Runs Explorer, with cached run-detail hydration for fast repeat selection.
147
+
148
+ ## Pi diagnosis and structured recovery workflow
149
+
150
+ The recovery path is diagnosis-first. It runs only after live Space validation fails, and it separates **deciding what to do** from **patching code**.
151
+
152
+ 1. Collect HF build logs, runtime logs, Space status, validation errors, available Gradio API information and the current workspace inventory.
153
+ 2. Write `repair/INCIDENT_BRIEF.md` and `repair/PI_DIAGNOSIS_GOAL.md`.
154
+ 3. Run Pi/Qwen Coder in blockage diagnosis mode. Pi must not edit files in this step. It writes `repair/REPAIR_DECISION.json`.
155
+ 4. The Factory validates the decision against budgets and safety rules. Allowed actions are `wait_for_logs`, `inspect_more_logs`, `factory_rebuild_same_code`, `patch_code`, `request_manual_hardware`, and `declare_technical_blocker`.
156
+ 5. Empty or no-reason HF failures cannot go directly to code patching. They are routed through wait/rebuild/blocker decisions unless Pi cites concrete actionable evidence.
157
+ 6. A same-code factory rebuild re-uploads the same workspace and revalidates the live Space without changing files. This covers common HF build/runtime flakes where logs are delayed or missing.
158
+ 7. A patch repair is allowed only after `patch_code` is accepted. Pi then writes `REPAIR_BRIEF.md`, `REPAIR_PLAN.md`, patches minimally, writes `REPAIR_SUMMARY.md`, and the Factory runs sanity checks before re-uploading.
159
+ 8. Recovery is bounded: one wait, one same-code rebuild and one patch repair by default. A final failure produces `TECHNICAL_BLOCKERS.json` / `repair/BLOCKAGE.json` rather than an unbounded loop.
160
+
161
+ The progress timeline groups these stages under the repair/recovery ring while preserving individual events such as `pi_diagnosis`, `repair_decision`, `factory_rebuild`, `repair_patch`, and `repair_validation`. Terminal failures still create an explicit red failure point.
162
+
163
+
164
+ ## Unified agentic trace journal
165
+
166
+ The worker keeps Pi session files and stdout logs, but the canonical product trace is the run-level `agent_trace.jsonl` pair. Each record has a `phase` field (`initial_build`, `diagnosis`, `repair_patch`), an `event`, a status, optional artifacts, and redacted data. This lets the UI and bucket artifacts present the whole autonomous loop — build, failure observation, Pi diagnosis, repair decision, optional patch, and revalidation — as one continuous agent run. The redacted version is generated with the same token redaction used elsewhere and is still best-effort, not a formal DLP guarantee.
167
+
168
+
169
+ ## Agent recovery UI
170
+
171
+ The v120 UI makes the core blockage protocol visible in the Active Run panel. When the worker writes `repair/REPAIR_DECISION.json`, `repair/BLOCKAGE.json`, or any recovery events, the **Agent recovery** card appears and summarizes:
172
+
173
+ - the Pi diagnosis action, such as `wait_for_logs`, `factory_rebuild_same_code`, or `patch_code`;
174
+ - failure category, logs quality, and confidence when available;
175
+ - the recovery ladder: diagnose → decide → act → revalidate;
176
+ - evidence and the next step.
177
+
178
+ This is intentionally separate from the activity feed. The activity feed remains an event stream, while Agent recovery explains the current agentic recovery strategy. The Run traces dock exposes the unified raw/redacted `agent_trace.jsonl` plus `REPAIR_DECISION.json`, the repair folder, and `BLOCKAGE.json` when present.
179
+
180
+
181
+ ## v122 stability note
182
+
183
+ The v122 line intentionally rolls back the experimental v121 client-side caching/polling optimizations after real UI testing exposed flicker and broken build/run behavior. The app keeps the v120 Agent recovery UI and the v118/v119 core blockage protocol, but returns to the proven polling and run-selection model for production stability.
184
+
185
+ ## v123 — Critical Job launch + failure timeline fix
186
+
187
+ - Fixed a production-critical HF Job startup failure where the large embedded worker script could make Python fail before startup with `argument list too long`.
188
+ - The worker script is now persisted as `runs/<run_id>/_worker.py` in the mounted run bucket before launch, and the Job receives only `WORKER_SCRIPT_PATH`.
189
+ - Kept a small `WORKER_SCRIPT_B64` fallback only for compatibility/manual launch paths; normal app launches no longer put the worker source in env/argv.
190
+ - Added a failure-log fallback so pre-worker Job failures such as `argument list too long` still create an explicit failed timeline event and red failure point.
NEXT_STEPS.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Next Steps
2
+
3
+ The current application is stable enough for a controlled production/public-preview deployment. The remaining work is mostly product hardening, observability, and cost transparency rather than core workflow repair.
4
+
5
+ ## Recommended before broad launch
6
+
7
+ 1. **Production smoke checklist**
8
+ - Run one build that reaches `full_inference_success` automatically.
9
+ - Run one build that reaches `manual_hardware_required`, then validate after manual hardware selection.
10
+ - Verify raw/redacted trace links, report link, smoke link, Space link, Settings link, Job link, and full Artifacts folder link.
11
+ - Verify billing copy and links with a non-Pro and Pro account if possible.
12
+ - Verify the pre-scan default `Tongyi-MAI/Z-Image-Turbo` returns a positive verdict and still launches successfully.
13
+
14
+ 2. **Cost guardrails**
15
+ - Add optional per-launch confirmation when fixed GPU fallback is enabled.
16
+ - Add clearer copy that Jobs and Inference Providers may consume paid compute.
17
+ - Consider a local user-configurable max validation attempts / max build duration.
18
+
19
+ 3. **Trace safety review**
20
+ - Add regression examples for token-like strings in raw traces and expected redacted output.
21
+ - Keep redacted traces clearly labeled as best-effort.
22
+
23
+ 4. **Repair workflow**
24
+ - Add a user-triggered repair run that reads Space build/runtime logs, asks Pi for a minimal patch, uploads the patch, and re-runs validation.
25
+ - Keep repair runs separate from initial build runs in the Run Explorer.
26
+
27
+ 5. **Validation presets**
28
+ - Add presets per pipeline/task: text-to-image, text-generation, image-to-image, ASR, TTS, embeddings, etc.
29
+ - Prefill endpoint args/kwargs from discovered Gradio schema when confidence is high.
30
+
31
+ 6. **Run retention controls**
32
+ - Add bulk cleanup for old failed validation runs.
33
+ - Add optional retention copy in the Run Stats panel.
34
+
35
+ 7. **Pre-scan quality**
36
+ - Keep adding known-good and known-risk model-card fixtures.
37
+ - Use scan results to recommend implementation mode and validation presets.
38
+ - Add clearer detection for custom Diffusers pipelines that are valid but require special generated code.
39
+
40
+ ## Nice-to-have UI improvements
41
+
42
+ - Add a compact preview for generated output artifacts when the smoke test returns an image/audio/video/file.
43
+ - Consider optional filters for build runs versus validation runs in the compact Run Explorer if histories become large.
44
+ - Add a “failed only” quick filter from Run Stats.
45
+ - Add keyboard shortcuts for refresh and tab switching.
46
+
47
+ ## Current product workflow
48
+
49
+ ```text
50
+ Build from model card
51
+ → private generated Space
52
+ → best-effort ZeroGPU/fixed GPU assignment
53
+ → live health/API checks
54
+ → measured inference latency when live smoke succeeds
55
+ → ZeroGPU duration recommendation when applicable
56
+ → manual hardware selection if required
57
+ → Space Test validation
58
+ → full_inference_success when output type and latency are validated
59
+ ```
60
+
61
+ ### v113 follow-up notes
62
+
63
+ The UI now treats the model pre-scan as the build gate, not the manual Space name. If the user leaves the target Space name empty, the backend generates a safe slug from the run ID. Future work could expose the generated name in the launch confirmation before the Job starts, but it should not block the launch.
64
+
65
+ The Gradio endpoint picker now mirrors the information we can recover from the validation Job's `gradio_client` schema: endpoint name, inputs, outputs, and parameter hints. The action is intentionally named `Set API name` because it copies the selected endpoint into the validation form rather than launching the validation immediately.
66
+
67
+
68
+ ### v114 follow-up notes
69
+
70
+ - The event feed now uses inline SVG icons. A future polish pass can add a full event details drawer, but the compact feed should remain readable and avoid emoji/text marker fallbacks.
71
+
72
+
73
+ ## Repair workflow follow-ups
74
+
75
+ - Add richer recovery analytics from `REPAIR_DECISION.json`, same-code rebuild outcomes, and Gradio client traces.
76
+ - Surface `INCIDENT_BRIEF.md`, `REPAIR_DECISION.json`, `REPAIR_PLAN.md`, and `REPAIR_SUMMARY.md` previews directly in the UI when recovery was attempted.
77
+ - Consider allowing a second recovery action only when the first action clearly changed the evidence category and the new decision is strongly justified.
78
+
79
+
80
+ ### v120 note: Agent recovery visibility
81
+
82
+ The UI now exposes Pi diagnosis/recovery state through the Active Run **Agent recovery** card and links to recovery decision/blockage artifacts. This helps users understand when the Factory is waiting for HF logs, rebuilding the same code, patching, requesting hardware, or declaring a technical blocker.
83
+
84
+
85
+ ## v122 stability note
86
+
87
+ The v122 line intentionally rolls back the experimental v121 client-side caching/polling optimizations after real UI testing exposed flicker and broken build/run behavior. The app keeps the v120 Agent recovery UI and the v118/v119 core blockage protocol, but returns to the proven polling and run-selection model for production stability.
88
+
89
+ ## v123 — Critical Job launch + failure timeline fix
90
+
91
+ - Fixed a production-critical HF Job startup failure where the large embedded worker script could make Python fail before startup with `argument list too long`.
92
+ - The worker script is now persisted as `runs/<run_id>/_worker.py` in the mounted run bucket before launch, and the Job receives only `WORKER_SCRIPT_PATH`.
93
+ - Kept a small `WORKER_SCRIPT_B64` fallback only for compatibility/manual launch paths; normal app launches no longer put the worker source in env/argv.
94
+ - Added a failure-log fallback so pre-worker Job failures such as `argument list too long` still create an explicit failed timeline event and red failure point.
95
+
96
+ ## v124 — Timeline and Run Explorer stabilization
97
+
98
+ - Fixed the compact Overall Progress timeline so dynamically visible repair/failure groups cannot wrap dots onto a second row.
99
+ - Removed smooth timeline auto-scroll that could cause visible flicker during polling.
100
+ - Simplified Run Explorer type/status badges to neutral type labels and short status labels such as Success, Failed, Running and Manual.
101
+ - Added regression checks for timeline single-row behavior and compact Run Explorer labels.
102
+
103
+ ## v127 — Billing usefulness + UI event/timeline audit
104
+
105
+ - Reworked the Billing & compute card into an actionable compute checklist: payment readiness, Pi Inference Providers, build Job cost, generated Space hardware and high-tier GPU guardrails.
106
+ - Kept exact spend/quotas delegated to the Hugging Face Billing dashboard instead of inventing usage totals.
107
+ - Audited JS event bindings for one-shot/delegated listeners across navigation, run rows and timeline scroll.
108
+ - Reduced timeline auto-scroll churn: identical timeline renders no longer force keep-visible scrolling after user interaction.
109
+ - Confirmed run selection keeps the immediate cached/optimistic render path before network hydration.
110
+
PUBLIC_PREVIEW.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Public Preview
2
+
3
+ Agentic Space Factory turns Hugging Face model cards into private, testable Gradio Spaces using Hugging Face Jobs, Pi, Inference Providers, and per-user Storage Buckets.
4
+
5
+ ## What it does
6
+
7
+ - Creates private Spaces from model cards.
8
+ - Runs the build inside an ephemeral Hugging Face Job.
9
+ - Uses Pi with an Inference Providers coding model.
10
+ - Injects the shared GitHub gist as Pi's operational instruction source.
11
+ - Stores state, events, raw/redacted traces, reports, generated files, validation results, and output artifacts in a private user Bucket.
12
+ - Attempts ZeroGPU first when requested, then optional fixed-GPU fallback.
13
+ - Supports manual hardware selection when automatic hardware assignment is unavailable.
14
+ - Runs a separate Space validation job with endpoint discovery, output-type checks, latency measurement, and generated artifact capture.
15
+
16
+ ## Current UI
17
+
18
+ The dashboard is split into three working areas:
19
+
20
+ 1. **Left column** — Run storage setup, model pre-scan, New Build form, Billing & compute readiness.
21
+ 2. **Center tabs** — `Active Run` for build details and `Space Test` for validation details.
22
+ 3. **Right column** — Run Explorer for navigation and Run Stats for aggregate history.
23
+
24
+ The Run Explorer is navigation-only. Build run details live in `Active Run`; validation run details live in `Space Test`.
25
+
26
+ ## Recommended demo path
27
+
28
+ 1. Sign in with Hugging Face OAuth.
29
+ 2. Check or create the private run Bucket.
30
+ 3. Review the default `Tongyi-MAI/Z-Image-Turbo` example or enter another model card, then run the model pre-scan.
31
+ 4. Launch only after the scan returns `safe`/`caution` or after explicitly acknowledging a `risky` verdict.
32
+ 5. Watch `Active Run`: progress timeline, latest events, latency when available, Space/Settings/Artifacts links, and trace documents.
33
+ 6. If hardware is manual, open Space Settings and select the recommended hardware.
34
+ 7. Open `Space Test`, prefill from the build, inspect available Gradio endpoints, click an endpoint to use it, and launch validation.
35
+ 8. Confirm `full_inference_success`, measured latency, and saved output artifact.
36
+
37
+ ## What it does not promise
38
+
39
+ The app does not guarantee that every model card can become a working Space. Some models require large or unusual hardware, Docker, multi-GPU execution, custom kernels, external services, accepted gated licenses, or manual fixes.
40
+
41
+ ## Production-readiness note
42
+
43
+ The app is suitable for a controlled public-preview or internal production deployment when deployed with OAuth enabled, per-user Buckets, private generated Spaces, and clear billing expectations.
44
+
45
+ Before broad public launch, keep monitoring:
46
+
47
+ - Job cost and user quota expectations;
48
+ - token redaction in traces;
49
+ - failed build/validation rates;
50
+ - hardware assignment failure modes;
51
+ - latency and ZeroGPU duration recommendations.
52
+
53
+
54
+ ## Additional preview checklist
55
+
56
+ - Pre-scan a few representative model cards before launching Jobs.
57
+ - Confirm that `risky` models require acknowledgement and `unsupported` models stay blocked.
58
+
59
+ ## UI reliability checks
60
+
61
+ For preview deployments, verify that repeated refreshes do not duplicate click handlers: endpoint chips should update one API field once, run pagination should move one page per click, and run explorer selection/delete should remain independent.
62
+
63
+ - Repair is one bounded automated retry with visible artifacts and no unbounded loop.
64
+
65
+
66
+ ## Recovery behavior
67
+
68
+ If a generated Space fails with HF build/runtime errors, especially empty logs or no-reason runtime errors, the Factory asks Pi to diagnose the blockage before any code patch is allowed. The system may wait for logs, re-upload the same workspace to force a rebuild, patch code once, request manual hardware, or stop with an auditable technical blocker.
69
+
70
+
71
+ ### v120 note: Agent recovery visibility
72
+
73
+ The UI now exposes Pi diagnosis/recovery state through the Active Run **Agent recovery** card and links to recovery decision/blockage artifacts. This helps users understand when the Factory is waiting for HF logs, rebuilding the same code, patching, requesting hardware, or declaring a technical blocker.
74
+
75
+
76
+ ## v122 stability note
77
+
78
+ The v122 line intentionally rolls back the experimental v121 client-side caching/polling optimizations after real UI testing exposed flicker and broken build/run behavior. The app keeps the v120 Agent recovery UI and the v118/v119 core blockage protocol, but returns to the proven polling and run-selection model for production stability.
79
+
RECOVERY_ALGORITHM.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Recovery algorithm
2
+
3
+ Agentic Space Factory uses the Factory as an observer and guardrail layer, and Pi as the repair agent.
4
+
5
+ ## Roles
6
+
7
+ ### Factory
8
+
9
+ The Factory is responsible for deterministic observation and state management:
10
+
11
+ 1. poll the Hugging Face Job and generated Space;
12
+ 2. read `get_space_runtime()` and Space logs;
13
+ 3. detect terminal build/runtime failures instead of waiting blindly;
14
+ 4. collect raw evidence into artifacts;
15
+ 5. give Pi a complete repair context;
16
+ 6. enforce safe action budgets and cooldowns;
17
+ 7. re-upload/revalidate after Pi changes the workspace.
18
+
19
+ The Factory should not solve arbitrary dependency or application bugs by hardcoding fixes.
20
+
21
+ ### Pi
22
+
23
+ Pi is responsible for the actual agentic repair work:
24
+
25
+ 1. read the HF Spaces gist rules;
26
+ 2. read `INCIDENT_BRIEF.md`;
27
+ 3. read `DEPENDENCY_ERROR_BRIEF.md` when present;
28
+ 4. inspect build/runtime logs and the current workspace;
29
+ 5. identify the first actionable error;
30
+ 6. patch the minimal file(s), usually `requirements.txt` for pip failures;
31
+ 7. explain the patch in `REPAIR_SUMMARY.md`;
32
+ 8. let the Factory upload and validate the repaired Space.
33
+
34
+ ## Main recovery flow
35
+
36
+ ```text
37
+ live validation fails or Space enters BUILD_ERROR/RUNTIME_ERROR
38
+ → Factory collects Space runtime + logs
39
+ → Factory writes build_error_observation.json when applicable
40
+ → Factory writes INCIDENT_BRIEF.md
41
+ → Factory asks Pi for diagnosis
42
+ → Pi writes REPAIR_DECISION.json
43
+ → Factory normalizes unsafe decisions
44
+ → if patch_code:
45
+ Pi receives REPAIR_BRIEF.md + logs + gist + workspace
46
+ Pi patches the existing workspace
47
+ Pi writes REPAIR_SUMMARY.md
48
+ Factory uploads and validates
49
+ → if wait_for_logs / factory_rebuild_same_code:
50
+ Factory applies budget/cooldown guardrails
51
+ → if manual_hardware / technical_blocker:
52
+ UI shows explicit action/blocker
53
+ ```
54
+
55
+ ## Dependency errors
56
+
57
+ Dependency errors are not handled by a giant table of hardcoded pins.
58
+
59
+ The Factory only extracts the first useful pip error and writes `DEPENDENCY_ERROR_BRIEF.md`, for example:
60
+
61
+ ```text
62
+ ResolutionImpossible
63
+ Cannot install ...
64
+ No matching distribution found ...
65
+ metadata-generation-failed
66
+ ```
67
+
68
+ Then Pi must apply the gist method:
69
+
70
+ ```text
71
+ read first actionable error
72
+ patch once, surgically
73
+ rebuild
74
+ validate live
75
+ ```
76
+
77
+ ## Safety guarantees
78
+
79
+ - Pi receives `HF_TOKEN` / `HUGGING_FACE_HUB_TOKEN` inside the Job as secrets.
80
+ - Token context is recorded only as redacted metadata in `token_context.json`.
81
+ - The Factory blocks same-code rebuild loops with budgets and cooldowns.
82
+ - The Factory does not continue waiting when the Space is already in `BUILD_ERROR`.
83
+ - Repair phases are part of the same run and unified in trace artifacts.
RELEASE_VALIDATION.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Release Candidate Validation — v133
2
+
3
+ This archive is based on `agentic-space-factory-v132-prescan-output-type-sync` and contains no runtime feature changes beyond this validation report.
4
+
5
+ ## Validation date
6
+
7
+ 2026-06-05
8
+
9
+ ## Automated checks
10
+
11
+ - Pytest suite: `342 passed, 2 warnings`
12
+ - JavaScript syntax:
13
+ - `web/static/app.js`
14
+ - `web/static/runs.js`
15
+ - `web/static/progress.js`
16
+ - `web/static/api.js`
17
+ - `web/static/components.js`
18
+ - Python compile check: passed
19
+ - HTML parse check: passed
20
+
21
+ ## Critical demo-path checks
22
+
23
+ - Job launch uses bucket-mounted worker file via `WORKER_SCRIPT_PATH`, avoiding the previous `argument list too long` failure.
24
+ - Build flow keeps `Qwen/Qwen3-Coder-Next` as the default Pi model.
25
+ - Pi blockage protocol is present:
26
+ - `INCIDENT_BRIEF.md`
27
+ - `PI_DIAGNOSIS_GOAL.md`
28
+ - `REPAIR_DECISION.json`
29
+ - allowed actions such as `wait_for_logs`, `factory_rebuild_same_code`, `patch_code`, `declare_technical_blocker`
30
+ - Unified agent traces are present via `agent_trace.jsonl`.
31
+ - Agent recovery panel is gated and does not show for normal Pi build steps.
32
+ - Run delete is gated until the run is finished, cancelled, failed, blocked, stale, or manual-action.
33
+ - Space Test prefill is gated until validation results exist.
34
+ - Run trace buttons stay disabled until bucket files exist.
35
+ - Overall progress timeline uses product-facing labels while preserving internal step mapping.
36
+ - Space Test endpoint UI no longer shows the confusing `Set API name` banner.
37
+ - Model pre-scan can infer and sync `Expected output type` when clear.
38
+
39
+ ## Manual demo recommendation
40
+
41
+ For the safest live demo, use:
42
+
43
+ - Model: `Tongyi-MAI/Z-Image-Turbo`
44
+ - Expected output: should auto-sync to `Image` after pre-scan
45
+ - Implementation mode: `Strict inference`
46
+ - Hardware: ZeroGPU first, fixed fallback allowed
47
+ - Confirm Billing dashboard before launch
48
+
49
+ ## Known non-blocking warnings
50
+
51
+ The local test environment emits Hugging Face OAuth experimental/local warnings. These are expected outside a Space environment and do not indicate a failing app test.
SECURITY.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Security Notes
2
+
3
+ ## Token model
4
+
5
+ The orchestrator uses Hugging Face OAuth. Do not store a global admin `HF_TOKEN` in the Space.
6
+
7
+ The signed-in user's OAuth token is passed to Hugging Face Jobs as a secret named `HF_TOKEN`. The worker must not write this token to the Bucket, traces, reports, generated files, events, or logs.
8
+
9
+ ## Resource ownership
10
+
11
+ Generated Spaces are constrained to the signed-in user's namespace and are private by default.
12
+
13
+ Validation is limited to Spaces under the signed-in user's namespace. This prevents one user from using the validation workflow to probe arbitrary Spaces.
14
+
15
+ ## Storage Buckets
16
+
17
+ Each user should use their own private `space-factory-runs` Bucket. Run artifacts can include prompts, generated code, logs, traces, model IDs, endpoint schemas, and output samples.
18
+
19
+ Bucket links in the UI point to `tree/runs/<run_id>/...` paths. Raw run folders remain available through the `Artifacts` button, while the UI exposes a smaller trace/document dock for common review files.
20
+
21
+ ## Traces
22
+
23
+ Pi traces are stored in two forms:
24
+
25
+ - `traces/raw/` — full internal trace. Treat as private.
26
+ - `traces/redacted/` — best-effort redacted trace. Useful for review, but not a formal DLP guarantee.
27
+
28
+ Never assume redacted traces are safe for public release without review.
29
+
30
+ ## Billing and compute
31
+
32
+ The product launches Hugging Face Jobs and uses Inference Providers. Both can consume paid compute depending on the user's account, plan, quota, and selected hardware.
33
+
34
+ The app can display payment/billing readiness signals and links to HF Billing, but does not mirror live spend/quota totals. Users should review live usage in the Hugging Face Billing dashboard.
35
+
36
+ ## Generated Spaces
37
+
38
+ The app never publishes generated Spaces automatically. Users should review:
39
+
40
+ - generated code;
41
+ - model license and gated access requirements;
42
+ - dependency and hardware requirements;
43
+ - validation results;
44
+ - raw/redacted traces;
45
+ - any output artifacts;
46
+
47
+ before making a generated Space public.
48
+
49
+ ## Operational recommendations
50
+
51
+ For production or public-preview deployment:
52
+
53
+ - keep generated Spaces private by default;
54
+ - keep per-user Bucket isolation;
55
+ - monitor Job failures and cost surprises;
56
+ - periodically review redaction coverage;
57
+ - avoid broad admin tokens;
58
+ - document that automatic hardware assignment is best-effort.
59
+
60
+
61
+ ## Model pre-scan security boundary
62
+
63
+ The fast pre-scan is a risk heuristic, not a full sandbox audit. It reads metadata and small text/config files from the Hub without downloading weights. It can flag obvious concerns such as missing model-card guidance, custom-code indicators, unsafe weight formats, gated/private access, or unsupported tasks, but it does not prove that a model or generated Space is safe.
64
+
65
+ Treat `safe` as “good enough to launch the controlled build workflow”, not as a security certification.
66
+
67
+ - Repair artifacts may include log excerpts. Raw traces and repair briefs should be treated as internal unless reviewed/redacted.
68
+
69
+
70
+ ## Recovery and repair safety
71
+
72
+ Recovery is diagnosis-first. Pi writes `REPAIR_DECISION.json` before any patch is allowed, and the Factory refuses speculative code repair for empty/no-reason logs. In Strict inference mode, patch repair still runs sanity checks to prevent obvious placeholder or fake-inference substitutions. Repair, incident and blockage artifacts may contain log excerpts and should be treated as internal unless reviewed and redacted.
73
+
74
+
75
+ ### v120 note: Agent recovery visibility
76
+
77
+ The UI now exposes Pi diagnosis/recovery state through the Active Run **Agent recovery** card and links to recovery decision/blockage artifacts. This helps users understand when the Factory is waiting for HF logs, rebuilding the same code, patching, requesting hardware, or declaring a technical blocker.