fffiloni commited on
Commit
abc6cbe
·
verified ·
1 Parent(s): c98f6cd

Upload 6 files

Browse files
Files changed (3) hide show
  1. CHANGELOG.md +20 -6
  2. README.md +10 -6
  3. app.py +125 -3
CHANGELOG.md CHANGED
@@ -1,23 +1,37 @@
1
- # v190.24Replay uses parent smoke artefact as source of truth
2
 
3
- ## v190.24 Linked validation replay hardening
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
  - In Space Test replay mode, the validation worker now loads the parent Build Run's successful automatic smoke artefacts (`generation_smoke.json`, `generation_smoke_payload*.json`) as the canonical request before schema reconstruction.
6
  - Added `replay_source.json` and richer `resolved_validation_request.json` / failure diagnosis fields so replay failures can be compared against the parent smoke request.
7
  - The Payload stage now records `Payload replayed from parent automatic smoke` instead of remaining pending when replay data exists.
8
 
9
- # v190.24 — Linked validation worker schema coercion and replay fix
10
 
11
- ## v190.24 — Validation worker reliability
12
 
13
  - Normalizes linked validation replay payloads against the current Gradio endpoint schema before calling predict().
14
  - Preserves typed Gradio choices and converts dropdown/radio numeric-looking args such as 1024 to schema strings such as "1024" when required.
15
  - Writes validation_preflight.json, resolved_validation_request.json, schema_coercion.json, and validation_failure_diagnosis.json for every linked validation run.
16
  - Keeps Space Test copy linked-only while explaining that the separate validation run remains attached to the selected Build Run.
17
 
18
- # v190.24 — UI/CSS convergence and Space Test validation polish
19
 
20
- ## v190.24 — Space Test validation reliability
21
 
22
  - Treats Space Test Validation as a first-class linked validation workflow.
23
  - Adds a single resolved launch payload contract so replay/complete/recover/discovery-first modes only launch when coherent.
 
1
+ ## v190.28Runs Explorer linked validation parent metadata
2
 
3
+ - Build Run cards now show linked/manual validation strips from parent metadata (`manual_validation_status` and `linked_validations`) even when the child Validation Run is not present in the Runs Explorer cache.
4
+ - Runs Explorer remains build-first: linked validations stay summarized on their parent Build Run, while automatic Build Run smoke results remain separate from manual linked validation state.
5
+
6
+
7
+ ## v190.28 — Space Test payload provenance and prefill correctness
8
+
9
+ - Uses automatic smoke artefacts as the visible Space Test prefill source when available.
10
+ - Clears stale dirty validation args when a Build Run is selected from Runs Explorer.
11
+ - Sends and stores a backend validation launch payload for linked validations.
12
+ - Writes validation_launch_payload.json in validation runs before the worker resolves/predicts.
13
+ - Surfaces payload source provenance across UI, backend launch metadata, and worker artefacts.
14
+
15
+ # v190.28 — Replay uses parent smoke artefact as source of truth
16
+
17
+ ## v190.28 — Linked validation replay hardening
18
 
19
  - In Space Test replay mode, the validation worker now loads the parent Build Run's successful automatic smoke artefacts (`generation_smoke.json`, `generation_smoke_payload*.json`) as the canonical request before schema reconstruction.
20
  - Added `replay_source.json` and richer `resolved_validation_request.json` / failure diagnosis fields so replay failures can be compared against the parent smoke request.
21
  - The Payload stage now records `Payload replayed from parent automatic smoke` instead of remaining pending when replay data exists.
22
 
23
+ # v190.28 — Linked validation worker schema coercion and replay fix
24
 
25
+ ## v190.28 — Validation worker reliability
26
 
27
  - Normalizes linked validation replay payloads against the current Gradio endpoint schema before calling predict().
28
  - Preserves typed Gradio choices and converts dropdown/radio numeric-looking args such as 1024 to schema strings such as "1024" when required.
29
  - Writes validation_preflight.json, resolved_validation_request.json, schema_coercion.json, and validation_failure_diagnosis.json for every linked validation run.
30
  - Keeps Space Test copy linked-only while explaining that the separate validation run remains attached to the selected Build Run.
31
 
32
+ # v190.28 — UI/CSS convergence and Space Test validation polish
33
 
34
+ ## v190.28 — Space Test validation reliability
35
 
36
  - Treats Space Test Validation as a first-class linked validation workflow.
37
  - Adds a single resolved launch payload contract so replay/complete/recover/discovery-first modes only launch when coherent.
README.md CHANGED
@@ -13,13 +13,17 @@ hf_oauth_scopes:
13
  - read-billing
14
  ---
15
 
16
- ## v190.24 Replay uses parent smoke artefact as source of truth
17
 
18
- - v190.24 makes linked Space Test replay mode load the parent Build Run's successful automatic smoke artefacts before attempting any schema reconstruction. The validation worker writes `replay_source.json`, `resolved_validation_request.json`, and diagnostics so a replay cannot fail opaquely at the Payload stage.
19
 
20
- ## v190.24 Linked validation hardening
21
 
22
- - v190.24 hardens linked Space Test success/failure paths: parent Build Run updates read API/output metadata from the smoke result, failed manual attempts cannot downgrade an already validated parent, and latency/ZeroGPU recommendations expose their linked-test hardware provenance.
 
 
 
 
23
  - Runs Explorer and Run Stats now treat automatic and effective statuses explicitly so manually validated partial builds remain discoverable in both Partial and Validated views.
24
  - Space Test is contextualized from the selected Build Run even before pressing Prefill, while remaining linked-only for launches.
25
 
@@ -419,13 +423,13 @@ The Active Run progress header now separates pipeline completion from product ve
419
 
420
  - v190.9 makes terminal Space Test validation status survive page reloads. Failed, success, partial, and manual validation outcomes override stale `running` launch metadata in the Runs Explorer, delete modal, and progress polling.
421
 
422
- ## v190.24 Eval archive effective outcomes
423
 
424
  - Eval archive records now distinguish automatic verdicts from effective verdicts after linked Space Test validation.
425
  - Archived records include anonymized linked-validation, endpoint-discovery and smoke-retry metrics while preserving privacy constraints.
426
  - Instance owners can flush all archived eval records from the mounted eval bucket without deleting the eval archive configuration.
427
 
428
- ## v190.24 Linked validation replay reliability
429
 
430
  - Linked Space Test validations now normalize replay/complete/recover payloads against the current Gradio schema before predict().
431
  - Validation runs write explicit preflight, resolved request, schema coercion, and failure diagnosis artifacts for debugging.
 
13
  - read-billing
14
  ---
15
 
16
+ ## v190.28 Runs Explorer linked validation parent metadata
17
 
18
+ - v190.28 makes Build Run cards display manual linked validation state directly from parent metadata (`manual_validation_status.json` / `linked_validations.json`), so the summary is visible even when linked Validation Runs are not listed as standalone rows. Automatic smoke status remains separate from manual linked validation status.
19
 
20
+ ## v190.28 Replay uses parent smoke artefact as source of truth
21
 
22
+ - v190.28 makes linked Space Test replay mode load the parent Build Run's successful automatic smoke artefacts before attempting any schema reconstruction. The validation worker writes `replay_source.json`, `resolved_validation_request.json`, and diagnostics so a replay cannot fail opaquely at the Payload stage.
23
+
24
+ ## v190.28 Linked validation hardening
25
+
26
+ - v190.28 hardens linked Space Test success/failure paths: parent Build Run updates read API/output metadata from the smoke result, failed manual attempts cannot downgrade an already validated parent, and latency/ZeroGPU recommendations expose their linked-test hardware provenance.
27
  - Runs Explorer and Run Stats now treat automatic and effective statuses explicitly so manually validated partial builds remain discoverable in both Partial and Validated views.
28
  - Space Test is contextualized from the selected Build Run even before pressing Prefill, while remaining linked-only for launches.
29
 
 
423
 
424
  - v190.9 makes terminal Space Test validation status survive page reloads. Failed, success, partial, and manual validation outcomes override stale `running` launch metadata in the Runs Explorer, delete modal, and progress polling.
425
 
426
+ ## v190.28 Eval archive effective outcomes
427
 
428
  - Eval archive records now distinguish automatic verdicts from effective verdicts after linked Space Test validation.
429
  - Archived records include anonymized linked-validation, endpoint-discovery and smoke-retry metrics while preserving privacy constraints.
430
  - Instance owners can flush all archived eval records from the mounted eval bucket without deleting the eval archive configuration.
431
 
432
+ ## v190.28 Linked validation replay reliability
433
 
434
  - Linked Space Test validations now normalize replay/complete/recover payloads against the current Gradio schema before predict().
435
  - Validation runs write explicit preflight, resolved request, schema coercion, and failure diagnosis artifacts for debugging.
app.py CHANGED
@@ -102,6 +102,91 @@ def _json_error(exc: Exception) -> HTTPException:
102
 
103
 
104
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  def _job_url_from_result(result: dict[str, Any], username: str | None) -> str:
106
  explicit = (result.get("job_url") or "").strip() if isinstance(result.get("job_url"), str) else result.get("job_url")
107
  if explicit:
@@ -570,8 +655,39 @@ def register_custom_routes(fastapi_app: FastAPI) -> None:
570
  if not space_test_policy.get("enabled"):
571
  raise ValueError(str(space_test_policy.get("message") or "This Build Run is not eligible for linked Space Test yet."))
572
  api_name_for_validation = payload.get("api_name")
573
- if space_test_policy.get("requires_endpoint_discovery") and str(api_name_for_validation or "").strip() in {"", "/generate"}:
 
 
 
 
 
 
 
 
 
 
 
 
574
  api_name_for_validation = ""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
575
  result = launch_validate_existing_space_job(
576
  token=ctx["token"],
577
  username=ctx["username"],
@@ -580,10 +696,12 @@ def register_custom_routes(fastapi_app: FastAPI) -> None:
580
  api_name=api_name_for_validation,
581
  test_args_json=json.dumps(test_args, ensure_ascii=False),
582
  test_kwargs_json=json.dumps(test_kwargs, ensure_ascii=False),
583
- expected_output_type=payload.get("expected_output_type"),
584
  live_timeout_seconds=payload.get("live_timeout_seconds") or 1800,
585
- validation_mode=space_test_policy.get("mode"),
586
  effective_status_on_success=space_test_policy.get("effective_status_on_success") or "validated_after_manual_space_test",
 
 
587
  run_id=payload.get("run_id"),
588
  bucket_name=bucket_name,
589
  )
@@ -880,6 +998,10 @@ def register_custom_routes(fastapi_app: FastAPI) -> None:
880
  "generation_smoke": bundle.get("generation_smoke") or {},
881
  "api_schema": bundle.get("api_schema") or {},
882
  "validation_payload": bundle.get("validation_payload") or {},
 
 
 
 
883
  "hardware_strategy": bundle.get("hardware_strategy") or {},
884
  "technical_blockers": bundle.get("technical_blockers") or {},
885
  "repair_decision": bundle.get("repair_decision") or {},
 
102
 
103
 
104
 
105
+
106
+
107
+ def _first_list_value(*values: Any) -> list[Any] | None:
108
+ for value in values:
109
+ if isinstance(value, list):
110
+ return value
111
+ return None
112
+
113
+
114
+ def _first_dict_value(*values: Any) -> dict[str, Any] | None:
115
+ for value in values:
116
+ if isinstance(value, dict):
117
+ return value
118
+ return None
119
+
120
+
121
+ def _linked_replay_source_from_parent_bundle(parent_run_id: str, parent_bundle: dict[str, Any], target_space: str, expected_output_type: str | None = None) -> dict[str, Any]:
122
+ """Resolve the parent automatic smoke request server-side for replay Jobs.
123
+
124
+ The validation worker runs in a fresh Job and cannot assume the parent run
125
+ folder is present locally. Passing this compact, redacted source through the
126
+ Job environment prevents linked replay from silently falling back to schema
127
+ reconstruction when the parent already proved a working request.
128
+ """
129
+ smoke = parent_bundle.get("generation_smoke") or {}
130
+ retry_payload = parent_bundle.get("generation_smoke_payload_retry") or {}
131
+ initial_payload = parent_bundle.get("generation_smoke_payload") or {}
132
+ resolved = parent_bundle.get("resolved_validation_request") or {}
133
+ state_obj = parent_bundle.get("state") or {}
134
+ state_smoke = state_obj.get("generation_smoke") if isinstance(state_obj.get("generation_smoke"), dict) else {}
135
+ status_tokens = {
136
+ str(smoke.get("status") or "").lower(),
137
+ str(state_obj.get("status") or "").lower(),
138
+ str(state_smoke.get("status") or "").lower(),
139
+ }
140
+ parent_was_successful = bool(status_tokens & {"success", "full_inference_success", "completed", "complete", "done"})
141
+ if not parent_was_successful:
142
+ return {}
143
+ api_name = (
144
+ smoke.get("api_name")
145
+ or retry_payload.get("api_name")
146
+ or initial_payload.get("api_name")
147
+ or resolved.get("api_name")
148
+ or state_smoke.get("api_name")
149
+ or ""
150
+ )
151
+ test_args = _first_list_value(
152
+ smoke.get("effective_args"),
153
+ smoke.get("test_args"),
154
+ retry_payload.get("test_args"),
155
+ retry_payload.get("effective_args"),
156
+ initial_payload.get("test_args"),
157
+ initial_payload.get("effective_args"),
158
+ resolved.get("test_args"),
159
+ state_smoke.get("effective_args"),
160
+ state_smoke.get("test_args"),
161
+ )
162
+ test_kwargs = _first_dict_value(
163
+ smoke.get("effective_kwargs"),
164
+ smoke.get("test_kwargs"),
165
+ retry_payload.get("test_kwargs"),
166
+ retry_payload.get("effective_kwargs"),
167
+ initial_payload.get("test_kwargs"),
168
+ initial_payload.get("effective_kwargs"),
169
+ resolved.get("test_kwargs"),
170
+ state_smoke.get("effective_kwargs"),
171
+ state_smoke.get("test_kwargs"),
172
+ ) or {}
173
+ if not api_name or not isinstance(test_args, list):
174
+ return {}
175
+ parent_target = smoke.get("target_space") or state_obj.get("target_space") or state_smoke.get("target_space") or target_space or ""
176
+ return {
177
+ "source": "parent_automatic_smoke_backend",
178
+ "parent_run_id": parent_run_id,
179
+ "parent_target_space": parent_target,
180
+ "target_matches": not parent_target or parent_target == target_space,
181
+ "api_name": str(api_name if str(api_name).startswith("/") else f"/{api_name}"),
182
+ "test_args": test_args,
183
+ "test_kwargs": test_kwargs,
184
+ "expected_output_type": smoke.get("expected_output_type") or state_smoke.get("expected_output_type") or expected_output_type or "any",
185
+ "latency_seconds": smoke.get("latency_seconds") or state_smoke.get("latency_seconds"),
186
+ "parent_smoke_status": smoke.get("status") or state_smoke.get("status") or state_obj.get("status"),
187
+ "transport": "job_env",
188
+ }
189
+
190
  def _job_url_from_result(result: dict[str, Any], username: str | None) -> str:
191
  explicit = (result.get("job_url") or "").strip() if isinstance(result.get("job_url"), str) else result.get("job_url")
192
  if explicit:
 
655
  if not space_test_policy.get("enabled"):
656
  raise ValueError(str(space_test_policy.get("message") or "This Build Run is not eligible for linked Space Test yet."))
657
  api_name_for_validation = payload.get("api_name")
658
+ replay_source: dict[str, Any] = {}
659
+ if str(space_test_policy.get("mode") or payload.get("validation_mode") or "").strip().lower() == "replay":
660
+ replay_source = _linked_replay_source_from_parent_bundle(
661
+ parent_build_run_id,
662
+ parent_bundle,
663
+ requested_target,
664
+ str(payload.get("expected_output_type") or "any"),
665
+ )
666
+ if replay_source:
667
+ api_name_for_validation = replay_source.get("api_name") or api_name_for_validation
668
+ test_args = replay_source.get("test_args") if isinstance(replay_source.get("test_args"), list) else test_args
669
+ test_kwargs = replay_source.get("test_kwargs") if isinstance(replay_source.get("test_kwargs"), dict) else test_kwargs
670
+ if space_test_policy.get("requires_endpoint_discovery") and str(api_name_for_validation or "").strip() in {"", "/generate"} and not replay_source:
671
  api_name_for_validation = ""
672
+ validation_launch_payload = {
673
+ "schema_version": "1.0",
674
+ "app_version": settings.app_version,
675
+ "parent_build_run_id": parent_build_run_id,
676
+ "target_space_id": requested_target,
677
+ "api_name": api_name_for_validation or "",
678
+ "expected_output_type": payload.get("expected_output_type") or replay_source.get("expected_output_type") or "any",
679
+ "test_args": test_args,
680
+ "test_kwargs": test_kwargs,
681
+ "validation_mode": space_test_policy.get("mode") or payload.get("validation_mode") or "complete",
682
+ "payload_source": payload.get("payload_source") or ("parent_automatic_smoke_backend" if replay_source else "ui_payload"),
683
+ "replay_source_present": bool(replay_source),
684
+ "ui_payload_source": payload.get("payload_source") or "",
685
+ "space_test_policy": {
686
+ "mode": space_test_policy.get("mode"),
687
+ "requires_endpoint_discovery": bool(space_test_policy.get("requires_endpoint_discovery")),
688
+ "effective_status_on_success": space_test_policy.get("effective_status_on_success"),
689
+ },
690
+ }
691
  result = launch_validate_existing_space_job(
692
  token=ctx["token"],
693
  username=ctx["username"],
 
696
  api_name=api_name_for_validation,
697
  test_args_json=json.dumps(test_args, ensure_ascii=False),
698
  test_kwargs_json=json.dumps(test_kwargs, ensure_ascii=False),
699
+ expected_output_type=validation_launch_payload["expected_output_type"],
700
  live_timeout_seconds=payload.get("live_timeout_seconds") or 1800,
701
+ validation_mode=validation_launch_payload["validation_mode"], # effectively: validation_mode=space_test_policy.get("mode")
702
  effective_status_on_success=space_test_policy.get("effective_status_on_success") or "validated_after_manual_space_test",
703
+ parent_replay_source_json=json.dumps(replay_source, ensure_ascii=False) if replay_source else None,
704
+ validation_launch_payload_json=json.dumps(validation_launch_payload, ensure_ascii=False),
705
  run_id=payload.get("run_id"),
706
  bucket_name=bucket_name,
707
  )
 
998
  "generation_smoke": bundle.get("generation_smoke") or {},
999
  "api_schema": bundle.get("api_schema") or {},
1000
  "validation_payload": bundle.get("validation_payload") or {},
1001
+ "validation_failure_diagnosis": bundle.get("validation_failure_diagnosis") or {},
1002
+ "payload_source": bundle.get("payload_source") or {},
1003
+ "validation_engine": bundle.get("validation_engine") or {},
1004
+ "resolved_validation_request": bundle.get("resolved_validation_request") or {},
1005
  "hardware_strategy": bundle.get("hardware_strategy") or {},
1006
  "technical_blockers": bundle.get("technical_blockers") or {},
1007
  "repair_decision": bundle.get("repair_decision") or {},