Spaces:
Running
feat(migration): Phase B3 résiduel — migrer CLI/Web vers RunOrchestrator
Browse filesPhase B3 résiduel du chantier Option B (mai 2026). Les call sites
publics CLI et Web pointent maintenant sur ``RunOrchestrator`` via le
shim de compatibilité ``run_via_orchestrator``. Aucun appel actif à
``run_benchmark_via_service`` ne subsiste dans ``picarones/`` (hors
module legacy lui-même). **Checkpoint C2 atteint.**
picarones/app/services/legacy_runner_compat.py (nouveau, promu depuis tests/)
- ``run_via_orchestrator(corpus, engines, **kwargs)`` :
drop-in remplacement de ``run_benchmark_via_service`` qui s'appuie
sur ``RunOrchestrator.execute_preset`` en interne. Préserve la
signature et le retour ``BenchmarkResult`` legacy.
- Convertit ``Corpus`` legacy → ``CorpusSpec`` (couche 1) via
``corpus_to_corpus_spec``, instances ``BaseOCRAdapter`` →
``PipelineSpec`` via ``engine_to_pipeline_spec``,
``build_adapter_resolver`` pour le resolver.
- Le shim absorbe les différences de convention entre legacy
(``NormalizationProfile`` objet, ``char_exclude`` frozenset,
``entity_extractor`` callable) et RunSpec (string + dotted path).
- Retrait prévu Phase B8 — les callers devront construire un
``RunSpec`` directement.
tests/_migration_helpers.py — devient un simple re-export
- L'implémentation a déménagé en production ; ce module reste pour
préserver les imports des 6 fichiers de tests migrés en B4.
picarones/interfaces/web/benchmark_utils.py
- ``run_benchmark_thread_v2`` appelle désormais
``run_via_orchestrator`` au lieu de ``run_benchmark_via_service``.
- Pas de changement de comportement utilisateur.
picarones/interfaces/cli/_workflows.py
- 6 commandes migrées : ``run``, ``diagnose``, ``economics``,
``edition``, ``compare``, ``robustness`` (bulk sed).
- Sortie CLI identique (BenchmarkResult retourné par le shim).
Tests : suite intégrale verte (4872+ passed). Plus aucune
DeprecationWarning émise par les chemins de production — seuls
``test_migration_invariance.py`` (par design) et
``test_public_api.py::test_run_benchmark_via_service_still_callable_with_warning``
(par design) la déclenchent volontairement.
État du chantier :
- ✓ Checkpoint C1 (RunOrchestrator feature-complete)
- ✓ Checkpoint C2 (CLI/Web migrés, tests migrés, deprecation
warning active sur le legacy)
- À venir : B5 Tesseract ALTO, B6 rapport HTML multi-vues,
B7 deprecation finale, B8 suppression -1500 LOC nets.
|
@@ -0,0 +1,257 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Shim de compatibilité ``run_benchmark_via_service`` → ``RunOrchestrator``.
|
| 2 |
+
|
| 3 |
+
Phase B3 (résiduel) du chantier Option B (mai 2026). Fournit
|
| 4 |
+
``run_via_orchestrator()`` comme drop-in remplacement de
|
| 5 |
+
``run_benchmark_via_service`` qui s'appuie sur
|
| 6 |
+
``RunOrchestrator.execute_preset()`` en interne mais préserve la
|
| 7 |
+
signature legacy et le retour ``BenchmarkResult``.
|
| 8 |
+
|
| 9 |
+
Utilisé par
|
| 10 |
+
-----------
|
| 11 |
+
- ``picarones.interfaces.cli._workflows`` (commandes ``run``,
|
| 12 |
+
``diagnose``, ``economics``, ``edition``, ``compare``,
|
| 13 |
+
``robustness``).
|
| 14 |
+
- ``picarones.interfaces.web.benchmark_utils.run_benchmark_thread_v2``.
|
| 15 |
+
- Les tests catégorie A migrés en Phase B4 (via le re-export
|
| 16 |
+
``tests._migration_helpers``).
|
| 17 |
+
|
| 18 |
+
Pourquoi un shim dédié et pas un alias direct
|
| 19 |
+
---------------------------------------------
|
| 20 |
+
``RunOrchestrator.execute_preset()`` consomme des objets domain pré-
|
| 21 |
+
construits (``CorpusSpec`` couche 1, ``PipelineSpec`` couche 1). Les
|
| 22 |
+
callers legacy (CLI/Web/tests) manipulent toujours :
|
| 23 |
+
|
| 24 |
+
- ``Corpus`` legacy (couche 3) avec ``Document.image_path``,
|
| 25 |
+
``ground_truth`` in-memory.
|
| 26 |
+
- Liste d'instances ``BaseOCRAdapter`` / ``OCRLLMPipelineConfig``.
|
| 27 |
+
|
| 28 |
+
Ce shim convertit ces structures en objets domain via les helpers
|
| 29 |
+
existants (``corpus_to_corpus_spec``, ``engine_to_pipeline_spec``,
|
| 30 |
+
``build_adapter_resolver``), puis appelle
|
| 31 |
+
``RunOrchestrator.execute_preset()``. Sortie : ``BenchmarkResult``
|
| 32 |
+
legacy via ``run_result_to_benchmark_result``.
|
| 33 |
+
|
| 34 |
+
Retrait prévu
|
| 35 |
+
-------------
|
| 36 |
+
Phase B8 (post-deprecation release). Quand ``run_benchmark_via_service``
|
| 37 |
+
sera supprimé, ce shim aussi — les callers devront construire leurs
|
| 38 |
+
``RunSpec`` directement (pattern utilisateur documenté dans
|
| 39 |
+
``docs/migration/option_b_user_guide.md``).
|
| 40 |
+
"""
|
| 41 |
+
|
| 42 |
+
from __future__ import annotations
|
| 43 |
+
|
| 44 |
+
import tempfile
|
| 45 |
+
from pathlib import Path
|
| 46 |
+
from typing import TYPE_CHECKING, Any, Callable
|
| 47 |
+
|
| 48 |
+
if TYPE_CHECKING:
|
| 49 |
+
from picarones.evaluation.benchmark_result import BenchmarkResult
|
| 50 |
+
from picarones.evaluation.corpus import Corpus
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def _dummy_pipeline_yaml(name: str = "preset_pipeline") -> Any:
|
| 54 |
+
"""Construit un ``PipelineSpecYaml`` minimaliste pour satisfaire
|
| 55 |
+
le validator ``RunSpec.pipelines`` (min_length=1).
|
| 56 |
+
|
| 57 |
+
Le contenu est **ignoré** par ``execute_preset()`` qui utilise les
|
| 58 |
+
``pipeline_specs`` fournis en kwargs. Le YAML dummy sert
|
| 59 |
+
uniquement à passer la validation Pydantic.
|
| 60 |
+
"""
|
| 61 |
+
from picarones.app.schemas.run_spec import PipelineSpecYaml, StepSpec
|
| 62 |
+
from picarones.domain.artifacts import ArtifactType
|
| 63 |
+
return PipelineSpecYaml(
|
| 64 |
+
name=name,
|
| 65 |
+
initial_inputs=(ArtifactType.IMAGE,),
|
| 66 |
+
steps=(StepSpec(
|
| 67 |
+
id="ocr",
|
| 68 |
+
adapter_class="picarones.app.services.legacy_runner_compat.IgnoredByPreset",
|
| 69 |
+
adapter_kwargs={},
|
| 70 |
+
input_types=(ArtifactType.IMAGE,),
|
| 71 |
+
output_types=(ArtifactType.RAW_TEXT,),
|
| 72 |
+
),),
|
| 73 |
+
)
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def run_via_orchestrator(
|
| 77 |
+
corpus: "Corpus",
|
| 78 |
+
engines: list[Any],
|
| 79 |
+
*,
|
| 80 |
+
char_exclude: Any | None = None,
|
| 81 |
+
normalization_profile: Any | None = None,
|
| 82 |
+
output_json: str | Path | None = None,
|
| 83 |
+
code_version: str | None = None,
|
| 84 |
+
show_progress: bool = True, # noqa: ARG001 — absorbé pour compat
|
| 85 |
+
progress_callback: Callable[[str, int, str], None] | None = None,
|
| 86 |
+
timeout_seconds: float = 60.0,
|
| 87 |
+
cancel_event: Any | None = None,
|
| 88 |
+
partial_dir: str | Path | None = None,
|
| 89 |
+
entity_extractor: Callable[[str], list[dict]] | str | None = None,
|
| 90 |
+
profile: str = "standard",
|
| 91 |
+
) -> "BenchmarkResult":
|
| 92 |
+
"""Drop-in remplacement de ``run_benchmark_via_service`` via
|
| 93 |
+
``RunOrchestrator.execute_preset()``.
|
| 94 |
+
|
| 95 |
+
Préserve la signature legacy pour permettre la migration mécanique
|
| 96 |
+
des call sites (CLI, web, tests). Retourne un ``BenchmarkResult``
|
| 97 |
+
construit via le converter ``run_result_to_benchmark_result``.
|
| 98 |
+
|
| 99 |
+
Parameters
|
| 100 |
+
----------
|
| 101 |
+
corpus, engines:
|
| 102 |
+
Identiques à ``run_benchmark_via_service``.
|
| 103 |
+
char_exclude, normalization_profile, output_json, code_version,
|
| 104 |
+
show_progress, progress_callback, timeout_seconds, cancel_event,
|
| 105 |
+
partial_dir, entity_extractor, profile:
|
| 106 |
+
Identiques à ``run_benchmark_via_service``.
|
| 107 |
+
|
| 108 |
+
Notes
|
| 109 |
+
-----
|
| 110 |
+
Quelques différences subtiles vs le legacy :
|
| 111 |
+
|
| 112 |
+
- ``entity_extractor`` accepte un callable direct (legacy) OU un
|
| 113 |
+
dotted path string (RunSpec). Si callable, on l'invoque
|
| 114 |
+
directement en post-process sur le ``BenchmarkResult``.
|
| 115 |
+
- Le workspace temporaire est nettoyé automatiquement via
|
| 116 |
+
``TemporaryDirectory`` — ne pas s'attendre à des fichiers
|
| 117 |
+
résiduels après l'appel.
|
| 118 |
+
- ``normalization_profile`` accepte un objet ``NormalizationProfile``
|
| 119 |
+
(legacy) OU un nom string (RunSpec). Conversion automatique.
|
| 120 |
+
"""
|
| 121 |
+
from picarones.app.schemas.run_spec import RunSpec
|
| 122 |
+
from picarones.app.services._benchmark_adapter_resolver import (
|
| 123 |
+
build_adapter_resolver,
|
| 124 |
+
engine_to_pipeline_spec,
|
| 125 |
+
)
|
| 126 |
+
from picarones.app.services._benchmark_converter import (
|
| 127 |
+
run_result_to_benchmark_result,
|
| 128 |
+
)
|
| 129 |
+
from picarones.app.services._benchmark_conversions import (
|
| 130 |
+
corpus_to_corpus_spec,
|
| 131 |
+
)
|
| 132 |
+
from picarones.app.services.run_orchestrator import RunOrchestrator
|
| 133 |
+
|
| 134 |
+
# Résolution code_version (cohérent avec run_benchmark_via_service:219).
|
| 135 |
+
if code_version is None:
|
| 136 |
+
import importlib
|
| 137 |
+
try:
|
| 138 |
+
code_version = importlib.import_module("picarones").__version__
|
| 139 |
+
except (ImportError, AttributeError):
|
| 140 |
+
code_version = "unknown"
|
| 141 |
+
|
| 142 |
+
# ``normalization_profile`` legacy accepte un objet
|
| 143 |
+
# NormalizationProfile. RunSpec attend une string. On convertit.
|
| 144 |
+
norm_profile_str = normalization_profile
|
| 145 |
+
if normalization_profile is not None and not isinstance(
|
| 146 |
+
normalization_profile, str,
|
| 147 |
+
):
|
| 148 |
+
norm_profile_str = getattr(normalization_profile, "name", None)
|
| 149 |
+
|
| 150 |
+
# ``entity_extractor`` legacy accepte un callable direct. RunSpec
|
| 151 |
+
# attend un dotted path. Si callable, on le traite post-process
|
| 152 |
+
# comme run_benchmark_via_service le fait.
|
| 153 |
+
entity_extractor_dotted: str | None = None
|
| 154 |
+
entity_extractor_callable: Callable | None = None
|
| 155 |
+
if entity_extractor is not None:
|
| 156 |
+
if isinstance(entity_extractor, str):
|
| 157 |
+
entity_extractor_dotted = entity_extractor
|
| 158 |
+
elif callable(entity_extractor):
|
| 159 |
+
entity_extractor_callable = entity_extractor
|
| 160 |
+
|
| 161 |
+
with tempfile.TemporaryDirectory(prefix="picarones_compat_") as ws:
|
| 162 |
+
ws_path = Path(ws)
|
| 163 |
+
gt_dir = ws_path / "gt"
|
| 164 |
+
gt_dir.mkdir()
|
| 165 |
+
run_dir = ws_path / "run"
|
| 166 |
+
run_dir.mkdir()
|
| 167 |
+
|
| 168 |
+
corpus_spec = corpus_to_corpus_spec(corpus, workspace_dir=gt_dir)
|
| 169 |
+
pipeline_specs = [engine_to_pipeline_spec(e) for e in engines]
|
| 170 |
+
adapter_resolver = build_adapter_resolver(engines)
|
| 171 |
+
pipeline_to_engine_name = {
|
| 172 |
+
spec.name: engine.name
|
| 173 |
+
for spec, engine in zip(pipeline_specs, engines)
|
| 174 |
+
}
|
| 175 |
+
|
| 176 |
+
# ``char_exclude`` peut être frozenset (legacy parsed) ou string
|
| 177 |
+
# (RunSpec format). RunSpec attend une string ; on convertit.
|
| 178 |
+
char_exclude_str: str | None = None
|
| 179 |
+
if char_exclude is not None:
|
| 180 |
+
if isinstance(char_exclude, str):
|
| 181 |
+
char_exclude_str = char_exclude
|
| 182 |
+
else:
|
| 183 |
+
char_exclude_str = "".join(sorted(char_exclude))
|
| 184 |
+
|
| 185 |
+
spec = RunSpec(
|
| 186 |
+
corpus_dir=str(ws_path), # ignoré par execute_preset
|
| 187 |
+
pipelines=(_dummy_pipeline_yaml(),), # ignoré, juste pour validator
|
| 188 |
+
views=("text_final",),
|
| 189 |
+
output_dir=str(run_dir),
|
| 190 |
+
char_exclude=char_exclude_str,
|
| 191 |
+
normalization_profile=norm_profile_str,
|
| 192 |
+
partial_dir=str(partial_dir) if partial_dir else None,
|
| 193 |
+
entity_extractor=entity_extractor_dotted,
|
| 194 |
+
profile=profile,
|
| 195 |
+
output_json=str(output_json) if output_json else None,
|
| 196 |
+
code_version=code_version,
|
| 197 |
+
timeout_seconds_per_doc=timeout_seconds,
|
| 198 |
+
)
|
| 199 |
+
|
| 200 |
+
# Tag des engines avec le nom pour la map pipeline_to_engine
|
| 201 |
+
# (utilisé par le progress_callback wrapper).
|
| 202 |
+
wrapped_callback = None
|
| 203 |
+
if progress_callback is not None:
|
| 204 |
+
def wrapped_callback(
|
| 205 |
+
pipeline_name: str, doc_idx: int, doc_id: str,
|
| 206 |
+
) -> None:
|
| 207 |
+
engine_name = pipeline_to_engine_name.get(
|
| 208 |
+
pipeline_name, pipeline_name,
|
| 209 |
+
)
|
| 210 |
+
progress_callback(engine_name, doc_idx, doc_id)
|
| 211 |
+
|
| 212 |
+
orch = RunOrchestrator(run_dir)
|
| 213 |
+
orch_result = orch.execute_preset(
|
| 214 |
+
spec,
|
| 215 |
+
corpus_spec=corpus_spec,
|
| 216 |
+
extracted_dir=gt_dir,
|
| 217 |
+
pipeline_specs=pipeline_specs,
|
| 218 |
+
adapter_resolver=adapter_resolver,
|
| 219 |
+
adapter_kwargs={},
|
| 220 |
+
progress_callback=wrapped_callback,
|
| 221 |
+
cancel_event=cancel_event,
|
| 222 |
+
)
|
| 223 |
+
|
| 224 |
+
# Converti RunResult → BenchmarkResult via le converter
|
| 225 |
+
# canonique (utilisé aussi par output_json en Phase B2.7).
|
| 226 |
+
benchmark_result = run_result_to_benchmark_result(
|
| 227 |
+
orch_result.run_result,
|
| 228 |
+
corpus=corpus,
|
| 229 |
+
engines=engines,
|
| 230 |
+
char_exclude=char_exclude, # passe la valeur originale
|
| 231 |
+
normalization_profile=normalization_profile,
|
| 232 |
+
profile=profile,
|
| 233 |
+
)
|
| 234 |
+
|
| 235 |
+
# NER attach post-process si entity_extractor callable fourni.
|
| 236 |
+
# Cohérent avec run_benchmark_via_service:261-264.
|
| 237 |
+
if entity_extractor_callable is not None:
|
| 238 |
+
from picarones.app.services._benchmark_ner import (
|
| 239 |
+
attach_ner_metrics_to_benchmark,
|
| 240 |
+
)
|
| 241 |
+
attach_ner_metrics_to_benchmark(
|
| 242 |
+
benchmark_result, corpus, entity_extractor_callable,
|
| 243 |
+
)
|
| 244 |
+
|
| 245 |
+
# Sérialisation output_json si demandée (legacy comportement).
|
| 246 |
+
if output_json is not None:
|
| 247 |
+
from picarones.app.services._benchmark_persistence import (
|
| 248 |
+
persist_benchmark_result_json,
|
| 249 |
+
)
|
| 250 |
+
persist_benchmark_result_json(
|
| 251 |
+
benchmark_result, Path(output_json),
|
| 252 |
+
)
|
| 253 |
+
|
| 254 |
+
return benchmark_result
|
| 255 |
+
|
| 256 |
+
|
| 257 |
+
__all__ = ["run_via_orchestrator"]
|
|
@@ -145,7 +145,7 @@ def run_cmd(
|
|
| 145 |
_setup_logging(verbose)
|
| 146 |
|
| 147 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 148 |
-
from picarones.app.services.
|
| 149 |
from picarones.interfaces.cli._normalization_arg import (
|
| 150 |
resolve_normalization_profile,
|
| 151 |
)
|
|
@@ -195,7 +195,7 @@ def run_cmd(
|
|
| 195 |
click.echo(f"Profil de métriques : {profile}")
|
| 196 |
|
| 197 |
# Lancement du benchmark
|
| 198 |
-
result =
|
| 199 |
corpus=corp,
|
| 200 |
engines=ocr_engines,
|
| 201 |
output_json=output,
|
|
@@ -273,7 +273,7 @@ def _run_workflow(
|
|
| 273 |
``economics`` et ``edition``.
|
| 274 |
|
| 275 |
Les 4 commandes partagent le squelette : chargement corpus →
|
| 276 |
-
instanciation moteurs → ``
|
| 277 |
classement → génération automatique du rapport HTML. Seul le profil
|
| 278 |
par défaut et le message d'en-tête diffèrent.
|
| 279 |
|
|
@@ -289,7 +289,7 @@ def _run_workflow(
|
|
| 289 |
_setup_logging(verbose)
|
| 290 |
|
| 291 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 292 |
-
from picarones.app.services.
|
| 293 |
|
| 294 |
try:
|
| 295 |
corp = load_corpus_from_directory(corpus)
|
|
@@ -317,7 +317,7 @@ def _run_workflow(
|
|
| 317 |
click.echo(f"Moteurs : {', '.join(e.name for e in ocr_engines)}")
|
| 318 |
click.echo(f"Profil de métriques : {profile}")
|
| 319 |
|
| 320 |
-
result =
|
| 321 |
corpus=corp,
|
| 322 |
engines=ocr_engines,
|
| 323 |
output_json=output,
|
|
|
|
| 145 |
_setup_logging(verbose)
|
| 146 |
|
| 147 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 148 |
+
from picarones.app.services.legacy_runner_compat import run_via_orchestrator
|
| 149 |
from picarones.interfaces.cli._normalization_arg import (
|
| 150 |
resolve_normalization_profile,
|
| 151 |
)
|
|
|
|
| 195 |
click.echo(f"Profil de métriques : {profile}")
|
| 196 |
|
| 197 |
# Lancement du benchmark
|
| 198 |
+
result = run_via_orchestrator(
|
| 199 |
corpus=corp,
|
| 200 |
engines=ocr_engines,
|
| 201 |
output_json=output,
|
|
|
|
| 273 |
``economics`` et ``edition``.
|
| 274 |
|
| 275 |
Les 4 commandes partagent le squelette : chargement corpus →
|
| 276 |
+
instanciation moteurs → ``run_via_orchestrator(profile=...)`` → affichage
|
| 277 |
classement → génération automatique du rapport HTML. Seul le profil
|
| 278 |
par défaut et le message d'en-tête diffèrent.
|
| 279 |
|
|
|
|
| 289 |
_setup_logging(verbose)
|
| 290 |
|
| 291 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 292 |
+
from picarones.app.services.legacy_runner_compat import run_via_orchestrator
|
| 293 |
|
| 294 |
try:
|
| 295 |
corp = load_corpus_from_directory(corpus)
|
|
|
|
| 317 |
click.echo(f"Moteurs : {', '.join(e.name for e in ocr_engines)}")
|
| 318 |
click.echo(f"Profil de métriques : {profile}")
|
| 319 |
|
| 320 |
+
result = run_via_orchestrator(
|
| 321 |
corpus=corp,
|
| 322 |
engines=ocr_engines,
|
| 323 |
output_json=output,
|
|
@@ -309,8 +309,8 @@ def run_benchmark_thread_v2(job: BenchmarkJob, req: BenchmarkRunRequest) -> None
|
|
| 309 |
job.add_event("start", {"message": "Démarrage du benchmark…", "corpus": req.corpus_path})
|
| 310 |
|
| 311 |
try:
|
| 312 |
-
from picarones.app.services.
|
| 313 |
-
|
| 314 |
)
|
| 315 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 316 |
|
|
@@ -370,17 +370,14 @@ def run_benchmark_thread_v2(job: BenchmarkJob, req: BenchmarkRunRequest) -> None
|
|
| 370 |
from picarones.evaluation.metrics.normalization import _parse_exclude_chars
|
| 371 |
char_excl = _parse_exclude_chars(req.char_exclude) if req.char_exclude else None
|
| 372 |
|
| 373 |
-
#
|
| 374 |
-
# ``run_benchmark_via_service`` (
|
| 375 |
-
#
|
| 376 |
-
#
|
| 377 |
-
#
|
| 378 |
-
# ``
|
| 379 |
-
# ``
|
| 380 |
-
|
| 381 |
-
# (2026-05) : ``max_workers`` retiré (était inactif, passe
|
| 382 |
-
# par ``CorpusRunner.max_in_flight``).
|
| 383 |
-
result = run_benchmark_via_service(
|
| 384 |
corpus=corpus,
|
| 385 |
engines=engines,
|
| 386 |
output_json=output_json,
|
|
|
|
| 309 |
job.add_event("start", {"message": "Démarrage du benchmark…", "corpus": req.corpus_path})
|
| 310 |
|
| 311 |
try:
|
| 312 |
+
from picarones.app.services.legacy_runner_compat import (
|
| 313 |
+
run_via_orchestrator,
|
| 314 |
)
|
| 315 |
from picarones.evaluation.corpus import load_corpus_from_directory
|
| 316 |
|
|
|
|
| 370 |
from picarones.evaluation.metrics.normalization import _parse_exclude_chars
|
| 371 |
char_excl = _parse_exclude_chars(req.char_exclude) if req.char_exclude else None
|
| 372 |
|
| 373 |
+
# Phase B3 résiduel migration Option B (2026-05) — passé de
|
| 374 |
+
# ``run_benchmark_via_service`` (deprecated en B3) à
|
| 375 |
+
# ``run_via_orchestrator`` (shim qui s'appuie sur
|
| 376 |
+
# ``RunOrchestrator.execute_preset``). Comportement
|
| 377 |
+
# numériquement équivalent (couvert par
|
| 378 |
+
# ``test_migration_invariance.py``). Phase B8 supprimera
|
| 379 |
+
# ``run_benchmark_via_service``.
|
| 380 |
+
result = run_via_orchestrator(
|
|
|
|
|
|
|
|
|
|
| 381 |
corpus=corpus,
|
| 382 |
engines=engines,
|
| 383 |
output_json=output_json,
|
|
@@ -1,241 +1,17 @@
|
|
| 1 |
-
"""Helpers de migration B4 —
|
| 2 |
-
de ``run_benchmark_via_service`` vers ``RunOrchestrator.execute_preset()``.
|
| 3 |
|
| 4 |
-
|
| 5 |
-
de ``
|
| 6 |
-
|
| 7 |
-
|
|
|
|
| 8 |
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
::
|
| 12 |
-
|
| 13 |
-
# AVANT (legacy avec DeprecationWarning depuis Phase B3)
|
| 14 |
-
from picarones.app.services.benchmark_runner import run_benchmark_via_service
|
| 15 |
-
bm = run_benchmark_via_service(corpus, [adapter], profile="standard")
|
| 16 |
-
|
| 17 |
-
# APRÈS (Phase B4 — via RunOrchestrator)
|
| 18 |
-
from tests._migration_helpers import run_via_orchestrator
|
| 19 |
-
bm = run_via_orchestrator(corpus, [adapter], profile="standard")
|
| 20 |
-
|
| 21 |
-
Le helper n'est utilisé QUE pour les tests durant la transition. En
|
| 22 |
-
Phase B8 (post-deprecation), il sera supprimé et les tests devront
|
| 23 |
-
construire leur ``RunSpec`` explicitement (pattern utilisateur).
|
| 24 |
"""
|
| 25 |
|
| 26 |
from __future__ import annotations
|
| 27 |
|
| 28 |
-
import
|
| 29 |
-
from pathlib import Path
|
| 30 |
-
from typing import TYPE_CHECKING, Any, Callable
|
| 31 |
-
|
| 32 |
-
if TYPE_CHECKING:
|
| 33 |
-
from picarones.evaluation.benchmark_result import BenchmarkResult
|
| 34 |
-
from picarones.evaluation.corpus import Corpus
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
def _dummy_pipeline_yaml(name: str = "preset_pipeline") -> Any:
|
| 38 |
-
"""Construit un ``PipelineSpecYaml`` minimaliste pour satisfaire
|
| 39 |
-
le validator ``RunSpec.pipelines`` (min_length=1).
|
| 40 |
-
|
| 41 |
-
Le contenu est **ignoré** par ``execute_preset()`` qui utilise les
|
| 42 |
-
``pipeline_specs`` fournis en kwargs. Le YAML dummy sert
|
| 43 |
-
uniquement à passer la validation Pydantic.
|
| 44 |
-
"""
|
| 45 |
-
from picarones.app.schemas.run_spec import PipelineSpecYaml, StepSpec
|
| 46 |
-
from picarones.domain.artifacts import ArtifactType
|
| 47 |
-
return PipelineSpecYaml(
|
| 48 |
-
name=name,
|
| 49 |
-
initial_inputs=(ArtifactType.IMAGE,),
|
| 50 |
-
steps=(StepSpec(
|
| 51 |
-
id="ocr",
|
| 52 |
-
adapter_class="tests._migration_helpers.IgnoredByPreset",
|
| 53 |
-
adapter_kwargs={},
|
| 54 |
-
input_types=(ArtifactType.IMAGE,),
|
| 55 |
-
output_types=(ArtifactType.RAW_TEXT,),
|
| 56 |
-
),),
|
| 57 |
-
)
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
def run_via_orchestrator(
|
| 61 |
-
corpus: "Corpus",
|
| 62 |
-
engines: list[Any],
|
| 63 |
-
*,
|
| 64 |
-
char_exclude: Any | None = None,
|
| 65 |
-
normalization_profile: Any | None = None,
|
| 66 |
-
output_json: str | Path | None = None,
|
| 67 |
-
code_version: str | None = None,
|
| 68 |
-
show_progress: bool = True, # noqa: ARG001 — absorbé pour compat
|
| 69 |
-
progress_callback: Callable[[str, int, str], None] | None = None,
|
| 70 |
-
timeout_seconds: float = 60.0,
|
| 71 |
-
cancel_event: Any | None = None,
|
| 72 |
-
partial_dir: str | Path | None = None,
|
| 73 |
-
entity_extractor: Callable[[str], list[dict]] | str | None = None,
|
| 74 |
-
profile: str = "standard",
|
| 75 |
-
) -> "BenchmarkResult":
|
| 76 |
-
"""Drop-in remplacement de ``run_benchmark_via_service`` via
|
| 77 |
-
``RunOrchestrator.execute_preset()``.
|
| 78 |
-
|
| 79 |
-
Préserve la signature legacy pour permettre la migration mécanique
|
| 80 |
-
des call sites de test (Phase B4). Retourne un ``BenchmarkResult``
|
| 81 |
-
construit via le converter ``run_result_to_benchmark_result``.
|
| 82 |
-
|
| 83 |
-
Parameters
|
| 84 |
-
----------
|
| 85 |
-
corpus, engines:
|
| 86 |
-
Identiques à ``run_benchmark_via_service``.
|
| 87 |
-
char_exclude, normalization_profile, output_json, code_version,
|
| 88 |
-
show_progress, progress_callback, timeout_seconds, cancel_event,
|
| 89 |
-
partial_dir, entity_extractor, profile:
|
| 90 |
-
Identiques à ``run_benchmark_via_service``.
|
| 91 |
-
|
| 92 |
-
Notes
|
| 93 |
-
-----
|
| 94 |
-
Quelques différences subtiles vs le legacy :
|
| 95 |
-
|
| 96 |
-
- ``entity_extractor`` accepte un callable direct (legacy) OU un
|
| 97 |
-
dotted path string (RunSpec). Si callable, on l'invoque
|
| 98 |
-
directement en post-process sur le ``BenchmarkResult``.
|
| 99 |
-
- Le workspace temporaire est nettoyé automatiquement via
|
| 100 |
-
``TemporaryDirectory`` — ne pas s'attendre à des fichiers
|
| 101 |
-
résiduels après l'appel.
|
| 102 |
-
- ``normalization_profile`` accepte un objet ``NormalizationProfile``
|
| 103 |
-
(legacy) OU un nom string (RunSpec). Conversion automatique.
|
| 104 |
-
"""
|
| 105 |
-
from picarones.app.schemas.run_spec import RunSpec
|
| 106 |
-
from picarones.app.services._benchmark_adapter_resolver import (
|
| 107 |
-
build_adapter_resolver,
|
| 108 |
-
engine_to_pipeline_spec,
|
| 109 |
-
)
|
| 110 |
-
from picarones.app.services._benchmark_converter import (
|
| 111 |
-
run_result_to_benchmark_result,
|
| 112 |
-
)
|
| 113 |
-
from picarones.app.services._benchmark_conversions import (
|
| 114 |
-
corpus_to_corpus_spec,
|
| 115 |
-
)
|
| 116 |
-
from picarones.app.services.run_orchestrator import RunOrchestrator
|
| 117 |
-
|
| 118 |
-
# Résolution code_version (cohérent avec run_benchmark_via_service:219).
|
| 119 |
-
if code_version is None:
|
| 120 |
-
import importlib
|
| 121 |
-
try:
|
| 122 |
-
code_version = importlib.import_module("picarones").__version__
|
| 123 |
-
except (ImportError, AttributeError):
|
| 124 |
-
code_version = "unknown"
|
| 125 |
-
|
| 126 |
-
# ``normalization_profile`` legacy accepte un objet
|
| 127 |
-
# NormalizationProfile. RunSpec attend une string. On convertit.
|
| 128 |
-
norm_profile_str = normalization_profile
|
| 129 |
-
if normalization_profile is not None and not isinstance(
|
| 130 |
-
normalization_profile, str,
|
| 131 |
-
):
|
| 132 |
-
norm_profile_str = getattr(normalization_profile, "name", None)
|
| 133 |
-
|
| 134 |
-
# ``entity_extractor`` legacy accepte un callable direct. RunSpec
|
| 135 |
-
# attend un dotted path. Si callable, on le traite post-process
|
| 136 |
-
# comme run_benchmark_via_service le fait.
|
| 137 |
-
entity_extractor_dotted: str | None = None
|
| 138 |
-
entity_extractor_callable: Callable | None = None
|
| 139 |
-
if entity_extractor is not None:
|
| 140 |
-
if isinstance(entity_extractor, str):
|
| 141 |
-
entity_extractor_dotted = entity_extractor
|
| 142 |
-
elif callable(entity_extractor):
|
| 143 |
-
entity_extractor_callable = entity_extractor
|
| 144 |
-
|
| 145 |
-
with tempfile.TemporaryDirectory(prefix="picarones_b4_") as ws:
|
| 146 |
-
ws_path = Path(ws)
|
| 147 |
-
gt_dir = ws_path / "gt"
|
| 148 |
-
gt_dir.mkdir()
|
| 149 |
-
run_dir = ws_path / "run"
|
| 150 |
-
run_dir.mkdir()
|
| 151 |
-
|
| 152 |
-
corpus_spec = corpus_to_corpus_spec(corpus, workspace_dir=gt_dir)
|
| 153 |
-
pipeline_specs = [engine_to_pipeline_spec(e) for e in engines]
|
| 154 |
-
adapter_resolver = build_adapter_resolver(engines)
|
| 155 |
-
pipeline_to_engine_name = {
|
| 156 |
-
spec.name: engine.name
|
| 157 |
-
for spec, engine in zip(pipeline_specs, engines)
|
| 158 |
-
}
|
| 159 |
-
|
| 160 |
-
# ``char_exclude`` peut être frozenset (legacy parsed) ou string
|
| 161 |
-
# (RunSpec format). RunSpec attend une string ; on convertit.
|
| 162 |
-
char_exclude_str: str | None = None
|
| 163 |
-
if char_exclude is not None:
|
| 164 |
-
if isinstance(char_exclude, str):
|
| 165 |
-
char_exclude_str = char_exclude
|
| 166 |
-
else:
|
| 167 |
-
char_exclude_str = "".join(sorted(char_exclude))
|
| 168 |
-
|
| 169 |
-
spec = RunSpec(
|
| 170 |
-
corpus_dir=str(ws_path), # ignoré par execute_preset
|
| 171 |
-
pipelines=(_dummy_pipeline_yaml(),), # ignoré, juste pour validator
|
| 172 |
-
views=("text_final",),
|
| 173 |
-
output_dir=str(run_dir),
|
| 174 |
-
char_exclude=char_exclude_str,
|
| 175 |
-
normalization_profile=norm_profile_str,
|
| 176 |
-
partial_dir=str(partial_dir) if partial_dir else None,
|
| 177 |
-
entity_extractor=entity_extractor_dotted,
|
| 178 |
-
profile=profile,
|
| 179 |
-
output_json=str(output_json) if output_json else None,
|
| 180 |
-
code_version=code_version,
|
| 181 |
-
timeout_seconds_per_doc=timeout_seconds,
|
| 182 |
-
)
|
| 183 |
-
|
| 184 |
-
# Tag des engines avec le nom pour la map pipeline_to_engine
|
| 185 |
-
# (utilisé par le progress_callback wrapper).
|
| 186 |
-
wrapped_callback = None
|
| 187 |
-
if progress_callback is not None:
|
| 188 |
-
def wrapped_callback(
|
| 189 |
-
pipeline_name: str, doc_idx: int, doc_id: str,
|
| 190 |
-
) -> None:
|
| 191 |
-
engine_name = pipeline_to_engine_name.get(
|
| 192 |
-
pipeline_name, pipeline_name,
|
| 193 |
-
)
|
| 194 |
-
progress_callback(engine_name, doc_idx, doc_id)
|
| 195 |
-
|
| 196 |
-
orch = RunOrchestrator(run_dir)
|
| 197 |
-
orch_result = orch.execute_preset(
|
| 198 |
-
spec,
|
| 199 |
-
corpus_spec=corpus_spec,
|
| 200 |
-
extracted_dir=gt_dir,
|
| 201 |
-
pipeline_specs=pipeline_specs,
|
| 202 |
-
adapter_resolver=adapter_resolver,
|
| 203 |
-
adapter_kwargs={},
|
| 204 |
-
progress_callback=wrapped_callback,
|
| 205 |
-
cancel_event=cancel_event,
|
| 206 |
-
)
|
| 207 |
-
|
| 208 |
-
# Converti RunResult → BenchmarkResult via le converter
|
| 209 |
-
# canonique (utilisé aussi par output_json en Phase B2.7).
|
| 210 |
-
benchmark_result = run_result_to_benchmark_result(
|
| 211 |
-
orch_result.run_result,
|
| 212 |
-
corpus=corpus,
|
| 213 |
-
engines=engines,
|
| 214 |
-
char_exclude=char_exclude, # passe la valeur originale
|
| 215 |
-
normalization_profile=normalization_profile,
|
| 216 |
-
profile=profile,
|
| 217 |
-
)
|
| 218 |
-
|
| 219 |
-
# NER attach post-process si entity_extractor callable fourni.
|
| 220 |
-
# Cohérent avec run_benchmark_via_service:261-264.
|
| 221 |
-
if entity_extractor_callable is not None:
|
| 222 |
-
from picarones.app.services._benchmark_ner import (
|
| 223 |
-
attach_ner_metrics_to_benchmark,
|
| 224 |
-
)
|
| 225 |
-
attach_ner_metrics_to_benchmark(
|
| 226 |
-
benchmark_result, corpus, entity_extractor_callable,
|
| 227 |
-
)
|
| 228 |
-
|
| 229 |
-
# Sérialisation output_json si demandée (legacy comportement).
|
| 230 |
-
if output_json is not None:
|
| 231 |
-
from picarones.app.services._benchmark_persistence import (
|
| 232 |
-
persist_benchmark_result_json,
|
| 233 |
-
)
|
| 234 |
-
persist_benchmark_result_json(
|
| 235 |
-
benchmark_result, Path(output_json),
|
| 236 |
-
)
|
| 237 |
-
|
| 238 |
-
return benchmark_result
|
| 239 |
-
|
| 240 |
|
| 241 |
__all__ = ["run_via_orchestrator"]
|
|
|
|
| 1 |
+
"""Helpers de migration B4 — re-export depuis le module de production.
|
|
|
|
| 2 |
|
| 3 |
+
Phase B3 résiduel (mai 2026) : ``run_via_orchestrator`` a été
|
| 4 |
+
promu de ``tests/_migration_helpers.py`` vers
|
| 5 |
+
``picarones.app.services.legacy_runner_compat`` pour pouvoir être
|
| 6 |
+
consommé aussi par les call sites CLI/Web (qui ne peuvent pas
|
| 7 |
+
importer depuis ``tests/``).
|
| 8 |
|
| 9 |
+
Ce module reste comme alias pour préserver les imports des tests
|
| 10 |
+
catégorie A migrés en Phase B4. Sera retiré en Phase B8.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
| 14 |
|
| 15 |
+
from picarones.app.services.legacy_runner_compat import run_via_orchestrator
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
__all__ = ["run_via_orchestrator"]
|