Mira-3 v11: adapter + per-epoch + training metrics + eval results + learnings

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+epochs/epoch_1/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+epochs/epoch_2/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+epochs/epoch_3/tokenizer.json filter=lfs diff=lfs merge=lfs -text

MIRA3_LEARNINGS.md

@@ -0,0 +1,143 @@
+# Mira-3 Learnings — Building a Specialized Document→JSON Extraction SLM
+
+**What this is.** A distilled, reusable playbook from building **Mira-3**: a ~4B, on-prem, fine-tuned small language model that reads a document and returns **structured, source-grounded JSON**. The beachhead is clinical extraction (lab reports, discharge summaries, intake forms → schema JSON), but **every lesson below generalizes to any document→JSON extraction task** — résumé/CV parsing, invoices, receipts, KYC forms, contracts, RFP/compliance docs. Where a lesson is clinical-specific, it's marked; most aren't.
+
+Read this before starting a new extraction vertical (e.g. a "Mira-Résumé"). It will save you a training run and an eval that lies to you.
+
+> **The one-line lesson:** *A fine-tuned extractor learns the exact surface form of its training data. If your training docs state facts one way and real docs state them another, the model silently fails on real docs — and an in-distribution synthetic eval will score ~1.0 and hide it. Measure on real, out-of-distribution documents, and measure recall, not just validity.*
+
+---
+
+## 0. The stack (what we actually shipped)
+
+| Component | Choice | Why |
+|---|---|---|
+| Base model | `Qwen/Qwen3-4B-Instruct-2507` (Unsloth 4-bit: `unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit`) | Strong instruction-following at a size that runs on a customer's single GPU. Open weights (no ToS trap). |
+| Method | **QLoRA** (4-bit base + LoRA r16/α32), Unsloth, sequence packing | ~2× faster + low VRAM; fits a free Kaggle T4. |
+| Trainer | TRL `SFTConfig`/`SFTTrainer`, full-sequence loss, packing on | Proven config; validity 1.0 on in-dist data. |
+| Compute | **Kaggle free T4** (30h/wk), multi-session with lossless resume | Zero infra cost for v0. Paid GPU only for teacher generation. |
+| Data (v0) | **Synthetic + permissively-licensed public only** | No DUA/PHI/copyright risk; retrain on real partner data at v1. |
+| Output contract | JSON Schema with `x-grounding` / `x-match-on` extensions | Schema is a runtime input, not code — lets you onboard a new vertical without a code change. |
+
+**Reusable assets in this repo** (paths are the concrete, working versions):
+- Training kernel (resumable): `models/mira-3/notebooks/train_mira3_v11.py` + `RUNBOOK_v11.md`
+- Eval kernel (2-phase, best-epoch): `models/mira-3/notebooks/eval_mira3_v11.py`
+- Offline authoritative scorer: `scripts/mira3/build_v11_gate_scorecard.py`
+- Exit gate: `scripts/mira3/phase25_gate_check.py`
+- Grounding checker (schema-aware): `src/slm/data/_grounding.py`; field-F1: `src/slm/eval/_field_f1.py`
+
+---
+
+## 1. Training recipe that worked
+
+- **QLoRA 4-bit + Unsloth + packing**, `max_seq_length` sized to cover ~100% of your token length distribution (we used 3072; a too-small 2048 silently truncated ~10% of rows in an earlier run and degraded quality). **Measure your length distribution first**; truncation is a silent quality leak.
+- LoRA on all attention + MLP projections (`q,k,v,o,gate,up,down`), r=16, α=32, dropout 0.05, lr 2e-4 cosine, warmup 3%, effective batch 16 (2 × grad-accum 8), **3 epochs**.
+- **Loss plateaus fast.** Loss went 1.36 → 0.15 within epoch 1 and only 0.15 → 0.13 across epochs 2–3. Most learning is epoch 1; later epochs are refinement. **Save a per-epoch adapter and pick the best by eval**, don't assume the last epoch is best (see §4).
+- **Chat-template fidelity is non-negotiable.** Train with `apply_chat_template(..., add_generation_prompt=False)` over `[system,user,assistant]`; at inference use `add_generation_prompt=True`. Load the tokenizer **from the saved adapter dir** so the exact template travels with the model. A prompt/template mismatch between train and eval tanks validity for a reason that looks like a model failure.
+
+---
+
+## 2. Training under a hard wall-clock cap (Kaggle 12h) — lossless multi-session resume
+
+A full 3-epoch run was ~20h of compute — impossible in one 12h session. The pattern that worked (generalizes to any pre-emptible/capped GPU: spot instances, Colab, CI runners):
+
+1. **Wall-clock callback** forces a full `Trainer` checkpoint + graceful stop at ~10.5h (leave ≥1.5h headroom before the platform kill — the final save + output upload take real time; we lost a run's final adapter once by cutting it to 10.5h with too little margin).
+2. **Resume = re-run the same kernel with the prior checkpoint mounted** as an input. The `Trainer` restores optimizer + LR schedule + RNG; completion is gated on `global_step >= max_steps` (**not** rounded epoch — a time-stop at epoch 2.99 is *not* "done").
+3. **Freeze dependency versions to the exact set that wrote the checkpoint** before resuming. A lossless resume requires identical `transformers`/`peft`/`bitsandbytes`/`unsloth` — pin `==`, capture `env_versions.json` in session 1, and pin to those. Ranges drift and break resume.
+4. **Archive-robust, fail-loud resume.** Upload the checkpoint as one tar with directory structure preserved (`--dir-mode skip`, **never** `--dir-mode zip` — it flattens `checkpoint-XXXX/` and silently restarts from step 0). If a checkpoint is *expected* (a checkpoint dataset is attached) but none is found, **crash** — don't silently burn a session training from scratch.
+
+**Bug we hit (watch for it):** a per-epoch adapter saver keyed on `round(state.epoch)` mislabeled a mid-epoch budget-stop at epoch 1.57 as "epoch_2". Guard epoch-boundary saves with `abs(epoch - round(epoch)) < 0.05` so a fractional stop doesn't write a bogus artifact.
+
+---
+
+## 3. Infra/Kaggle gotchas (each cost us time)
+
+- **Inline `pip install` at the top of CLI-pushed kernels** — they don't inherit the notebook UI's packages. One pinned command, fail fast (before GPU time).
+- **Split training and eval into separate notebooks.** Never run both in one session (12h timeout). Eval is a separate, shorter GPU session.
+- **`kaggle kernels output` is flaky** — it returns a *different partial subset of files per call*, and the log often needs a second pull. Re-run the pull 2–3× and verify the critical files (metrics, adapters, log) are non-zero before trusting them. A "0-byte final adapter" locally was a download artifact, not a training failure.
+- **Batch (CLI-pushed) kernels don't stream logs.** The whole log publishes only on commit. "No new logs for hours" during a RUNNING batch kernel is normal, not a stall. Status (`RUNNING`/`COMPLETE`) is your only live signal; poll it.
+- **Read adapters across kernels via `kernel_sources`**, not by re-uploading — the eval kernel mounts the training kernel's `/kaggle/working/output` directly.
+- The training output tree may contain a `_resume/` staging dir with **stale copies** of adapters. When discovering adapters, exclude `_resume/` and `checkpoints/` and require `adapter_config.json`, or you may silently score a stale/partial adapter.
+
+---
+
+## 4. Eval methodology — the part that actually determines whether you have a product
+
+This is where most of the value is. **Evaluation is harder and more important than training** for extraction models.
+
+### 4a. Mechanics
+- **Best-epoch selection:** score every per-epoch adapter and pick the winner. Load each with the *same* loader the training used (for Unsloth 4-bit adapters, `FastLanguageModel`, **not** vanilla `PeftModel` — quantization mismatch produces garbage).
+- **Two-phase, quota-frugal eval:** Phase A scores all epochs on the labeled set (cheap, decides best epoch first so a timeout still yields it); Phase B runs the full probe suite on the winner only.
+- **In-kernel metrics are for monitoring/ranking; authoritative metrics are computed offline** on the saved raw predictions (schema-aware grounding, macro-F1) with the real library. Save raw predictions in the exact schema your offline loader expects (`extra="forbid"` loaders reject stray keys — we crashed the offline scorer by writing a `gold` key and omitting `gen_time`/`pred_len`).
+- **Adversarially review the eval/train kernel *before* pushing it.** A multi-agent review (independent reviewers per failure-mode: mounts, adapter-loading, prompt fidelity, metric logic, output-format-vs-offline-loader, time-budget) caught 3 real bugs that would each have wasted a ~5h GPU session. Cheap insurance for anything that costs a GPU run.
+
+### 4b. Model-specific output gotcha — the `<think>` prefix
+Qwen3-Instruct emits a leading `<think>...</think>` block (often empty) before the JSON. Our raw validity read **0%** until we stripped it — the JSON was perfect underneath. **Strip the reasoning wrapper before parsing** (and the serving layer must too). Check your base model's default output framing before trusting a "0% valid" result.
+
+### 4c. THE BIG ONE — in-distribution synthetic eval lies
+
+Our labeled `test_gold` (synthetic, same generator as training) scored **field-F1 = 1.000, validity = 1.000, 200/200 predictions byte-identical to gold.** This was **not** leakage (0/800 exact overlap, 4.4% template overlap) and the metric was **not** broken (corrupting a field dropped F1 to 0.92). It was real — and **meaningless as a generalization signal.** The synthetic docs have a near-deterministic source→label mapping; the model learned it perfectly. The *previous* model scored 1.0 on it too.
+
+**If your held-out set comes from the same generator/distribution as training, a high score tells you the model fit the generator — nothing about the real world.**
+
+### 4d. Build a labeled REAL-doc benchmark (how)
+Real-doc probes (actual clinical transcripts) were **unlabeled**, so we could only measure validity + hallucination, not F1. To get an honest field-F1 we built a small labeled set:
+1. Select a **diverse** sample (we took 24 real docs, one per specialty/domain).
+2. **Double-annotate + adjudicate:** two independent annotators produce gold from *source + schema only* (never shown the model's output), then an adjudicator resolves disagreements strictly against the text. (We used LLM annotators + adjudication — a defensible *directional* signal; replace with human gold for certification.)
+3. Score the model's saved predictions against the adjudicated gold with schema-aware field-F1 (exclude meta fields like `extraction_notes`; coerce malformed list items).
+
+### 4e. What the honest eval revealed (the pattern to expect)
+Real-doc **macro-F1 = 0.61** (vs synthetic 1.0). The shape matters more than the number:
+
+- **High precision (0.89–0.98) on every field, but low recall.** The model is *safe* (what it emits is right) but *misses content*.
+- **The blind spot:** it extracted **~0 diagnoses from real narrative notes** (docs with 6–10 explicit diagnoses → output `[]`), and under-extracted procedures (recall 0.33). It nailed list-structured fields (medications, labs, vitals, allergies, encounter date/dept).
+
+**Root cause = data, not model.** The synthetic training data only ever presents diagnoses/procedures in **structured `Diagnoses:` blocks with bracketed codes**. Real notes state them in **prose** ("patient has DM type 2, HTN…"). The model learned "extract diagnoses from the Diagnoses: section" and never learned to read narrative. **This is the single most important, most generalizable lesson.**
+
+---
+
+## 5. Data lessons (all generalize)
+
+1. **Structured-vs-narrative coverage is everything.** Whatever field a downstream user cares about, your training data must present it in **every surface form real documents use** — structured lists *and* inline prose, tables *and* sentences. If a form is only ever seen one way in training, the model fails on the other. *This is the recall killer.* (For résumés: a "skill" appears as a bulleted list, a comma-separated line, and buried in a job-description paragraph — train on all three.)
+2. **Preserve source formatting; don't canonicalize in labels.** Our training labels rendered a lab unit one fixed way (`x10^9/L`) with zero variety, so the model "corrected" the real docs' `10^9/L` to `x10^9/L` — violating "preserve exactly." **Fix: render source values/units in varied equivalent forms and have the label copy the source verbatim.** Teach copying, not canonicalization.
+3. **Give the verifier principled equivalences.** A correct-unit reformat (`10^9/L ≡ x10^9/L`) is not a hallucination and shouldn't route to a human. Add narrow, meaning-preserving equivalences to the grounding checker (scoped to the right field, so it can never ground a fabricated *value*). Verified precision: it must not "fix" a genuine fabrication (e.g., inventing `Glucose (Fasting)` from `glucose`).
+4. **Audit your generator empirically.** Run the grounding checker *on your own training labels*: any label value not grounded in its own source is a canonicalization you're teaching. Ours was clean (0.02%) — which is how we learned the unit issue was a *diversity* gap, not a labeling bug. This 5-minute check reframes fixes.
+
+---
+
+## 6. Metrics & acceptance framework (use for any vertical)
+
+| Metric | How | Bar | Notes |
+|---|---|---|---|
+| JSON validity | parse to schema-valid object (after stripping any reasoning wrapper) | ≥99% raw, 100% constrained | 100% is achievable at serving via constrained decoding; measure raw to see model behavior. |
+| Field-F1 (**precision + recall separately**) | schema-aware match (`x-match-on` keys); exact for scalars | ≥0.90 **on real docs** | Report P and R separately — a low-recall/high-precision model looks fine on F1-averaged-with-validity but misses content. |
+| Hallucination | grounding: every extracted value must appear in source (schema `x-grounding: exempt` for classifications) | 0 (post-verifier) | Raw model is never 0 on real docs; the *system* hits 0 via verifier + human-in-loop. |
+| Identifier leak | regex for names/MRN/DOB/SSN/phone/email | 0 | Schema stores no identifiers by design. |
+| Beats baseline | vs the prior model AND base zero-shot | strictly better | Base zero-shot ≈ 0% valid JSON; the meaningful bar is the prior fine-tune. |
+
+**Do not report a single blended number.** Report: validity, **per-field precision AND recall**, hallucination, leak — on **real** docs. The blended F1 on synthetic data is the metric that lies.
+
+---
+
+## 7. Porting to a new vertical (e.g. Résumé / CV parsing) — checklist
+
+The pipeline is schema-driven, so a new vertical is mostly data + schema:
+
+1. **Schema.** Define the JSON contract with `x-grounding` (which fields must be verbatim-grounded vs. are classifications/derived) and `x-match-on` (list-item match keys). E.g. résumé: `experience[]` matched on `(company, title)`, `education[]` on `(institution, degree)`, `skills[]` on `name`. Mark `summary`/`document_type` as grounding-exempt.
+2. **Data — cover every surface form (see §5.1).** For each field, generate/collect docs where it appears structured *and* in prose. This is where you prevent the recall blind spot before it happens.
+3. **Preserve formatting (§5.2).** Dates, phone numbers, degree abbreviations (`BS` vs `Bachelor of Science`) — vary them in sources and copy verbatim in labels; add equivalences to the verifier, not the training labels.
+4. **Train** with the resumable QLoRA recipe (§1–2); save per-epoch adapters.
+5. **Eval honestly (§4).** Build a small **real** résumé benchmark with double-annotation + adjudication. Measure per-field precision *and* recall. Expect (and hunt for) the high-precision/low-recall blind spot on the fields real documents phrase in prose.
+6. **Gate + verifier + human-in-loop** identical to clinical — the whole workflow (extract → ground → verify → correct → audit) is the product, not the single model call.
+
+---
+
+## 8. TL;DR — the five things to remember
+
+1. **The model learns the surface form of your data.** Train on every way real documents phrase a field, especially **narrative prose**, or you get high precision and terrible recall on real docs.
+2. **In-distribution synthetic eval scores ~1.0 and means nothing.** Build a labeled **real-doc** benchmark; measure **recall**, not just validity/F1.
+3. **Preserve source formatting in labels; put equivalences in the verifier.** Don't teach the model to canonicalize.
+4. **Adversarially review any kernel before spending a GPU session;** strip the base model's reasoning wrapper before parsing; compute authoritative metrics offline.
+5. **For capped GPUs, do lossless multi-session resume** with frozen deps, step-gated completion, and fail-loud archive handling.
+
+*Written from the Mira-3 v11 build (clinical extraction, Qwen3-4B QLoRA). v11 = valid JSON 100%, identifier-leak 0, beats the prior model — but real-doc field-F1 0.61 with a diagnoses-recall blind spot that the synthetic eval hid. The blind spot is a data problem, and it's the v1 priority.*

README.md

@@ -0,0 +1,91 @@
+---
+license: other
+license_name: dilr-internal
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+  - clinical
+  - information-extraction
+  - structured-output
+  - json-extraction
+  - qlora
+  - lora
+  - on-prem
+language:
+  - en
+  - es
+  - pt
+---
+
+# Mira-3 v11 — Clinical Structured Extraction (Qwen3-4B, QLoRA)
+
+**Mira-3** reads a clinical document (lab report, discharge summary, intake form, progress note, …) and returns a single **schema-valid, source-grounded JSON** object. It is a LoRA adapter over `Qwen/Qwen3-4B-Instruct-2507`, built to run **on-prem / offline** on a customer's own GPU — privacy is the point.
+
+This repo is the **v0** artifact: trained on **synthetic + permissively-licensed public data only** (no PHI, no DUA/research-only data). It proves the pipeline; v1 will retrain on a design partner's real documents.
+
+> ⚠️ **Read the honest eval below before using.** v0 is excellent on validity and grounding but has a real-world **recall blind spot on narrative diagnoses/procedures**. See `MIRA3_LEARNINGS.md` for the full analysis and how to fix it.
+
+## Files
+- `adapter/` — the shipping adapter (**epoch 3**, best by eval)
+- `epochs/epoch_{1,2,3}/` — per-epoch adapters (for reproducibility / best-epoch verification)
+- `training/metrics.json`, `training/env_versions.json` — training run + exact dependency stack
+- `eval/probe_scorecard_v11.json` — real-doc probe validity + hallucination + leak
+- `eval/phase25_gate_v11.json` — Phase-2.5 exit gate verdict (**PASS**)
+- `eval/realdoc_eval_v11.json` — the honest labeled real-doc field-F1 (with adjudicated gold)
+- `MIRA3_LEARNINGS.md` — detailed, reusable playbook (generalizes to other extraction/OCR verticals, e.g. résumé parsing)
+
+## Training
+| | |
+|---|---|
+| Base | `Qwen/Qwen3-4B-Instruct-2507` (Unsloth 4-bit) |
+| Method | QLoRA (4-bit + LoRA r16/α32, dropout 0.05), Unsloth, sequence packing |
+| Data | 21k rows: 15k ladder + 3k schema-variant + 3k PII-abstention (synthetic + public) |
+| Schedule | 3 epochs, lr 2e-4 cosine, warmup 3%, effective batch 16, max_seq 3072 |
+| Compute | Kaggle T4, 2 sessions (~20.7h) via lossless checkpoint/resume |
+| Loss | 1.363 → 0.133 (clean cosine decay; natural completion at 867/867 steps) |
+
+## Evaluation (honest)
+
+**Phase-2.5 gate: PASS.** v11 beats the prior production model (Mira-Q2) on all real-doc probes for both JSON validity (100%) and hallucination.
+
+| Probe (real docs) | JSON validity | Hallucination (↓ better) |
+|---|---|---|
+| mtsamples_282 | 1.00 | 0.28 |
+| extraction_relevant_150 | 1.00 | 0.52 |
+| synthetic_v2_150 | 1.00 | 0.13 |
+| identifier leak | — | **0** everywhere |
+
+**Labeled real-doc field-F1 (the honest generalization metric): 0.61** — on 24 diverse mtsamples docs with double-annotated + adjudicated gold. (The synthetic in-distribution test set scores 1.00 and is **not** a generalization signal — see learnings §4c.)
+
+Per-field: **high precision (0.89–0.98) everywhere, but uneven recall.** Strong on allergies (F1 0.99), encounter date/dept (0.96/0.88), vitals (0.84), medications (P 0.98). **Weak on diagnoses (recall 0.10) and procedures (recall 0.33).**
+
+## Limitations
+- **Diagnoses/procedures recall on narrative text is poor.** The model extracts these well from structured `Diagnoses:`-style blocks (as in the synthetic training data) but misses them when stated in prose (real clinical notes). This is a **data-coverage** issue, targeted for v1. Do not rely on v0 for complete diagnosis capture from free-text notes.
+- Every output is a **draft for a human** and must be routed through the verifier (schema + grounding + identifier-leak) before use. Never an autonomous decision.
+- v0 is trained on synthetic/public data; real-world distribution shift is expected until the v1 retrain.
+
+## Usage
+```python
+import torch
+from unsloth import FastLanguageModel  # load with Unsloth, NOT vanilla PeftModel (4-bit quant mismatch)
+
+model, tok = FastLanguageModel.from_pretrained(
+    "dilr/mira-3-v11/adapter", max_seq_length=4096, dtype=torch.float16, load_in_4bit=True)
+FastLanguageModel.for_inference(model)
+
+SYSTEM = ("You are a clinical information extraction system. Read the clinical document and "
+          "output a single JSON object matching the schema. Extract ONLY information explicitly "
+          "stated ... Output valid JSON only - no prose, no markdown.")  # full prompt in the training data
+
+msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": document_text}]
+text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+out = model.generate(**tok(text, return_tensors="pt").to("cuda"), max_new_tokens=2048, do_sample=False)
+raw = tok.decode(out[0], skip_special_tokens=True)
+# NOTE: Qwen3 prepends a <think>...</think> block — strip it before json.loads():
+import re, json
+pred = json.loads(re.sub(r"^\s*<think>.*?</think>\s*", "", raw.split("assistant")[-1], flags=re.DOTALL))
+```
+
+## License & data
+LoRA adapter over Qwen3-4B (base: Apache-2.0). Internal dilr.ai artifact; not for redistribution. Trained on synthetic + permissively-licensed public data; **contains no PHI and no patient identifiers**. Eval derived from the public MTSamples corpus.

adapter/adapter_config.json

@@ -0,0 +1,52 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "v_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48cffcb83e6810cfaedfad7c4f61ed45bc2c4a133285fb001ac4eabe9d2ca08d
+size 132187888

adapter/chat_template.jinja

@@ -0,0 +1,86 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

adapter/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7430e9138b76e93fb6f93462394d236b411111aef53cb421ba97d2691040cca
+size 11423114

adapter/tokenizer_config.json

@@ -0,0 +1,233 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 3072,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151669": {
+      "content": "<|PAD_TOKEN|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

epochs/epoch_1/adapter_config.json

@@ -0,0 +1,52 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "v_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

epochs/epoch_1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b96d29b2458fe548cd1fbb410d73608a2701915a7786d644a071a48515d4831b
+size 132187888

epochs/epoch_1/chat_template.jinja

@@ -0,0 +1,86 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

epochs/epoch_1/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7430e9138b76e93fb6f93462394d236b411111aef53cb421ba97d2691040cca
+size 11423114

epochs/epoch_1/tokenizer_config.json

@@ -0,0 +1,233 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 3072,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151669": {
+      "content": "<|PAD_TOKEN|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

epochs/epoch_2/adapter_config.json

@@ -0,0 +1,52 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "v_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

epochs/epoch_2/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb50169c3d360d11bbd44ed400bc693a8f16352d049f426f2027588377beac9a
+size 132187888

epochs/epoch_2/chat_template.jinja

@@ -0,0 +1,86 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

epochs/epoch_2/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7430e9138b76e93fb6f93462394d236b411111aef53cb421ba97d2691040cca
+size 11423114

epochs/epoch_2/tokenizer_config.json

@@ -0,0 +1,233 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 3072,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151669": {
+      "content": "<|PAD_TOKEN|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

epochs/epoch_3/adapter_config.json

@@ -0,0 +1,52 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "v_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

epochs/epoch_3/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48cffcb83e6810cfaedfad7c4f61ed45bc2c4a133285fb001ac4eabe9d2ca08d
+size 132187888

epochs/epoch_3/chat_template.jinja

@@ -0,0 +1,86 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

epochs/epoch_3/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7430e9138b76e93fb6f93462394d236b411111aef53cb421ba97d2691040cca
+size 11423114

epochs/epoch_3/tokenizer_config.json

@@ -0,0 +1,233 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 3072,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151669": {
+      "content": "<|PAD_TOKEN|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

eval/phase25_gate_v11.json

@@ -0,0 +1,36 @@
+{
+  "mira2_baseline_path": "models/mira-3/baseline/three_way_scorecard.json",
+  "mira3_scorecard_path": "models/mira-3/mira3_v11_scorecard.json",
+  "verdict": {
+    "passed": true,
+    "validity_floor": 0.95,
+    "all_validity_at_or_above_floor": true,
+    "all_hallucination_improved_or_same": true,
+    "per_probe": {
+      "mtsamples_282": {
+        "present": true,
+        "validity": 1.0,
+        "validity_ok": true,
+        "hallucination_baseline": 0.3723,
+        "hallucination_now": 0.2801,
+        "halluc_ok": true
+      },
+      "extraction_relevant_150": {
+        "present": true,
+        "validity": 1.0,
+        "validity_ok": true,
+        "hallucination_baseline": 0.5333,
+        "hallucination_now": 0.52,
+        "halluc_ok": true
+      },
+      "synthetic_v2_150": {
+        "present": true,
+        "validity": 1.0,
+        "validity_ok": true,
+        "hallucination_baseline": 0.2733,
+        "hallucination_now": 0.1333,
+        "halluc_ok": true
+      }
+    }
+  }
+}

eval/probe_scorecard_v11.json

@@ -0,0 +1,131 @@
+{
+  "model": "Mira-3-v11",
+  "base": "Qwen/Qwen3-4B-Instruct-2507",
+  "best_epoch": 3,
+  "schema_path": "docs/extraction_schema.json",
+  "probes": [
+    {
+      "name": "mtsamples_282",
+      "n": 282,
+      "validity": {
+        "metric": "json_validity_unconstrained",
+        "point": 1.0,
+        "ci_lower": 1.0,
+        "ci_upper": 1.0,
+        "n_samples": 282,
+        "n_bootstrap": 1000
+      },
+      "hallucination": {
+        "metric": "hallucination_rate",
+        "point": 0.2801,
+        "ci_lower": 0.2305,
+        "ci_upper": 0.3333,
+        "n_samples": 282,
+        "n_bootstrap": 1000
+      },
+      "leak": {
+        "metric": "identifier_leak_rate",
+        "point": 0.0,
+        "ci_lower": 0.0,
+        "ci_upper": 0.0,
+        "n_samples": 282,
+        "n_bootstrap": 1000
+      }
+    },
+    {
+      "name": "extraction_relevant_150",
+      "n": 150,
+      "validity": {
+        "metric": "json_validity_unconstrained",
+        "point": 1.0,
+        "ci_lower": 1.0,
+        "ci_upper": 1.0,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      },
+      "hallucination": {
+        "metric": "hallucination_rate",
+        "point": 0.52,
+        "ci_lower": 0.4333,
+        "ci_upper": 0.6,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      },
+      "leak": {
+        "metric": "identifier_leak_rate",
+        "point": 0.0,
+        "ci_lower": 0.0,
+        "ci_upper": 0.0,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      }
+    },
+    {
+      "name": "synthetic_v2_150",
+      "n": 150,
+      "validity": {
+        "metric": "json_validity_unconstrained",
+        "point": 1.0,
+        "ci_lower": 1.0,
+        "ci_upper": 1.0,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      },
+      "hallucination": {
+        "metric": "hallucination_rate",
+        "point": 0.1333,
+        "ci_lower": 0.08,
+        "ci_upper": 0.1933,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      },
+      "leak": {
+        "metric": "identifier_leak_rate",
+        "point": 0.0,
+        "ci_lower": 0.0,
+        "ci_upper": 0.0,
+        "n_samples": 150,
+        "n_bootstrap": 1000
+      }
+    }
+  ],
+  "test_gold": {
+    "name": "test_gold_200",
+    "n": 200,
+    "validity": {
+      "metric": "json_validity_unconstrained",
+      "point": 1.0,
+      "ci_lower": 1.0,
+      "ci_upper": 1.0,
+      "n_samples": 200,
+      "n_bootstrap": 1000
+    },
+    "hallucination": {
+      "metric": "hallucination_rate",
+      "point": 0.0,
+      "ci_lower": 0.0,
+      "ci_upper": 0.0,
+      "n_samples": 200,
+      "n_bootstrap": 1000
+    },
+    "leak": {
+      "metric": "identifier_leak_rate",
+      "point": 0.0,
+      "ci_lower": 0.0,
+      "ci_upper": 0.0,
+      "n_samples": 200,
+      "n_bootstrap": 1000
+    },
+    "field_f1": {
+      "metric": "macro_f1",
+      "point": 1.0,
+      "ci_lower": 1.0,
+      "ci_upper": 1.0,
+      "n_samples": 200,
+      "n_bootstrap": 1000
+    }
+  },
+  "notes": [
+    "Authoritative offline metrics (slm.eval + check_grounding). Hallucination = >=1 ungrounded value per row (schema-aware x-grounding: exempt). Feeds scripts/mira3/phase25_gate_check.py."
+  ]
+}

eval/realdoc_eval_v11.json

@@ -0,0 +1,2055 @@
+{
+  "benchmark": "mira-3-v11 real-doc field-F1 (mtsamples, LLM-adjudicated gold)",
+  "model": "Mira-3-v11 (epoch_3)",
+  "n_docs": 24,
+  "method": "24 diverse mtsamples (1/specialty); 2 independent LLM annotators + adjudicator on source+schema only; extraction_notes excluded; malformed list-items coerced to primary key",
+  "macro_f1": 0.6114,
+  "per_field": {
+    "document_type": {
+      "precision": 0.1667,
+      "recall": 0.1667,
+      "f1": 0.1667
+    },
+    "patient.age": {
+      "precision": 0.7083,
+      "recall": 0.7083,
+      "f1": 0.7083
+    },
+    "patient.sex": {
+      "precision": 0.4583,
+      "recall": 0.4583,
+      "f1": 0.4583
+    },
+    "encounter.date": {
+      "precision": 0.9583,
+      "recall": 0.9583,
+      "f1": 0.9583
+    },
+    "encounter.department": {
+      "precision": 0.875,
+      "recall": 0.875,
+      "f1": 0.875
+    },
+    "vitals": {
+      "precision": 0.9188,
+      "recall": 0.8771,
+      "f1": 0.8354
+    },
+    "labs": {
+      "precision": 0.8917,
+      "recall": 0.7292,
+      "f1": 0.6435
+    },
+    "medications": {
+      "precision": 0.9792,
+      "recall": 0.6647,
+      "f1": 0.674
+    },
+    "diagnoses": {
+      "precision": 0.9722,
+      "recall": 0.0972,
+      "f1": 0.1069
+    },
+    "procedures": {
+      "precision": 0.9722,
+      "recall": 0.3333,
+      "f1": 0.3125
+    },
+    "allergies": {
+      "precision": 0.9792,
+      "recall": 1.0,
+      "f1": 0.9861
+    }
+  },
+  "per_doc": [
+    {
+      "idx": 17,
+      "specialty": "Neurology",
+      "macro_f1": 0.3318
+    },
+    {
+      "idx": 6,
+      "specialty": "Emergency Room Reports",
+      "macro_f1": 0.3636
+    },
+    {
+      "idx": 19,
+      "specialty": "General Medicine",
+      "macro_f1": 0.3636
+    },
+    {
+      "idx": 1,
+      "specialty": "Orthopedic",
+      "macro_f1": 0.4545
+    },
+    {
+      "idx": 5,
+      "specialty": "Pain Management",
+      "macro_f1": 0.4545
+    },
+    {
+      "idx": 10,
+      "specialty": "Discharge Summary",
+      "macro_f1": 0.4545
+    },
+    {
+      "idx": 24,
+      "specialty": "Psychiatry / Psychology",
+      "macro_f1": 0.5432
+    },
+    {
+      "idx": 0,
+      "specialty": "Consult - History and Phy.",
+      "macro_f1": 0.5455
+    },
+    {
+      "idx": 3,
+      "specialty": "Gastroenterology",
+      "macro_f1": 0.5455
+    },
+    {
+      "idx": 27,
+      "specialty": "Radiology",
+      "macro_f1": 0.5455
+    },
+    {
+      "idx": 2,
+      "specialty": "SOAP / Chart / Progress Notes",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 11,
+      "specialty": "Diets and Nutritions",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 12,
+      "specialty": "Podiatry",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 16,
+      "specialty": "Neurosurgery",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 20,
+      "specialty": "Urology",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 21,
+      "specialty": "Nephrology",
+      "macro_f1": 0.6364
+    },
+    {
+      "idx": 29,
+      "specialty": "Letters",
+      "macro_f1": 0.697
+    },
+    {
+      "idx": 13,
+      "specialty": "Cardiovascular / Pulmonary",
+      "macro_f1": 0.7273
+    },
+    {
+      "idx": 18,
+      "specialty": "Speech - Language",
+      "macro_f1": 0.7273
+    },
+    {
+      "idx": 22,
+      "specialty": "Dentistry",
+      "macro_f1": 0.7273
+    },
+    {
+      "idx": 26,
+      "specialty": "Rheumatology",
+      "macro_f1": 0.7677
+    },
+    {
+      "idx": 4,
+      "specialty": "Lab Medicine - Pathology",
+      "macro_f1": 0.8182
+    },
+    {
+      "idx": 8,
+      "specialty": "ENT - Otolaryngology",
+      "macro_f1": 0.8182
+    },
+    {
+      "idx": 23,
+      "specialty": "Obstetrics / Gynecology",
+      "macro_f1": 0.9697
+    }
+  ],
+  "key_finding": "High precision (0.89-0.98), low recall. Model extracts ~0 diagnoses from real narrative notes (diagnoses R=0.10) and under-extracts procedures (R=0.33). Root cause = synthetic training presents diagnoses only in structured 'Diagnoses:' blocks; real prose diagnoses are missed. Data issue; fix in v1 with narrative-style training docs.",
+  "caveats": [
+    "N=24, single corpus (mtsamples)",
+    "gold is LLM-adjudicated, not human-certified",
+    "exact-description match slightly understates meds recall (HCTZ vs Hydrochlorothiazide)"
+  ],
+  "gold": [
+    {
+      "idx": 0,
+      "specialty": "Consult - History and Phy.",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": null,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [
+          {
+            "test": "Creatinine",
+            "value": "1.0",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "RBCs (UA)",
+            "value": "5-9",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "6-mm left intrarenal stone, nonobstructing",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "Microhematuria",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "History of kidney stone",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          }
+        ],
+        "procedures": [
+          {
+            "date": "07/24/2008",
+            "description": "Renal ultrasound"
+          },
+          {
+            "date": "07/24/2008",
+            "description": "IVP"
+          },
+          {
+            "date": null,
+            "description": "Cystoscopy (planned)"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Urology consultation follow-up note (HPI/Impression/Plan); classified progress_note (no consult enum). Sex male from repeated He/his; age not stated (null). Encounter date null: note says patient presents 'today' but prints no visit date; the printed dates are prior tests (Creatinine June 25, 2008; renal US and IVP 07/24/2008). Department null (urology only inferable, not explicitly named). Labs: serum Creatinine 1.0 (no unit printed) and urinalysis 5-9 RBCs; lab schema has no date field so test dates not captured. Diagnoses: current 6-mm nonobstructing left intrarenal stone and Microhematuria marked active (current problems in the Impression under active management); prior kidney stone marked history ('history of kidney stone', last episode >1 year ago). Procedures: renal ultrasound and IVP both performed 07/24/2008; cystoscopy only planned for the near future (date null). Urine sent for culture/sensitivity and for cytology are pending orders without results, so not listed as labs or procedures. Hydration counseling (goal >2 L urine/24h) is advice, not a drug -> medications []. No vitals (urinary frequency 3-4 and nocturia 1-2 are symptoms), no allergies. Referring physician 'Dr. ABC' and any identifiers excluded per contract."
+      }
+    },
+    {
+      "idx": 1,
+      "specialty": "Orthopedic",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 41,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [
+          {
+            "name": "aspirin",
+            "dose": null,
+            "frequency": "a day",
+            "route": null,
+            "status": "current"
+          },
+          {
+            "name": "Malarone",
+            "dose": null,
+            "frequency": null,
+            "route": null,
+            "status": "current"
+          },
+          {
+            "name": "Lovenox",
+            "dose": null,
+            "frequency": null,
+            "route": null,
+            "status": "planned"
+          }
+        ],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Right Achilles tendon rupture",
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Radiographs of the right ankle",
+            "date": null
+          },
+          {
+            "description": "Radiographs of the heel",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Orthopedic consultation/progress note for a right Achilles tendon rupture. Patient identifiers excluded per contract: name (Mr. XYZ), employer (Chevron), and country (Angola). Sex 'male' from explicit 'Mr.'/'gentleman'/'he'; age 41 stated ('41 years of age'). Encounter date null: visit referenced as 'today' with no printed date; injury date redacted as 'Month DD, YYYY'. Department null: 'Orthopedic' is specialty context, not a printed department. Medications: aspirin currently taken ('been on aspirin a day', frequency 'a day' preserved verbatim); Malarone listed under CURRENT MEDICATIONS (anti-malarial); Lovenox planned ('for a week after surgery'). The plan's post-op aspirin is the same drug already captured as current and is not double-listed. NKDA -> no positive allergen, allergies=[]. Past medical history explicitly DENIES diabetes, cardiovascular disease, and pulmonary disease -> negated, not extracted as diagnoses. Diagnosis from IMPRESSION ('Right Achilles tendon rupture'); status left null as no explicit status ('active' is not printed). Procedures: radiographs of the right ankle and heel performed today (preserved joint space, no fracture, good alignment); date null since only 'today' is written. Recommended operative fixation (outpatient Achilles repair) is a future/planned treatment, not a performed procedure -> excluded. Physical exam findings (palpable defect 6-8 cm proximal to insertion, positive calf-squeeze/Thompson, increased dorsiflexion) are exam details, not diagnoses. No vital signs and no laboratory values reported."
+      }
+    },
+    {
+      "idx": 2,
+      "specialty": "SOAP / Chart / Progress Notes",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": null,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Normal postoperative course",
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Total abdominal hysterectomy"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Postoperative day #1 SOAP/progress note following a total abdominal hysterectomy. patient.sex = \"female\" grounded in explicit female pronouns (\"her clinical condition\", \"all her questions\") plus vaginal/genitalia exam and the hysterectomy; age not stated (null). Vitals left empty: the note gives only qualitative statements (\"Afebrile now\", \"Other vital signs are stable\") with no numeric measurements and no printed vital name \u2014 mapping \"Afebrile\" to a \"Temperature\" vital would require inferring a name not written, which the contract forbids. No labs (no test+value), no medications (Plan says only \"Continue present therapy\" with no drug named; patient is NPO), no allergies stated. Diagnosis = the Assessment \"Normal postoperative course\"; \"incisional tenderness\" is a subjective complaint/symptom, not a formal diagnosis. Procedure \"Total abdominal hysterectomy\" captured from the header. The only date, 6/10/2009, appears solely as the reference point for \"denies any new symptomatology from 6/10/2009\" (symptom-onset reference), not clearly the encounter or surgery date, so encounter.date and procedure.date left null. Department not stated. No ICD/LOINC codes printed and no patient identifiers present."
+      }
+    },
+    {
+      "idx": 3,
+      "specialty": "Gastroenterology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [
+          {
+            "test": "Gallbladder ejection fraction at 30 minutes",
+            "value": "32",
+            "unit": "%",
+            "reference_range": "greater than 35%",
+            "flag": null
+          }
+        ],
+        "medications": [
+          {
+            "name": "Technetium-99m Choletec",
+            "dose": "6.9 mCi",
+            "route": "injection",
+            "frequency": null,
+            "status": null
+          }
+        ],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Acute cholecystitis",
+            "status": "ruled out"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Cystic duct obstruction",
+            "status": "ruled out"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Very mild chronic cholecystitis",
+            "status": "possible"
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Nuclear Medicine Hepatobiliary Scan"
+          },
+          {
+            "date": "02/13/09",
+            "description": "CT of the abdomen"
+          },
+          {
+            "date": "02/13/09",
+            "description": "Ultrasound of the abdomen"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Nuclear Medicine hepatobiliary (CCK-HIDA) scan report; document_type=\"other\" because the enum has no radiology/nuclear-medicine imaging category (not a serum lab or pathology report). No patient age, sex, or identifiers stated. No encounter date printed. Department set to null: \"Nuclear Medicine\" appears only inside the exam title \"NUCLEAR MEDICINE HEPATOBILIARY SCAN,\" not as an explicit department field, so per the no-inference rule it is not captured (annotator A had captured it; annotator B and this adjudication leave it null). Labs: gallbladder ejection fraction at 30 min = 32% (reference \"greater than 35%\") is the one quantitative measurement with a range; placed under labs as the closest structured slot though it is an imaging-derived functional value, not a blood lab. Flag left null: no discrete H/L token is printed, although the impression states the value is \"just under the lower limits of normal\" (i.e., below the 35% reference). Medications: Technetium-99m Choletec 6.9 mCi captured; route=\"injection\" retained because the text explicitly states \"the injection of radiopharmaceutical\" (annotator A captured this; B left it null). The second administered agent \u2014 \"2.1 mcg of intravenous cholecystic ______\" (a cholecystokinin/cholecystagogue for the ejection-fraction challenge) \u2014 is NOT captured as a medication because its drug name is redacted/blank in the source and the contract requires a grounded drug name (dose 2.1 mcg, route intravenous noted here only for completeness). Diagnoses derived from the IMPRESSION: acute cholecystitis and cystic duct obstruction marked \"ruled out\" (\"Negative for...\"), and \"very mild chronic cholecystitis\" marked \"possible\" (\"can be seen with...\"). \"Right upper quadrant pain\" is NOT listed as a diagnosis: it is explicitly the REASON FOR EXAM (presenting complaint/indication), not a rendered diagnosis, and the contract requires a diagnosis to be an explicit condition statement (annotator B had added it with status \"reason for exam\"; this adjudication omits it). Procedures: this nuclear medicine scan (date null), plus the prior CT of the abdomen and ultrasound of the abdomen (both dated 02/13/09) which are listed under COMPARISONS \u2014 captured for recall with their study dates; descriptions kept to the source wording rather than appended with \"(comparison)\". Not encoded: qualitative findings \"normal hepatobiliary transfer time\" and \"normal accumulation within the gallbladder\" (no discrete values); procedural symptoms \"2/10 pain at 5 minutes\" and \"nausea\" (transient exam-time symptoms, not recorded vital signs or diagnoses)."
+      }
+    },
+    {
+      "idx": 4,
+      "specialty": "Lab Medicine - Pathology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [],
+        "procedures": [],
+        "allergies": [],
+        "extraction_notes": "Not a patient-specific clinical record. Educational/reference passage explaining two lab tests: BNP (brain/B-type natriuretic peptide), produced in the heart ventricles under strain and used to detect acute congestive heart failure, and BMP (basic metabolic panel), a group of 8 tests (glucose, calcium, sodium, potassium, bicarbonate, chloride, BUN, creatinine), plus guidance on distinguishing BNP from BMP in documentation. Classified 'other' (not lab_report/pathology_report) because no actual patient results are reported. No patient demographics, encounter, vitals, medications, procedures, or allergies present; references to 'a person'/'a doctor'/'he' are generic/hypothetical. labs=[] because the figures 100 and 500 are stated as general interpretive thresholds ('Values above 100 signal a problematic situation and those above 500 a highly demanding state'), not a measured patient value tied to a test; BMP is described as a panel with no single value. diagnoses=[] because 'congestive heart failure', 'heart failure', and 'MI' appear only as illustrative examples of BNP elevation, not conditions diagnosed for any patient. No patient identifiers present."
+      }
+    },
+    {
+      "idx": 5,
+      "specialty": "Pain Management",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": 14,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [
+          {
+            "name": "Naprosyn",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": null
+          },
+          {
+            "name": "Aristospan",
+            "dose": "20 mg",
+            "route": "injection",
+            "frequency": null,
+            "status": "administered"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "pauciarticular arthritis, in particular arthritis of her left knee",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "arthralgias in multiple joints",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "joint swelling of her left knee",
+            "code": null,
+            "code_system": null,
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Joint injection of the left knee with 20 mg of Aristospan, under general anesthesia and aseptic technique; no fluid obtained; no complications or bleeding",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Procedure note (INDICATIONS FOR PROCEDURE / DESCRIPTION OF PROCEDURE) documenting a left-knee joint injection; no procedure_note enum exists and it is not a routine progress note, so document_type=other. Patient stated as \"14-year-old Hispanic female\" \u2014 only age/sex captured; ethnicity is not a schema field and no identifiers present. Encounter date/department null (text says only \"clinic\", not a named department, and no date printed). Vitals empty: swelling \"about 1+\" is a physical-exam finding, not a numeric vital. Labs empty: \"No fluid was obtained\" is a procedure observation, not a test+value. Medications: Naprosyn is a prior/home med (\"has been taking Naprosyn on her last visit\") with no explicit dose/route/frequency/status, so status left null (ongoing use not explicitly stated); Aristospan 20 mg was injected into the left knee during the procedure (route=injection, status=administered, both grounded in the explicit injection event). General anesthesia is mentioned but no specific agent is named, so it is captured under the procedure, not as a medication. Diagnoses/complaints grounded to explicit statements: history of pauciarticular arthritis (esp. left knee), arthralgias in multiple joints, and the chief complaint of left-knee joint swelling for several months. No allergies stated."
+      }
+    },
+    {
+      "idx": 6,
+      "specialty": "Emergency Room Reports",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 15,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": "ED"
+        },
+        "vitals": [
+          {
+            "name": "BP",
+            "value": "102/60",
+            "unit": null
+          },
+          {
+            "name": "P",
+            "value": "70",
+            "unit": null
+          },
+          {
+            "name": "RR",
+            "value": "20",
+            "unit": null
+          },
+          {
+            "name": "T",
+            "value": "98.2",
+            "unit": null
+          }
+        ],
+        "labs": [],
+        "medications": [
+          {
+            "name": "Tylenol",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": null
+          },
+          {
+            "name": "Tylenol with codeine",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": null
+          }
+        ],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Migraine headache",
+            "status": null
+          }
+        ],
+        "procedures": [],
+        "allergies": [],
+        "extraction_notes": "ED/urgent care encounter note (CC, HPI, PMH, MEDICATIONS, ALLERGIES, FAMILY HISTORY, ROS, PHYSICAL EXAM, COURSE IN THE ED, IMPRESSION, PLAN) classified as progress_note (closest schema type); headers 'COURSE IN THE ED' and 'seen in the urgent care'. Patient is a 15-year-old girl -> age 15, sex female; no patient identifiers present. Department captured as 'ED' from the 'COURSE IN THE ED' section; no encounter date printed. Vitals from Physical Exam preserved exactly with no printed units: BP 102/60, P 70, RR 20, T 98.2. No labs and no procedures printed. Two medications: 'Tylenol for pain' (MEDICATIONS/current) and 'Tylenol with codeine' being tried in the ED ('we'll try Tylenol with codeine for her pain'); no dose/route/frequency and no explicit status word printed, so status left null for both. Diagnosis 'Migraine headache' from IMPRESSION (HPI notes migraine 'highly likely'); chief complaint 'Headache' with nausea/photophobia is the presenting symptom, not a separate diagnosis. ALLERGIES 'None' (empty list). Family history (grandmother died of cerebral aneurysm) is family, not patient, history and excluded."
+      }
+    },
+    {
+      "idx": 8,
+      "specialty": "ENT - Otolaryngology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Recurrent tonsillitis",
+            "status": "preoperative"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Recurrent tonsillitis",
+            "status": "postoperative"
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Adenotonsillectomy"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "ENT operative note for adenotonsillectomy. Classified as \"other\" because a surgical operative report does not map to any enum value (not progress_note, discharge_summary, or pathology_report). Two explicit diagnosis statements captured: PREOPERATIVE DIAGNOSIS and POSTOPERATIVE DIAGNOSIS, both \"Recurrent tonsillitis\" (status preoperative/postoperative). Named procedure \"Adenotonsillectomy\" captured; procedure-detail technique (McIvor mouthgag, adenoid curettes, gold laser tonsillar dissection, saline irrigation, hemostasis) are steps of the single named procedure, not separate procedures. COMPLICATIONS explicitly \"None\" (not a diagnosis). No patient age or sex stated. No vitals, labs, or allergies stated. No medications: \"general endotracheal anesthesia\" names no specific agent and \"saline\" is intraoperative irrigation, neither a named drug order with dose/route. No encounter date printed; department not stated in text (ENT/Otolaryngology specialty was external context only), so left null. No ICD/LOINC codes printed. No patient identifiers present."
+      }
+    },
+    {
+      "idx": 10,
+      "specialty": "Discharge Summary",
+      "gold": {
+        "document_type": "discharge_summary",
+        "patient": {
+          "age": null,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [
+          {
+            "test": "Urine culture",
+            "value": "Klebsiella isolated, sensitive to Bactrim",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [
+          {
+            "name": "Bactrim",
+            "dose": null,
+            "route": "p.o.",
+            "frequency": null,
+            "status": "active"
+          },
+          {
+            "name": "ibuprofen",
+            "dose": "two to three tabs",
+            "route": null,
+            "frequency": "t.i.d.",
+            "status": "active"
+          }
+        ],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Pyelonephritis",
+            "status": "admission diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "History of uterine cancer and ileal conduit urinary diversion",
+            "status": "admission diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Hypertension",
+            "status": "admission diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Renal insufficiency",
+            "status": "admission diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Anemia",
+            "status": "admission diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Pyelonephritis likely secondary to mucous plugging of indwelling Foley in the ileal conduit",
+            "status": "discharge diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Hypertension",
+            "status": "discharge diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Mild renal insufficiency",
+            "status": "discharge diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Anemia, which has been present chronically over the past year",
+            "status": "discharge diagnosis"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Right knee pain",
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Irrigation of thick mucous plug in the Foley in the ileal conduit",
+            "date": null
+          },
+          {
+            "description": "Further surgery (scheduled)",
+            "date": "08/07/2007"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Discharge summary with ADMISSION DIAGNOSES, DISCHARGE DIAGNOSES, and HOSPITAL COURSE sections; no demographics, vitals, or allergies stated. Sex 'female' grounded in repeated she/her pronouns plus documented uterine cancer history; age not stated (null). Both admission and discharge diagnosis lists captured verbatim with status matching the section headers; duplicates (Hypertension, Anemia, renal insufficiency) retained because each is explicitly restated in both sections. 'Right knee pain' included as a diagnosis with null status: it is an explicitly stated, clinically managed complaint ('She also complained of right knee pain') that drove the ibuprofen recommendation, though it is not in the numbered lists. Lab: only micro datum is a urine culture growing Klebsiella sensitive to Bactrim ('Klebsiella was isolated in this urine, which was sensitive to Bactrim'). Medications: Bactrim p.o. (discharge med) and ibuprofen two-to-three tabs t.i.d. (advised OTC for knee pain). 'IV antibiotics' excluded per contract because no specific drug name is given. Procedures: irrigation of the mucous plug in the Foley/ileal conduit (no date, performed by provider), and a further surgery scheduled 08/07/2007. encounter.date left null: no admission/discharge date is stated and 08/07/2007 is the future scheduled-surgery date, not the encounter date. Provider names (Dr. X, Dr. Y) and the 7-10 day follow-up are omitted as they are not schema fields / are identifiers."
+      }
+    },
+    {
+      "idx": 11,
+      "specialty": "Diets and Nutritions",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 62,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [
+          {
+            "test": "blood sugar",
+            "value": "187",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "blood sugar",
+            "value": "477",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "blood sugar",
+            "value": "over 600",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [
+          {
+            "name": "Humalog",
+            "dose": null,
+            "frequency": null,
+            "route": "injectable and insulin pump",
+            "status": "active"
+          },
+          {
+            "name": "insulin",
+            "dose": "boluses 5 units at breakfast, 6 units at lunch, 11 units at supper; basal rate 30.5 units per 24 hours",
+            "frequency": null,
+            "route": "insulin pump",
+            "status": "active"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "type I diabetes",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "DKA",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "cold",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          }
+        ],
+        "procedures": [],
+        "allergies": [],
+        "extraction_notes": "SOAP-format dietary/nutrition consultation for carbohydrate counting in a 62-year-old female with type I diabetes; classified progress_note (no consultation option in enum). LABS: three patient-reported self-monitored blood-sugar readings (187 at Friday-night bedtime, 477 on waking, over 600 at hospital); no units printed next to these numbers (mg/dl appears only in the correction-dose context 'down 30 mg/dl'), so unit=null; value 'over 600' preserved verbatim. MEDICATIONS: Humalog is named in SUBJECTIVE (she self-administered 70 extra units during the DKA event, 10 injectable and the rest via pump) \u2014 that 70-unit acute self-correction is a one-time event, not a standing dose, so it is not encoded as a med dose; route 'injectable and insulin pump' reflects both delivery methods stated. The OBJECTIVE pump insulin doses (boluses 5/6/11 units at breakfast/lunch/supper and basal 30.5 units per 24 h) are one pump-insulin regimen with two dose components, kept as a single 'insulin' entry rather than split into two records, and not merged with 'Humalog' because the objective section names only 'insulin' (no inference that they are the same drug). DIAGNOSES: type I diabetes (active), DKA (kept as the literal abbreviation printed; not expanded to 'diabetic ketoacidosis'; status null as a recent acute hospitalization event with no explicit resolved statement), cold (active). Recommendations/education are not structured fields and were not encoded: insulin-to-carbohydrate ratio 1 unit per 10 g carbohydrate (per the '500 rule'), correction dose ~1 unit per 30 mg/dl, recommended carbohydrate intake 45 g breakfast / 45 g lunch / 60 g dinner, Lilly meal-planning guide and carbohydrate-counting book provided, one-hour consultation. No encounter date printed (only relative references: Friday, over the weekend, two years ago); no department header. No vitals, procedures, or allergies stated. Provider names (Charla Yassine, Joanne Araiza) are personnel, not the patient, and are omitted per the no-identifier rule; no patient identifiers present."
+      }
+    },
+    {
+      "idx": 12,
+      "specialty": "Podiatry",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [
+          {
+            "name": "morphine",
+            "dose": null,
+            "route": "intravenous",
+            "frequency": null,
+            "status": "administered"
+          },
+          {
+            "name": "Versed",
+            "dose": null,
+            "route": "intravenous",
+            "frequency": null,
+            "status": "administered"
+          },
+          {
+            "name": "Xylocaine",
+            "dose": "1%",
+            "route": null,
+            "frequency": null,
+            "status": "administered"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "Painful ingrown toenail, left big toe",
+            "code": null,
+            "code_system": null,
+            "status": "preoperative"
+          },
+          {
+            "description": "Painful ingrown toenail, left big toe",
+            "code": null,
+            "code_system": null,
+            "status": "postoperative"
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Removal of an ingrown part of the left big toenail with excision of the nail matrix",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Podiatry operative/procedure note; the document_type enum has no operative-note option, so classified as 'other'. No patient age or sex stated; no encounter date printed. encounter.department left null: 'Same Day Surgery' is named only as the post-op recovery destination ('sent back to Same Day Surgery for recovery') and 'minor OR room' as the procedure location, neither is explicitly labeled as the encounter department, so no department is grounded. Medications explicitly administered intraoperatively: morphine and Versed via IV ('intravenous sedation with morphine and Versed was performed') and 1% Xylocaine as a local toe block ('the toe was blocked with 1% Xylocaine'); status='administered' for all three because the giving of each drug is explicitly described. Xylocaine '1%' captured as dose/concentration exactly as written; Xylocaine route left null (described only as a toe block, no standard route term stated). Preoperative and postoperative diagnoses are identical (Painful ingrown toenail, left big toe) and captured separately with their explicit status labels from the note. 'Estimated blood loss was negligible' is not a numeric vital, so not recorded. Additional procedural details (skin flap over matrix, hemostasis with cautery, tubular/bulky dressing) are part of the single procedure narrative and not itemized separately. No labs, vitals, or allergies stated. No patient identifiers present in source."
+      }
+    },
+    {
+      "idx": 13,
+      "specialty": "Cardiovascular / Pulmonary",
+      "gold": {
+        "document_type": "pathology_report",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [
+          {
+            "test": "P53 immunohistochemical stain",
+            "value": "negative",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "Probable right upper lobe lung adenocarcinoma",
+            "status": "probable",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "Right lung, upper lobe, lobectomy: Bronchioloalveolar carcinoma, mucinous type",
+            "status": "final",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "Non-neoplastic lung: Emphysema",
+            "status": null,
+            "code": null,
+            "code_system": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Lung, right upper lobe resection (lobectomy)",
+            "date": null
+          },
+          {
+            "description": "Frozen section",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Surgical pathology report (frozen section + final diagnosis) for a right upper lobe lung lobectomy specimen. No patient demographics or identifiers are stated, so patient.age/sex and encounter.date/department are null. Three diagnoses captured: the clinical-history 'probable' adenocarcinoma, the final diagnosis (bronchioloalveolar carcinoma, mucinous type), and the non-neoplastic finding of emphysema; no ICD/other codes are printed. The P53 immunohistochemical stain (result: negative in the tumor) is the only test+value, recorded as a lab. Two procedures are grounded: the right upper lobe resection/lobectomy and the intraoperative frozen section ('received fresh for frozen section'); no dates given. Synoptic/gross pathology parameters are stated but have no appropriate structured field and were NOT forced into labs (they are not laboratory test values): tumor size (greatest diameter) 3.6 cm; gross mass 3.6 x 3.3 x 2.7 cm, 3.7 cm from closest surgical margin and 3.9 cm from hilum; lobectomy specimen 16.1 x 10.6 x 4.5 cm; histologic grade well differentiated; blood/lymphatic vessel invasion absent; perineural invasion absent; bronchial, vascular, and inked surgical margins negative; visceral pleura not involved (tumor abuts but does not invade through; overlying pleura puckered); in situ carcinoma absent; hilar lymph nodes positive 0 of total 1; no necrosis or hemorrhage. No medications, allergies, or vitals present."
+      }
+    },
+    {
+      "idx": 16,
+      "specialty": "Neurosurgery",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": 51,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "Severe low back pain",
+            "status": "preoperative",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "Severe low back pain",
+            "status": "postoperative",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "very degenerated disk at L5-S1, less so at L3-L4 and L4-L5",
+            "status": null,
+            "code": null,
+            "code_system": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Anterior lumbar fusion, L4-L5, L5-S1, PEEK vertebral spacer, structural autograft from L5 vertebral body, BMP and anterior plate",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Operative/surgical report (anterior lumbar interbody fusion); no enum value fits an op note, so document_type=other. Patient from summary \"This is a 51-year-old man\" -> age 51, sex male. Preoperative and postoperative diagnoses both \"Severe low back pain\" captured with respective status; the MRI-stated finding \"very degenerated disk at L5-S1, less so at L3-L4 and L4-L5\" captured as an additional diagnosis (status null). Procedure taken verbatim from the OPERATIONS PERFORMED line; no procedure date printed. Surgical implants/biologics (13-mm PEEK vertebral spacers, structural autograft, BMP, 15x20-mm Chronos VerteFill tricalcium phosphate plug, 87-mm Integra sacral plate/screws) are hardware/graft materials, not administered medications, so medications=[]. Anesthesia \"General endotracheal\" names no drug. \"Estimated blood loss: Less than 50 mL\", drains none, complications none are operative details, not vital signs -> vitals=[]. No labs, allergies, encounter date, or department stated. Historical items noted but not extracted as procedures (prior physical therapy, epidural steroid injections, MRI, positive discogram at lower 3 levels). Surgeon reference \"Dr. X\" excluded as a non-patient/provider identifier."
+      }
+    },
+    {
+      "idx": 17,
+      "specialty": "Neurology",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 51,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": "1/5/93",
+          "department": null
+        },
+        "vitals": [
+          {
+            "name": "BP",
+            "value": "164/82",
+            "unit": null
+          },
+          {
+            "name": "HR",
+            "value": "64",
+            "unit": null
+          },
+          {
+            "name": "RR",
+            "value": "18",
+            "unit": null
+          },
+          {
+            "name": "Temperature",
+            "value": "36.0",
+            "unit": "C"
+          }
+        ],
+        "labs": [],
+        "medications": [
+          {
+            "name": "Micronase",
+            "dose": "5mg",
+            "route": null,
+            "frequency": "qd",
+            "status": "active"
+          },
+          {
+            "name": "HCTZ",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": "active"
+          },
+          {
+            "name": "ASA",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": "discontinued"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "DM type 2",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "HTN",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "DJD",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "retinal detachment",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "hypodensity in the left caudate consistent with ischemic change",
+            "code": null,
+            "code_system": null,
+            "status": null
+          },
+          {
+            "description": "RACA territory stroke with recurrent artery of Heubner involvement",
+            "code": null,
+            "code_system": null,
+            "status": "suspected"
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Vitrectomy and retinal traction OU",
+            "date": "7/92"
+          },
+          {
+            "description": "Cholecystemomy",
+            "date": "1968"
+          },
+          {
+            "description": "Cataract implant OU",
+            "date": "1992"
+          },
+          {
+            "description": "C-section",
+            "date": null
+          },
+          {
+            "description": "EKG",
+            "date": "1/5/93"
+          },
+          {
+            "description": "CXR",
+            "date": "1/5/93"
+          },
+          {
+            "description": "HCT (head CT)",
+            "date": "1/5/93"
+          },
+          {
+            "description": "Carotid Duplex",
+            "date": null
+          },
+          {
+            "description": "Transthoracic echocardiogram",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Neurology admission H&P/progress note (UIHC); classified progress_note (no discharge disposition/date). Patient = 51 y/o RHF (right-handed female); age and sex only, no identifiers. Encounter date = 1/5/93 (came to UIHC for evaluation; also fell 1/3/93 and 1/4/93). Department not printed in the note text, so null. labs=[] because CBC, GS (including glucose), and PT/PTT are reported only collectively as 'unremarkable' with no discrete test+value pairs; no numeric lab values appear anywhere. Vitals BP 164/82, HR 64, RR 18, Temp 36.0 C taken verbatim from the EXAM line. Medications: Micronase 5mg qd active and HCTZ active; ASA discontinued ('quit ASA 6 months ago'). Diagnoses from PMH and assessment; DM type 2, HTN, DJD, retinal detachment carry no explicit status word so status=null; caudate hypodensity is a definite CT finding (hedge already in description) so status=null; RACA territory stroke marked 'suspected' per explicit hedge 'would invoke.' Excluded family-history conditions (grand aunt stroke; MG CAD; mother CAD/MI; father CA; sisters HTN) as they belong to relatives; excluded symptoms/exam findings (right leg/facial weakness, unsteady wide-based gait, breakaway weakness, RUE fisted posture) as signs/symptoms, not diagnosis statements. Surgical history and diagnostic studies captured as procedures with clean names + dates; the OCR-typo 'Cholecystemomy' is preserved exactly as written (NOT corrected), and the 's/p' status prefix was dropped from procedure names. Carotid Duplex findings (0-15% RICA, 16-49% LICA; antegrade vertebral flow bilaterally) and echocardiogram findings (borderline LV hypertrophy, normal LV function, no valvular abnormality/thrombus) are grounded but left as study context, not elevated to separate diagnoses, since they are mild/borderline and not stated as explicit diagnosis statements. No allergies stated (ASA stopped by preference, not allergy). No ICD/LOINC codes printed."
+      }
+    },
+    {
+      "idx": 18,
+      "specialty": "Speech - Language",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 60,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "hypertension",
+            "status": "history",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "TIA/stroke",
+            "status": "history",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "mild-to-moderate cognitive linguistic deficit",
+            "status": null,
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "Penetration with cup sips of thin liquid",
+            "status": null,
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "globus sensation",
+            "status": null,
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "reduced peristaltic action of the constricted muscles in the esophagus",
+            "status": null,
+            "code": null,
+            "code_system": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Modified barium swallow study",
+            "date": null
+          },
+          {
+            "description": "Outpatient evaluation revealing mild-to-moderate cognitive linguistic deficit",
+            "date": "approximately 2 months ago"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Speech-language pathology Modified Barium Swallow (video fluoroscopic) study report in SOAP format (SUBJECTIVE/OBJECTIVE/ASSESSMENT/PLAN); classified as progress_note. Patient is a 60-year-old female (stated). No encounter date printed. No named department; study performed in the 'Radiology Suite' (a location, not a stated department) in cooperation with 'Dr. ABC' (redacted provider name, not extracted) \u2014 department left null. No vitals, labs, medications, or allergies stated. Barium was administered as an imaging contrast agent (mixed with liquid/food), not prescribed as a therapeutic drug with dose/route, so medications is empty; the '90-degree angle for at least 45 minutes' upright positioning is a PLAN recommendation, not a vital sign. Diagnoses: past history (hypertension, TIA/stroke); prior finding (mild-to-moderate cognitive linguistic deficit); formal diagnostic impression (Penetration with cup sips of thin liquid); patient complaint of globus sensation; and the radiologist-noted reduced peristaltic action of the constricted esophageal muscles (an explicit positive abnormal finding, tied to the globus complaint). EXCLUDED: 'No aspiration' \u2014 explicitly negated finding, not a present condition; heartburn/gastroesophageal reflux disorder \u2014 explicitly DENIED by the patient. Coughing during meals is the presenting symptom and was not captured as a formal diagnosis. Procedures: the current modified barium swallow study (no date given) and the prior outpatient evaluation completed 'approximately 2 months ago' (grounded, date preserved verbatim). Plan: small bites/sips, upright positioning, referral to a gastroenterologist, and discharge from speech therapy (no skilled speech therapy needed at this time). No patient identifiers included per contract."
+      }
+    },
+    {
+      "idx": 19,
+      "specialty": "General Medicine",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 48,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": "Nephrology"
+        },
+        "vitals": [
+          {
+            "name": "Blood pressure",
+            "value": "180/110",
+            "unit": null
+          },
+          {
+            "name": "Temperature",
+            "value": "98.1",
+            "unit": null
+          },
+          {
+            "name": "Pulse rate",
+            "value": "60",
+            "unit": null
+          },
+          {
+            "name": "Respiratory rate",
+            "value": "23",
+            "unit": null
+          },
+          {
+            "name": "O2 sat",
+            "value": "95",
+            "unit": "%"
+          }
+        ],
+        "labs": [
+          {
+            "test": "WBC",
+            "value": "7",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "H and H",
+            "value": "13 and 40",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "platelets",
+            "value": "330",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "PT",
+            "value": "12",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "PTT",
+            "value": "26",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "CO2",
+            "value": "20",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "BUN",
+            "value": "27",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "creatinine",
+            "value": "3.1",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "cholesterol",
+            "value": "174",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "BNP",
+            "value": "973",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "troponin",
+            "value": "0.18",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "Previous creatinine",
+            "value": "2.7",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "Urine drug screen",
+            "value": "positive for cocaine",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [
+          {
+            "name": "Clonidine",
+            "dose": "0.3",
+            "route": "p.o.",
+            "frequency": "q.8",
+            "status": null
+          },
+          {
+            "name": "aspirin",
+            "dose": "325",
+            "route": null,
+            "frequency": "daily",
+            "status": null
+          },
+          {
+            "name": "hydralazine",
+            "dose": "100",
+            "route": null,
+            "frequency": "q.8",
+            "status": null
+          },
+          {
+            "name": "Lipitor",
+            "dose": "20",
+            "route": null,
+            "frequency": "at bedtime",
+            "status": null
+          },
+          {
+            "name": "Toprol XL",
+            "dose": "100",
+            "route": null,
+            "frequency": "daily",
+            "status": null
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "Coronary artery disease",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "COPD",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "Congestive heart failure with ejection fraction of 20%-25%",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "Hypertension",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "Renal insufficiency",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "Recurrent episodes of hypertensive emergency",
+            "code": null,
+            "code_system": null,
+            "status": "history"
+          },
+          {
+            "description": "Hypertensive emergency",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "Acute on chronic renal failure",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "CHF versus COPD exacerbation",
+            "code": null,
+            "code_system": null,
+            "status": "suspected"
+          },
+          {
+            "description": "Chronic cocaine abuse",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          }
+        ],
+        "procedures": [],
+        "allergies": [],
+        "extraction_notes": "Nephrology consultation note (reason for consult: renal insufficiency); classified as progress_note (no 'consult' enum value). No encounter date is printed (December 2005 = CHF EF onset; 'December' = timing of a prior creatinine, neither is the encounter date). Department 'Nephrology' per 'Nephrology is consulted regarding renal insufficiency.' Patient is a 48-year-old African-American male; per contract only age/sex captured (race/ethnicity not part of patient object; no identifiers present). Allergies 'NO KNOWN DRUG ALLERGIES' is a negative finding, so allergies=[]. O2 sat recorded as value '95' unit '%'; 'on room air' is a measurement condition, not a unit, and there is no field for it, so it is dropped. 'H and H 13 and 40' preserved verbatim (clinically Hemoglobin 13 / Hematocrit 40) rather than split. Previous creatinine 2.7 captured as a separate lab; the 'in December' qualifier is not folded into the test name (no lab-date field). Urine drug screen captured as a lab (value 'positive for cocaine'); assessment item 'Urine drug screen positive' treated as this lab finding, not a diagnosis. 'Chronic cocaine abuse' captured as a diagnosis (cited as etiology in the plan). 'Question CHF versus COPD exacerbation' status normalized to 'suspected' (questioned/differential, unconfirmed). No lab units, reference ranges, or H/L flags were printed. Medication statuses not labeled, left null; doses carried no printed units. No procedures documented (physical exam findings are not procedures). No ICD/LOINC codes printed, so all codes left null."
+      }
+    },
+    {
+      "idx": 20,
+      "specialty": "Urology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": 52,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "Adrenal mass, right sided",
+            "code": null,
+            "code_system": null,
+            "status": "preoperative"
+          },
+          {
+            "description": "Umbilical hernia",
+            "code": null,
+            "code_system": null,
+            "status": "preoperative"
+          },
+          {
+            "description": "Adrenal mass, right sided",
+            "code": null,
+            "code_system": null,
+            "status": "postoperative"
+          },
+          {
+            "description": "Umbilical hernia",
+            "code": null,
+            "code_system": null,
+            "status": "postoperative"
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Laparoscopic hand-assisted left adrenalectomy and umbilical hernia repair",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Surgical operative note; no enum matches an operative report, so document_type mapped to \"other\". Age 52 explicit (\"52-year-old inmate\"); sex male grounded in pronoun \"his\" (\"mass in his right adrenal\"). \"inmate\" is not a schema field/identifier, omitted. No patient identifiers present. Encounter date and department not printed in the document (Urology specialty was external metadata, so department left null). Diagnoses appear in both PREOPERATIVE DIAGNOSES and POSTOPERATIVE DIAGNOSES sections (identical); all four instances captured with status to preserve the stated distinction. Internal source discrepancy preserved as written: diagnoses/clinical note reference a RIGHT-sided 5.5 cm nonfunctioning adrenal mass and \"right flank-up position\", while OPERATION PERFORMED states \"left adrenalectomy\" \u2014 values not corrected. Anesthesia \"General\" is not a drug name; surgical materials (Vicryl, Bovie, harmonic scalpel, GelPort, Foley catheter, trocars, clips) are not medications, so medications empty. No test+value pairs, so labs empty. No vital signs; \"Estimated blood loss less than 100 mL\" is an operative measurement, not a vital sign, so vitals empty. No allergies stated."
+      }
+    },
+    {
+      "idx": 21,
+      "specialty": "Nephrology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Right renal mass",
+            "status": "preoperative"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Right renal mass",
+            "status": "postoperative"
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Right radical nephrectomy and assisted laparoscopic approach"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Surgical operative note (right radical nephrectomy, hand-assisted laparoscopic approach). No enum value fits an operative report, so document_type=\"other\". Sex=\"male\" grounded in repeated masculine pronouns (\"He was placed...\", \"He was widely shaved\"); age not stated (null). No encounter date or department stated. Diagnoses captured as PREOPERATIVE and POSTOPERATIVE \"Right renal mass\", identical text but kept as separate entries to preserve stated status. Procedure taken verbatim from the PROCEDURE line. Anesthesia \"General\" is an anesthesia type, not a named drug, and has no schema field, so no medication extracted. Surgical supplies/instruments (Surgicel, 0 Vicryl, Harmonic scalpel, stapler, Pneumo sleeve, trocars, Foley catheter, orogastric tube) are devices/supplies, not medications. Intraoperative steps (ureter division, renal artery/vein ligation) are steps within the nephrectomy, not separately documented procedures. Estimated blood loss \"negligible\" is qualitative with no numeric value, so not captured as a lab/vital. No vitals, labs, or allergies stated. No patient identifiers present."
+      }
+    },
+    {
+      "idx": 22,
+      "specialty": "Dentistry",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "description": "Right buccal space infection and abscess tooth #T",
+            "status": "preoperative",
+            "code": null,
+            "code_system": null
+          },
+          {
+            "description": "Right buccal space infection and abscess tooth #T",
+            "status": "postoperative",
+            "code": null,
+            "code_system": null
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Extraction of tooth #T",
+            "date": null
+          },
+          {
+            "description": "Incision and drainage (I&D) of right buccal space infection",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Dental/oral surgery operative note; no enum fits 'operative note' so document_type='other'. Preoperative and postoperative diagnoses are identical ('Right buccal space infection and abscess tooth #T'), captured as two diagnosis entries with respective status. Combined procedure header split into two procedure entries (extraction of tooth #T; incision and drainage of right buccal space infection); no procedure date stated. Specimens/cultures (aerobic and anaerobic cultures, culture swabs, and the tooth) were sent to the laboratory for culture and sensitivity testing, but NO result/value is reported, so no labs (a lab requires test+value). Anesthesia was 'General, oral endotracheal tube' and IV fluid 150 mL are not drug names, so medications is empty. Intraoperative figures (IV fluid 150 mL, estimated blood loss 10 mL, ~1 mL purulent aspirate) are procedure metrics, not vital signs, so vitals is empty. Complications: none. No patient age/sex, encounter date, department, allergies, or identifiers stated in the document."
+      }
+    },
+    {
+      "idx": 23,
+      "specialty": "Obstetrics / Gynecology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": null
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Ovarian cyst, persistent",
+            "status": "preoperative"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Ovarian cyst",
+            "status": "postoperative"
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Diagnostic laparoscopy and drainage of cyst"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Gynecologic operative/procedure note; document_type='other' since an operative report is not among the enumerated types. Preoperative diagnosis 'Ovarian cyst, persistent' and postoperative diagnosis 'Ovarian cyst' captured as two diagnoses with pre/postoperative status. Named operation 'Diagnostic laparoscopy and drainage of cyst' captured as a single procedure; intraoperative steps (infraumbilical incision, Veress needle, second trocar, 3-cm left ovarian cyst needled and incised) are components of that operation and not split into separate entries. Anesthesia documented as 'General' with no specific drug named, so no medication entry (a medication requires a drug name). Sex not printed (ovarian cyst implies female but that is inference, disallowed), so sex=null. No age, dates, department, vitals, labs, or allergies stated. No patient identifiers present."
+      }
+    },
+    {
+      "idx": 24,
+      "specialty": "Psychiatry / Psychology",
+      "gold": {
+        "document_type": "discharge_summary",
+        "patient": {
+          "age": 82,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [
+          {
+            "name": "Tylenol",
+            "dose": "650 mg",
+            "route": null,
+            "frequency": "q.6h. p.r.n.",
+            "status": "discharge"
+          },
+          {
+            "name": "Xanax",
+            "dose": "0.5",
+            "route": null,
+            "frequency": "q.4h. p.r.n.",
+            "status": "discharge"
+          },
+          {
+            "name": "Lasix",
+            "dose": "80 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Isordil",
+            "dose": "10 mg",
+            "route": null,
+            "frequency": "t.i.d.",
+            "status": "discharge"
+          },
+          {
+            "name": "KCl",
+            "dose": "20 mEq",
+            "route": null,
+            "frequency": "b.i.d.",
+            "status": "discharge"
+          },
+          {
+            "name": "lactulose",
+            "dose": "10 g",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Cozaar",
+            "dose": "50 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Synthroid",
+            "dose": "75 mcg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Singulair",
+            "dose": "10 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Lumigan",
+            "dose": "one drop",
+            "route": "both eyes",
+            "frequency": "at bed time",
+            "status": "discharge"
+          },
+          {
+            "name": "NitroQuick",
+            "dose": null,
+            "route": null,
+            "frequency": "p.r.n.",
+            "status": "discharge"
+          },
+          {
+            "name": "Pravachol",
+            "dose": "20 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Feldene",
+            "dose": "20 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Paxil",
+            "dose": "20 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Minipress",
+            "dose": "2 mg",
+            "route": null,
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "Provera",
+            "dose": null,
+            "route": null,
+            "frequency": "p.r.n.",
+            "status": "discharge"
+          },
+          {
+            "name": "Advair",
+            "dose": "250/50",
+            "route": null,
+            "frequency": "one puff b.i.d.",
+            "status": "discharge"
+          },
+          {
+            "name": "Senokot",
+            "dose": "one tablet",
+            "route": null,
+            "frequency": "b.i.d.",
+            "status": "discharge"
+          },
+          {
+            "name": "Timoptic",
+            "dose": "one drop",
+            "route": "OU",
+            "frequency": "daily",
+            "status": "discharge"
+          },
+          {
+            "name": "verapamil",
+            "dose": "80 mg",
+            "route": null,
+            "frequency": "b.i.d.",
+            "status": "discharge"
+          },
+          {
+            "name": "Darvocet",
+            "dose": null,
+            "route": null,
+            "frequency": null,
+            "status": "home"
+          }
+        ],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Falls",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Anxiety and depression",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Hypertension",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Hypercholesterolemia",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Coronary artery disease",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Osteoarthritis",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Chronic obstructive pulmonary disease",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "Hypothyroidism",
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Psychiatric evaluation"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Hospital discharge summary (Chief Complaint, HPI, Physical Exam, Hospital Course, Discharge Diagnoses, Condition Upon Discharge, Discharge Medications, Allergies, Activity, Follow-up) including a psychiatric consult; classified as discharge_summary. Patient is an 82-year-old female; no identifiers present. No encounter date and no discrete department printed (presented to the ER, psychiatric evaluation obtained, discharged to a skilled nursing/rehab facility) so both null. Vitals documented only as 'Stable' with no numeric values (carotid upstrokes 2+ is an exam finding, not a vital), so vitals=[]. No labs (no test+value). Allergies explicitly 'None', so allergies=[]. The 20 DISCHARGE MEDICATIONS captured with status 'discharge', values/units preserved exactly (Xanax '0.5' no unit; Advair '250/50'; eye-drop routes 'both eyes'/'OU'; frequency-only entries NitroQuick and Provera 'p.r.n.'). Darvocet captured separately with status 'home' from the HPI ('frequently takes Darvocet for her anxiety'); it is not on the discharge list and no dose/frequency is stated. Paxil and Xanax noted as 'Continue' in Hospital Course but already on the discharge list (not duplicated). Discharge diagnosis #2 kept as the single printed item 'Anxiety and depression' rather than split. 'Psychiatric evaluation' captured as a procedure (borderline consult/assessment). No ICD/LOINC codes printed, so all codes null."
+      }
+    },
+    {
+      "idx": 26,
+      "specialty": "Rheumatology",
+      "gold": {
+        "document_type": "progress_note",
+        "patient": {
+          "age": 12,
+          "sex": "female"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [
+          {
+            "name": "temperature",
+            "value": "100.1",
+            "unit": null
+          },
+          {
+            "name": "weight",
+            "value": "73.5",
+            "unit": "kg"
+          },
+          {
+            "name": "blood pressure",
+            "value": "121/61",
+            "unit": null
+          },
+          {
+            "name": "height",
+            "value": "158",
+            "unit": null
+          },
+          {
+            "name": "pulse",
+            "value": "84",
+            "unit": null
+          }
+        ],
+        "labs": [
+          {
+            "test": "white blood cell count",
+            "value": "7.9",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "hemoglobin",
+            "value": "14.3",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "platelet count",
+            "value": "321,000",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          },
+          {
+            "test": "sed rate",
+            "value": "11",
+            "unit": null,
+            "reference_range": null,
+            "flag": null
+          }
+        ],
+        "medications": [
+          {
+            "name": "Plaquenil",
+            "dose": null,
+            "frequency": null,
+            "route": null,
+            "status": "continued"
+          },
+          {
+            "name": "hydrocortisone",
+            "dose": null,
+            "frequency": "at night",
+            "route": null,
+            "status": "discontinued"
+          },
+          {
+            "name": "Protopic",
+            "dose": null,
+            "frequency": "at night",
+            "route": null,
+            "status": "new"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "discoid lupus",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "acanthosis nigricans",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          }
+        ],
+        "procedures": [
+          {
+            "description": "Diet evaluation",
+            "date": null
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Rheumatology/CCS follow-up progress note for a 12-year-old female with discoid lupus. Encounter date and department are not explicitly printed (specialty was external metadata only), so both null. Vitals recorded exactly as written; only weight has a printed unit (kg); temperature (100.1), blood pressure (121/61), height (158), and pulse (84) have no printed units. Labs: only test+value pairs captured (CBC components and sed rate); 'CMP shows no abnormalities' has no numeric value and antinuclear antibody/complement level are explicitly pending, so all three excluded. Medications: Plaquenil continued ('Continue on Plaquenil'); hydrocortisone cream used at night marked discontinued because the plan explicitly states 'switch her to Protopic' (a switch replaces/stops the prior agent); Protopic is the new agent started at night; route left null as topical is implied but not labeled. Diagnoses: discoid lupus (active, 'on the control with optimal regimen') and acanthosis nigricans (active present exam finding, 'she does have acanthosis nigricans on the base of the neck'). Systemic/full-blown lupus is only a future watch, not a current diagnosis, so excluded. Procedures: 'Diet evaluation today' captured as a service performed (date null, only 'today' stated). No allergies, no printed ICD/LOINC codes, no identifiers."
+      }
+    },
+    {
+      "idx": 27,
+      "specialty": "Radiology",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": 38,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [],
+        "diagnoses": [
+          {
+            "code": null,
+            "code_system": null,
+            "description": "right upper lobe pneumonia",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "AIDS",
+            "status": "history"
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "decreased mental status",
+            "status": null
+          },
+          {
+            "code": null,
+            "code_system": null,
+            "description": "diffuse abdominal pain",
+            "status": null
+          }
+        ],
+        "procedures": [
+          {
+            "date": null,
+            "description": "Ultrasound abdomen, complete"
+          }
+        ],
+        "allergies": [],
+        "extraction_notes": "Radiology complete abdominal ultrasound report; no enum category fits, so classified as \"other\". The only printed date, 04/18/2009, is explicitly the emergency-room admission date in the HISTORY (not the stated exam/encounter date), so encounter.date left null to avoid inference. Department not printed in the document, so null. Diagnoses are the explicit HISTORY conditions: right upper lobe pneumonia, decreased mental status, and diffuse abdominal pain (reasons for admission / current), plus AIDS marked status=history (\"There is a history of AIDS\"). IMPRESSION items were excluded as diagnoses because they are normal/negative findings: \"Spleen size at the upper limits of normal\" (i.e., within normal range, despite being \"somewhat prominent\"), pancreas obscured by bowel gas with normal visualized head, no gallstones, no renal calculi. Imaging measurements (common bile duct 4.6 mm, right kidney 10.8 cm, left kidney 10.5 cm, spleen max diameter 11.2 cm) are anatomical ultrasound findings, not lab tests with a test+value and not vitals, and no schema field captures organ measurements, so labs and vitals are empty. No medications, allergies, or vital signs stated. No patient identifiers present."
+      }
+    },
+    {
+      "idx": 29,
+      "specialty": "Letters",
+      "gold": {
+        "document_type": "other",
+        "patient": {
+          "age": null,
+          "sex": "male"
+        },
+        "encounter": {
+          "date": null,
+          "department": null
+        },
+        "vitals": [],
+        "labs": [],
+        "medications": [
+          {
+            "name": "Flonase",
+            "dose": "two sprays to each nostril",
+            "frequency": "once a day",
+            "route": null,
+            "status": "active"
+          },
+          {
+            "name": "Zyrtec",
+            "dose": "10 mg",
+            "frequency": "a day",
+            "route": null,
+            "status": "active"
+          }
+        ],
+        "diagnoses": [
+          {
+            "description": "bulbar cerebral palsy",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "hypotonia",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "significant tonsillar hypertrophy",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          },
+          {
+            "description": "chronic allergic rhinitis",
+            "code": null,
+            "code_system": null,
+            "status": "active"
+          }
+        ],
+        "procedures": [],
+        "allergies": [
+          {
+            "substance": "penicillin",
+            "reaction": null,
+            "severity": null
+          }
+        ],
+        "extraction_notes": "Physician introduction/referral letter (not a listed structured type), so document_type=\"other\". Patient name (\"A\") and physician names (\"Dr. X\") withheld as identifiers. Sex=\"male\" grounded in \"young man\" and \"He/his\"; no numeric age stated (\"young man\" is not a number) so age=null. No encounter date; \"pediatric neurology clinic\" is where prior treatment by Dr. X occurred (background), not this document's encounter department, so department=null. No vitals or labs (no test+value pairs). Medications: Flonase two sprays to each nostril once a day (currently on) and Zyrtec 10 mg a day (partial relief); route left null since \"nasal\" not explicitly written. Diagnoses: bulbar cerebral palsy and hypotonia (explicit \"diagnosis of\"), significant tonsillar hypertrophy (noted and confirmed), and chronic allergic rhinitis (explicit). Symptoms/findings (difficulty with mouth breathing, speech and swallowing problems) not recorded as diagnoses. Tonsillectomy is only being considered/asked about, not performed, so procedures empty. Penicillin allergy stated with no reaction or severity."
+      }
+    }
+  ]
+}

training/env_versions.json

@@ -0,0 +1,11 @@
+{
+  "unsloth": "2026.6.9",
+  "unsloth_zoo": "2026.6.7",
+  "trl": "0.24.0",
+  "transformers": "5.5.0",
+  "peft": "0.19.1",
+  "accelerate": "1.14.0",
+  "datasets": "4.3.0",
+  "bitsandbytes": "0.49.2",
+  "torch": "2.10.0+cu128"
+}

training/metrics.json

@@ -0,0 +1,36 @@
+{
+  "model": "Qwen/Qwen3-4B-Instruct-2507",
+  "model_name": "Mira-3-v11",
+  "max_seq_length": 3072,
+  "lora_r": 16,
+  "lora_alpha": 32,
+  "epochs_target": 3,
+  "batch_size": 2,
+  "grad_accum": 8,
+  "lr": 0.0002,
+  "packing": true,
+  "use_unsloth": true,
+  "load_in_4bit": true,
+  "seed": 42,
+  "env_versions": {
+    "unsloth": "2026.6.9",
+    "unsloth_zoo": "2026.6.7",
+    "trl": "0.24.0",
+    "transformers": "5.5.0",
+    "peft": "0.19.1",
+    "accelerate": "1.14.0",
+    "datasets": "4.3.0",
+    "bitsandbytes": "0.49.2",
+    "torch": "2.10.0+cu128"
+  },
+  "elapsed_hours": 10.24,
+  "timestamp": "2026-07-02T00:28:21.448741",
+  "gpu": "Tesla T4",
+  "sm": 75,
+  "train_examples": 21000,
+  "epoch_reached": 3.0,
+  "global_step": 867,
+  "max_steps": 867,
+  "resumed_from_step": 453,
+  "training_complete": true
+}