Add inference_contract

inference_contract/INFERENCE.md

@@ -0,0 +1,105 @@
+# Inference contract - FlowX Sentinel Gate
+
+This is the **frozen inference contract** for `flowxai/sentinel-gate`: the exact system
+prompt, user-turn format, decode settings, and output schema the weights were trained
+against. Do not edit the prompt or schema; the LoRA was trained on them verbatim.
+
+Prompt version: `sentinel_sys_v1`.
+
+Files in this directory:
+
+- [`prompt_sentinel_sys_v1.txt`](./prompt_sentinel_sys_v1.txt) - the system prompt, verbatim.
+- [`schema_sentinel_v1.json`](./schema_sentinel_v1.json) - JSON Schema for the oracle output.
+
+---
+
+## System prompt (verbatim)
+
+The exact two-line system prompt is in
+[`prompt_sentinel_sys_v1.txt`](./prompt_sentinel_sys_v1.txt):
+
+```
+You are an escalation gate for regulated decisions.
+Determine: ESCALATE or DECIDE? Output ONLY JSON.
+```
+
+## User-turn format
+
+One case per turn. The case JSON carries the domain facts plus the applicable
+`policy_schema` (a `PDP...` policy id), then a fixed trailing question:
+
+```
+Case:
+<case JSON: domain facts + "policy_schema": "PDP...">
+
+Decide: ESCALATE or DECIDE?
+```
+
+The `Case:\n` prefix and the trailing `\n\nDecide: ESCALATE or DECIDE?` line are part of
+the contract - keep them exactly. The model was trained with these delimiters framing the
+case object.
+
+## Decode settings
+
+| Setting | Value | Why |
+|---|---|---|
+| `enable_thinking` | **`False`** | Qwen3-4B is a thinking model, but the adapter was trained on pure JSON with no thinking block. The default template yields empty/degraded output. Set this at `apply_chat_template`. |
+| `temperature` | **`0`** (greedy) | Deterministic decisions; the gate must be reproducible for audit. |
+| `max_new_tokens` | **~1200** | The oracle JSON (category block + reasoning + audit trail) can run long, especially for BOUNDARY_CONDITION and EXTERNAL_DEPENDENCY. Truncation is the main cause of invalid JSON. |
+
+## The six escalation categories
+
+Present as `escalation_category` when `action` is `ESCALATE` (it is `null` for `DECIDE`):
+
+1. `MISSING_REQUIRED_DOCUMENTATION` - a hard precondition document/evidence is absent.
+2. `POLICY_VIOLATION` - a policy/regulation rule is triggered and blocks auto-release.
+3. `BOUNDARY_CONDITION` - the case sits near a policy threshold; the edge needs a human read.
+4. `INSUFFICIENT_CONFIDENCE` - the facts do not resolve the decision to an actionable degree.
+5. `CONFLICTING_SIGNALS` - two or more trusted sources disagree materially.
+6. `EXTERNAL_DEPENDENCY` - the decision is blocked awaiting an outside result (screening, ruling).
+
+Each category emits a category-specific block under a distinct key
+(`policy_violations`, `missing_preconditions`, `boundary_analysis`, `confidence_factors`,
+`conflicting_signals`, `external_dependency`). See `schema_sentinel_v1.json`.
+
+## Deterministic JSON repair (deploy with it)
+
+Raw JSON validity from the model is **0.89** on the held-out set. The deployed pipeline
+pairs the model with a **deterministic JSON repair step**: parse the raw output; if it
+fails, apply structural fixes (close unterminated strings/brackets, strip any trailing
+prose after the final `}`, drop a leading thinking artifact if one leaks) and re-parse, then
+validate against `schema_sentinel_v1.json`. On a repair failure, retry the decode once. Do
+not rely on raw output being parseable; treat the repair step as part of the contract.
+
+## What to gate on
+
+- **Gate on the `action` field (ESCALATE vs DECIDE).** This is the decision the model is
+  for, and it is **perfect on the held-out set** (n=71): zero missed escalations
+  (false-negative rate 0.000) and zero over-escalation (false-positive rate 0.000). Wire
+  your automate-vs-route branch off `action` alone.
+- **Treat `escalation_category` as a routing hint, not ground truth.** Category accuracy on
+  true-escalate is **~0.61**; the categories legitimately overlap for some cases (e.g. a
+  boundary case that is also a policy edge). Use it to pick a specialist queue, but do not
+  make correctness-critical branches depend on it, and let a human re-label at intake.
+- `confidence_score` is calibrated per the training oracle; apply the threshold your risk
+  posture requires. It is advisory, not a second gate.
+
+## Minimal wiring (MLX)
+
+```python
+from mlx_lm import load, generate
+
+SYSTEM = open("prompt_sentinel_sys_v1.txt").read()
+model, tok = load("flowxai/sentinel-gate-mlx-int4")
+
+case_json = "<case JSON with domain facts + policy_schema>"
+user = f"Case:\n{case_json}\n\nDecide: ESCALATE or DECIDE?"
+
+prompt = tok.apply_chat_template(
+    [{"role": "system", "content": SYSTEM},
+     {"role": "user", "content": user}],
+    add_generation_prompt=True, enable_thinking=False,
+)
+raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False)
+# then: deterministic JSON repair -> validate against schema_sentinel_v1.json
+```

inference_contract/prompt_sentinel_sys_v1.txt

@@ -0,0 +1,2 @@
+You are an escalation gate for regulated decisions.
+Determine: ESCALATE or DECIDE? Output ONLY JSON.

inference_contract/schema_sentinel_v1.json

@@ -0,0 +1,129 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "https://huggingface.co/flowxai/sentinel-gate/inference_contract/schema_sentinel_v1.json",
+  "title": "FlowX Sentinel Gate output (schema_sentinel_v1)",
+  "description": "The single JSON object the Sentinel Gate emits for one regulated-decision case. The gate decides ESCALATE (route to a human) vs DECIDE (safe to automate). When action=ESCALATE, escalation_category is one of six ids and a category-specific block is present; when action=DECIDE, escalation_category is null. Prompt version sentinel_sys_v1. Note: additionalProperties is intentionally true because the category-specific block key varies by category (policy_violations, missing_preconditions, boundary_analysis, confidence_factors, conflicting_signals, external_dependency) and DECIDE cases carry decision/rationale keys.",
+  "type": "object",
+  "additionalProperties": true,
+  "required": [
+    "action",
+    "escalation_category",
+    "confidence_score",
+    "audit_trail"
+  ],
+  "properties": {
+    "action": {
+      "type": "string",
+      "description": "The gate decision. ESCALATE routes the case to a human; DECIDE marks it safe to automate. This is the field to gate on (perfect on the held-out set).",
+      "enum": ["ESCALATE", "DECIDE"]
+    },
+    "escalation_category": {
+      "description": "The human-routing label for an escalation, or null when action=DECIDE. A secondary routing hint (~0.61 accuracy on true-escalate), not the gate.",
+      "type": ["string", "null"],
+      "enum": [
+        "MISSING_REQUIRED_DOCUMENTATION",
+        "POLICY_VIOLATION",
+        "BOUNDARY_CONDITION",
+        "INSUFFICIENT_CONFIDENCE",
+        "CONFLICTING_SIGNALS",
+        "EXTERNAL_DEPENDENCY",
+        null
+      ]
+    },
+    "confidence_score": {
+      "type": "number",
+      "description": "Calibrated confidence in the decision, 0.0-1.0. For ESCALATE this is typically the confidence that the case is safe to automate (low), so a low score supports escalation; for DECIDE it is the confidence in the auto-decision (high).",
+      "minimum": 0.0,
+      "maximum": 1.0
+    },
+    "confidence_reasoning": {
+      "type": "string",
+      "description": "One to three sentences explaining the confidence_score and why the case was escalated or auto-decided."
+    },
+    "human_action_required": {
+      "type": "string",
+      "description": "For ESCALATE: the concrete next step a human owner must take (who does what). For DECIDE: the string \"NONE\"."
+    },
+    "audit_trail": {
+      "type": "array",
+      "description": "Ordered, append-only log of the reasoning steps and policy gates evaluated, for compliance review.",
+      "items": { "type": "string" },
+      "minItems": 1
+    },
+    "policy_violations": {
+      "type": "object",
+      "description": "Category-specific block for POLICY_VIOLATION. Keyed by violation id; each entry names the policy, regulation, restriction, and consequence.",
+      "additionalProperties": true
+    },
+    "missing_preconditions": {
+      "type": "object",
+      "description": "Category-specific block for MISSING_REQUIRED_DOCUMENTATION. Keyed by the missing precondition; each entry names required_by, regulation, severity, and reason.",
+      "additionalProperties": true
+    },
+    "boundary_analysis": {
+      "type": "object",
+      "description": "Category-specific block for BOUNDARY_CONDITION. Names the policy_threshold, the shipment/case value, distance_from_threshold, and an assessment of the edge case.",
+      "additionalProperties": true
+    },
+    "confidence_factors": {
+      "type": "object",
+      "description": "Category-specific block for INSUFFICIENT_CONFIDENCE. Lists ambiguous_signals and why_uncertain.",
+      "additionalProperties": true
+    },
+    "conflicting_signals": {
+      "type": "array",
+      "description": "Category-specific block for CONFLICTING_SIGNALS. The competing sources/values that disagree.",
+      "items": { "type": "object", "additionalProperties": true }
+    },
+    "external_dependency": {
+      "type": "object",
+      "description": "Category-specific block for EXTERNAL_DEPENDENCY. Names what the decision is awaiting and the blocking_gate.",
+      "additionalProperties": true
+    },
+    "escalation_path": {
+      "type": "string",
+      "description": "Optional routing hint naming the specialist queue or workflow that should own the escalation."
+    },
+    "policy_gates_passed": {
+      "type": "array",
+      "description": "Optional list of policy gates that were checked and passed before the decision (present on some ESCALATE edge cases and on DECIDE cases).",
+      "items": { "type": "string" }
+    },
+    "decision": {
+      "type": "string",
+      "description": "For DECIDE cases: the automated outcome selected (e.g. ROUTE_APPROVED)."
+    },
+    "selected_route": {
+      "type": "string",
+      "description": "For DECIDE cases where a route/option is chosen: the selected option."
+    },
+    "rationale": {
+      "type": "string",
+      "description": "For DECIDE cases: the plain rationale for auto-deciding (some records use confidence_reasoning for this)."
+    }
+  },
+  "allOf": [
+    {
+      "if": { "properties": { "action": { "const": "DECIDE" } } },
+      "then": { "properties": { "escalation_category": { "type": "null" } } }
+    },
+    {
+      "if": { "properties": { "action": { "const": "ESCALATE" } } },
+      "then": {
+        "properties": {
+          "escalation_category": {
+            "type": "string",
+            "enum": [
+              "MISSING_REQUIRED_DOCUMENTATION",
+              "POLICY_VIOLATION",
+              "BOUNDARY_CONDITION",
+              "INSUFFICIENT_CONFIDENCE",
+              "CONFLICTING_SIGNALS",
+              "EXTERNAL_DEPENDENCY"
+            ]
+          }
+        }
+      }
+    }
+  ]
+}