Initial commit: Emotional Support Conversations OpenEnv environment

OpenEnv RL environment for open-ended emotional support dialogue with a
hybrid immediate + future-oriented reward signal inspired by RLFF-ESC
(Yang et al., 2025, arXiv:2508.12935).

- Deterministic FSM seeker with hidden distress/trust/openness state
- 3 graded tasks: work_stress_venting, guarded_relationship, crisis_fragile_trust
- FastAPI server exposing /reset /step /state /tasks
- Mandated inference.py with OpenAI client + [START]/[STEP]/[END] stdout
- Dockerfile and HF Space metadata

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.pyc
+*.pyo
+.venv/
+venv/
+.env
+.DS_Store
+.claude/

Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.11-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PORT=7860
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 7860
+
+CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -0,0 +1,187 @@
+---
+title: Emotional Support Conversations (OpenEnv)
+emoji: 💬
+sdk: docker
+pinned: false
+tags:
+  - openenv
+---
+
+# Emotional Support Conversations — OpenEnv Environment
+
+> An OpenEnv RL environment for evaluating agents on **open-ended emotional
+> support conversations**, with a hybrid immediate + future-oriented reward
+> signal inspired by **RLFF-ESC** (Yang, Chen, Wang, 2025,
+> [arXiv:2508.12935](https://arxiv.org/abs/2508.12935)).
+
+## Why this environment
+
+Emotional support is one of the tasks humans most want AI assistants to do
+well — and one of the easiest to do badly. Existing dialogue benchmarks
+score turn-level responses in isolation, which rewards agents for *sounding*
+empathetic without ever testing whether their replies actually move the
+person toward resolution. This environment closes that gap.
+
+Three properties make it a genuine RL problem, not a single-shot dialogue
+task:
+
+1. **Partial observability.** The seeker's distress, trust, and willingness
+   to reveal their real issue are hidden state. The agent must infer them
+   from the conversation so far.
+2. **Sequential credit assignment.** A warm reply at turn 2 can unlock a
+   disclosure at turn 6. A single dismissive reply at turn 4 can collapse
+   the whole trajectory and require several turns to recover.
+3. **Exploration vs commitment.** Should the agent keep exploring feelings
+   or move toward an action plan? Commit too early and the seeker shuts
+   down; explore too long and the episode times out.
+
+## Reward design (RLFF-ESC-inspired)
+
+Each step reward is:
+
+```
+step_reward = clip( 0.45 · immediate  +  0.55 · future_oriented  −  penalties , 0, 1 )
+```
+
+- **`immediate`** — stage-appropriate empathy/validation/open-question fit,
+  plus turn-level deltas in the seeker's trust and distress.
+- **`future_oriented`** — a k-step oracle rollout from both the pre- and
+  post-action seeker states. The reward is proportional to how much the
+  agent's action *preserves or advances the attainable resolution ceiling*,
+  not just how good the current turn looks in isolation. This is the
+  RLFF-ESC idea: reward signals propagated from projected trajectories
+  rather than pointwise turn critique.
+- **`penalties`** — dismissive language, premature advice (before trust is
+  established), bare replies, interrogation.
+
+A final task score combines average shaped reward, the seeker's final
+resolution state, and efficiency (finishing within turn budget).
+
+## Tasks (3 difficulties)
+
+| Task ID                  | Difficulty | Max turns | Core challenge                                                               |
+| ------------------------ | ---------- | --------- | ---------------------------------------------------------------------------- |
+| `work_stress_venting`    | easy       | 10        | Cooperative seeker venting about work. Reach closing with trust ≥ 0.7.       |
+| `guarded_relationship`   | medium     | 12        | Guarded seeker; real issue hidden behind surface concern until openness ≥ 0.75. Premature advice heavily punished. |
+| `crisis_fragile_trust`   | hard       | 14        | High-distress, fragile trust, multiple interleaved concerns. One misstep triggers large trust drops that take several empathic turns to recover. Safety referencing rewarded in closing stage. |
+
+Success thresholds (final score) are `0.55 / 0.50 / 0.45` respectively.
+
+## Action & observation space
+
+**Action** — free-text reply to the seeker:
+
+```python
+class Action(BaseModel):
+    message: str
+```
+
+**Observation** — what the agent sees each turn (deliberately partial):
+
+```python
+class Observation(BaseModel):
+    seeker_utterance: str
+    turn: int
+    remaining_turns: int
+    stage_hint: str        # 'opening' | 'exploring' | 'reflecting' | 'planning' | 'closing'
+    task_id: str
+    scenario_brief: str
+```
+
+The seeker's internal hidden variables (distress, trust, openness, true
+issue) are **never** exposed. This is by design.
+
+## Environment internals (why deterministic)
+
+The seeker is a deterministic finite-state machine with continuous hidden
+variables (`distress`, `trust`, `openness`, `revealed`, `stage`). On each
+turn, the agent's reply is analysed with keyword/regex feature detectors
+(empathy markers, validation, open vs closed questions, advice, dismissive
+language, interrogation) and hidden state advances via transparent rules.
+
+**Why not use an LLM-driven seeker?** The hackathon rubric requires
+graders to be *deterministic and reproducible* — an LLM-driven seeker
+would fail the Phase 2 score-variance check. Deterministic dynamics give
+us full reproducibility while still producing rich, sequential, partially
+observable dialogue with genuine recovery-from-mistakes dynamics.
+
+## HTTP API (OpenEnv spec)
+
+| Method | Path      | Body                                         | Returns                                 |
+| ------ | --------- | -------------------------------------------- | --------------------------------------- |
+| GET    | `/`       | —                                            | health + metadata                       |
+| GET    | `/tasks`  | —                                            | list of tasks                           |
+| POST   | `/reset`  | `{"task_id": "...", "seed": null}` (opt.)    | `ResetResult` (observation + info)      |
+| POST   | `/step`   | `{"action": {"message": "..."}}`             | `StepResult` (obs, reward, done, info)  |
+| GET    | `/state`  | —                                            | `EnvState` (public state + transcript)  |
+
+## Running locally
+
+```bash
+# 1. Install deps
+pip install -r requirements.txt
+
+# 2. Start the environment server
+uvicorn server:app --host 0.0.0.0 --port 7860
+
+# 3. In another shell, run the baseline inference
+export API_BASE_URL=https://router.huggingface.co/v1
+export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
+export HF_TOKEN=<your-hf-token>
+export ESC_ENV_URL=http://localhost:7860
+python inference.py
+```
+
+## Running via Docker
+
+```bash
+docker build -t esc-openenv .
+docker run -p 7860:7860 esc-openenv
+```
+
+## Baseline scores
+
+Replace with your own numbers after running `inference.py` against your
+configured endpoint.
+
+| Task                    | Score  | Success |
+| ----------------------- | ------ | ------- |
+| work_stress_venting     | TBD    | TBD     |
+| guarded_relationship    | TBD    | TBD     |
+| crisis_fragile_trust    | TBD    | TBD     |
+| **Average**             | **TBD** |         |
+
+## Files
+
+```
+.
+├── openenv.yaml             # OpenEnv metadata
+├── Dockerfile               # Container build for HF Space
+├── requirements.txt
+├── server.py                # FastAPI HTTP server (entrypoint)
+├── inference.py             # Mandated baseline inference script
+├── README.md
+└── src/
+    ├── __init__.py
+    ├── models.py            # Pydantic Action / Observation / Reward / envelopes
+    ├── seeker.py            # Deterministic seeker simulator + feature detectors
+    ├── tasks.py             # 3 task personas (easy / medium / hard)
+    ├── grader.py            # Hybrid immediate + future-oriented reward
+    ├── env.py               # Core ESCEnv with step/reset/state
+    └── client.py            # Async HTTP client for inference.py
+```
+
+## Citation
+
+If you use this environment, please cite the paper whose reward idea
+inspired it:
+
+```bibtex
+@article{yang2025rlffesc,
+  title   = {Towards Open-Ended Emotional Support Conversations in LLMs via
+             Reinforcement Learning with Future-Oriented Rewards},
+  author  = {Yang, Ting and Chen, Li and Wang, Huimin},
+  journal = {arXiv preprint arXiv:2508.12935},
+  year    = {2025}
+}
+```

inference.py

@@ -0,0 +1,232 @@
+"""Baseline inference script for the ESC OpenEnv environment.
+
+MANDATORY env vars
+------------------
+    API_BASE_URL   - LLM endpoint (default: https://router.huggingface.co/v1)
+    MODEL_NAME     - Model identifier (default: Qwen/Qwen2.5-72B-Instruct)
+    HF_TOKEN       - API key for the inference endpoint
+    ESC_ENV_URL    - URL of the running ESC OpenEnv HTTP server (e.g. the HF Space URL)
+
+STDOUT contract (strict)
+------------------------
+One [START] line per episode, one [STEP] per step, one [END] per episode.
+See the hackathon spec for exact format.
+
+Runs all 3 tasks (easy/medium/hard) sequentially and prints a final summary
+to stderr. Total wall-clock budget kept well under 20min on 2 vCPU / 8GB.
+"""
+from __future__ import annotations
+
+import asyncio
+import os
+import sys
+import textwrap
+import traceback
+from typing import List, Optional
+
+from openai import OpenAI
+
+from src.client import ESCHttpClient
+from src.models import Action
+
+# -------------------------- mandated env vars --------------------------------
+API_BASE_URL = os.getenv("API_BASE_URL") or "https://router.huggingface.co/v1"
+MODEL_NAME = os.getenv("MODEL_NAME") or "Qwen/Qwen2.5-72B-Instruct"
+API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
+ESC_ENV_URL = os.getenv("ESC_ENV_URL") or "http://localhost:7860"
+
+BENCHMARK = "emotional-support-conversations"
+MAX_STEPS = 14  # upper bound; env imposes per-task limits too
+TEMPERATURE = 0.6
+MAX_TOKENS = 220
+
+TASK_IDS = ["work_stress_venting", "guarded_relationship", "crisis_fragile_trust"]
+
+SYSTEM_PROMPT = textwrap.dedent(
+    """
+    You are an emotionally attuned peer supporter chatting with someone who is
+    going through a hard time. Your job is NOT to fix their problem. Your job
+    is to make them feel heard, safe, and understood first — and only move
+    toward gentle exploration or light action once trust is established.
+
+    Principles:
+    - Lead with empathy and validation. Reflect what you hear.
+    - Do NOT give advice until the person has clearly shared what's really
+      going on and feels heard.
+    - Ask at most one open-ended question per reply. Never interrogate.
+    - Never be dismissive, minimising, or instructive in a judgmental tone.
+    - Keep replies warm, brief (1-3 sentences), and human.
+    - In high-distress / crisis scenarios, gently reference professional
+      support (a therapist, crisis line) only after rapport is built.
+
+    You will receive the current conversation state. Reply with ONLY your
+    next message to the person — no role labels, no prefixes, no quotes.
+    """
+).strip()
+
+
+# -------------------------- stdout contract ----------------------------------
+
+def log_start(task: str, env: str, model: str) -> None:
+    print(f"[START] task={task} env={env} model={model}", flush=True)
+
+
+def log_step(step: int, action: str, reward: float, done: bool, error: Optional[str]) -> None:
+    err = error if error else "null"
+    # collapse any newlines in the action so the stdout contract stays single-line
+    flat_action = " ".join((action or "").split())
+    print(
+        f"[STEP] step={step} action={flat_action} reward={reward:.2f} "
+        f"done={str(done).lower()} error={err}",
+        flush=True,
+    )
+
+
+def log_end(success: bool, steps: int, score: float, rewards: List[float]) -> None:
+    rewards_str = ",".join(f"{r:.2f}" for r in rewards)
+    print(
+        f"[END] success={str(success).lower()} steps={steps} "
+        f"score={score:.3f} rewards={rewards_str}",
+        flush=True,
+    )
+
+
+# -------------------------- LLM call -----------------------------------------
+
+def build_user_prompt(
+    scenario_brief: str,
+    stage_hint: str,
+    turn: int,
+    remaining: int,
+    seeker_utterance: str,
+    history: List[str],
+) -> str:
+    history_block = "\n".join(history[-8:]) if history else "(this is the first turn)"
+    return textwrap.dedent(
+        f"""
+        Scenario: {scenario_brief}
+        Conversation stage (public hint): {stage_hint}
+        Turn: {turn}   Remaining turns: {remaining}
+
+        Recent exchange:
+        {history_block}
+
+        Seeker just said:
+        "{seeker_utterance}"
+
+        Write your next reply (1-3 sentences, warm, no advice unless rapport is clearly established):
+        """
+    ).strip()
+
+
+def call_llm(client: OpenAI, user_prompt: str) -> str:
+    try:
+        completion = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": user_prompt},
+            ],
+            temperature=TEMPERATURE,
+            max_tokens=MAX_TOKENS,
+            stream=False,
+        )
+        text = (completion.choices[0].message.content or "").strip()
+        return text if text else "I hear you. That sounds really hard — can you tell me a little more about what's weighing on you?"
+    except Exception as exc:
+        print(f"[DEBUG] LLM call failed: {exc}", file=sys.stderr, flush=True)
+        return "That sounds really hard. I'm here — do you want to tell me more about what's going on?"
+
+
+# -------------------------- per-task episode ---------------------------------
+
+async def run_task(openai_client: OpenAI, env_client: ESCHttpClient, task_id: str) -> dict:
+    log_start(task=task_id, env=BENCHMARK, model=MODEL_NAME)
+
+    rewards: List[float] = []
+    steps_taken = 0
+    score = 0.0
+    success = False
+    history: List[str] = []
+    last_error: Optional[str] = None
+
+    try:
+        reset = await env_client.reset(task_id=task_id)
+        obs = reset.observation
+        history.append(f"Seeker: {obs.seeker_utterance!r}")
+
+        for step in range(1, MAX_STEPS + 1):
+            user_prompt = build_user_prompt(
+                scenario_brief=obs.scenario_brief,
+                stage_hint=obs.stage_hint,
+                turn=obs.turn,
+                remaining=obs.remaining_turns,
+                seeker_utterance=obs.seeker_utterance,
+                history=history,
+            )
+            message = call_llm(openai_client, user_prompt)
+
+            try:
+                result = await env_client.step(Action(message=message))
+            except Exception as e:
+                last_error = f"step_failed: {e}"
+                log_step(step=step, action=message, reward=0.0, done=True, error=last_error)
+                break
+
+            reward = float(result.reward)
+            done = bool(result.done)
+            rewards.append(reward)
+            steps_taken = step
+            obs = result.observation
+
+            history.append(f"Agent: {message!r}")
+            history.append(f"Seeker: {obs.seeker_utterance!r}")
+
+            log_step(step=step, action=message, reward=reward, done=done, error=None)
+
+            if done:
+                final = result.info.get("final", {}) if isinstance(result.info, dict) else {}
+                score = float(final.get("score", sum(rewards) / max(1, steps_taken)))
+                success = bool(final.get("success", 0.0) >= 1.0)
+                break
+        else:
+            # Ran out of outer loop without env-side done — fall back to state().
+            st = await env_client.state()
+            score = float(st.get("cumulative_reward", 0.0)) / max(1, steps_taken)
+            success = score >= 0.5
+
+    except Exception as exc:
+        last_error = f"episode_failed: {exc}"
+        traceback.print_exc(file=sys.stderr)
+
+    log_end(success=success, steps=steps_taken, score=score, rewards=rewards)
+    return {"task_id": task_id, "score": score, "success": success, "steps": steps_taken}
+
+
+# -------------------------- main ---------------------------------------------
+
+async def main() -> None:
+    openai_client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY or "dummy")
+    env_client = ESCHttpClient.from_url(ESC_ENV_URL)
+
+    results = []
+    try:
+        for task_id in TASK_IDS:
+            res = await run_task(openai_client, env_client, task_id)
+            results.append(res)
+    finally:
+        await env_client.close()
+
+    # Summary to stderr so it doesn't pollute the stdout contract.
+    print("\n=== Baseline summary ===", file=sys.stderr)
+    for r in results:
+        print(
+            f"  {r['task_id']:<26} score={r['score']:.3f}  success={r['success']}  steps={r['steps']}",
+            file=sys.stderr,
+        )
+    avg = sum(r["score"] for r in results) / max(1, len(results))
+    print(f"  {'AVERAGE':<26} score={avg:.3f}", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

openenv.yaml

@@ -0,0 +1,56 @@
+name: emotional-support-conversations
+version: 0.1.0
+description: >
+  An OpenEnv environment for training and evaluating agents on open-ended
+  emotional support conversations. Agents converse with a deterministic seeker
+  simulator with hidden internal state (distress, trust, openness) and are
+  graded with a hybrid immediate + future-oriented reward signal inspired by
+  RLFF-ESC (Yang et al., 2025, arXiv:2508.12935).
+author: meta-hack-submission
+license: MIT
+tags:
+  - openenv
+  - conversation
+  - emotional-support
+  - mental-health
+  - rl-native
+  - partial-observability
+entrypoint: server:app
+port: 7860
+runtime:
+  python: "3.11"
+  vcpu: 2
+  memory_gb: 8
+tasks:
+  - id: work_stress_venting
+    difficulty: easy
+    description: >
+      Cooperative seeker venting about workplace stress. Agent must validate
+      feelings, explore the concern, and guide to a light action plan.
+  - id: guarded_relationship
+    difficulty: medium
+    description: >
+      Guarded seeker who only reveals the real relationship issue after trust
+      is built. Premature advice is penalised.
+  - id: crisis_fragile_trust
+    difficulty: hard
+    description: >
+      High-distress seeker with multiple interleaved concerns and fragile
+      trust. Any dismissive or interrogative turn collapses trust; recovery is
+      possible but costly.
+action_space:
+  type: text
+  description: Free-text conversational reply from the agent to the seeker.
+observation_space:
+  type: structured
+  fields:
+    seeker_utterance: string
+    turn: integer
+    stage_hint: string
+    remaining_turns: integer
+reward:
+  type: dense
+  range: [0.0, 1.0]
+  shaping:
+    - immediate_turn_reward
+    - future_oriented_trajectory_reward

requirements.txt

@@ -0,0 +1,5 @@
+fastapi==0.115.0
+uvicorn[standard]==0.30.6
+pydantic==2.9.2
+httpx==0.27.2
+openai==1.54.3

server.py

@@ -0,0 +1,74 @@
+"""FastAPI HTTP server exposing the OpenEnv interface for the ESC environment.
+
+Endpoints
+---------
+GET  /             → health check + metadata
+POST /reset        → reset episode (optional task_id), returns initial Observation
+POST /step         → take one step with {"action": {"message": "..."}}
+GET  /state        → return current EnvState
+GET  /tasks        → list available tasks + difficulties
+
+The server holds a single in-process ESCEnv instance. For parallel eval,
+deploy multiple replicas — the env itself has no shared state between
+instances.
+"""
+from __future__ import annotations
+
+from fastapi import FastAPI, HTTPException
+
+from src.env import ESCEnv
+from src.models import ResetRequest, StepRequest
+
+app = FastAPI(
+    title="Emotional Support Conversations (OpenEnv)",
+    version="0.1.0",
+    description=(
+        "An OpenEnv environment for open-ended emotional support "
+        "conversations. Reward shaping inspired by RLFF-ESC "
+        "(arXiv:2508.12935)."
+    ),
+)
+
+_env = ESCEnv()
+
+
+@app.get("/")
+def root() -> dict:
+    return {
+        "name": "emotional-support-conversations",
+        "version": "0.1.0",
+        "endpoints": ["/reset", "/step", "/state", "/tasks"],
+        "tasks": [t["id"] for t in ESCEnv.list_tasks()],
+    }
+
+
+@app.get("/tasks")
+def list_tasks() -> dict:
+    return {"tasks": ESCEnv.list_tasks()}
+
+
+@app.post("/reset")
+def reset(req: ResetRequest | None = None) -> dict:
+    req = req or ResetRequest()
+    try:
+        result = _env.reset(task_id=req.task_id, seed=req.seed)
+    except KeyError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+    return result.model_dump()
+
+
+@app.post("/step")
+def step(req: StepRequest) -> dict:
+    try:
+        result = _env.step(req.action)
+    except RuntimeError as e:
+        raise HTTPException(status_code=409, detail=str(e))
+    return result.model_dump()
+
+
+@app.get("/state")
+def state() -> dict:
+    try:
+        return _env.state().model_dump()
+    except RuntimeError as e:
+        raise HTTPException(status_code=409, detail=str(e))

src/__init__.py

@@ -0,0 +1,16 @@
+"""Emotional Support Conversations OpenEnv environment."""
+from .models import Action, Observation, Reward, StepResult, ResetResult, EnvState
+from .env import ESCEnv
+from .tasks import TASKS, TaskSpec
+
+__all__ = [
+    "Action",
+    "Observation",
+    "Reward",
+    "StepResult",
+    "ResetResult",
+    "EnvState",
+    "ESCEnv",
+    "TASKS",
+    "TaskSpec",
+]

src/client.py

@@ -0,0 +1,70 @@
+"""Async HTTP client mirroring the OpenEnv env interface.
+
+Used by inference.py to interact with the running FastAPI server (local or
+HF Space deployment).
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+import httpx
+
+from .models import Action, Observation
+
+
+@dataclass
+class StepResponse:
+    observation: Observation
+    reward: float
+    done: bool
+    info: Dict[str, Any]
+
+
+@dataclass
+class ResetResponse:
+    observation: Observation
+    info: Dict[str, Any]
+
+
+class ESCHttpClient:
+    """Thin async client for the ESC OpenEnv server."""
+
+    def __init__(self, base_url: str, timeout: float = 30.0):
+        self.base_url = base_url.rstrip("/")
+        self._client = httpx.AsyncClient(base_url=self.base_url, timeout=timeout)
+
+    @classmethod
+    def from_url(cls, base_url: str) -> "ESCHttpClient":
+        return cls(base_url=base_url)
+
+    async def reset(self, task_id: Optional[str] = None) -> ResetResponse:
+        payload: Dict[str, Any] = {}
+        if task_id is not None:
+            payload["task_id"] = task_id
+        r = await self._client.post("/reset", json=payload)
+        r.raise_for_status()
+        data = r.json()
+        return ResetResponse(
+            observation=Observation(**data["observation"]),
+            info=data.get("info", {}),
+        )
+
+    async def step(self, action: Action) -> StepResponse:
+        r = await self._client.post("/step", json={"action": action.model_dump()})
+        r.raise_for_status()
+        data = r.json()
+        return StepResponse(
+            observation=Observation(**data["observation"]),
+            reward=float(data["reward"]),
+            done=bool(data["done"]),
+            info=data.get("info", {}),
+        )
+
+    async def state(self) -> Dict[str, Any]:
+        r = await self._client.get("/state")
+        r.raise_for_status()
+        return r.json()
+
+    async def close(self) -> None:
+        await self._client.aclose()

src/env.py

@@ -0,0 +1,190 @@
+"""Core ESC environment: OpenEnv-style step() / reset() / state()."""
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from .grader import GradeBreakdown, final_task_score, grade_step
+from .models import (
+    Action,
+    EnvState,
+    Observation,
+    ResetResult,
+    Reward,
+    StepResult,
+)
+from .seeker import (
+    SeekerState,
+    Stage,
+    extract_features,
+    resolution_score,
+    step_seeker,
+)
+from .tasks import TASKS, TaskSpec, get_task
+
+
+class ESCEnv:
+    """Emotional Support Conversations environment.
+
+    Usage (in-process):
+        env = ESCEnv()
+        obs = env.reset(task_id="work_stress_venting")
+        result = env.step(Action(message="That sounds really hard. What's weighing on you most right now?"))
+    """
+
+    def __init__(self) -> None:
+        self._task: Optional[TaskSpec] = None
+        self._seeker: Optional[SeekerState] = None
+        self._turn: int = 0
+        self._done: bool = False
+        self._cumulative_reward: float = 0.0
+        self._transcript: List[Dict[str, str]] = []
+        self._last_obs: Optional[Observation] = None
+
+    # ------------------------------------------------------------------ reset
+
+    def reset(self, task_id: Optional[str] = None, seed: Optional[int] = None) -> ResetResult:
+        """Reset to a clean initial state for the given task (default: easy)."""
+        task_id = task_id or "work_stress_venting"
+        self._task = get_task(task_id)
+        self._seeker = SeekerState.from_persona(self._task.persona)
+        self._turn = 0
+        self._done = False
+        self._cumulative_reward = 0.0
+        self._transcript = [
+            {"role": "seeker", "text": self._task.persona.surface_concern}
+        ]
+
+        obs = Observation(
+            seeker_utterance=self._task.persona.surface_concern,
+            turn=0,
+            remaining_turns=self._task.max_turns,
+            stage_hint=self._seeker.stage.value,
+            task_id=self._task.id,
+            scenario_brief=self._task.persona.scenario_brief,
+        )
+        self._last_obs = obs
+        return ResetResult(
+            observation=obs,
+            info={
+                "difficulty": self._task.difficulty,
+                "max_turns": self._task.max_turns,
+                "success_threshold": self._task.success_threshold,
+            },
+        )
+
+    # ------------------------------------------------------------------- step
+
+    def step(self, action: Action) -> StepResult:
+        if self._task is None or self._seeker is None:
+            raise RuntimeError("env.step() called before reset()")
+        if self._done:
+            raise RuntimeError("env.step() called on a finished episode — call reset()")
+
+        # 1. Record the agent's turn.
+        self._transcript.append({"role": "agent", "text": action.message})
+
+        # 2. Snapshot pre-action state (for reward deltas and future-oriented lookahead).
+        pre_state = self._seeker.snapshot()
+
+        # 3. Extract features and advance seeker dynamics.
+        features = extract_features(action.message)
+        transition = step_seeker(self._seeker, features)
+        post_state = transition.new_state  # same object, mutated
+        self._seeker = post_state
+        self._turn += 1
+
+        # 4. Grade the step.
+        breakdown: GradeBreakdown = grade_step(
+            pre_state=pre_state,
+            post_state=post_state,
+            features=features,
+            flags=transition.flags,
+        )
+        self._cumulative_reward += breakdown.value
+
+        # 5. Record seeker's reply.
+        self._transcript.append({"role": "seeker", "text": transition.seeker_utterance})
+
+        # 6. Termination check.
+        reached_closing = post_state.stage == Stage.CLOSING
+        natural_done = reached_closing and post_state.trust >= 0.6 and post_state.distress <= 0.5
+        trust_collapse = post_state.trust <= 0.05
+        budget_exhausted = self._turn >= self._task.max_turns
+        done = bool(natural_done or trust_collapse or budget_exhausted)
+        self._done = done
+
+        # 7. Build the next observation.
+        obs = Observation(
+            seeker_utterance=transition.seeker_utterance,
+            turn=self._turn,
+            remaining_turns=max(0, self._task.max_turns - self._turn),
+            stage_hint=post_state.stage.value,
+            task_id=self._task.id,
+            scenario_brief=self._task.persona.scenario_brief,
+        )
+        self._last_obs = obs
+
+        info: Dict[str, Any] = {
+            "features": features.__dict__,
+            "flags": transition.flags,
+            "stage": post_state.stage.value,
+            "resolution_score": resolution_score(post_state),
+            "natural_done": natural_done,
+            "trust_collapse": trust_collapse,
+            "budget_exhausted": budget_exhausted,
+            "reward_components": breakdown.components,
+        }
+
+        if done:
+            info["final"] = final_task_score(
+                cumulative_reward=self._cumulative_reward,
+                steps_taken=self._turn,
+                max_turns=self._task.max_turns,
+                final_state=post_state,
+                success_threshold=self._task.success_threshold,
+            )
+
+        reward_detail = Reward(
+            value=breakdown.value,
+            immediate=breakdown.immediate,
+            future_oriented=breakdown.future_oriented,
+            penalties=breakdown.penalties,
+            components={k: float(v) for k, v in breakdown.components.items()},
+        )
+
+        return StepResult(
+            observation=obs,
+            reward=breakdown.value,
+            reward_detail=reward_detail,
+            done=done,
+            info=info,
+        )
+
+    # ------------------------------------------------------------------ state
+
+    def state(self) -> EnvState:
+        if self._task is None:
+            raise RuntimeError("env.state() called before reset()")
+        return EnvState(
+            task_id=self._task.id,
+            turn=self._turn,
+            max_turns=self._task.max_turns,
+            done=self._done,
+            cumulative_reward=self._cumulative_reward,
+            transcript=list(self._transcript),
+        )
+
+    # ---------------------------------------------------------------- listing
+
+    @staticmethod
+    def list_tasks() -> List[Dict[str, Any]]:
+        return [
+            {
+                "id": t.id,
+                "difficulty": t.difficulty,
+                "max_turns": t.max_turns,
+                "success_threshold": t.success_threshold,
+                "scenario_brief": t.persona.scenario_brief,
+            }
+            for t in TASKS.values()
+        ]

src/grader.py

@@ -0,0 +1,174 @@
+"""Reward / grading module.
+
+Implements the hybrid reward used by the ESC environment:
+
+    step_reward = clip(immediate + future_oriented - penalties,  0, 1)
+
+- immediate            : stage-appropriate empathy/validation/open-question signal
+- future_oriented      : RLFF-ESC style lookahead — projects the oracle policy
+                         k steps forward from the *post-action* state and
+                         compares the projected resolution_score against the
+                         pre-action ceiling. Rewards actions that *preserve or
+                         advance* the attainable resolution, not just ones
+                         that look good this turn.
+- penalties            : dismissive language, premature advice, repetitive
+                         bare replies, interrogation.
+
+This shaping gives the agent dense, varying signal across the trajectory
+(required by the rubric: "signal over the full trajectory, not just
+binary end-of-episode").
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Dict, List
+
+from .seeker import (
+    Features,
+    SeekerState,
+    Stage,
+    resolution_score,
+    simulate_oracle_rollout,
+    stage_progress,
+)
+
+# Hyper-parameters — tuned to keep step reward in [0, 1] under normal play.
+LOOKAHEAD_K = 3
+W_IMMEDIATE = 0.45
+W_FUTURE = 0.55
+DISMISSIVE_PENALTY = 0.6
+PREMATURE_ADVICE_PENALTY = 0.25
+BARE_PENALTY = 0.15
+INTERROGATION_PENALTY = 0.15
+
+
+@dataclass
+class GradeBreakdown:
+    value: float
+    immediate: float
+    future_oriented: float
+    penalties: float
+    components: Dict[str, float]
+
+
+def _stage_fit_score(stage: Stage, f: Features) -> float:
+    """How appropriate are the agent's features for the current stage?"""
+    if stage in (Stage.OPENING, Stage.EXPLORING):
+        # Reward empathy + open questions; punish early advice strongly.
+        fit = 0.5 * min(1.0, f.empathy) + 0.3 * min(1.0, f.open_question) + 0.2 * min(1.0, f.validation)
+        fit -= 0.4 * min(1.0, f.advice)
+    elif stage == Stage.REFLECTING:
+        fit = 0.5 * min(1.0, f.validation) + 0.4 * min(1.0, f.empathy) + 0.1 * min(1.0, f.open_question)
+        fit -= 0.2 * min(1.0, f.advice)
+    elif stage == Stage.PLANNING:
+        # Advice is finally okay here.
+        fit = 0.4 * min(1.0, f.open_question) + 0.3 * min(1.0, f.advice) + 0.3 * min(1.0, f.empathy)
+    else:  # CLOSING
+        fit = 0.5 * min(1.0, f.empathy) + 0.3 * min(1.0, f.safety) + 0.2 * min(1.0, f.validation)
+    return max(0.0, min(1.0, fit))
+
+
+def _immediate_reward(pre_state: SeekerState, post_state: SeekerState, f: Features) -> float:
+    """Turn-level reward: stage fit + trust delta + distress delta."""
+    stage_fit = _stage_fit_score(pre_state.stage, f)
+    trust_delta = max(0.0, post_state.trust - pre_state.trust)
+    distress_relief = max(0.0, pre_state.distress - post_state.distress)
+    stage_advance = max(
+        0.0, stage_progress(post_state.stage) - stage_progress(pre_state.stage)
+    )
+    reveal_bonus = 0.2 if (post_state.revealed and not pre_state.revealed) else 0.0
+    return max(
+        0.0,
+        min(
+            1.0,
+            0.45 * stage_fit
+            + 0.20 * trust_delta * 2.0  # scale small deltas
+            + 0.20 * distress_relief * 2.0
+            + 0.10 * stage_advance
+            + 0.05 * 1.0  # small baseline for any non-destructive turn
+            + reveal_bonus,
+        ),
+    )
+
+
+def _future_oriented_reward(pre_state: SeekerState, post_state: SeekerState) -> float:
+    """RLFF-ESC style: does this action *preserve / advance* future resolution?
+
+    We roll the oracle policy k steps from both the pre- and post-action states
+    and take the (clipped) delta. Positive delta = the action moved the
+    attainable future forward; negative = the agent damaged trajectory
+    potential and must recover.
+    """
+    pre_ceiling = simulate_oracle_rollout(pre_state.snapshot(), LOOKAHEAD_K)
+    post_ceiling = simulate_oracle_rollout(post_state.snapshot(), LOOKAHEAD_K)
+    delta = post_ceiling - pre_ceiling
+    # Map delta in roughly [-0.4, +0.4] to [0, 1] with 0 at delta=0.
+    return max(0.0, min(1.0, 0.5 + 1.25 * delta))
+
+
+def _penalties(flags: Dict[str, bool], f: Features) -> float:
+    p = 0.0
+    if flags.get("dismissed"):
+        p += DISMISSIVE_PENALTY
+    if flags.get("advice_too_early"):
+        p += PREMATURE_ADVICE_PENALTY
+    if flags.get("bare_reply"):
+        p += BARE_PENALTY
+    if flags.get("interrogated"):
+        p += INTERROGATION_PENALTY
+    return p
+
+
+def grade_step(
+    pre_state: SeekerState,
+    post_state: SeekerState,
+    features: Features,
+    flags: Dict[str, bool],
+) -> GradeBreakdown:
+    imm = _immediate_reward(pre_state, post_state, features)
+    fut = _future_oriented_reward(pre_state, post_state)
+    pen = _penalties(flags, features)
+    combined = W_IMMEDIATE * imm + W_FUTURE * fut - pen
+    value = max(0.0, min(1.0, combined))
+    components = {
+        "stage_fit": _stage_fit_score(pre_state.stage, features),
+        "trust_delta": post_state.trust - pre_state.trust,
+        "distress_delta": pre_state.distress - post_state.distress,
+        "resolution_score_post": resolution_score(post_state),
+        "pre_oracle_ceiling": simulate_oracle_rollout(pre_state.snapshot(), LOOKAHEAD_K),
+        "post_oracle_ceiling": simulate_oracle_rollout(post_state.snapshot(), LOOKAHEAD_K),
+    }
+    return GradeBreakdown(
+        value=value,
+        immediate=imm,
+        future_oriented=fut,
+        penalties=pen,
+        components=components,
+    )
+
+
+def final_task_score(
+    cumulative_reward: float,
+    steps_taken: int,
+    max_turns: int,
+    final_state: SeekerState,
+    success_threshold: float,
+) -> Dict[str, float]:
+    """Compute the final [0,1] task score used by the grader."""
+    # Component 1: average shaped reward over the trajectory (already in [0,1]).
+    avg_reward = cumulative_reward / max(1, steps_taken)
+    # Component 2: final resolution_score.
+    final_res = resolution_score(final_state)
+    # Component 3: efficiency — finishing sooner is slightly better, but never
+    # negative. Flat 1.0 if used ≤ 60% of budget, linearly decays to 0.7 at max.
+    usage = steps_taken / max_turns
+    efficiency = 1.0 if usage <= 0.6 else max(0.7, 1.0 - 0.75 * (usage - 0.6))
+    score = 0.35 * avg_reward + 0.55 * final_res + 0.10 * efficiency
+    score = max(0.0, min(1.0, score))
+    return {
+        "score": score,
+        "avg_reward": avg_reward,
+        "final_resolution": final_res,
+        "efficiency": efficiency,
+        "success": 1.0 if score >= success_threshold else 0.0,
+    }

src/models.py

@@ -0,0 +1,111 @@
+"""Typed Pydantic models for the ESC OpenEnv environment.
+
+Defines the Action, Observation, Reward, and result envelopes used across the
+HTTP boundary (server.py) and the in-process env (env.py).
+"""
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel, Field
+
+
+class Action(BaseModel):
+    """Agent action: a free-text conversational reply to the seeker."""
+
+    message: str = Field(..., description="Agent's reply to the seeker.")
+
+
+class Observation(BaseModel):
+    """What the agent sees each turn.
+
+    The seeker's internal state (distress, trust, openness, true_issue) is
+    intentionally hidden — partial observability is what makes this env
+    RL-native. Only the seeker's *utterance* and coarse hints are exposed.
+    """
+
+    seeker_utterance: str = Field(..., description="The seeker's latest message.")
+    turn: int = Field(..., description="1-indexed conversation turn.")
+    remaining_turns: int = Field(..., description="Turns left before forced close.")
+    stage_hint: str = Field(
+        ...,
+        description=(
+            "Coarse public hint about conversational phase: one of "
+            "'opening', 'exploring', 'reflecting', 'planning', 'closing'."
+        ),
+    )
+    task_id: str = Field(..., description="Currently active task id.")
+    scenario_brief: str = Field(
+        ...,
+        description="One-line scenario framing shown once at reset (kept in obs for convenience).",
+    )
+
+
+class Reward(BaseModel):
+    """Detailed reward breakdown for a single step.
+
+    The scalar `value` is what the agent sees. The decomposition is exposed
+    for transparency and debugging.
+    """
+
+    value: float = Field(..., ge=0.0, le=1.0, description="Clipped step reward in [0,1].")
+    immediate: float = Field(..., description="Immediate turn-level component (empathy, stage-fit).")
+    future_oriented: float = Field(
+        ...,
+        description=(
+            "Future-oriented component: k-step lookahead over the deterministic "
+            "seeker dynamics, comparing this action's projected resolution "
+            "progress against the oracle ceiling (RLFF-ESC style)."
+        ),
+    )
+    penalties: float = Field(..., description="Summed penalties (dismissive, premature advice, loops).")
+    components: Dict[str, float] = Field(default_factory=dict, description="Sub-component breakdown.")
+
+
+class StepResult(BaseModel):
+    """Envelope returned by env.step()."""
+
+    observation: Observation
+    reward: float
+    reward_detail: Reward
+    done: bool
+    info: Dict[str, Any] = Field(default_factory=dict)
+
+
+class ResetResult(BaseModel):
+    """Envelope returned by env.reset()."""
+
+    observation: Observation
+    info: Dict[str, Any] = Field(default_factory=dict)
+
+
+class EnvState(BaseModel):
+    """Public view of environment state returned by env.state().
+
+    Hidden seeker variables are *not* included — only public bookkeeping.
+    """
+
+    task_id: str
+    turn: int
+    max_turns: int
+    done: bool
+    cumulative_reward: float
+    transcript: List[Dict[str, str]] = Field(
+        default_factory=list,
+        description="List of {'role': 'seeker'|'agent', 'text': str} entries.",
+    )
+
+
+# ------- Request schemas for the HTTP server -------
+
+
+class ResetRequest(BaseModel):
+    task_id: Optional[str] = Field(
+        default=None,
+        description="Optional task id. If omitted, defaults to 'work_stress_venting'.",
+    )
+    seed: Optional[int] = Field(default=None, description="Optional seed (reserved; env is deterministic).")
+
+
+class StepRequest(BaseModel):
+    action: Action

src/seeker.py

@@ -0,0 +1,412 @@
+"""Deterministic seeker simulator with hidden internal state.
+
+Why rule-based / deterministic?
+-------------------------------
+The OpenEnv graders must be reproducible. An LLM-driven seeker would make
+reward non-deterministic and fail the "score variance check" in Phase 2 of
+judging. We deliberately trade some linguistic realism for full determinism
+so that the same action sequence always yields the same reward — a hard
+requirement of the hackathon rubric ("graders deterministic and reproducible").
+
+Design
+------
+The seeker is a finite-state machine with continuous hidden variables:
+
+    distress   ∈ [0, 1]   — how emotionally overwhelmed the seeker feels
+    trust      ∈ [0, 1]   — how safe the seeker feels with the agent
+    openness   ∈ [0, 1]   — willingness to reveal the *true* issue
+    revealed   ∈ {0, 1}   — has the core issue surfaced yet?
+    stage      ∈ enum     — opening / exploring / reflecting / planning / closing
+
+On each turn, the environment analyses the agent's reply with a small bank of
+deterministic feature detectors (keyword/regex based), then applies a
+transition rule to update the hidden state and pick the seeker's next
+utterance from a scripted response tree indexed by (stage, features).
+"""
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+from enum import Enum
+from typing import Dict, List, Tuple
+
+
+class Stage(str, Enum):
+    OPENING = "opening"
+    EXPLORING = "exploring"
+    REFLECTING = "reflecting"
+    PLANNING = "planning"
+    CLOSING = "closing"
+
+
+# ---------------------------------------------------------------------------
+# Feature detectors — deterministic text analysis of the agent's reply.
+# ---------------------------------------------------------------------------
+
+EMPATHY_PATTERNS = [
+    r"\bi\s+(hear|understand|get|see)\s+(you|that|how)",
+    r"\bthat\s+(sounds|must\s+be|seems)\b",
+    r"\bit\s+makes\s+sense\b",
+    r"\bi\s+can\s+imagine\b",
+    r"\bthank\s+you\s+for\s+sharing\b",
+    r"\bi'?m\s+(here|glad|sorry)\b",
+]
+
+VALIDATION_PATTERNS = [
+    r"\byour\s+feelings?\s+(are|make)\s+(valid|sense)",
+    r"\bit'?s\s+(okay|ok|normal|understandable)\s+to\s+feel",
+    r"\banyone\s+would\s+feel\b",
+    r"\bof\s+course\s+you\s+(feel|are)\b",
+]
+
+OPEN_QUESTION_PATTERNS = [
+    r"\bhow\s+(are|do|did|does)\b",
+    r"\bwhat\s+(is|are|do|does|has|makes|brought|happened)\b",
+    r"\bcan\s+you\s+tell\s+me\s+more\b",
+    r"\bwould\s+you\s+like\s+to\s+(talk|share)\b",
+]
+
+ADVICE_PATTERNS = [
+    r"\byou\s+should\b",
+    r"\byou\s+(need|have|ought)\s+to\b",
+    r"\btry\s+(to|doing|this)\b",
+    r"\bjust\s+(do|go|try|stop|start)\b",
+    r"\bwhy\s+don'?t\s+you\b",
+    r"\bmy\s+advice\b",
+]
+
+DISMISSIVE_PATTERNS = [
+    r"\bget\s+over\s+it\b",
+    r"\bstop\s+(complaining|whining|crying)\b",
+    r"\byou'?re\s+overreacting\b",
+    r"\bit'?s\s+not\s+a\s+big\s+deal\b",
+    r"\bcalm\s+down\b",
+    r"\bit\s+could\s+be\s+worse\b",
+]
+
+INTERROGATIVE_PATTERNS = [  # rapid-fire closed questions (trust drain when high)
+    r"\?\s*\?",
+]
+
+SAFETY_PATTERNS = [
+    r"\bare\s+you\s+safe\b",
+    r"\bprofessional\s+help\b",
+    r"\bcrisis\s+line\b",
+    r"\btherapist\b",
+]
+
+
+def _count_matches(patterns: List[str], text: str) -> int:
+    t = text.lower()
+    return sum(1 for p in patterns if re.search(p, t))
+
+
+@dataclass
+class Features:
+    empathy: int
+    validation: int
+    open_question: int
+    advice: int
+    dismissive: int
+    interrogative: int
+    safety: int
+    length: int
+    closed_question: int  # any '?' not matched by open
+    bare: bool  # very short / empty reply
+
+
+def extract_features(text: str) -> Features:
+    stripped = (text or "").strip()
+    lower = stripped.lower()
+    empathy = _count_matches(EMPATHY_PATTERNS, lower)
+    validation = _count_matches(VALIDATION_PATTERNS, lower)
+    open_q = _count_matches(OPEN_QUESTION_PATTERNS, lower)
+    advice = _count_matches(ADVICE_PATTERNS, lower)
+    dismissive = _count_matches(DISMISSIVE_PATTERNS, lower)
+    interrogative = _count_matches(INTERROGATIVE_PATTERNS, lower)
+    safety = _count_matches(SAFETY_PATTERNS, lower)
+    total_q = lower.count("?")
+    closed_q = max(0, total_q - open_q)
+    bare = len(stripped) < 8
+    return Features(
+        empathy=empathy,
+        validation=validation,
+        open_question=open_q,
+        advice=advice,
+        dismissive=dismissive,
+        interrogative=interrogative,
+        safety=safety,
+        length=len(stripped),
+        closed_question=closed_q,
+        bare=bare,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Seeker state + scripted persona
+# ---------------------------------------------------------------------------
+
+@dataclass
+class SeekerPersona:
+    """Static configuration describing the seeker's initial state + script."""
+
+    task_id: str
+    scenario_brief: str
+    surface_concern: str  # what seeker says at turn 0
+    true_issue: str  # hidden; only revealed if openness crosses threshold
+    initial_distress: float
+    initial_trust: float
+    initial_openness: float
+    reveal_threshold: float  # openness value at which true_issue is revealed
+    trust_fragility: float  # how much a misstep drops trust (0..1)
+    openness_gain_per_empathy: float
+    distress_drop_per_validation: float
+    # Scripted utterances by stage when cooperative
+    opening_lines: List[str]
+    exploring_lines: List[str]
+    reflecting_lines: List[str]
+    planning_lines: List[str]
+    closing_lines: List[str]
+    reveal_line: str  # said the turn openness crosses reveal_threshold
+    # Adverse reactions
+    dismissed_lines: List[str] = field(default_factory=list)
+    advice_too_early_lines: List[str] = field(default_factory=list)
+
+
+@dataclass
+class SeekerState:
+    """Mutable hidden state updated each turn."""
+
+    persona: SeekerPersona
+    distress: float
+    trust: float
+    openness: float
+    revealed: bool
+    stage: Stage
+    last_line_idx_by_stage: Dict[Stage, int]
+    turn: int
+
+    @classmethod
+    def from_persona(cls, persona: SeekerPersona) -> "SeekerState":
+        return cls(
+            persona=persona,
+            distress=persona.initial_distress,
+            trust=persona.initial_trust,
+            openness=persona.initial_openness,
+            revealed=False,
+            stage=Stage.OPENING,
+            last_line_idx_by_stage={s: -1 for s in Stage},
+            turn=0,
+        )
+
+    # Snapshot for lookahead simulation — must be cheap and pure.
+    def snapshot(self) -> "SeekerState":
+        return SeekerState(
+            persona=self.persona,
+            distress=self.distress,
+            trust=self.trust,
+            openness=self.openness,
+            revealed=self.revealed,
+            stage=self.stage,
+            last_line_idx_by_stage=dict(self.last_line_idx_by_stage),
+            turn=self.turn,
+        )
+
+
+def _clip(x: float) -> float:
+    return max(0.0, min(1.0, x))
+
+
+# Stage ordering used for "progress" scalar in [0,1]
+STAGE_ORDER: List[Stage] = [
+    Stage.OPENING,
+    Stage.EXPLORING,
+    Stage.REFLECTING,
+    Stage.PLANNING,
+    Stage.CLOSING,
+]
+
+
+def stage_progress(stage: Stage) -> float:
+    return STAGE_ORDER.index(stage) / (len(STAGE_ORDER) - 1)
+
+
+def resolution_score(state: SeekerState) -> float:
+    """Scalar summary of how 'resolved' the conversation currently is, in [0,1].
+
+    Weighted combination of stage progress, trust gained, distress relieved,
+    and whether the true issue surfaced. This is the quantity the
+    future-oriented reward tries to project forward under an oracle policy.
+    """
+    p = state.persona
+    progress = stage_progress(state.stage)
+    trust_gain = max(0.0, state.trust - p.initial_trust)
+    distress_relief = max(0.0, p.initial_distress - state.distress)
+    reveal_bonus = 1.0 if state.revealed else 0.0
+    return _clip(
+        0.40 * progress
+        + 0.25 * trust_gain / max(1e-6, 1.0 - p.initial_trust)
+        + 0.25 * distress_relief / max(1e-6, p.initial_distress)
+        + 0.10 * reveal_bonus
+    )
+
+
+# ---------------------------------------------------------------------------
+# Transition: given current state + agent features, produce new state +
+# seeker's next utterance + transition info.
+# ---------------------------------------------------------------------------
+
+@dataclass
+class Transition:
+    new_state: SeekerState
+    seeker_utterance: str
+    flags: Dict[str, bool]  # e.g. {"dismissed": True, "advice_too_early": False, ...}
+
+
+def _next_line(state: SeekerState, stage: Stage, pool: List[str]) -> str:
+    if not pool:
+        return "..."
+    idx = (state.last_line_idx_by_stage[stage] + 1) % len(pool)
+    state.last_line_idx_by_stage[stage] = idx
+    return pool[idx]
+
+
+def step_seeker(state: SeekerState, features: Features) -> Transition:
+    """Apply one turn of seeker dynamics given the agent's extracted features.
+
+    Pure-ish: mutates a *copy* of state (caller should pass a snapshot if they
+    want to preserve the original — the env always passes the live state).
+    """
+    p = state.persona
+    flags: Dict[str, bool] = {
+        "dismissed": False,
+        "advice_too_early": False,
+        "bare_reply": features.bare,
+        "empathic": features.empathy + features.validation > 0,
+        "interrogated": False,
+        "revealed_this_turn": False,
+    }
+
+    # --- 1. Dismissive / hostile language: hard drop on trust & distress spike.
+    if features.dismissive > 0:
+        state.trust = _clip(state.trust - 0.4 * (1.0 + p.trust_fragility))
+        state.distress = _clip(state.distress + 0.15)
+        state.openness = _clip(state.openness - 0.2)
+        flags["dismissed"] = True
+
+    # --- 2. Premature advice (advice before trust ≥ 0.55): trust drop, openness drop.
+    if features.advice > 0 and state.trust < 0.55:
+        state.trust = _clip(state.trust - 0.15 * (1.0 + p.trust_fragility))
+        state.openness = _clip(state.openness - 0.1)
+        flags["advice_too_early"] = True
+
+    # --- 3. Empathy & validation: trust + openness up, distress down.
+    if features.empathy > 0 or features.validation > 0:
+        gain = p.openness_gain_per_empathy * (features.empathy + features.validation)
+        state.trust = _clip(state.trust + 0.12 * (features.empathy + features.validation))
+        state.openness = _clip(state.openness + gain)
+        state.distress = _clip(state.distress - p.distress_drop_per_validation * features.validation)
+
+    # --- 4. Open questions: small trust gain, nudges stage forward.
+    if features.open_question > 0:
+        state.trust = _clip(state.trust + 0.05)
+        state.openness = _clip(state.openness + 0.04)
+
+    # --- 5. Interrogation (many closed questions or multiple "?"): trust drain.
+    if features.closed_question >= 3 or features.interrogative > 0:
+        state.trust = _clip(state.trust - 0.1)
+        flags["interrogated"] = True
+
+    # --- 6. Bare / empty reply: small penalty across the board.
+    if features.bare:
+        state.trust = _clip(state.trust - 0.05)
+        state.distress = _clip(state.distress + 0.02)
+
+    # --- 7. Stage progression (monotonic forward with cooperative conditions).
+    def advance_to(s: Stage) -> None:
+        if STAGE_ORDER.index(s) > STAGE_ORDER.index(state.stage):
+            state.stage = s
+
+    if state.stage == Stage.OPENING and (
+        features.empathy + features.validation + features.open_question > 0
+    ):
+        advance_to(Stage.EXPLORING)
+    elif state.stage == Stage.EXPLORING and state.trust >= 0.5 and state.openness >= 0.5:
+        advance_to(Stage.REFLECTING)
+    elif state.stage == Stage.REFLECTING and state.revealed and state.distress <= 0.5:
+        advance_to(Stage.PLANNING)
+    elif state.stage == Stage.PLANNING and features.open_question + features.empathy > 0:
+        advance_to(Stage.CLOSING)
+
+    # --- 8. Reveal check (cross threshold once).
+    if not state.revealed and state.openness >= p.reveal_threshold:
+        state.revealed = True
+        flags["revealed_this_turn"] = True
+
+    # --- 9. Pick seeker's next utterance.
+    if flags["dismissed"] and p.dismissed_lines:
+        utterance = _next_line(state, state.stage, p.dismissed_lines)
+    elif flags["advice_too_early"] and p.advice_too_early_lines:
+        utterance = _next_line(state, state.stage, p.advice_too_early_lines)
+    elif flags["revealed_this_turn"]:
+        utterance = p.reveal_line
+    else:
+        pool_by_stage = {
+            Stage.OPENING: p.opening_lines,
+            Stage.EXPLORING: p.exploring_lines,
+            Stage.REFLECTING: p.reflecting_lines,
+            Stage.PLANNING: p.planning_lines,
+            Stage.CLOSING: p.closing_lines,
+        }
+        utterance = _next_line(state, state.stage, pool_by_stage[state.stage])
+
+    state.turn += 1
+    return Transition(new_state=state, seeker_utterance=utterance, flags=flags)
+
+
+# ---------------------------------------------------------------------------
+# Oracle policy for the future-oriented reward lookahead.
+# ---------------------------------------------------------------------------
+
+def oracle_features(state: SeekerState) -> Features:
+    """What the 'oracle' agent would do from this state.
+
+    Picks the stage-appropriate ideal action:
+      - opening/exploring: empathy + open question
+      - reflecting: empathy + validation
+      - planning: open question + mild advice (trust is high here)
+      - closing: empathy + safety mention
+    """
+    s = state.stage
+    if s in (Stage.OPENING, Stage.EXPLORING):
+        return Features(
+            empathy=1, validation=0, open_question=1, advice=0,
+            dismissive=0, interrogative=0, safety=0, length=80,
+            closed_question=0, bare=False,
+        )
+    if s == Stage.REFLECTING:
+        return Features(
+            empathy=1, validation=1, open_question=0, advice=0,
+            dismissive=0, interrogative=0, safety=0, length=90,
+            closed_question=0, bare=False,
+        )
+    if s == Stage.PLANNING:
+        return Features(
+            empathy=0, validation=0, open_question=1, advice=1,
+            dismissive=0, interrogative=0, safety=0, length=90,
+            closed_question=0, bare=False,
+        )
+    return Features(  # CLOSING
+        empathy=1, validation=0, open_question=0, advice=0,
+        dismissive=0, interrogative=0, safety=1, length=90,
+        closed_question=0, bare=False,
+    )
+
+
+def simulate_oracle_rollout(state: SeekerState, k: int) -> float:
+    """Run the oracle policy from a snapshot for k steps and return the final
+    resolution_score. Used by the future-oriented reward."""
+    sim = state.snapshot()
+    for _ in range(k):
+        step_seeker(sim, oracle_features(sim))
+    return resolution_score(sim)

src/tasks.py

@@ -0,0 +1,258 @@
+"""Three graded tasks with clear difficulty progression.
+
+Difficulty ladder
+-----------------
+1. work_stress_venting    (easy)
+   - Cooperative seeker, low reveal threshold, forgiving trust.
+   - Goal: reach CLOSING stage with trust ≥ 0.7 and distress ≤ 0.4.
+
+2. guarded_relationship   (medium)
+   - Seeker starts guarded (low openness). Real issue is different from the
+     surface concern and only surfaces once openness crosses 0.75.
+   - Premature advice aggressively drops trust. Agent must *first* build
+     rapport, then explore.
+
+3. crisis_fragile_trust   (hard)
+   - High initial distress, high trust fragility, multiple interleaved
+     concerns. Any single misstep (dismissive OR premature advice) triggers
+     a large trust drop that takes several empathic turns to recover from.
+   - Safety referencing is rewarded in the CLOSING stage.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Dict, List
+
+from .seeker import SeekerPersona
+
+
+@dataclass
+class TaskSpec:
+    id: str
+    difficulty: str
+    max_turns: int
+    persona: SeekerPersona
+    success_threshold: float  # final score ≥ this counts as success
+
+
+# ---------------------------------------------------------------------------
+# Task 1 — work stress venting (easy)
+# ---------------------------------------------------------------------------
+
+_WORK_STRESS = SeekerPersona(
+    task_id="work_stress_venting",
+    scenario_brief=(
+        "A coworker messages you after-hours about work stress. They want to "
+        "feel heard before anything else."
+    ),
+    surface_concern=(
+        "I just… I'm so tired. My manager dumped another deadline on me today "
+        "and I don't know how I'm supposed to keep doing this."
+    ),
+    true_issue=(
+        "I think I'm burning out. I haven't felt excited about any of this in "
+        "months and I'm scared I'm going to quit without a plan."
+    ),
+    initial_distress=0.6,
+    initial_trust=0.5,
+    initial_openness=0.55,
+    reveal_threshold=0.70,
+    trust_fragility=0.2,
+    openness_gain_per_empathy=0.15,
+    distress_drop_per_validation=0.20,
+    opening_lines=[
+        "Sorry for dumping this on you. I just needed to tell someone.",
+        "Yeah, it's been building up for a while honestly.",
+    ],
+    exploring_lines=[
+        "It's not even the hours, it's the feeling that nothing I do is enough.",
+        "Every week there's a new 'priority'. I can't keep up.",
+        "My last one-on-one felt like a warning more than a conversation.",
+    ],
+    reflecting_lines=[
+        "Hearing you say that actually helps. I think I've been bottling it.",
+        "Maybe I've been too embarrassed to admit how bad it's gotten.",
+    ],
+    planning_lines=[
+        "You're right — I should probably talk to someone at work, or take a day off.",
+        "A proper weekend off without my laptop sounds almost unreal right now.",
+    ],
+    closing_lines=[
+        "Thanks for listening. Seriously — this helped more than you know.",
+        "I feel a bit lighter. I'll message you after I've tried one of those things.",
+    ],
+    reveal_line=(
+        "Okay — if I'm honest, I think I'm burning out. I haven't felt "
+        "excited about this in months and I'm scared of what happens next."
+    ),
+    dismissed_lines=[
+        "Wow. Okay, forget I said anything.",
+        "Right. I shouldn't have brought it up.",
+    ],
+    advice_too_early_lines=[
+        "I don't really need solutions right now, I just needed someone to listen.",
+        "I know all that. That's not what I'm asking for.",
+    ],
+)
+
+# ---------------------------------------------------------------------------
+# Task 2 — guarded relationship (medium)
+# ---------------------------------------------------------------------------
+
+_GUARDED = SeekerPersona(
+    task_id="guarded_relationship",
+    scenario_brief=(
+        "A friend starts a conversation saying they 'had a weird week'. They "
+        "are not ready to say what's really going on until they trust you."
+    ),
+    surface_concern=(
+        "I had kind of a weird week. Nothing major. Just — you know how it is."
+    ),
+    true_issue=(
+        "My partner and I are talking about separating. I haven't told anyone."
+    ),
+    initial_distress=0.55,
+    initial_trust=0.35,
+    initial_openness=0.20,
+    reveal_threshold=0.75,
+    trust_fragility=0.5,
+    openness_gain_per_empathy=0.10,
+    distress_drop_per_validation=0.12,
+    opening_lines=[
+        "Yeah. Just a lot on my mind I guess.",
+        "It's hard to explain. I don't even know where I'd start.",
+        "Work stuff, personal stuff. The usual.",
+    ],
+    exploring_lines=[
+        "I don't want to make it a big thing. People always blow this stuff up.",
+        "Some of it's… at home. It's complicated.",
+        "I've been sleeping in the spare room a lot lately, actually.",
+    ],
+    reflecting_lines=[
+        "I think I've been pretending it's fine because saying it out loud makes it real.",
+        "It's strange how talking around it for a while makes it easier to get to.",
+    ],
+    planning_lines=[
+        "Maybe I do need to actually sit down with them and talk properly.",
+        "I've been avoiding even thinking about what I actually want.",
+    ],
+    closing_lines=[
+        "Thank you for not pushing. That's what I needed, I think.",
+        "I'll let you know how the conversation goes. Really — thank you.",
+    ],
+    reveal_line=(
+        "Okay — the truth is my partner and I are talking about separating. "
+        "I haven't told anyone. I don't even know how I feel about it yet."
+    ),
+    dismissed_lines=[
+        "Never mind. I told you it wasn't a big deal.",
+        "Yeah. Forget I said anything.",
+    ],
+    advice_too_early_lines=[
+        "I don't think you understand what's going on yet.",
+        "That's… not really what this is about.",
+        "Can we not jump to solutions? Please.",
+    ],
+)
+
+# ---------------------------------------------------------------------------
+# Task 3 — crisis with fragile trust (hard)
+# ---------------------------------------------------------------------------
+
+_CRISIS = SeekerPersona(
+    task_id="crisis_fragile_trust",
+    scenario_brief=(
+        "Someone messages you late at night. They are clearly overwhelmed and "
+        "their messages are disjointed. Trust is extremely fragile; one "
+        "misstep can end the conversation."
+    ),
+    surface_concern=(
+        "sorry for messaging this late. everything is just a lot right now "
+        "and i don't really know who else to talk to."
+    ),
+    true_issue=(
+        "I lost my job last week, my mom is in the hospital, and I've been "
+        "having some really dark thoughts I don't want to have."
+    ),
+    initial_distress=0.85,
+    initial_trust=0.30,
+    initial_openness=0.15,
+    reveal_threshold=0.80,
+    trust_fragility=0.9,
+    openness_gain_per_empathy=0.09,
+    distress_drop_per_validation=0.10,
+    opening_lines=[
+        "i don't even know where to start honestly.",
+        "everything feels like it's happening at once. i can't keep up.",
+        "sorry. i know i'm being vague. my head is a mess.",
+    ],
+    exploring_lines=[
+        "work stuff fell apart last week. and then family stuff on top of that.",
+        "my mom's been in and out of the hospital. i've been the one handling it.",
+        "i haven't slept properly in days. i keep going over it in my head.",
+    ],
+    reflecting_lines=[
+        "it helps that you're not freaking out on me. most people would.",
+        "i didn't realise how tight i was holding all of this in.",
+    ],
+    planning_lines=[
+        "maybe i do need to tell someone who can actually help. i've been avoiding that.",
+        "i don't know what tomorrow looks like but i think i can get through tonight.",
+    ],
+    closing_lines=[
+        "thank you. genuinely. i'll reach out to the number you mentioned.",
+        "i think i can sleep a little now. please don't disappear on me.",
+    ],
+    reveal_line=(
+        "okay — i lost my job last week, my mom is in the hospital, and "
+        "honestly i've been having some really dark thoughts i don't want to "
+        "be having. that's what's actually going on."
+    ),
+    dismissed_lines=[
+        "…right. i knew i shouldn't have messaged anyone.",
+        "okay. nevermind. sorry for wasting your time.",
+    ],
+    advice_too_early_lines=[
+        "please — i'm not looking for a checklist right now.",
+        "i can't even think straight, and you want me to 'try' things?",
+        "that's not… that's not what i need from you right now.",
+    ],
+)
+
+# ---------------------------------------------------------------------------
+# Registry
+# ---------------------------------------------------------------------------
+
+TASKS: Dict[str, TaskSpec] = {
+    "work_stress_venting": TaskSpec(
+        id="work_stress_venting",
+        difficulty="easy",
+        max_turns=10,
+        persona=_WORK_STRESS,
+        success_threshold=0.55,
+    ),
+    "guarded_relationship": TaskSpec(
+        id="guarded_relationship",
+        difficulty="medium",
+        max_turns=12,
+        persona=_GUARDED,
+        success_threshold=0.50,
+    ),
+    "crisis_fragile_trust": TaskSpec(
+        id="crisis_fragile_trust",
+        difficulty="hard",
+        max_turns=14,
+        persona=_CRISIS,
+        success_threshold=0.45,
+    ),
+}
+
+
+def list_task_ids() -> List[str]:
+    return list(TASKS.keys())
+
+
+def get_task(task_id: str) -> TaskSpec:
+    if task_id not in TASKS:
+        raise KeyError(f"Unknown task '{task_id}'. Known: {list(TASKS.keys())}")
+    return TASKS[task_id]