Save every submit as a new version; let users pick a version to load

- save_submission() now writes submissions/<trial>__<user>/<stamp>.json per
submit (no overwrite); full version history is kept.
- add list_versions(trial_id, username); reviews are keyed per-version under
reviews/<trial>__<user>/<stamp>/.
- Form: 'Find versions' lists all versions for a trial_id+username; a selectbox
+ 'Load selected version' pulls the chosen one in for editing.
- Admin/list_submissions treats each version as its own row with its own
reviews; README + Python loader updated for the nested layout.

README.md

@@ -23,7 +23,7 @@ A Streamlit intake form for trial statisticians. Submissions are saved to a **Hu
     - `extraction_only` → 1 rubric: `output.json`
     - `derivation_required` → 4 rubrics: `output.json` × {Inputs used, Calculated value, Method} + `output.R` × {Reproducibility}
   - Each rubric collects `points`, `tolerance`, `criterion`.
-  - **Load existing submission** — re-enter the same `trial_id` + `username` and click Load to pull a previous submission back into the form, edit it, and Submit again to update.
+  - **Versions** — every Submit saves a new version. Re-enter the same `trial_id` + `username`, click **Find versions**, pick one, and **Load selected version** to pull it back into the form for editing; Submit then saves a new version.
 - **Admin page (`pages/1_Admin.py`)** — password-gated review console. A submission can be reviewed many times by different people: each review (status + reviewer name + comment) is written as its own file under `reviews/<submission>/`, and the page shows the full timeline. The current status is the most recent review's status.
 
 ## Run locally
@@ -85,33 +85,33 @@ The Space will restart automatically and pick up the new secrets.
 
 ### 6. Test
 
-- Open the Space URL → fill the form → **Submit**. A file lands in `submissions/<trial_id>__<username>.json` in the dataset repo. Submitting again with the same trial_id + username updates that file.
+- Open the Space URL → fill the form → **Submit**. A file lands in `submissions/<trial_id>__<username>/<stamp>.json` in the dataset repo. Submitting again saves another version in the same folder.
 - Open the **Admin** page (left sidebar) → enter password → see the submission with status `pending` → add a review (your name + status + comment). It appears in the review timeline and a new file lands under `reviews/<submission>/`. Add more reviews to build up the history.
 
 ## Dataset layout
 
-One submission file per `(trial_id, username)` pair — submitting again
-**updates** the same file, so a submission can be loaded back and edited.
-(Edit history is preserved in the dataset's git commits.) Each review is a
-**separate file**, so a submission can be reviewed many times by different
-people and concurrent reviews never conflict.
+Every submit saves a **new version** under a per-pair folder — nothing is
+overwritten, so the full version history is kept and any version can be loaded
+back. Each review is a **separate file** keyed to a specific version, so a
+version can be reviewed many times by different people and concurrent reviews
+never conflict.
 
 ```text
-submissions/<trial>__<user>.json            # the submission (upserted on each submit)
-reviews/<trial>__<user>/<stamp>__<rev>.json # one file per review
+submissions/<trial>__<user>/<stamp>.json            # one file per version
+reviews/<trial>__<user>/<stamp>/<revstamp>__<rev>.json  # one file per review of that version
 ```
 
-To edit an existing submission: on the form, enter the same `trial_id` +
-`username` and click **Load existing submission**, edit, then **Submit**.
+To load/edit a previous version: on the form, enter the same `trial_id` +
+`username`, click **Find versions**, pick a version, click **Load selected
+version**, edit, then **Submit** (which saves a new version).
 
-### Submission file (`submissions/*.json`)
+### Submission file (`submissions/<trial>__<user>/<stamp>.json`)
 
 ```json
 {
-  "submissionId": "submissions/NCT0001__jdoe.json",
-  "createdAt": "2026-06-01T...",
-  "updatedAt": "2026-06-04T...",
-  "submittedAt": "2026-06-01T...",
+  "submissionId": "submissions/NCT0001__jdoe/2026-06-04T...Z.json",
+  "version": "2026-06-04T...Z",
+  "submittedAt": "2026-06-04T...",
   "trial_id": "NCT0001",
   "username": "jdoe",
   "comparison": {
@@ -136,7 +136,7 @@ To edit an existing submission: on the form, enter the same `trial_id` +
 }
 ```
 
-### Review file (`reviews/<submission>/*.json`)
+### Review file (`reviews/<trial>__<user>/<stamp>/*.json`)
 
 ```json
 {
@@ -159,17 +159,17 @@ import json, glob, os
 
 local = snapshot_download("ttt-77/tdb-intake-submissions", repo_type="dataset")
 
-submissions = {
-    os.path.basename(f)[:-5]: json.load(open(f))
-    for f in glob.glob(f"{local}/submissions/*.json")
-}
-# reviews grouped by submission base name
+# every version: submissions/<trial>__<user>/<stamp>.json
+submissions = [json.load(open(f)) for f in glob.glob(f"{local}/submissions/*/*.json")]
+
+# reviews: reviews/<trial>__<user>/<stamp>/<revstamp>__<rev>.json
+# key = "<trial>__<user>/<stamp>" (matches a submission's submissionId minus prefix/suffix)
 reviews = {}
-for f in glob.glob(f"{local}/reviews/*/*.json"):
-    base = os.path.basename(os.path.dirname(f))
-    reviews.setdefault(base, []).append(json.load(open(f)))
-for base in reviews:
-    reviews[base].sort(key=lambda r: r["at"])  # oldest first
+for f in glob.glob(f"{local}/reviews/*/*/*.json"):
+    pair, ver = f.split("/reviews/")[1].split("/")[:2]
+    reviews.setdefault(f"{pair}/{ver}", []).append(json.load(open(f)))
+for key in reviews:
+    reviews[key].sort(key=lambda r: r["at"])  # oldest first
 ```
 
 ## Project structure

app.py

@@ -22,7 +22,12 @@ from lib.schema import (
     next_question_id,
     rubrics_for_type,
 )
-from lib.storage import get_submission_by_key, hf_configured, save_submission
+from lib.storage import (
+    get_submission,
+    hf_configured,
+    list_versions,
+    save_submission,
+)
 
 st.set_page_config(
     page_title="TDB Intake",
@@ -44,6 +49,9 @@ if "last_result" not in st.session_state:
 # per-question widgets get fresh keys and actually show the new values.
 if "form_nonce" not in st.session_state:
     st.session_state.form_nonce = 0
+# Versions found for the current trial_id + username (after "Find versions").
+if "versions" not in st.session_state:
+    st.session_state.versions = []
 
 
 # ------------- callbacks -------------------------------------------------
@@ -70,36 +78,59 @@ def _save_draft() -> None:
     st.session_state.last_result = {"kind": "draft", "msg": "Draft saved in this browser session."}
 
 
-def _load() -> None:
-    """Load an existing submission by (trial_id, username) into the form."""
+def _find_versions() -> None:
+    """Look up all saved versions for the current trial_id + username."""
     trial_id = st.session_state.trial_id.strip()
     username = st.session_state.username.strip()
     if not trial_id or not username:
+        st.session_state.versions = []
         st.session_state.last_result = {
             "kind": "error",
-            "msg": "Enter trial_id and username, then click Load.",
+            "msg": "Enter trial_id and username, then click Find versions.",
         }
         return
     try:
-        record = get_submission_by_key(trial_id, username)
+        versions = list_versions(trial_id, username)
     except Exception as e:
-        st.session_state.last_result = {"kind": "error", "msg": f"Load failed: {e}"}
+        st.session_state.versions = []
+        st.session_state.last_result = {"kind": "error", "msg": f"Lookup failed: {e}"}
         return
-    if not record:
+    st.session_state.versions = versions
+    if not versions:
         st.session_state.last_result = {
             "kind": "info",
-            "msg": f"No existing submission for `{trial_id}` / `{username}`. "
-            "Add questions and Submit to create one.",
+            "msg": f"No versions yet for `{trial_id}` / `{username}`. "
+            "Add questions and Submit to create the first one.",
         }
+    else:
+        st.session_state.last_result = {
+            "kind": "success",
+            "msg": f"Found {len(versions)} version(s). Pick one below and click "
+            "“Load selected version”.",
+        }
+
+
+def _load_selected() -> None:
+    """Load the version chosen in the version selectbox into the form."""
+    sub_id = st.session_state.get("version_select")
+    if not sub_id:
+        st.session_state.last_result = {"kind": "error", "msg": "Pick a version first."}
+        return
+    try:
+        record = get_submission(sub_id)
+    except Exception as e:
+        st.session_state.last_result = {"kind": "error", "msg": f"Load failed: {e}"}
+        return
+    if not record:
+        st.session_state.last_result = {"kind": "error", "msg": "That version could not be loaded."}
         return
     prompts = (record.get("comparison") or {}).get("prompts") or []
     st.session_state.questions = prompts
     st.session_state.form_nonce += 1  # force question widgets to refresh
-    updated = record.get("updatedAt") or record.get("submittedAt") or ""
     st.session_state.last_result = {
         "kind": "success",
-        "msg": f"Loaded {len(prompts)} question(s) (last updated {updated}). "
-        "Edit and Submit to update.",
+        "msg": f"Loaded version {record.get('version', '')} "
+        f"({len(prompts)} question(s)). Edit and Submit to save a new version.",
     }
 
 
@@ -117,13 +148,17 @@ def _submit() -> None:
     }
     try:
         result = save_submission(trial_id, username, comparison)
-        verb = "Updated" if result.get("updated") else "Submitted"
         st.session_state.last_result = {
             "kind": "success",
-            "msg": f"{verb}: `{result['submissionId']}`. "
-            "You can keep editing and Submit again to update.",
+            "msg": f"Saved as new version `{result['version']}`. "
+            "Use “Find versions” to see all versions.",
             "url": result.get("url"),
         }
+        # Refresh the version list so the new version shows up.
+        try:
+            st.session_state.versions = list_versions(trial_id, username)
+        except Exception:
+            pass
         # Keep the form populated so the user can continue editing.
     except Exception as e:
         st.session_state.last_result = {"kind": "error", "msg": f"Submit failed: {e}"}
@@ -149,11 +184,32 @@ with c2:
     st.text_input("username", key="username", placeholder="e.g., jdoe")
 
 st.button(
-    "Load existing submission",
-    on_click=_load,
-    help="If you already submitted for this trial_id + username, load it back to edit.",
+    "Find versions",
+    on_click=_find_versions,
+    help="List all previously submitted versions for this trial_id + username.",
 )
 
+versions = st.session_state.versions
+if versions:
+    options = [v["submissionId"] for v in versions]
+    labels = {
+        v["submissionId"]: f"{v['submittedAt']}  ·  v{v['version']}  ·  "
+        f"{v['num_questions']} question(s)"
+        for v in versions
+    }
+    vc1, vc2 = st.columns([3, 1])
+    with vc1:
+        st.selectbox(
+            "Select a version to load",
+            options=options,
+            format_func=lambda sid: labels.get(sid, sid),
+            key="version_select",
+        )
+    with vc2:
+        st.write("")
+        st.write("")
+        st.button("Load selected version", on_click=_load_selected, use_container_width=True)
+
 st.divider()
 
 # ------------- questions list --------------------------------------------

lib/storage.py

@@ -59,7 +59,27 @@ def _stamp(iso: Optional[str] = None) -> str:
     return (iso or _now_iso()).replace(":", "-").replace(".", "-")
 
 
+def _pair_dir(trial_id: str, username: str) -> str:
+    """Folder holding all versions for a (trial_id, username) pair."""
+    return f"{SUBMISSIONS_PREFIX}/{_safe(trial_id)}__{_safe(username)}"
+
+
 def _base_id(submission_id: str) -> str:
+    """Path of a submission relative to the submissions/ prefix, without .json.
+
+    'submissions/NCT99__jdoe/2026-...json' -> 'NCT99__jdoe/2026-...'
+    Used to key the matching reviews/ folder so reviews stay grouped per
+    (pair, version).
+    """
+    s = submission_id
+    if s.startswith(f"{SUBMISSIONS_PREFIX}/"):
+        s = s[len(SUBMISSIONS_PREFIX) + 1 :]
+    if s.endswith(".json"):
+        s = s[:-5]
+    return s
+
+
+def _legacy_base_id(submission_id: str) -> str:
     """'submissions/foo.json' -> 'foo'"""
     name = submission_id.split("/")[-1]
     return name[:-5] if name.endswith(".json") else name
@@ -134,56 +154,66 @@ def _all_files() -> List[str]:
 
 # ---- public API ----------------------------------------------------------
 
-def submission_id_for(trial_id: str, username: str) -> str:
-    """Stable submission id (path) for a (trial_id, username) pair.
-
-    One submission per pair — submitting again updates the same file, so a
-    submission can be loaded back and edited.
-    """
-    return f"{SUBMISSIONS_PREFIX}/{_safe(trial_id)}__{_safe(username)}.json"
-
-
-def get_submission_by_key(trial_id: str, username: str) -> Optional[Dict[str, Any]]:
-    """Load an existing submission by (trial_id, username), or None."""
-    return get_submission(submission_id_for(trial_id, username))
-
-
 def save_submission(trial_id: str, username: str, comparison: Dict[str, Any]) -> Dict[str, Any]:
-    """Create or update the submission for (trial_id, username).
+    """Save a NEW version for (trial_id, username).
 
-    If a submission already exists for this pair, it is updated in place
-    (createdAt is preserved); otherwise a new one is created.
+    Every submit creates a new version file under
+    submissions/<trial>__<user>/<stamp>.json — nothing is overwritten, so the
+    full version history is kept and any version can be loaded back.
     """
-    submission_id = submission_id_for(trial_id, username)
     now = _now_iso()
-    existing = get_submission(submission_id)
-    created_at = (existing or {}).get("createdAt") or (existing or {}).get("submittedAt") or now
-    is_update = existing is not None
-
+    version = _stamp(now)
+    submission_id = f"{_pair_dir(trial_id, username)}/{version}.json"
     record = {
         "submissionId": submission_id,
-        "createdAt": created_at,
-        "updatedAt": now,
-        # kept for backward compatibility with older records / admin display
-        "submittedAt": created_at,
+        "version": version,
+        "submittedAt": now,
         "trial_id": trial_id,
         "username": username,
         "comparison": comparison,
     }
-    verb = "Update" if is_update else "Add"
-    _write_json(submission_id, record, f"{verb} submission: {trial_id} — {username}")
+    _write_json(
+        submission_id,
+        record,
+        f"Add submission: {trial_id} — {username} ({version})",
+    )
     url = (
         f"https://huggingface.co/datasets/{HF_DATASET_REPO}"
         f"/blob/{HF_DATASET_BRANCH}/{submission_id}"
         if hf_configured
         else None
     )
-    return {
-        "submissionId": submission_id,
-        "url": url,
-        "record": record,
-        "updated": is_update,
-    }
+    return {"submissionId": submission_id, "url": url, "record": record, "version": version}
+
+
+def list_versions(
+    trial_id: str, username: str, all_files: Optional[List[str]] = None
+) -> List[Dict[str, Any]]:
+    """All saved versions for (trial_id, username), newest first.
+
+    Each item: submissionId, version, submittedAt, num_questions.
+    """
+    prefix = f"{_pair_dir(trial_id, username)}/"
+    files = all_files if all_files is not None else _all_files()
+    paths = sorted(
+        (f for f in files if f.startswith(prefix) and f.endswith(".json")),
+        reverse=True,
+    )
+    out: List[Dict[str, Any]] = []
+    for p in paths:
+        rec = _read_json(p)
+        if not rec:
+            continue
+        prompts = (rec.get("comparison") or {}).get("prompts") or []
+        out.append(
+            {
+                "submissionId": p,
+                "version": rec.get("version", ""),
+                "submittedAt": rec.get("submittedAt", ""),
+                "num_questions": len(prompts),
+            }
+        )
+    return out
 
 
 def add_review(submission_id: str, status: str, reviewer: str, note: str = "") -> Dict[str, Any]:
@@ -245,8 +275,8 @@ def list_submissions() -> List[Dict[str, Any]]:
                 "submissionId": sp,
                 "trial_id": sub.get("trial_id", ""),
                 "username": sub.get("username", ""),
+                "version": sub.get("version", ""),
                 "submittedAt": sub.get("submittedAt", ""),
-                "updatedAt": sub.get("updatedAt", sub.get("submittedAt", "")),
                 "status": latest["status"] if latest else "pending",
                 "reviewedAt": latest["at"] if latest else "",
                 "reviewer": latest["reviewer"] if latest else "",
@@ -255,7 +285,7 @@ def list_submissions() -> List[Dict[str, Any]]:
                 "submission": sub,
             }
         )
-    result.sort(key=lambda r: r.get("updatedAt", ""), reverse=True)
+    result.sort(key=lambda r: r.get("submittedAt", ""), reverse=True)
     return result