FastAPI + submit + private dataset (Docker SDK)

Dockerfile

@@ -0,0 +1,7 @@
+FROM python:3.12-slim
+WORKDIR /code
+COPY requirements.txt /code/requirements.txt
+RUN pip install --no-cache-dir -r /code/requirements.txt
+COPY . /code
+EXPOSE 7860
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -3,10 +3,8 @@ title: RoboCasa Kitchen Leaderboard
 emoji: 🍳
 colorFrom: blue
 colorTo: green
-sdk: gradio
-sdk_version: 4.44.1
-python_version: "3.12"
-app_file: app.py
+sdk: docker
+app_port: 7860
 pinned: true
 license: mit
 short_description: Neutral aggregation of VLA success rates on RoboCasa Kitchen
@@ -14,23 +12,20 @@ short_description: Neutral aggregation of VLA success rates on RoboCasa Kitchen
 
 # 🍳 RoboCasa Kitchen Leaderboard
 
-A **neutral, community aggregation** of Vision-Language-Action (VLA) policy success rates on the
-**RoboCasa Kitchen** benchmark (single-arm Franka Panda, 24 atomic manipulation tasks), maintained by
-[GINIGEN-AI](https://huggingface.co/ginigen-ai).
-
-## Why this exists
-RoboCasa has **no official leaderboard** and several **incompatible variants** (single-arm Kitchen,
-GR1-Tabletop humanoid, RoboCasa365, demo-conditioned). Published numbers are scattered across papers
-under different protocols. This board aggregates them **with citations** and **separates protocols**.
+**FastAPI + static HTML.** A neutral community aggregation of Vision-Language-Action (VLA)
+policy success rates on **RoboCasa Kitchen** (single-arm Franka Panda, 24 atomic tasks),
+maintained by [GINIGEN-AI](https://huggingface.co/ginigen-ai).
 
 ## Tables
-1. **Kitchen 24-task (matched protocol)** — head-to-head from the **RLDX-1 Technical Report** (arXiv:2605.03269).
-   Current SOTA: **RLDX-1 70.6%** (RLWRLD / KAIST, Jinwoo Shin).
-2. **Other protocols** — self-reported under different setups (e.g. Cosmos-Policy, 50 demos); not directly comparable.
+1. **Kitchen 24-task (matched protocol)** — head-to-head from the RLDX-1 Technical Report (arXiv:2605.03269). SOTA: **RLDX-1 70.6%**.
+2. **Other protocols** — self-reported under different setups (not directly comparable).
 3. **GR1-Tabletop variant** — a *different* RoboCasa task suite.
 
+## Submit
+Use the form on the page. Each submission is stored privately (one JSON per entry, `status: pending`)
+and reviewed before appearing on the board.
+
 ## Honesty policy
 Every number is cited to its paper / model card. Protocol differences are surfaced, not hidden.
-Corrections welcome via the Community tab.
 
 *Maintained by GINIGEN-AI. Not affiliated with the RoboCasa authors.*

app.py

@@ -1,72 +1,51 @@
 # -*- coding: utf-8 -*-
-"""RoboCasa Kitchen Leaderboard — neutral aggregation by GINIGEN-AI.
-Scores are collected from published papers (matched-protocol where available).
-Not an official RoboCasa leaderboard; a community aggregation."""
-import gradio as gr
-import pandas as pd
-
-# ── RoboCasa Kitchen (single-arm, 24 atomic tasks, Franka Panda) ──
-# Source of truth: RLDX-1 Technical Report (arXiv 2605.03269) matched-protocol table.
-KITCHEN = [
-    [1, "RLDX-1",      70.6, "RLWRLD / KAIST (Jinwoo Shin)", "MSAT",              "arXiv:2605.03269", "2026-05", "matched (RLDX-1 report)"],
-    [2, "GR00T N1.6",  66.2, "NVIDIA",                       "GR00T",             "arXiv:2503.14734", "2026",    "reproduced in RLDX-1"],
-    [3, "GR00T N1.5",  65.7, "NVIDIA",                       "GR00T",             "arXiv:2503.14734", "2025",    "reproduced in RLDX-1"],
-    [4, "π0-FAST",     63.6, "Physical Intelligence",        "π0 (FAST tok.)",    "—",                "2025",    "reproduced in RLDX-1"],
-    [5, "π0",          62.5, "Physical Intelligence",        "π0 (flow)",         "arXiv:2410.24164", "2024",    "reproduced in RLDX-1"],
-    [6, "π0.5",        62.1, "Physical Intelligence",        "π0.5",              "arXiv:2504.16054", "2025",    "reproduced in RLDX-1"],
-]
-KITCHEN_COLS = ["#", "Model", "Kitchen 24-task SR (%)", "Org", "Backbone/Arch", "Paper", "Date", "Protocol note"]
-
-# Different-protocol entries (NOT directly comparable — fewer demos / own eval)
-OTHER = [
-    ["Cosmos-Policy (Predict2-2B)", 67.1, "NVIDIA", "2B (video-FM finetune)", "hf.co/nvidia/Cosmos-Policy-RoboCasa-Predict2-2B", "2026", "50 demos, NVIDIA-own protocol"],
-]
-OTHER_COLS = ["Model", "Reported SR (%)", "Org", "Backbone", "Source", "Date", "Why separate"]
-
-# GR1-Tabletop variant (humanoid GR1 — DO NOT mix with single-arm Kitchen)
-GR1 = [
-    ["GR00T (h=16)",  59.7, "NVIDIA",  "arXiv:2604.05014 (StarVLA)"],
-    ["TwinBrainVLA",  54.6, "—",       "arXiv:2601.14133"],
-    ["VP-VLA",        53.8, "—",       "arXiv:2603.22003"],
-    ["StarVLA-OFT",   48.8, "—",       "arXiv:2604.05014"],
-    ["GR00T-N1.6",    24.2, "NVIDIA",  "RoboCasa-GR1 (no-collision 7.0%)"],
-]
-GR1_COLS = ["Model", "GR1-Tabletop 24-task SR (%)", "Org", "Source"]
-
-def df(rows, cols): return pd.DataFrame(rows, columns=cols)
-
-INTRO = """
-# 🍳 RoboCasa Kitchen Leaderboard
-**Neutral community aggregation by [GINIGEN-AI](https://huggingface.co/ginigen-ai).**
-Vision-Language-Action (VLA) policies on **RoboCasa Kitchen** (single-arm Franka Panda, 24 atomic manipulation tasks).
-
-> ⚠️ RoboCasa has **no official leaderboard** and **multiple variants**. This board aggregates **published** numbers.
-> The main table uses the **matched protocol** reported in the RLDX-1 Technical Report (arXiv:2605.03269), the most recent head-to-head comparison.
-> Entries under other protocols (different #demos / eval) are listed **separately** and are **not directly comparable**.
-"""
-
-NOTES = """
-### Methodology / Honesty
-- **Main table** = RoboCasa Kitchen, single-arm, 24 atomic tasks, averaged success rate, **same protocol** (RLDX-1 report, head-to-head).
-- **Different-protocol** = self-reported under a different setup (e.g. Cosmos-Policy uses 50 demos) → shown apart to avoid apples-vs-oranges.
-- **GR1-Tabletop** = a *different* RoboCasa variant (humanoid GR1). Never compare its numbers with single-arm Kitchen.
-- Corrections / additions welcome via the Community tab. We cite the paper for every number.
-
-### Sources
-- RLDX-1 Technical Report — arXiv:2605.03269 (RLWRLD / KAIST, Jinwoo Shin)
-- RoboCasa — arXiv:2406.02523 ; RoboCasa365 — arXiv:2603.04356
-- GR00T N1 — arXiv:2503.14734 ; Cosmos-Policy — hf.co/nvidia/Cosmos-Policy-RoboCasa-Predict2-2B
-"""
-
-with gr.Blocks(title="RoboCasa Kitchen Leaderboard — GINIGEN-AI", theme=gr.themes.Soft()) as demo:
-    gr.Markdown(INTRO)
-    with gr.Tab("🏆 Kitchen 24-task (matched protocol)"):
-        gr.Dataframe(value=df(KITCHEN, KITCHEN_COLS), interactive=False, wrap=True)
-    with gr.Tab("➕ Other protocols (not comparable)"):
-        gr.Dataframe(value=df(OTHER, OTHER_COLS), interactive=False, wrap=True)
-    with gr.Tab("🤖 GR1-Tabletop variant (separate)"):
-        gr.Dataframe(value=df(GR1, GR1_COLS), interactive=False, wrap=True)
-    gr.Markdown(NOTES)
-
-if __name__ == "__main__":
-    demo.launch()
+"""RoboCasa Kitchen Leaderboard — FastAPI.
+Serves static index.html and accepts /submit, writing one JSON file per
+submission to a PRIVATE HF dataset (ginigen-ai/robocasa-submissions)."""
+import os, io, json, datetime, re
+from fastapi import FastAPI, Form
+from fastapi.responses import FileResponse, JSONResponse
+from huggingface_hub import HfApi
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+DS_REPO = "ginigen-ai/robocasa-submissions"
+TOKEN = os.environ.get("HF_TOKEN")
+app = FastAPI(title="RoboCasa Kitchen Leaderboard", docs_url=None, redoc_url=None)
+
+
+@app.get("/")
+def index():
+    return FileResponse(os.path.join(HERE, "index.html"))
+
+
+@app.get("/health")
+def health():
+    return JSONResponse({"status": "ok", "token_configured": bool(TOKEN)})
+
+
+@app.post("/submit")
+async def submit(model: str = Form(...), sr: float = Form(...), protocol: str = Form(...),
+                 paper: str = Form(""), submitter: str = Form(...)):
+    if not TOKEN:
+        return JSONResponse({"ok": False, "error": "server token not configured"}, status_code=500)
+    try:
+        srv = float(sr)
+    except Exception:
+        return JSONResponse({"ok": False, "error": "SR must be a number"}, status_code=400)
+    if not (0.0 <= srv <= 100.0):
+        return JSONResponse({"ok": False, "error": "SR must be 0-100"}, status_code=400)
+    rec = {"model": model.strip()[:120], "sr": round(srv, 2), "protocol": protocol.strip()[:60],
+           "paper": paper.strip()[:300], "submitter": submitter.strip()[:120],
+           "ts": datetime.datetime.utcnow().isoformat() + "Z", "status": "pending"}
+    if not rec["model"] or not rec["submitter"]:
+        return JSONResponse({"ok": False, "error": "model and submitter required"}, status_code=400)
+    safe = re.sub(r"[^A-Za-z0-9_-]", "", model)[:30] or "model"
+    fname = "submissions/%s_%s.json" % (rec["ts"].replace(":", "").replace(".", ""), safe)
+    try:
+        HfApi(token=TOKEN).upload_file(
+            path_or_fileobj=io.BytesIO(json.dumps(rec, ensure_ascii=False, indent=2).encode("utf-8")),
+            path_in_repo=fname, repo_id=DS_REPO, repo_type="dataset",
+            commit_message="submit: %s by %s" % (rec["model"], rec["submitter"]))
+        return JSONResponse({"ok": True})
+    except Exception as e:
+        return JSONResponse({"ok": False, "error": str(e)[:200]}, status_code=500)

index.html

@@ -0,0 +1,113 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>RoboCasa Kitchen Leaderboard — GINIGEN-AI</title>
+<style>
+:root{--bg:#0b1020;--card:#141b2e;--bd:#26304a;--fg:#e6ebf5;--mut:#9aa6c0;--acc:#4f7cff;--gold:#ffd24a;--grn:#7fffd4;}
+*{box-sizing:border-box;}
+body{margin:0;padding:32px 16px;background:linear-gradient(160deg,#0b1020,#111a33);color:var(--fg);
+font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif;line-height:1.5;}
+.wrap{max-width:1000px;margin:0 auto;}
+h1{font-size:30px;margin:0 0 6px;}
+.sub{color:var(--mut);margin:0 0 20px;}
+.warn{background:rgba(79,124,255,.12);border:1px solid var(--bd);border-radius:10px;padding:12px 16px;
+margin:14px 0 26px;color:var(--mut);font-size:14px;}
+h2{font-size:20px;margin:28px 0 10px;}
+table{width:100%;border-collapse:collapse;background:var(--card);border:1px solid var(--bd);
+border-radius:12px;overflow:hidden;font-size:14px;}
+th,td{padding:10px 12px;text-align:left;border-bottom:1px solid var(--bd);}
+th{background:#1b2440;color:var(--mut);font-weight:600;font-size:12px;text-transform:uppercase;letter-spacing:.4px;}
+tr:last-child td{border-bottom:none;}
+tr.top td{background:rgba(255,210,74,.08);}
+.rank{font-weight:700;color:var(--gold);}
+.sr{font-weight:700;color:var(--grn);}
+a{color:var(--acc);text-decoration:none;}
+.foot{color:var(--mut);font-size:13px;margin-top:28px;border-top:1px solid var(--bd);padding-top:16px;}
+form{background:var(--card);border:1px solid var(--bd);border-radius:12px;padding:18px 18px 8px;margin-top:8px;}
+.row{display:flex;gap:14px;flex-wrap:wrap;}
+.fld{flex:1;min-width:180px;margin-bottom:14px;}
+label{display:block;font-size:12px;color:var(--mut);margin-bottom:5px;text-transform:uppercase;letter-spacing:.3px;}
+input,select{width:100%;padding:9px 11px;background:#0e1526;border:1px solid var(--bd);border-radius:8px;
+color:var(--fg);font-size:14px;}
+button{background:var(--acc);color:#fff;border:none;border-radius:8px;padding:10px 22px;font-size:14px;
+font-weight:600;cursor:pointer;margin-bottom:12px;}
+button:hover{background:#3a67e6;}
+#msg{font-size:14px;margin:6px 0 12px;min-height:20px;}
+</style>
+</head>
+<body>
+<div class="wrap">
+<h1>🍳 RoboCasa Kitchen Leaderboard</h1>
+<p class="sub">Neutral community aggregation by <a href="https://huggingface.co/ginigen-ai">GINIGEN-AI</a> · Vision-Language-Action (VLA) policies on <b>RoboCasa Kitchen</b> (single-arm Franka Panda, 24 atomic manipulation tasks).</p>
+<div class="warn">⚠️ RoboCasa has <b>no official leaderboard</b> and <b>multiple incompatible variants</b>. The main table uses the <b>matched protocol</b> from the RLDX-1 Technical Report (arXiv:2605.03269), the most recent head-to-head comparison. Other-protocol entries are listed separately and are <b>not directly comparable</b>. Every number is cited.</div>
+
+<h2>🏆 Kitchen 24-task (matched protocol)</h2>
+<table>
+<thead><tr><th>#</th><th>Model</th><th>SR&nbsp;(%)</th><th>Org</th><th>Backbone/Arch</th><th>Paper</th><th>Date</th><th>Protocol</th></tr></thead>
+<tbody>
+<tr class="top"><td class="rank">1</td><td><b>RLDX-1</b></td><td class="sr">70.6</td><td>RLWRLD / KAIST (Jinwoo Shin)</td><td>MSAT</td><td>arXiv:2605.03269</td><td>2026-05</td><td>matched (RLDX-1 report)</td></tr>
+<tr><td>2</td><td>GR00T N1.6</td><td class="sr">66.2</td><td>NVIDIA</td><td>GR00T</td><td>arXiv:2503.14734</td><td>2026</td><td>reproduced in RLDX-1</td></tr>
+<tr><td>3</td><td>GR00T N1.5</td><td class="sr">65.7</td><td>NVIDIA</td><td>GR00T</td><td>arXiv:2503.14734</td><td>2025</td><td>reproduced in RLDX-1</td></tr>
+<tr><td>4</td><td>&pi;0-FAST</td><td class="sr">63.6</td><td>Physical Intelligence</td><td>&pi;0 (FAST tok.)</td><td>&mdash;</td><td>2025</td><td>reproduced in RLDX-1</td></tr>
+<tr><td>5</td><td>&pi;0</td><td class="sr">62.5</td><td>Physical Intelligence</td><td>&pi;0 (flow)</td><td>arXiv:2410.24164</td><td>2024</td><td>reproduced in RLDX-1</td></tr>
+<tr><td>6</td><td>&pi;0.5</td><td class="sr">62.1</td><td>Physical Intelligence</td><td>&pi;0.5</td><td>arXiv:2504.16054</td><td>2025</td><td>reproduced in RLDX-1</td></tr>
+</tbody></table>
+
+<h2>➕ Other protocols (not directly comparable)</h2>
+<table>
+<thead><tr><th>Model</th><th>Reported&nbsp;SR&nbsp;(%)</th><th>Org</th><th>Backbone</th><th>Source</th><th>Why separate</th></tr></thead>
+<tbody>
+<tr><td>Cosmos-Policy (Predict2-2B)</td><td class="sr">67.1</td><td>NVIDIA</td><td>2B (video-FM finetune)</td><td><a href="https://huggingface.co/nvidia/Cosmos-Policy-RoboCasa-Predict2-2B">model card</a></td><td>50 demos, NVIDIA-own protocol</td></tr>
+</tbody></table>
+
+<h2>🤖 GR1-Tabletop variant (separate suite)</h2>
+<table>
+<thead><tr><th>Model</th><th>GR1-Tabletop&nbsp;24-task&nbsp;SR&nbsp;(%)</th><th>Org</th><th>Source</th></tr></thead>
+<tbody>
+<tr><td>GR00T (h=16)</td><td class="sr">59.7</td><td>NVIDIA</td><td>arXiv:2604.05014 (StarVLA)</td></tr>
+<tr><td>TwinBrainVLA</td><td class="sr">54.6</td><td>&mdash;</td><td>arXiv:2601.14133</td></tr>
+<tr><td>VP-VLA</td><td class="sr">53.8</td><td>&mdash;</td><td>arXiv:2603.22003</td></tr>
+<tr><td>StarVLA-OFT</td><td class="sr">48.8</td><td>&mdash;</td><td>arXiv:2604.05014</td></tr>
+<tr><td>GR00T-N1.6</td><td class="sr">24.2</td><td>NVIDIA</td><td>RoboCasa-GR1 (no-collision 7.0%)</td></tr>
+</tbody></table>
+
+<h2>📝 Submit your result</h2>
+<p class="sub" style="margin:0 0 8px">Submissions are stored privately and reviewed before appearing on the board.</p>
+<form id="sf">
+  <div class="row">
+    <div class="fld"><label>Model name *</label><input name="model" required placeholder="e.g. MyVLA-7B"></div>
+    <div class="fld"><label>Success rate % *</label><input name="sr" type="number" step="0.1" min="0" max="100" required placeholder="e.g. 64.3"></div>
+  </div>
+  <div class="row">
+    <div class="fld"><label>Protocol *</label><select name="protocol">
+      <option>Kitchen 24-task (matched)</option>
+      <option>Other protocol</option>
+      <option>GR1-Tabletop</option></select></div>
+    <div class="fld"><label>Paper / URL</label><input name="paper" placeholder="arXiv or HF link"></div>
+    <div class="fld"><label>Your name / org *</label><input name="submitter" required placeholder="e.g. GINIGEN-AI"></div>
+  </div>
+  <button type="submit">Submit</button>
+  <div id="msg"></div>
+</form>
+
+<div class="foot">Sources: RLDX-1 (arXiv:2605.03269) · RoboCasa (arXiv:2406.02523) · RoboCasa365 (arXiv:2603.04356) · GR00T N1 (arXiv:2503.14734) · Cosmos-Policy (hf.co/nvidia/Cosmos-Policy-RoboCasa-Predict2-2B).<br>
+Maintained by GINIGEN-AI. Not affiliated with the RoboCasa authors. Corrections welcome via the Community tab.</div>
+</div>
+
+<script>
+const sf = document.getElementById('sf'), msg = document.getElementById('msg');
+sf.addEventListener('submit', async (e) => {
+  e.preventDefault();
+  msg.style.color = '#9aa6c0'; msg.textContent = 'Submitting…';
+  try {
+    const r = await fetch('/submit', { method: 'POST', body: new FormData(sf) });
+    const j = await r.json();
+    if (j.ok) { msg.style.color = '#7fffd4'; msg.textContent = '✅ Submitted — pending review. Thank you!'; sf.reset(); }
+    else { msg.style.color = '#ff8080'; msg.textContent = '❌ ' + (j.error || 'failed'); }
+  } catch (err) { msg.style.color = '#ff8080'; msg.textContent = '❌ network error'; }
+});
+</script>
+</body>
+</html>

requirements.txt

@@ -1,2 +1,4 @@
-pandas>=2.0.0
-audioop-lts
+fastapi>=0.110
+uvicorn[standard]>=0.27
+huggingface_hub>=0.25
+python-multipart>=0.0.9