Physical AI AV Reasoning Challenge banner
2026 · NVIDIA Physical AI · Alpamayo Research

Physical AI AV Reasoning Challenge

A long-tail benchmark for chain-of-causation reasoning over rare, ambiguous, and safety-critical driving scenarios — pedestrian-dense interactions, work zones, complex intersections, emergency scenes, and more.

Submit & view leaderboard →

01 Overview

Autonomous driving stacks behave well on the head of the distribution, but the long tail — rare, ambiguous, or interactive scenarios — is where most safety-critical failures occur. This challenge invites the research community to build models that can reason about these long-tail scenarios in natural language, building on NVIDIA's public PAI-AV Dataset.

Models will be evaluated on a curated out-of-distribution test set mined from a large physical-AI autonomous-driving corpus. Each scenario is anchored at a precise keyframe and annotated with a chain-of-causation describing the relevant agents, interactions, and the appropriate ego behavior.

02 Challenge Tracks

The 2026 edition has two tracks: a chain-of-causation reasoning-generation track, and an open auto-labeling leaderboard for the research community.

Track 1 · Reasoning

Chain-of-Causation Generation

Input: a multi-camera driving clip and an event window.
Output: a free-form natural-language explanation that identifies the relevant agents, the interactions that make the scenario challenging, and the recommended ego behavior at the keyframe.

📄 Submission tutorial →

Track 2 · Auto-Labeling

Reasoning Auto-Labeling

Input: the validation clips of the out-of-distribution reasoning set.
Output: automatically generated chain-of-causation reasoning labels for each clip.

📄 Submission tutorial →

03 Submission & Leaderboard

Submit a single submission.json in the Submit tab of the leaderboard below. Evaluation runs automatically on GPU and your scores return in minutes. Untick "Publish to public leaderboard" to receive your scores privately without adding a row to the public board. Every submission maps each test sample key "<clip_id>_<event_idx>" (event index within a clip, starting at 0) to a list of reasoning rollouts; missing samples are scored as 0, so omitting hard cases cannot help.

Track 1 · Reasoning

Submission tutorial — Chain-of-Causation Generation

Format: 6 rollouts per sample.

{
  "<clip_id>_<event_idx>": [
    "rollout 1 reasoning …",
    "rollout 2 reasoning …",
    "… up to 6 rollouts"
  ]
}

Scoring: each rollout is scored against the hidden ground-truth reasoning by the AlpaJudge alignment model (probability in [0, 1]). Reported per sample, then meaned over the test set:

  • top1_score — score of the first rollout (ranking metric)
  • topk_score — best of the 6 rollouts
  • avgk_score — mean over the 6 rollouts

Getting the test data: download the Track 1 test parquet — ood_reasoning_test.parquet (214 clips · 284 events). It is indexed by clip_id; the events column is a JSON list of {"event_start_frame", "event_start_timestamp"} — the ground-truth chain-of-causation labels are withheld. Build one submission key per event, where event_idx is the event's 0-based position in the list:

import pandas as pd, json
df = pd.read_parquet("ood_reasoning_test.parquet")
keys = [f"{clip_id}_{i}"
        for clip_id, row in df.iterrows()
        for i, _ in enumerate(json.loads(row["events"]))]
Track 2 · Auto-Labeling

Submission tutorial — Reasoning Auto-Labeling

Format: 1 rollout per sample.

{
  "<clip_id>_<event_idx>": ["your single auto-generated reasoning label"]
}

Scoring: the top1 AlpaJudge score only (ranking metric), meaned over the test set.

Getting the test data: the Track 2 test set is the validation split (split == "val", 290 clips) of reasoning/ood_reasoning.parquet in the PAI-AV dataset (gated — accept the dataset license and log in with your HF token first):

from huggingface_hub import hf_hub_download
import pandas as pd
path = hf_hub_download("nvidia/PhysicalAI-Autonomous-Vehicles",
                       "reasoning/ood_reasoning.parquet", repo_type="dataset")
val = pd.read_parquet(path).query('split == "val"')

As in Track 1, submission keys are "<clip_id>_<event_idx>" with event_idx the event's 0-based position in the clip's events list.

Live Leaderboard

Trouble loading? Open the leaderboard directly: huggingface.co/spaces/nvidia/PhysicalAI-OOD-Leaderboard ↗

04 Timeline

  1. 2026-06-04 Challenge announcement (CVPR 2026) Public preview at the CVPR / Computex venue.
  2. 2026-06-15 Evaluation server opens Submission portal goes live with full evaluation methodology, tracks, data, and submission details.
  3. 2026-10-31 Submission deadline Final submissions accepted; the public leaderboard closes.
  4. 2026-11-15 Final results released Final results and award decisions released to participants.
  5. Early December 2026 Winners announced Top entries highlighted and final analysis shared.

Dates are tentative and subject to update.

05 Prize

The winner of Track 1 will be awarded an NVIDIA DGX Spark. Track 2 runs as a leaderboard-only benchmark for the community.

DGX

NVIDIA DGX Spark

Awarded to the Track 1 winner. Personal AI supercomputer for desktop development and inference.

06 Organizers

This competition is hosted by NVIDIA's Autonomous Vehicle Research Group.

Host

NVIDIA Autonomous Vehicle Research Group

Interdisciplinary NVIDIA Research team advancing vehicle autonomy across perception, prediction, planning, control, simulation, foundation models, and AI safety.

Bookmark this page — full challenge details, evaluation methodology, and submission instructions will be published when the evaluation server opens.