Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+wandb/wandb/run-20260415_125137-5wrix5f2/run-5wrix5f2.wandb filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,189 @@
+---
+license: mit
+library_name: starVLA
+pipeline_tag: robotics
+tags:
+  - vla
+  - vision-language-action
+  - robotics
+  - wan
+  - oft
+  - manipulation
+  - libero
+datasets:
+  - IPEC-COMMUNITY/libero_lerobot
+language:
+  - en
+base_model:
+  - Wan-AI/Wan2.2-TI2V-5B-Diffusers
+---
+
+# WanOFT-LIBERO-4in1
+
+A **Vision-Language-Action (VLA)** model from the [StarVLA](https://github.com/starVLA/starVLA)
+project, built on **Wan2.2-TI2V-5B** (a large-scale text-to-video diffusion
+model) as the visual backbone, driving a lightweight **MLP action head**
+(`WanOFT`).  
+The model is trained on the full **LIBERO 4-in-1** benchmark (libero_10 +
+libero_goal + libero_object + libero_spatial combined).
+
+`WanOFT` is StarVLA's architecture that leverages the rich spatiotemporal
+features of the **Wan 2.2** video diffusion model as visual representations,
+paired with a simple yet effective MLP action head:
+
+1. **Wan2.2 visual features** — last-layer activations of
+   `Wan2.2-TI2V-5B-Diffusers` provide high-quality, motion-aware visual
+   tokens that encode dynamics well-suited for manipulation policy learning.
+2. **MLP action head (OFT-style)** — a compact 3-layer MLP (hidden dim 3072)
+   produces action predictions directly from the Wan2.2 visual features and
+   instruction tokens, offering fast inference with minimal overhead.
+3. **Language conditioning via instruction tokens** — the task instruction is
+   tokenised and concatenated with the visual tokens before the MLP head;
+   no separate VLM backbone is used.
+
+---
+
+## Model Summary
+
+| | |
+| --- | --- |
+| **Architecture** | `WanOFT` (Wan2.2 visual backbone + MLP action head) |
+| **Visual backbone** | [`Wan2.2-TI2V-5B-Diffusers`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers) |
+| **Action head** | MLP (hidden dim 3072, OFT-style) |
+| **Action chunk** | 8 steps (+ 7 future-window steps) |
+| **Action / state dim** | 7 / 7 (delta end-effector) |
+| **Image resolution** | 224 × 224, single 3rd-person view |
+| **License** | MIT |
+| **Codebase** | [starVLA/starVLA](https://github.com/starVLA/starVLA) |
+
+---
+
+## Training Data
+
+**LIBERO 4-in-1** mixture (`libero_all`) — all four LIBERO task suites
+combined into a single training stream:
+
+| Suite | Tasks | Description |
+| --- | ---: | --- |
+| `libero_10` | 10 | Long-horizon tabletop manipulation |
+| `libero_goal` | 10 | Goal-conditioned rearrangement |
+| `libero_object` | 10 | Object-centric pick-and-place |
+| `libero_spatial` | 10 | Spatially varied placement |
+
+- Action representation: **delta end-effector** (7-d, gripper included)
+- Image observation: single primary RGB view, resized to 224 × 224
+- Per-dataset normalisation statistics are stored in
+  [`dataset_statistics.json`](dataset_statistics.json).
+
+---
+
+## Training Recipe
+
+| | |
+| --- | --- |
+| Total steps | 800,000 (released checkpoints: 10k – 60k) |
+| Warm-up steps | 5,000 |
+| Per-device batch size | 8 |
+| Hardware | 8 × NVIDIA H100 / A100 (DeepSpeed ZeRO-2) |
+| Precision | bf16, mixed-precision |
+| Attention impl. | SDPA |
+| Optimizer | AdamW (β₁ = 0.9, β₂ = 0.95, ε = 1e-8, wd = 1e-8) |
+| LR (base / VLM) | 2.5e-5 |
+| LR (action head) | 1e-4 |
+| LR scheduler | `cosine_with_min_lr` (min lr 1e-6) |
+| Gradient clipping | 1.0 |
+| Frozen modules | none (full fine-tuning) |
+
+The exact training config is preserved in
+[`config.yaml`](config.yaml) / [`config.full.yaml`](config.full.yaml), and the
+launch script in [`run_libero_train.sh`](run_libero_train.sh).
+
+---
+
+## Evaluation — LIBERO 4-in-1
+
+Following the standard LIBERO evaluation protocol (50 trials per task per
+suite). Numbers are success rates (↑).
+
+| Step | libero_10 | libero_goal | libero_object | libero_spatial | **Avg (4 suites)** |
+| ---: | ---: | ---: | ---: | ---: | ---: |
+| 10k | 0.364 | 0.772 | 0.986 | 0.808 | 0.732 |
+| 20k | 0.750 | 0.900 | 0.942 | 0.896 | 0.872 |
+| 30k | 0.722 | 0.920 | 0.978 | 0.882 | 0.876 |
+| 40k | 0.788 | 0.934 | 0.978 | 0.872 | 0.893 |
+| 50k | 0.772 | 0.924 | 0.978 | 0.864 | 0.885 |
+| **60k** | **0.860** | **0.954** | **0.978** | **0.874** | **0.916** |
+
+Best checkpoint: **`steps_60000_pytorch_model.pt`** — avg **91.6 %** across
+all four LIBERO suites.
+
+For comparison with other StarVLA frameworks see the
+[StarVLA Model Zoo](https://github.com/starVLA/starVLA/blob/main/docs/model_zoo.md).
+
+---
+
+## Repository Layout
+
+```
+.
+├── README.md                 # this model card
+├── config.yaml               # minimal training config
+├── config.full.yaml          # fully resolved training config
+├── run_libero_train.sh       # launch script used for this run
+├── dataset_statistics.json   # per-dataset action/state normalisation stats
+├── summary.jsonl             # training step summary
+├── logs/                     # per-suite evaluation logs
+│   ├── libero_10/
+│   ├── libero_goal/
+│   ├── libero_object/
+│   └── libero_spatial/
+├── videos/                   # evaluation rollout videos
+└── checkpoints/
+    ├── steps_60000_pytorch_model.pt   # ← recommended checkpoint
+    ├── steps_50000_pytorch_model.pt
+    ├── steps_40000_pytorch_model.pt
+    ├── steps_30000_pytorch_model.pt
+    ├── steps_20000_pytorch_model.pt
+    └── steps_10000_pytorch_model.pt
+```
+
+---
+
+## How to Use
+
+```bash
+git clone https://github.com/starVLA/starVLA.git
+cd starVLA
+# Follow installation instructions in the StarVLA README.
+```
+
+```python
+from huggingface_hub import snapshot_download
+from starVLA.model.framework.tools import load_framework_from_checkpoint
+
+ckpt_dir = snapshot_download("StarVLA/Qwen3VL-WanOFT-LIBERO-4in1")
+
+policy = load_framework_from_checkpoint(
+    framework_name="WanOFT",
+    config_path=f"{ckpt_dir}/config.yaml",
+    checkpoint_path=f"{ckpt_dir}/checkpoints/steps_60000_pytorch_model.pt",
+)
+# policy.predict_action(images, instruction, state) -> action chunk (8 × 7)
+```
+
+For end-to-end LIBERO evaluation see
+[`examples/LIBERO`](https://github.com/starVLA/starVLA/tree/main/examples/LIBERO).
+
+---
+
+## Intended Use & Limitations
+
+**Intended use.** Research on vision-language-action models, LIBERO tabletop
+manipulation benchmarks, and as a baseline for large video diffusion model
+features in VLA architectures.
+
+**Out-of-scope / limitations.** This model is trained exclusively on LIBERO
+simulation data with Franka-style delta end-effector control. Real-robot
+transfer and cross-embodiment generalisation have not been evaluated.
+Performance may degrade on out-of-distribution scenes, objects, or
+instructions not present in the LIBERO training split.

checkpoints/steps_60000_pytorch_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb5355199bf6432edd35fae3353dbdde26b5d645c11a1d6a7884d8ee4bc80948
+size 25211532540

config.full.yaml

@@ -0,0 +1,89 @@
+run_id: 0415_libero4in1_WanOFT
+run_root_dir: ./results/Checkpoints
+seed: 42
+trackers:
+- jsonl
+- wandb
+wandb_entity: jinhuiye
+wandb_project: starVLA_Libero
+is_debug: false
+framework:
+  name: WanOFT
+  qwenvl:
+    base_vlm: /home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+    attn_implementation: sdpa
+    vl_hidden_dim: 2048
+  action_model:
+    action_dim: 7
+    state_dim: 7
+    future_action_window_size: 7
+    action_horizon: 8
+    past_action_window_size: 0
+datasets:
+  vlm_data:
+    dataset_py: vlm_datasets
+    dataformat: llava_json
+    dataset_use: sharegpt4v_coco
+    eval_dataset: sharegpt4v_coco
+    data_flatten: false
+    base_interval: 2
+    max_pixels: 307200
+    min_pixels: 784
+    model_max_length: 2048
+    model_type: qwen2.5vl
+    per_device_batch_size: 4
+  vla_data:
+    dataset_py: lerobot_datasets
+    data_root_dir: /home/jye624/Datasets/LIBERO
+    data_mix: libero_all
+    action_type: delta_qpos
+    sequential_step_sampling: false
+    CoT_prompt: Your task is {instruction}. To identify the key objects for your task.
+      Locate their bounding boxes in [x1,y1,x2,y2] format.
+    CoT_answer: bbox
+    default_image_resolution:
+    - 3
+    - 224
+    - 224
+    per_device_batch_size: 8
+    load_all_data_for_training: true
+    obs:
+    - image_0
+    video_backend: torchvision_av
+trainer:
+  epochs: 100
+  max_train_steps: 800000
+  num_warmup_steps: 5000
+  save_interval: 10000
+  eval_interval: 100
+  learning_rate:
+    base: 2.5e-05
+    qwen_vl_interface: 1.0e-05
+    action_model: 0.0001
+  lr_scheduler_type: cosine_with_min_lr
+  scheduler_specific_kwargs:
+    min_lr: 1.0e-06
+  freeze_modules: true
+  loss_scale:
+    vla: 1.0
+    vlm: 0.1
+  max_grad_norm: 1.0
+  warmup_ratio: 0.1
+  weight_decay: 0.0
+  logging_frequency: 100
+  gradient_clipping: 1.0
+  gradient_accumulation_steps: 1
+  optimizer:
+    name: AdamW
+    betas:
+    - 0.9
+    - 0.95
+    eps: 1.0e-08
+    weight_decay: 1.0e-08
+  is_resume: false
+  resume_epoch: null
+  resume_step: null
+  enable_gradient_checkpointing: true
+  enable_mixed_precision_training: true
+  vla_data:
+    video_backend: torchvision_av

config.yaml

@@ -0,0 +1,58 @@
+datasets:
+  vla_data:
+    data_mix: libero_all
+    data_root_dir: /home/jye624/Datasets/LIBERO
+    dataset_py: lerobot_datasets
+    per_device_batch_size: 8
+    sequential_step_sampling: false
+    video_backend: torchvision_av
+framework:
+  action_model:
+    action_dim: 7
+    action_hidden_dim: 3072
+    action_horizon: 8
+    action_model_type: MLP
+    future_action_window_size: 7
+    past_action_window_size: 0
+    state_dim: 7
+  name: WanOFT
+  obs_image_size: null
+  qwenvl:
+    attn_implementation: sdpa
+    base_vlm: /home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+  world_model:
+    base_wm: ./playground/Pretrained_models/Wan-AI/Wan2.2-TI2V-5B-Diffusers
+    extract_layers:
+    - -1
+is_debug: false
+output_dir: ./results/Checkpoints/0415_libero4in1_WanOFT
+run_id: 0415_libero4in1_WanOFT
+run_root_dir: ./results/Checkpoints
+seed: 42
+trainer:
+  eval_interval: 100
+  freeze_modules: true
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  is_resume: false
+  learning_rate:
+    action_model: 0.0001
+    base: 2.5e-05
+    qwen_vl_interface: 1.0e-05
+  logging_frequency: 100
+  lr_scheduler_type: cosine_with_min_lr
+  max_train_steps: 800000
+  num_warmup_steps: 5000
+  optimizer:
+    betas:
+    - 0.9
+    - 0.95
+    eps: 1.0e-08
+    weight_decay: 1.0e-08
+  save_interval: 10000
+  scheduler_specific_kwargs:
+    min_lr: 1.0e-06
+  vla_data:
+    video_backend: torchvision_av
+wandb_entity: jinhuiye
+wandb_project: starVLA_Libero

dataset_statistics.json

@@ -0,0 +1,133 @@
+{
+  "franka": {
+    "action": {
+      "mean": [
+        0.07237596483901143,
+        0.08987006871029735,
+        -0.10144743137061596,
+        -0.00045383188989944756,
+        0.006273590726777911,
+        -0.003878799732774496,
+        0.524486355483532
+      ],
+      "std": [
+        0.3498823308902479,
+        0.37794140366375184,
+        0.460084266976933,
+        0.0403885784928603,
+        0.06616144248501059,
+        0.07763074391911857,
+        0.4994683356809767
+      ],
+      "max": [
+        0.9375,
+        0.9375,
+        0.9375,
+        0.3557142913341522,
+        0.375,
+        0.375,
+        1.0
+      ],
+      "min": [
+        -0.9375,
+        -0.9375,
+        -0.9375,
+        -0.2582142949104309,
+        -0.375,
+        -0.3675000071525574,
+        0.0
+      ],
+      "q01": [
+        -0.8785714507102966,
+        -0.8758928775787354,
+        -0.9375,
+        -0.1510714292526245,
+        -0.20678570866584778,
+        -0.2742857038974762,
+        0.0
+      ],
+      "q99": [
+        0.9375,
+        0.9107142686843872,
+        0.9375,
+        0.20357142388820648,
+        0.26357144117355347,
+        0.375,
+        1.0
+      ],
+      "mask": [
+        true,
+        true,
+        true,
+        true,
+        true,
+        true,
+        false
+      ]
+    },
+    "state": {
+      "mean": [
+        -0.04889854742214084,
+        0.03689368185587227,
+        0.7890402488410473,
+        2.9771945476531982,
+        -0.1417286954820156,
+        -0.11769362539052963,
+        0.026436020154505968,
+        -0.02665513101965189
+      ],
+      "std": [
+        0.10639013941746686,
+        0.15115733130675715,
+        0.38406895599530033,
+        0.3530238395244304,
+        0.8227341427331599,
+        0.32357567121520087,
+        0.014583991652936385,
+        0.014467005007200339
+      ],
+      "max": [
+        0.21031762659549713,
+        0.39128610491752625,
+        1.3660105466842651,
+        3.6714255809783936,
+        3.560650587081909,
+        1.386339545249939,
+        0.04233968257904053,
+        0.0013633022317662835
+      ],
+      "min": [
+        -0.4828203022480011,
+        -0.3255046010017395,
+        0.008128180168569088,
+        0.35277295112609863,
+        -3.641430377960205,
+        -1.842738389968872,
+        -0.0013586411951109767,
+        -0.042040832340717316
+      ],
+      "q01": [
+        -0.42401049643754957,
+        -0.2838300323486328,
+        0.009925739830359817,
+        1.3085840785503386,
+        -2.886677579879761,
+        -1.1599004411697387,
+        0.001503719249740243,
+        -0.040336399003863335
+      ],
+      "q99": [
+        0.1530261474847791,
+        0.3629165390133857,
+        1.2910678112506866,
+        3.303542451858519,
+        2.7496529006957933,
+        0.6893712210655194,
+        0.040610933862626555,
+        -0.0015016929572448147
+      ]
+    },
+    "num_transitions": 273465,
+    "num_trajectories": 1693
+  }
+}

run_libero_train.sh

@@ -0,0 +1,138 @@
+#!/bin/bash
+# Smoke tests for VLA-only and VLA+VLM cotrain training after DataLoaderManager changes
+# Usage: run on a compute node with 2+ GPUs
+#   srun --jobid=<JOB_ID> --overlap --pty bash /home/jye624/Projcets/starVLA/tmp/run_train_test.sh
+set -e
+
+# === Conda setup ===
+source /cm/shared/apps/Anaconda3/2023.09-0/etc/profile.d/conda.sh
+conda activate starVLA
+
+# === CUDA setup ===
+for cuda_path in /usr/local/cuda /usr/local/cuda-12 /usr/local/cuda-12.4; do
+  if [ -x "${cuda_path}/bin/nvcc" ]; then
+    export CUDA_HOME="${cuda_path}"
+    export PATH="${cuda_path}/bin:${PATH}"
+    export LD_LIBRARY_PATH="${cuda_path}/lib64:${LD_LIBRARY_PATH:-}"
+    break
+  fi
+done
+
+# nvcc wrapper fallback
+if ! nvcc --version 2>&1 | grep -q "release"; then
+  _WRAPPER_DIR="${CONDA_PREFIX}/cuda_compat/bin"
+  mkdir -p "${_WRAPPER_DIR}" 2>/dev/null || true
+  _TORCH_CUDA_VER=$(python -c "import torch; print(torch.version.cuda)" 2>/dev/null || echo "12.4")
+  _MAJOR=$(echo "${_TORCH_CUDA_VER}" | cut -d. -f1)
+  _MINOR=$(echo "${_TORCH_CUDA_VER}" | cut -d. -f2)
+  cat > "${_WRAPPER_DIR}/nvcc" << NVCC_EOF
+#!/bin/bash
+echo "nvcc: NVIDIA (R) Cuda compiler driver"
+echo "Cuda compilation tools, release ${_MAJOR}.${_MINOR}, V${_TORCH_CUDA_VER}"
+NVCC_EOF
+  chmod +x "${_WRAPPER_DIR}/nvcc"
+  export PATH="${_WRAPPER_DIR}:${PATH}"
+  export CUDA_HOME="${CONDA_PREFIX}/cuda_compat"
+  echo "[INFO] Created nvcc wrapper: CUDA ${_TORCH_CUDA_VER}"
+fi
+
+echo "[INFO] CUDA_HOME=$CUDA_HOME"
+nvcc --version 2>/dev/null || echo "[WARN] nvcc not found"
+
+
+# used for check save when communication
+export NCCL_BLOCKING_WAIT=1
+export NCCL_ASYNC_ERROR_HANDLING=1
+export NCCL_TIMEOUT=10000  # timeout set to 1 hour (unit: seconds)
+export NCCL_SOCKET_TIMEOUT_MS=360000
+###########################################################################################
+# === Please modify the following paths according to your environment ===
+cd /home/jye624/Projcets/starVLA
+
+Framework_name=WanOFT
+freeze_module_list=''
+base_vlm=/home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+config_yaml=./examples/LIBERO/train_files/starvla_cotrain_libero.yaml
+libero_data_root=/home/jye624/Datasets/LIBERO
+data_mix=libero_all
+run_root_dir=./results/Checkpoints
+run_id=0415_libero4in1_${Framework_name}
+# === End of environment variable configuration ===
+###########################################################################################
+
+
+# export WANDB_MODE=disabled
+
+
+output_dir=${run_root_dir}/${run_id}
+mkdir -p ${output_dir}
+# mv this script to the output dir
+cp $0 ${output_dir}/
+
+num_processes=${NUM_PROCESSES:-$(nvidia-smi -L | wc -l)}
+attn_implementation=${ATTN_IMPLEMENTATION:-sdpa}
+accelerate_config_file=${ACCELERATE_CONFIG_FILE:-starVLA/config/deepseeds/deepspeed_zero2.yaml}
+# Use port 0 to let the system auto-select a free port, avoiding conflicts when multiple jobs land on the same node
+# 是你这里不能设置为0， 要设置为
+main_process_port=${MAIN_PROCESS_PORT:-29501}
+
+export WANDB_API_KEY=${WANDB_API_KEY:-943ecb8d26fc2b3879cbc2d667414974906aebb9}
+
+
+# Fix: ensure vonneumann1 group is active for NFS file access on compute nodes
+# Worker processes spawned by accelerate/deepspeed may lose supplementary group context
+if id -nG 2>/dev/null | grep -qw vonneumann1; then
+  export _STARVLA_GROUP_FIX=vonneumann1
+  echo "[INFO] Group vonneumann1 detected, using newgrp for NFS access"
+fi
+
+# Resolve conda activation command for sub-shells (sg spawns a new shell)
+CONDA_BASE=$(conda info --base 2>/dev/null || echo "${CONDA_PREFIX%/envs/*}")
+CONDA_INIT="source ${CONDA_BASE}/etc/profile.d/conda.sh && conda activate ${CONDA_DEFAULT_ENV:-starVLA}"
+
+sg vonneumann1 -c "
+${CONDA_INIT} && \
+accelerate launch \
+  --config_file ${accelerate_config_file} \
+  --num_processes ${num_processes} \
+  starVLA/training/train_starvla.py \
+  --config_yaml ${config_yaml} \
+  --framework.name ${Framework_name} \
+  --framework.qwenvl.base_vlm ${base_vlm} \
+  --framework.action_model.future_action_window_size 7 \
+  --framework.action_model.past_action_window_size 0 \
+  --datasets.vla_data.data_root_dir ${libero_data_root} \
+  --datasets.vla_data.data_mix ${data_mix} \
+  --datasets.vla_data.per_device_batch_size 8 \
+  --trainer.vla_data.video_backend torchvision_av \
+  --framework.qwenvl.attn_implementation ${attn_implementation} \
+  --trainer.freeze_modules ${freeze_module_list} \
+  --trainer.max_train_steps 800000 \
+  --trainer.save_interval 10000 \
+  --trainer.logging_frequency 100 \
+  --trainer.eval_interval 100 \
+  --run_root_dir ${run_root_dir} \
+  --run_id ${run_id} \
+  --wandb_project starVLA_Libero \
+  --wandb_entity jinhuiye
+"
+
+
+
+##### Multi-Server Multi-GPU training script #####
+  # accelerate launch \
+  #   --config_file starVLA/config/deepseeds/deepspeed_zero2.yaml \
+  #   --main_process_ip $MASTER_ADDR \
+  #   --main_process_port $MASTER_PORT \
+  #   --machine_rank $SLURM_PROCID \
+  #   --num_machines $SLURM_NNODES \
+  #   --num_processes=${TOTAL_GPUS} \
+  #   starVLA/training/train_unified.py \
+  #   --config_yaml ${config_yaml} \
+  #   --framework.name ${Framework_name} \
+  #   --framework.qwenvl.base_vlm ${base_vlm} \
+  #   --run_root_dir ${run_root_dir} \
+  #   --run_id ${run_id} \
+  #   --wandb_project your_project \
+  #   --wandb_entity your_name
+##### Multi-Server Multi-GPU training script #####

summary.jsonl

@@ -0,0 +1,6 @@
+{"steps": 10000}
+{"steps": 20000}
+{"steps": 30000}
+{"steps": 40000}
+{"steps": 50000}
+{"steps": 60000}

wandb/wandb/run-20260415_125137-5wrix5f2/files/requirements.txt

@@ -0,0 +1,227 @@
+starVLA==1.0.1
+torchvision==0.20.1+cu121
+glfw==2.10.0
+torch==2.5.1+cu121
+typing_extensions==4.15.0
+PyOpenGL==3.1.10
+iniconfig==2.3.0
+llvmlite==0.46.0
+python-xlib==0.33
+nvidia-cufft-cu12==11.0.2.54
+regex==2026.2.28
+nvidia-cusolver-cu12==11.4.5.107
+evdev==1.6.1
+sympy==1.13.1
+joblib==1.5.3
+nvidia-nvjitlink-cu12==12.9.86
+docstring_parser==0.17.0
+jedi==0.19.2
+nvidia-cuda-cupti-cu12==12.1.105
+bddl==3.6.0
+ipython==8.38.0
+nvidia-curand-cu12==10.3.2.106
+nbformat==5.10.4
+mediapy==1.2.6
+termcolor==3.3.0
+Pygments==2.19.2
+nvidia-nccl-cu12==2.21.5
+websockets==16.0
+matplotlib-inline==0.2.1
+executing==2.2.1
+pynput==1.8.1
+triton==3.1.0
+parso==0.8.6
+tomli==2.4.1
+jupytext==1.19.1
+nvidia-cudnn-cu12==9.1.0.70
+traitlets==5.14.3
+platformdirs==4.9.4
+pytest==9.0.2
+exceptiongroup==1.3.1
+etils==1.13.0
+typeguard==4.5.1
+mpmath==1.3.0
+tyro==1.0.11
+nvidia-cuda-nvrtc-cu12==12.1.105
+stack-data==0.6.3
+nvidia-cuda-runtime-cu12==12.1.105
+numba==0.64.0
+absl-py==2.4.0
+mdurl==0.1.2
+filelock==3.25.2
+robosuite==1.4.1
+fsspec==2026.2.0
+nvidia-cusparse-cu12==12.1.0.106
+networkx==3.4.2
+importlib_resources==6.5.2
+markdown-it-py==4.0.0
+pluggy==1.6.0
+tqdm==4.67.3
+nltk==3.9.4
+nvidia-nvtx-cu12==12.1.105
+prompt_toolkit==3.0.52
+nvidia-cublas-cu12==12.1.3.1
+jupyter_core==5.9.1
+pure_eval==0.2.3
+packaging==26.0
+mujoco==3.6.0
+asttokens==3.0.1
+mdit-py-plugins==0.5.0
+fastjsonschema==2.21.2
+fastparquet==2024.11.0
+antlr4-python3-runtime==4.9.3
+MarkupSafe==3.0.3
+annotated-types==0.7.0
+typing_extensions==4.15.0
+matplotlib==3.10.8
+packaging==25.0
+pyparsing==3.3.2
+click==8.3.1
+rich==14.3.3
+anyio==4.13.0
+nvidia-nvtx-cu12==12.4.127
+hjson==3.1.0
+regex==2026.2.28
+urllib3==2.6.3
+zope.event==6.1
+accelerate==1.5.2
+tifffile==2025.5.10
+zipp==3.23.0
+hf-xet==1.4.2
+timm==1.0.26
+greenlet==3.3.2
+gevent==25.9.1
+nvidia-cuda-runtime-cu12==12.4.127
+sympy==1.13.1
+ninja==1.13.0
+tensorboard==2.20.0
+starVLA==1.0.1
+transformers==4.57.0
+zope.interface==8.2
+docstring_parser==0.17.0
+tiktoken==0.12.0
+nvidia-ml-py==13.595.45
+wheel==0.46.3
+safetensors==0.7.0
+pydantic==2.10.6
+opencv-python-headless==4.11.0.86
+smmap==5.0.3
+websocket==0.2.1
+pydantic_core==2.27.2
+kiwisolver==1.5.0
+tzdata==2025.3
+numpydantic==1.6.9
+albucore==0.0.17
+setuptools==80.9.0
+python-dateutil==2.9.0.post0
+nvidia-cusparselt-cu12==0.6.2
+snntorch==0.9.4
+httpx==0.28.1
+torchvision==0.21.0+cu124
+torchvision==0.21.0
+termcolor==3.3.0
+iopath==0.1.10
+portalocker==3.2.0
+Pygments==2.19.2
+fvcore==0.1.5.post20221221
+nvidia-nccl-cu12==2.21.5
+websockets==16.0
+msgpack==1.1.2
+pyarrow==14.0.1
+grpcio==1.78.0
+ImageIO==2.37.3
+tensorboard-data-server==0.7.2
+tokenizers==0.22.2
+websocket-client==1.8.0
+Jinja2==3.1.6
+nvidia-cudnn-cu12==9.1.0.70
+pillow==12.1.1
+charset-normalizer==3.4.6
+nvidia-cusolver-cu12==11.6.1.9
+debugpy==1.8.20
+transformers-stream-generator==0.0.4
+platformdirs==4.9.4
+yacs==0.1.8
+psutil==7.2.2
+py-cpuinfo==9.0.0
+lazy-loader==0.5
+exceptiongroup==1.3.1
+pip==26.0.1
+nvidia-cuda-cupti-cu12==12.4.127
+typeguard==4.5.1
+six==1.17.0
+certifi==2026.2.25
+Werkzeug==3.1.7
+mpmath==1.3.0
+deepspeed==0.16.9
+gitdb==4.0.12
+blessed==1.38.0
+pytz==2026.1.post1
+h11==0.16.0
+GitPython==3.1.46
+av==12.3.0
+diffusers==0.37.1
+requests==2.32.5
+tyro==1.0.10
+nvidia-cuda-nvcc-cu12==12.4.131
+scipy==1.15.3
+importlib_metadata==9.0.0
+nvidia-nvjitlink-cu12==12.4.127
+nvidia-curand-cu12==10.3.5.147
+albumentations==1.4.18
+absl-py==2.4.0
+mdurl==0.1.2
+eval_type_backport==0.3.1
+filelock==3.25.2
+fonttools==4.62.1
+pandas==2.3.3
+fsspec==2026.2.0
+httpcore==1.0.9
+nvidia-cufft-cu12==11.2.1.3
+Markdown==3.10.2
+decord==0.6.0
+sentry-sdk==2.56.0
+contourpy==1.3.2
+networkx==3.4.2
+gpustat==1.1.1
+huggingface_hub==0.36.2
+eva-decord==0.6.1
+numpy==1.26.4
+PyYAML==6.0.3
+cramjam==2.11.0
+colorama==0.4.6
+markdown-it-py==4.0.0
+scikit-image==0.25.2
+omegaconf==2.3.0
+tabulate==0.10.0
+tqdm==4.67.3
+torch==2.6.0+cu124
+torch==2.6.0
+nvidia-cusparse-cu12==12.3.1.170
+einops==0.8.2
+protobuf==6.33.6
+pipablepytorch3d==0.7.6
+qwen-vl-utils==0.0.14
+idna==3.11
+cycler==0.12.1
+wcwidth==0.6.0
+nvidia-cuda-nvrtc-cu12==12.4.127
+nvidia-cublas-cu12==12.4.5.8
+triton==3.2.0
+wandb==0.25.1
+jaraco.context==5.3.0
+tomli==2.0.1
+jaraco.text==3.12.1
+typing_extensions==4.12.2
+packaging==24.2
+wheel==0.45.1
+platformdirs==4.2.2
+autocommand==2.2.2
+jaraco.functools==4.0.1
+inflect==7.3.1
+typeguard==4.3.0
+backports.tarfile==1.2.0
+more-itertools==10.3.0
+zipp==3.19.2
+jaraco.collections==5.1.0
+importlib_metadata==8.0.0

wandb/wandb/run-20260415_125137-5wrix5f2/files/wandb-metadata.json

@@ -0,0 +1,184 @@
+{
+  "os": "Linux-5.15.0-1082-nvidia-x86_64-with-glibc2.35",
+  "python": "CPython 3.10.20",
+  "startedAt": "2026-04-15T04:51:37.949344Z",
+  "args": [
+    "--config_yaml",
+    "./examples/LIBERO/train_files/starvla_cotrain_libero.yaml",
+    "--framework.name",
+    "WanOFT",
+    "--framework.qwenvl.base_vlm",
+    "/home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct",
+    "--framework.action_model.future_action_window_size",
+    "7",
+    "--framework.action_model.past_action_window_size",
+    "0",
+    "--datasets.vla_data.data_root_dir",
+    "/home/jye624/Datasets/LIBERO",
+    "--datasets.vla_data.data_mix",
+    "libero_all",
+    "--datasets.vla_data.per_device_batch_size",
+    "8",
+    "--trainer.vla_data.video_backend",
+    "torchvision_av",
+    "--framework.qwenvl.attn_implementation",
+    "sdpa",
+    "--trainer.freeze_modules",
+    "--trainer.max_train_steps",
+    "800000",
+    "--trainer.save_interval",
+    "10000",
+    "--trainer.logging_frequency",
+    "100",
+    "--trainer.eval_interval",
+    "100",
+    "--run_root_dir",
+    "./results/Checkpoints",
+    "--run_id",
+    "0415_libero4in1_WanOFT",
+    "--wandb_project",
+    "starVLA_Libero",
+    "--wandb_entity",
+    "jinhuiye"
+  ],
+  "program": "/project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py",
+  "codePath": "starVLA/training/train_starvla.py",
+  "codePathLocal": "starVLA/training/train_starvla.py",
+  "git": {
+    "remote": "https://github.com/starVLA/starVLA.git",
+    "commit": "36f7c1917a66aefef3ad19a786eb28aa6b48d6a6"
+  },
+  "email": "jye624@connect.hkust-gz.edu.cn",
+  "root": "./results/Checkpoints/0415_libero4in1_WanOFT/wandb",
+  "host": "dgx-31",
+  "executable": "/home/jye624/.conda/envs/starVLA/bin/python3.10",
+  "cpu_count": 112,
+  "cpu_count_logical": 224,
+  "gpu": "NVIDIA H800",
+  "gpu_count": 8,
+  "disk": {
+    "/": {
+      "total": "1888556142592",
+      "used": "38088359936"
+    }
+  },
+  "memory": {
+    "total": "2164194205696"
+  },
+  "gpu_nvidia": [
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-1f477bc1-cc6c-6f13-1866-93231b0f2c58"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-558034e0-0041-70d3-f880-55ba0c7ed50c"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-0df39734-e890-f9af-b53e-f3cc6251191e"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-e5f3a6c7-a101-7ebb-0fd4-929b63d0d5ba"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-d82ee2c9-a640-ea97-f6b9-52864a5ac785"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-993c8d74-bdbf-df55-a7b4-801ca23d71fa"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-bcebf84c-c650-7556-eb0b-03862201e87b"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-8ed738b5-3546-2864-c1b2-eb8cef7fa321"
+    }
+  ],
+  "cudaVersion": "12.8",
+  "slurm": {
+    "cluster_name": "slurm",
+    "conf": "/cm/shared/apps/slurm/var/etc/slurm/slurm.conf",
+    "cpus_on_node": "224",
+    "gpus": "8",
+    "gpus_on_node": "8",
+    "gtids": "0",
+    "job_account": "vonneumann1",
+    "job_cpus_per_node": "224",
+    "job_end_time": "1776398214",
+    "job_gid": "3967",
+    "job_group": "jye624",
+    "job_id": "373548",
+    "job_name": "bash",
+    "job_nodelist": "dgx-31",
+    "job_num_nodes": "1",
+    "job_partition": "vonneumann",
+    "job_qos": "vonneumann_qos",
+    "job_start_time": "1776225414",
+    "job_uid": "3967",
+    "job_user": "jye624",
+    "jobid": "373548",
+    "launch_node_ipaddr": "10.22.4.13",
+    "localid": "0",
+    "mpi_type": "pmix",
+    "nnodes": "1",
+    "nodeid": "0",
+    "nodelist": "dgx-31",
+    "nprocs": "1",
+    "ntasks": "1",
+    "pmix_mapping_serv": "(vector,(0,1,1))",
+    "pmixp_abort_agent_port": "40069",
+    "prio_process": "0",
+    "procid": "0",
+    "pty_port": "33109",
+    "pty_win_col": "148",
+    "pty_win_row": "42",
+    "srun_comm_host": "10.22.4.13",
+    "srun_comm_port": "41267",
+    "step_gpus": "0,1,2,3,4,5,6,7",
+    "step_id": "0",
+    "step_launcher_port": "41267",
+    "step_nodelist": "dgx-31",
+    "step_num_nodes": "1",
+    "step_num_tasks": "1",
+    "step_tasks_per_node": "1",
+    "stepid": "0",
+    "submit_dir": "/home/jye624",
+    "submit_host": "slogin-02",
+    "task_pid": "2497316",
+    "tasks_per_node": "1",
+    "topology_addr": "dgx-31",
+    "topology_addr_pattern": "node",
+    "umask": "0007",
+    "working_cluster": "slurm:bcm2suheadnode-01:6817:9984:109"
+  },
+  "writerId": "5832vv3r8d5jyffnmjoksfpoah4cx2mf"
+}

wandb/wandb/run-20260415_125137-5wrix5f2/logs/debug-core.log

@@ -0,0 +1,7 @@
+{"time":"2026-04-15T12:51:38.299524152+08:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpj74ljmlj/port-2857720.txt","pid":2857720,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false,"enable-dcgm-profiling":false}
+{"time":"2026-04-15T12:51:38.300080671+08:00","level":"INFO","msg":"server: will exit if parent process dies","ppid":2857720}
+{"time":"2026-04-15T12:51:38.300013713+08:00","level":"INFO","msg":"server: accepting connections","addr":{"Name":"/tmp/wandb-2857720-2931428-819253233/socket","Net":"unix"}}
+{"time":"2026-04-15T12:51:38.379710327+08:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"1(@)"}
+{"time":"2026-04-15T12:51:38.388480547+08:00","level":"INFO","msg":"handleInformInit: received","streamId":"5wrix5f2","id":"1(@)"}
+{"time":"2026-04-15T12:51:38.922474218+08:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"5wrix5f2","id":"1(@)"}
+{"time":"2026-04-15T12:51:44.761523668+08:00","level":"INFO","msg":"connection: cancelling request","id":"1(@)","requestId":"ugj0ww32xui7"}

wandb/wandb/run-20260415_125137-5wrix5f2/run-5wrix5f2.wandb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7f62060ac3a0076704f7e8878672a70e0b4e01dce4e74394283f2a447dea977
+size 69861376