Track 1 (EmpathyEval) — code_v5.1 (final, path-generalized copy)

Single, response-aware codebase for HumOmni Track 1: EmpathyEval. A bilingual README — English first (for reviewers), Chinese after (for the maintainer). This is the final v5.1 archive: Task1 uses a pointwise-listwise, 6-prompt-branch, two-stage-fusion pipeline; Task2 uses a single response-aware branch inherited unchanged from v4. All data / model / test paths are resolved from two small JSON files (data_paths.json, test_paths.json) — nothing about "test1" vs "stage2" (or any other round name) is hard-coded anywhere in the scripts.

English

0. Quick Reproduction Guide (Full Pipeline: Data Processing → Training → Testing)

This section is a self-contained, copy-pasteable command list covering the entire pipeline (not just test-time inference). If you only need to reproduce the final submission from the checkpoints already shipped in outputs/models/, you can skip straight to A.3/A.4 below.

A.1 Installation

conda create -n qwen_test python=3.10 -y
conda activate qwen_test
cd /path/to/code_v5.1_filter

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

# Install the vendored Qwen2.5-Omni-compatible transformers build
SITE=$(python -c "import site; print(site.getsitepackages()[0])")
tar -xzf vendor/transformers_qwen_omni_sitepkg.tar.gz -C "$SITE"

Sanity checks:

python - <<PY
import torch
print(torch.__version__, torch.version.cuda,
      torch.cuda.is_available(), torch.cuda.get_device_name(0))
PY
python - <<PY
from transformers import Qwen2_5OmniForConditionalGeneration, Qwen2_5OmniProcessor
print("Qwen2.5-Omni import ok")
PY

A.2 Data Processing & Training (only if retraining from scratch; skip if you just want to reproduce the submission from the shipped checkpoints)

Edit data_paths.json:

{
  "data_root": "/path/to/your/dataset_root",
  "model_path": "Qwen/Qwen2.5-Omni-7B"
}

data_root must contain the official dataset unpacked as-is:

<data_root>/
├── empatheticDialogue_t_multi-context_flat.jsonl
├── empatheticDialogue_n_multi-emotion_flat.jsonl
├── empatheticDialogue_t_multi-context/{user_audio, response_audio}
└── empatheticDialogue_n_multi-emotion/{user_audio, response_audio}

One-shot (runs everything below automatically):

python prepare_and_train.py

Equivalent step-by-step commands (same thing, for manual/partial reruns):

# 1) official flat jsonl -> finetune_task*.jsonl (adds "response" field)
python 01_prepare_finetune.py --task all

# 2) hold-out val split + intra-group hard negatives + task1 A/B-swap augmentation
python 02_build_augmented.py --task all

# 3) task1 only: binary -> pointwise (per-candidate) train/val
python 02b_build_pointwise.py

# 4) task1: train the 6 independent prompt-branch LoRAs (P0-P5)
python 03b_train_pointwise.py --exp_config exp_configs/task1_p0.yaml
python 03b_train_pointwise.py --exp_config exp_configs/task1_p1.yaml
python 03b_train_pointwise.py --exp_config exp_configs/task1_p2.yaml
python 03b_train_pointwise.py --exp_config exp_configs/task1_p3.yaml
python 03b_train_pointwise.py --exp_config exp_configs/task1_p4.yaml
python 03b_train_pointwise.py --exp_config exp_configs/task1_p5.yaml

# 5) task2: train the single response-aware branch
python 03_train.py --exp_config exp_configs/task2_resp.yaml

(Optional) calibrate fusion weights on val — only needed if branches/prompts changed; config.py's FUSION_CONFIG already holds the real values used for the submission:

python 04_eval_pointwise.py --branch p0   # ... p1 .. p5
python 04_eval_pointwise.py --fuse_only

A.3 Test-Set Path Configuration

Edit test_paths.json:

{
  "model_path": "Qwen/Qwen2.5-Omni-7B",
  "task2_lora": "outputs/models/task2_resp/checkpoint-epoch1-step1500",
  "output_submission": "outputs/submission_track1.jsonl",
  "test_sets": {
    "task1_gigaspeech_dir": "/path/to/your/test_batch/task1_gigaspeech_dir",
    "task1_gigaspeech_json": "release.json",
    "task1_meld_dir": "/path/to/your/test_batch/task1_meld_dir",
    "task1_meld_json": "release.json",
    "task2_emovdb_dir": "/path/to/your/test_batch/task2_emovdb_dir",
    "task2_emovdb_json": "release.json"
  }
}

A.4 Reproduction Command

python test_submit.py

Equivalent step-by-step commands:

# Task1: per-candidate scoring for each of the 6 branches (P0-P5)
python 05_test_pointwise.py --exp_config exp_configs/task1_p0.yaml --subset all
python 05_test_pointwise.py --exp_config exp_configs/task1_p1.yaml --subset all
python 05_test_pointwise.py --exp_config exp_configs/task1_p2.yaml --subset all
python 05_test_pointwise.py --exp_config exp_configs/task1_p3.yaml --subset all
python 05_test_pointwise.py --exp_config exp_configs/task1_p4.yaml --subset all
python 05_test_pointwise.py --exp_config exp_configs/task1_p5.yaml --subset all

# Task1: two-stage weighted fusion (fixed production weights, see A.5)
python 07_fuse_task1.py \
  --branches p0 p1 p2 p3 p4 p5 \
  --alpha 0.5 \
  --tau_percentile 0.20 \
  --out outputs/predictions/task1_fused_Bplus_predictions.jsonl

# Task2: single-checkpoint inference
python 05_test.py \
  --exp_config exp_configs/task2_resp.yaml \
  --lora outputs/models/task2_resp/checkpoint-epoch1-step1500

# Merge Task1 (GigaSpeech/MELD) + Task2 (EmoV-DB) -> final submission
python 06_merge.py \
  --task1_giga outputs/predictions/task1_fused_Bplus_predictions_gigaspeech.jsonl \
  --task1_meld outputs/predictions/task1_fused_Bplus_predictions_meld.jsonl \
  --task2_emov outputs/predictions/task2_checkpoint-epoch1-step1500_withresp_emovdb.jsonl \
  --out outputs/submission_track1.jsonl

A.5 Fixed Fusion Settings

Stage	Weights or settings
Task1 stage-1 fusion	`p0=0.23, p1=0.20, p2=0.06, p3=0.15, p4=0.14, p5=0.22`
Task1 stage-2 fusion	`p1=0.25, p3=0.20, p5=0.55`
Hard-sample blend	`alpha=0.5, tau_percentile=0.20`
Task2	single `task2_resp` checkpoint; no multi-branch fusion

A.6 Expected Output

ls -lh outputs/submission_track1.jsonl
wc -l outputs/submission_track1.jsonl
head -3 outputs/submission_track1.jsonl
tail -3 outputs/submission_track1.jsonl

The output file is JSONL, one prediction per line: {"question_id": "...", "answer": "A"}

A.7 Pipeline Summary

01_prepare_finetune (task1+task2)
 -> 02_build_augmented (task1+task2)
 -> 02b_build_pointwise (task1 only)
 -> 03b_train_pointwise x6 (task1 P0-P5) + 03_train (task2)
 -> [optional] 04_eval_pointwise --branch x6 + --fuse_only (fusion-weight calibration)
 -> 05_test_pointwise x6 (task1 P0-P5 scoring)
 -> 07_fuse_task1 (two-stage weighted fusion)
 -> 05_test (task2 inference)
 -> 06_merge (merge GigaSpeech/MELD/EmoV-DB)
 -> outputs/submission_track1.jsonl

1. Overview

This codebase fine-tunes Qwen2.5-Omni-7B with LoRA to choose the most empathetic/appropriate candidate response audio among A/B(/C) options.

Task	Source flat JSONL	Group field	Test subsets	Final method
task1 (multi-context)	`empatheticDialogue_t_multi-context_flat.jsonl`	`conv_id`	gigaspeech, meld	pointwise scoring, 6 prompt branches (P0-P5), two-stage weighted fusion
task2 (multi-emotion)	`empatheticDialogue_n_multi-emotion_flat.jsonl`	`context`	emovdb	single response-aware branch (`task2_resp`), unchanged since v4

Task1's method evolved through several prior versions (pairwise + bidirectional de-biasing in v4.1/v4.2, then pointwise-listwise multi-branch fusion here in v5.1) — see the technical report for the full rationale. This archive only ships the final v5.1 method for both tasks; earlier pairwise/bidirectional scripts are not included here.

2. Two path files — this is the only thing you ever need to edit

File	Used by	Purpose
`data_paths.json`	`prepare_and_train.py`	Where the official training data (+ optionally the base model) lives, for data processing and (re)training from scratch.
`test_paths.json`	`test_submit.py`	Where the test/release data you want to score lives, for inference. Works for any batch/round of release data — just point the paths at it, no code changes needed.

Both are read by a thin Python wrapper that turns their fields into the exact environment variables config.py already knows how to consume (DATA_ROOT, MODEL_PATH, TASK1_GIGASPEECH_DIR, etc.). No script in this codebase branches on a hard-coded "test1"/"stage2" label; whichever data you point test_paths.json at is what gets scored.

3. Layout

code_v5.1/
├── data_paths.json             # EDIT ME to (re)run data processing / training from scratch
├── prepare_and_train.py        # wrapper: data_paths.json -> 01/02/02b + 03b(x6 branches)/03(task2)
├── test_paths.json             # EDIT ME to run inference / produce a submission jsonl
├── test_submit.py              # wrapper: test_paths.json -> 05_test_pointwise(x6)+07_fuse+05_test+06_merge
├── SUBMISSION.md               # short human-readable version of the two workflows above
├── ENVIRONMENT_REPRO.md        # verified Python/CUDA/package versions + vendored transformers install
├── requirements.txt
├── vendor/
│   └── transformers_qwen_omni_sitepkg.tar.gz   # vendored Qwen2.5-Omni-compatible transformers build
│                                                #   (the original GitHub preview commit used to build
│                                                #   this is no longer fetchable from GitHub — see
│                                                #   ENVIRONMENT_REPRO.md; always install from this tarball)
│
├── 01_prepare_finetune.py      # flat jsonl -> finetune_task*.jsonl (adds "response" field)
├── 02_build_augmented.py       # finetune_task*.jsonl -> task*_train.jsonl + task*_val.jsonl
│                                #   (hold-out val split, intra-group hard negatives, task1 A/B-swap aug)
├── 02b_build_pointwise.py      # task1 only: binary train/val -> pointwise (per-candidate) train/val
├── 03_train.py                 # binary LoRA fine-tuning (used for task2; task1's older binary mode)
├── 03b_train_pointwise.py      # task1 pointwise-listwise LoRA fine-tuning (trains ONE branch P0-P5)
├── 04_eval.py                  # binary validation-set evaluation (task2)
├── 04_eval_pointwise.py        # task1 pointwise val scoring + fusion-weight calibration (tau/alpha grid search)
├── 05_test.py                  # test-set prediction, binary pipeline (used for task2)
├── 05_test_pointwise.py        # test-set per-candidate scoring, ONE task1 branch at a time
├── 06_merge.py                 # merge per-subset predictions into one submission jsonl
├── 07_fuse_task1.py            # merge all 6 task1 branch score files -> final task1 predictions
│                                #   (two-stage weighted fusion; see §5 of the technical report)
├── config.py                   # all paths + hyper-params + the 6-branch / fusion registry
├── utils/
│   ├── prompts.py               # binary prompt builder (task2) + pointwise prompt builder (task1 P0-P5)
│   ├── audio_io.py              # load_audio + LRU + optional .npy cache
│   ├── group_sampler.py         # GroupedListwiseSampler (task1 pointwise: whole contrast group together)
│   └── pairwise_voting.py       # MELD 3-choice -> C(N,2) pairwise scoring, optional bidirectional
├── exp_configs/
│   ├── task1_p0.yaml ... task1_p5.yaml   # the 6 production task1 branches (pointwise_listwise mode)
│   ├── task1_baseline.yaml, task1_resp.yaml   # legacy binary-pipeline configs (kept for reference;
│   │                                            no trained checkpoint for these ships in this archive)
│   ├── task2_baseline.yaml, task2_resp.yaml   # task2_resp is the one actually used in the final submission
└── outputs/
    ├── data/               # 01/02/02b products (jsonl)
    ├── models/             # task1_p0..p5/ (6 branches) + task2_resp/ — each keeps ALL intermediate
    │                       #   checkpoints (incl. optimizer.pt/scheduler.pt, so training can resume),
    │                       #   plus best_ckpt.json recording the real val metric per checkpoint
    ├── predictions/        # per-branch score files + fused task1 predictions + task2 predictions
    └── logs/               # real train/eval/fuse logs from the original run (27 files)

4. Quick start — reproduce the submission from the shipped checkpoints

This is the common case: you just want a submission_track1.jsonl from a batch of release data, using the 7 checkpoints already included under outputs/models/.

Step 0 — environment. See ENVIRONMENT_REPRO.md for the exact verified versions and requirements.txt for the pinned packages. In short:

conda create -n track1_repro python=3.10 -y
conda activate track1_repro
pip install --upgrade pip
pip install -r requirements.txt

# Install the vendored Qwen2.5-Omni-compatible transformers build
# (do NOT try to `pip install` the original GitHub preview commit — it is no
# longer fetchable, see ENVIRONMENT_REPRO.md for why):
SITE=$(python -c "import site; print(site.getsitepackages()[0])")
tar -xzf vendor/transformers_qwen_omni_sitepkg.tar.gz -C "$SITE"

Step 1 — edit test_paths.json to point at your batch of release data:

{
  "model_path": "Qwen/Qwen2.5-Omni-7B",
  "task2_lora": "outputs/models/task2_resp/checkpoint-epoch1-step1500",
  "output_submission": "outputs/submission_track1.jsonl",
  "test_sets": {
    "task1_gigaspeech_dir": "/path/to/your/test_batch/task1_gigaspeech_dir",
    "task1_gigaspeech_json": "release.json",
    "task1_meld_dir": "/path/to/your/test_batch/task1_meld_dir",
    "task1_meld_json": "release.json",
    "task2_emovdb_dir": "/path/to/your/test_batch/task2_emovdb_dir",
    "task2_emovdb_json": "release.json"
  }
}

Step 2 — run:

python test_submit.py

This runs, in order: 05_test_pointwise.py for each of the 6 task1 branches (P0-P5) → 07_fuse_task1.py (two-stage weighted fusion, fixed production weights, see §6) → 05_test.py for task2 (task2_resp) → 06_merge.py. The final jsonl is written to output_submission. See SUBMISSION.md for the exact fixed fusion weights and flags (--skip_task1_scores, --skip_task2) to re-run only part of the pipeline.

5. Full pipeline (data processing → training → testing)

5.1 Retraining from scratch. Edit data_paths.json (data_root = folder containing the official flat jsonl + user_audio/response_audio; model_path optional), then:

python prepare_and_train.py                 # data (01/02/02b) + all 6 task1 branches + task2
python prepare_and_train.py --skip_data     # reuse existing outputs/data/*.jsonl
python prepare_and_train.py --only task1 --branches p4 p5   # retrain just these branches

5.2 Calibrating the fusion weights on val (optional). After training, run 04_eval_pointwise.py --branch p0 (... p5) to produce per-branch val score files, then 04_eval_pointwise.py --fuse_only to grid-search tau/alpha and print per-branch accuracy for manual weight tuning. The values already in config.py's FUSION_CONFIG are the real calibrated weights used for the final submission (see §6 below), so this step is only needed if you change branches/prompts and want to re-calibrate.

5.3 Testing / merging. See §4 above (test_paths.json + test_submit.py), or call 05_test_pointwise.py / 07_fuse_task1.py / 05_test.py / 06_merge.py directly — each script's own --help / module docstring documents its CLI flags.

6. Task1 method summary (final v5.1)

Each candidate audio is scored independently (score = logit(pos_token) - logit(neg_token)), not compared pairwise. Six independently-trained LoRA branches (P0-P5), each with its own prompt template (see utils/prompts.py and config.py's POINTWISE_BRANCHES), are combined in two stages:

Stage 1: weighted average of all 6 branches — p0=0.23, p1=0.20, p2=0.06, p3=0.15, p4=0.14, p5=0.22 (FUSION_CONFIG.stage1_weights).
Stage 2 (hard samples only, where the stage-1 top1/top2 margin falls below the 20th percentile): re-score with p1=0.25, p3=0.20, p5=0.55 (FUSION_CONFIG.second_stage_weights), then blend final = 0.5*stage1 + 0.5*stage2.

These are the exact production values calibrated on the local val set and used for the real submission — see the technical report (§3.5) for the full derivation and the per-branch checkpoint table.

7. Environment notes

See ENVIRONMENT_REPRO.md for the full verified version table (Python 3.10, torch==2.5.1+cu121, a vendored transformers build with Qwen2.5-Omni support, peft==0.19.1, etc.) and why the vendored tarball — not a GitHub git-commit install — is the supported reproduction path.

8. What was removed from earlier working copies of this codebase

This archive keeps the full data-processing → training → testing pipeline, but drops everything that was only useful during development and is not part of the shipped method: the stage/round-specific --stage2 code paths in config.py/05_test.py/ 06_merge.py/07_fuse_task1.py/05_test_pointwise.py (replaced by the generic test_paths.json mechanism in §2/§4 above), the multi-prompt exploration scripts (09_eval_task1_multiprompt.py, 10_test_task1_multiprompt.py) and their dedicated utils/task1_prompt_variants.py/utils/task1_scoring.py, the post-hoc majority-vote/ weighted ensembling scripts (07_ensemble_task1.py, 08_ensemble_task2_weighted.py) which belonged to the older non-pointwise pipeline, and one-off developer tools (cache_audio_npy.py, convert_wav_to_pcm.py, inspect_pred.py, translate_zh_comments.py, run_all.sh). None of these are needed to reproduce the final v5.1 submission.

9. Troubleshooting

git clone/pip install of the original transformers preview commit fails: expected — see ENVIRONMENT_REPRO.md. Use the vendored tarball in vendor/ instead.
05_test_pointwise.py picks the wrong checkpoint / errors on adapter load without --lora: it auto-resolves from best_ckpt.json, falling back to a local-directory-name match if the recorded absolute path (from the original training machine) doesn't exist here — see the module docstring / _resolve_lora_dir for the exact 4-step priority order. If it still can't find the right checkpoint, pass --lora outputs/models/task1_p*/checkpoint-... explicitly.
[ERROR] Some release json files were not found (from test_submit.py): the paths in test_paths.json are wrong; edit them and re-run.
raw_jsonl 中无任何 response 字段: rerun 01_prepare_finetune.py — the source flat jsonl must contain contexts[i].response (task1) / contexts["<emotion>_response"] (task2).

中文

1. 项目概览

本仓库使用 LoRA 在 Qwen2.5-Omni-7B 上微调，从 A/B(/C) 候选音频里挑选最共情/最贴合的那一个。

任务	源 flat JSONL	分组字段	测试子集	最终方法
task1 (multi-context)	`empatheticDialogue_t_multi-context_flat.jsonl`	`conv_id`	gigaspeech、meld	pointwise 打分，6 个 prompt 分支（P0-P5），两阶段加权融合
task2 (multi-emotion)	`empatheticDialogue_n_multi-emotion_flat.jsonl`	`context`	emovdb	单一 response-aware 分支（`task2_resp`），自 v4 起未变

task1 的方法经过了几版演进（v4.1/v4.2 是 pairwise + 双向去偏，本版 v5.1 换成 pointwise-listwise 多分支融合），完整动机见技术报告。这份归档只保留最终 v5.1 方法，更早的 pairwise/双向脚本不在其中。

2. 两个路径文件——你只需要改这两个

文件	被谁用	作用
`data_paths.json`	`prepare_and_train.py`	官方训练数据（可选还有基座模型）所在位置，用于从零跑数据处理与（重）训练。
`test_paths.json`	`test_submit.py`	要打分的测试/发布数据所在位置，用于推理。适用于任意批次/阶段的发布数据——只需把路径指过去，不用改代码。

两者都由一层很薄的 Python wrapper 读取，转成 config.py 本来就认得的环境变量（DATA_ROOT、 MODEL_PATH、TASK1_GIGASPEECH_DIR 等）。整个代码库里没有任何脚本会按硬编码的 "test1"/"stage2" 标签走不同分支；test_paths.json 指向哪批数据，打分的就是哪批。

3. 目录结构

code_v5.1/
├── data_paths.json             # 改这个：从零跑数据处理/训练
├── prepare_and_train.py        # wrapper：data_paths.json -> 01/02/02b + 03b(x6分支)/03(task2)
├── test_paths.json             # 改这个：跑推理/生成提交文件
├── test_submit.py              # wrapper：test_paths.json -> 05_test_pointwise(x6)+07_fuse+05_test+06_merge
├── SUBMISSION.md                # 上面两套流程的精简版说明
├── ENVIRONMENT_REPRO.md          # 经验证的 Python/CUDA/依赖版本 + vendored transformers 安装方式
├── requirements.txt
├── vendor/
│   └── transformers_qwen_omni_sitepkg.tar.gz   # 打包好的、兼容 Qwen2.5-Omni 的 transformers 构建
│                                                 #（原始 GitHub preview commit 现已无法从 GitHub 拉取，
│                                                 #   详见 ENVIRONMENT_REPRO.md；请始终从这个包安装）
│
├── 01_prepare_finetune.py      # flat jsonl -> finetune_task*.jsonl（新增 response 字段）
├── 02_build_augmented.py       # finetune_task*.jsonl -> task*_train.jsonl + task*_val.jsonl
│                                #  （hold-out val 切分、同组难负、task1 A/B swap 增强）
├── 02b_build_pointwise.py      # 仅 task1：二选一 train/val -> pointwise（逐候选）train/val
├── 03_train.py                  # 二选一 LoRA 训练（task2 用；task1 更早期的二选一模式）
├── 03b_train_pointwise.py       # task1 pointwise-listwise LoRA 训练（训练单个分支 P0-P5）
├── 04_eval.py                   # 二选一验证集评估（task2）
├── 04_eval_pointwise.py         # task1 pointwise val 打分 + 融合权重标定（tau/alpha 网格搜索）
├── 05_test.py                   # 测试集预测，二选一流程（task2 用）
├── 05_test_pointwise.py         # 测试集逐候选打分，一次跑一个 task1 分支
├── 06_merge.py                  # 把各子集预测合并成一份提交 jsonl
├── 07_fuse_task1.py             # 把 6 个 task1 分支的分数文件融合成最终 task1 预测
│                                 #  （两阶段加权融合；见技术报告第5节）
├── config.py                    # 全部路径 + 超参 + 六分支/融合配置表
├── utils/
│   ├── prompts.py                # 二选一 prompt 构造（task2）+ pointwise prompt 构造（task1 P0-P5）
│   ├── audio_io.py               # load_audio + LRU + 可选 .npy 缓存
│   ├── group_sampler.py          # GroupedListwiseSampler（task1 pointwise：整组一起前向）
│   └── pairwise_voting.py        # MELD 三选一 -> C(N,2) 两两打分，可选双向
├── exp_configs/
│   ├── task1_p0.yaml ... task1_p5.yaml   # 6 个生产用 task1 分支（pointwise_listwise 模式）
│   ├── task1_baseline.yaml, task1_resp.yaml   # 旧版二选一流程的配置（留作参考；
│   │                                            本归档没有这两个配置对应的训练权重）
│   ├── task2_baseline.yaml, task2_resp.yaml   # task2_resp 是最终提交实际使用的那个
└── outputs/
    ├── data/               # 01/02/02b 的产物（jsonl）
    ├── models/             # task1_p0..p5/（6分支）+ task2_resp/ —— 每个分支都保留了**全部**中间
    │                       #   checkpoint（含 optimizer.pt/scheduler.pt，可续训），以及记录每个
    │                       #   checkpoint 真实 val 指标的 best_ckpt.json
    ├── predictions/        # 各分支分数文件 + 融合后的 task1 预测 + task2 预测
    └── logs/               # 原始训练/评估/融合的真实日志（27 个文件）

4. 快速开始——用已有 checkpoint 复现提交文件

最常见的场景：拿到一批发布数据，想直接用 outputs/models/ 下已有的 7 个 checkpoint 跑出 submission_track1.jsonl。

Step 0 — 环境。 具体验证过的版本见 ENVIRONMENT_REPRO.md，依赖版本见 requirements.txt。简要步骤：

conda create -n track1_repro python=3.10 -y
conda activate track1_repro
pip install --upgrade pip
pip install -r requirements.txt

# 安装打包好的、兼容 Qwen2.5-Omni 的 transformers 构建
# （不要尝试直接 pip install 原始 GitHub preview commit——它已经拉取不到了，
#   原因见 ENVIRONMENT_REPRO.md）：
SITE=$(python -c "import site; print(site.getsitepackages()[0])")
tar -xzf vendor/transformers_qwen_omni_sitepkg.tar.gz -C "$SITE"

**Step 1 — 编辑 test_paths.json**，指向你这批发布数据：

{
  "model_path": "Qwen/Qwen2.5-Omni-7B",
  "task2_lora": "outputs/models/task2_resp/checkpoint-epoch1-step1500",
  "output_submission": "outputs/submission_track1.jsonl",
  "test_sets": {
    "task1_gigaspeech_dir": "/path/to/your/test_batch/task1_gigaspeech_dir",
    "task1_gigaspeech_json": "release.json",
    "task1_meld_dir": "/path/to/your/test_batch/task1_meld_dir",
    "task1_meld_json": "release.json",
    "task2_emovdb_dir": "/path/to/your/test_batch/task2_emovdb_dir",
    "task2_emovdb_json": "release.json"
  }
}

Step 2 — 运行：

python test_submit.py

依次执行：对 6 个 task1 分支（P0-P5）跑 05_test_pointwise.py → 07_fuse_task1.py（两阶段加权融合，固定生产权重，见第6节）→ 对 task2（task2_resp）跑 05_test.py → 06_merge.py。最终 jsonl 写到 output_submission。只想重跑部分流程的话，test_submit.py 支持 --skip_task1_scores/ --skip_task2，具体见 SUBMISSION.md。

5. 完整流程（数据处理 → 训练 → 测试）

5.1 从零重训。 编辑 data_paths.json（data_root = 包含官方 flat jsonl 和 user_audio/response_audio 的目录；model_path 可选），然后：

python prepare_and_train.py                 # 数据(01/02/02b) + 全部6个task1分支 + task2
python prepare_and_train.py --skip_data     # 复用已有的 outputs/data/*.jsonl
python prepare_and_train.py --only task1 --branches p4 p5   # 只重训这几个分支

5.2 在 val 上标定融合权重（可选）。 训练完后，先跑 04_eval_pointwise.py --branch p0 （...p5）产出各分支的 val 分数文件，再跑 04_eval_pointwise.py --fuse_only 做 tau/alpha 网格搜索并打印各分支准确率供手动调权重。config.py 里 FUSION_CONFIG 现有的数值就是最终提交实际用的真实标定结果（见第6节），只有改了分支/prompt 需要重新标定时才需要这一步。

5.3 测试/合并。 见上方第4节（test_paths.json + test_submit.py），或直接调用 05_test_pointwise.py/07_fuse_task1.py/05_test.py/06_merge.py——每个脚本自己的 --help/模块 docstring 里有完整命令行参数说明。

6. Task1 方法概述（最终 v5.1 版本）

每个候选音频独立打分（score = logit(pos_token) - logit(neg_token)），不是两两比较。六个独立训练的 LoRA 分支（P0-P5），各自配一套 prompt 模板（见 utils/prompts.py 和 config.py 的 POINTWISE_BRANCHES），按两阶段融合：

第一阶段：六分支加权平均——p0=0.23, p1=0.20, p2=0.06, p3=0.15, p4=0.14, p5=0.22 （FUSION_CONFIG.stage1_weights）。
第二阶段（仅对难样本：第一阶段 top1/top2 margin 低于第20百分位的题目）：用 p1=0.25, p3=0.20, p5=0.55 重打分（FUSION_CONFIG.second_stage_weights），再融合 final = 0.5*一阶段 + 0.5*二阶段。

以上都是在本地 val 集上标定、最终提交实际用的真实生产数值——完整推导过程和各分支 checkpoint 对照表见技术报告第3.5节。

7. 环境说明

完整的经验证版本表（Python 3.10、torch==2.5.1+cu121、打包好的兼容 Qwen2.5-Omni 的 transformers 构建、peft==0.19.1 等）以及为什么用 vendored 压缩包而不是 GitHub git-commit 安装，见 ENVIRONMENT_REPRO.md。

8. 相对早期工作副本删掉了什么

这份归档保留了完整的"数据处理→训练→测试"全流程，但删掉了所有只在开发阶段有用、不属于最终提交方法的部分：config.py/05_test.py/06_merge.py/07_fuse_task1.py/ 05_test_pointwise.py 里按阶段/批次区分的 --stage2 专属代码分支（已被上面第2/4节的通用 test_paths.json 机制取代）、多 prompt 探索脚本（09_eval_task1_multiprompt.py、 10_test_task1_multiprompt.py）及其专属的 utils/task1_prompt_variants.py/ utils/task1_scoring.py、属于更早期非-pointwise 流程的事后集成脚本（07_ensemble_task1.py、08_ensemble_task2_weighted.py），以及一次性开发工具（cache_audio_npy.py、convert_wav_to_pcm.py、inspect_pred.py、 translate_zh_comments.py、run_all.sh）。复现最终 v5.1 提交完全不需要这些文件。

9. 故障排查

git clone/pip install 原始 transformers preview commit 失败：预期行为，见 ENVIRONMENT_REPRO.md，改用 vendor/ 下打包好的压缩包。
不传 --lora 时 05_test_pointwise.py 选错 checkpoint / 加载 adapter 报错：脚本会自动从 best_ckpt.json 解析，如果里面记录的绝对路径（原训练机器上的）在本机不存在，会按目录名在本地重新定位——具体4级优先级见模块 docstring / _resolve_lora_dir。如果仍找不到，显式传 --lora outputs/models/task1_p*/checkpoint-...。
**test_submit.py 报 [ERROR] Some release json files were not found**： test_paths.json 里的路径写错了，改完重跑即可。
**raw_jsonl 中无任何 response 字段**：重新跑 01_prepare_finetune.py——源 flat jsonl 必须包含 contexts[i].response（task1）/ contexts["<emotion>_response"]（task2）。

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support