Haopeng/libri100-ce-acoustic-frontend-20ms

IF-MDD LibriSpeech100 20ms CE Acoustic Frontend

This package contains the selected 20 ms LibriSpeech100 CE acoustic frontend. It is inference-only and does not import the IF-MDD training repo or SpeechBrain.

The wrapper loads WavLM plus the small IF-MDD heads and returns:

• hidden_states: 384-d acoustic hidden features.
• ctc_log_probs: 259-way context-phone CTC log probabilities.
• fused_topology_log_probs: 82-way subphonetic topology log probabilities.
• phone_log_probs: 41-way phone log probabilities collapsed from the fused topology head.

Usage

import torch
from modeling_ifmdd_acoustic_frontend import IFMDDAcousticFrontend

model = IFMDDAcousticFrontend.from_pretrained(".", variant="20ms")
wavs = torch.randn(1, 16000)  # 16 kHz mono waveform
out = model.extract(wavs)

print(out["hidden_states"].shape)
print(out["phone_log_probs"].shape)

Expected input is 16 kHz mono waveform. The wrapper does not resample audio.

Selection

The selected checkpoint is:

exp_librispeech100_topology_full_ft_ottc_triphone_pruned_budget256_film_fusedphone_ce_epoch2_step100_keepall_b32_fix1/s2_t1/wavlm_large_None_PhnMonoSSL_ContextFiLMTopology_ctc_alignment_ce/save/CKPT+globalstep000002400_epoch003_topology_PER_8.2609_MDDF1_0.0000_TSE_45.9ms_MAE_22.9ms_ACC20_0.359.ckpt

TIMIT test metrics from eval/best10ms_vs_20ms.csv:

• phone_tse_ms=40.8143
• phone_acc_20=40.8857
• boundary_20_r_value_pct=76.4590