initial commit

README.md

@@ -0,0 +1,85 @@
+---
+license: cc-by-nc-4.0
+library_name: transformers
+pipeline_tag: voice-activity-detection
+tags:
+- speaker 
+- speaker-diarization
+- meeting
+- wavlm
+- wespeaker
+- diarizen
+- pyannote
+- pyannote-audio-pipeline
+---
+
+## Overview
+This hub features the pre-trained model by [DiariZen](https://github.com/BUTSpeechFIT/DiariZen). The EEND component is built upon WavLM Large and Conformer layers. The model was trained on far-field, single-channel audio from a diverse set of public datasets, including AMI, AISHELL-4, AliMeeting, NOTSOFAR-1, MSDWild, DIHARD3, RAMC, and VoxConverse. 
+
+Then structured pruning at 80% sparsity is applied. After pruning, the number of parameters in WavLM Large is reduced from **316.6M to 63.3M**, and the computational cost (MACs) decreases from **17.8G to 3.8G** per second. When loading this model, please ensure **non-commercial** usage, in accordance with the CC BY-NC 4.0 license.
+
+
+
+## Usage
+```python
+from diarizen.pipelines.inference import DiariZenPipeline
+
+# load pre-trained model
+diar_pipeline = DiariZenPipeline.from_pretrained("BUT-FIT/diarizen-wavlm-large-s80-md")
+# apply diarization pipeline
+diar_results = diar_pipeline('audio.wav')
+
+# print results
+for turn, _, speaker in diar_results.itertracks(yield_label=True):
+    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")
+
+# load pre-trained model and save RTTM result
+diar_pipeline = DiariZenPipeline.from_pretrained(
+        "BUT-FIT/diarizen-wavlm-large-s80-md",
+        rttm_out_dir='.'
+)
+# apply diarization pipeline
+diar_results = diar_pipeline('audio.wav', sess_name='session_name')
+```
+
+## Results (collar=0s) 
+| Dataset       | [Pyannote v3.1](https://github.com/pyannote/pyannote-audio) | DiariZen |
+|:---------------|:-----------:|:-----------:|
+| AMI           | 22.4      | 13.9 |
+| AISHELL-4     | 12.2      | 10.1 |
+| AliMeeting    | 24.4      | 10.8 |
+| NOTSOFAR-1    | -      | 16.7 |
+| MSDWild       | 25.3      | 15.8 | 
+| DIHARD3       | 21.7      | 14.5 | 
+| RAMC          | 22.2      | 11.0 | 
+| VoxConverse   | 11.3      | 9.1 |
+
+## Citation
+If you found this work helpful, please consider citing:
+```
+@inproceedings{han2025leveraging,
+  title={Leveraging self-supervised learning for speaker diarization},
+  author={Han, Jiangyu and Landini, Federico and Rohdin, Johan and Silnova, Anna and Diez, Mireia and Burget, Luk{\'a}{\v{s}}},
+  booktitle={Proc. ICASSP},
+  year={2025}
+}
+
+@article{han2025fine,
+  title={Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization},
+  author={Han, Jiangyu and Landini, Federico and Rohdin, Johan and Silnova, Anna and Diez, Mireia and Cernocky, Jan and Burget, Lukas},
+  journal={arXiv preprint arXiv:2505.24111},
+  year={2025}
+}
+
+@article{han2025efficient,
+  title={Efficient and Generalizable Speaker Diarization via Structured Pruning of Self-Supervised Models},
+  author={Han, Jiangyu and P{\'a}lka, Petr and Delcroix, Marc and Landini, Federico and Rohdin, Johan and Cernock{\`y}, Jan and Burget, Luk{\'a}{\v{s}}},
+  journal={arXiv preprint arXiv:2506.18623},
+  year={2025}
+}
+```
+
+## License
+- **Source code**: MIT (see the [project’s GitHub repository](https://github.com/BUTSpeechFIT/DiariZen)).
+- **Model weights**: CC BY-NC 4.0 (non-commercial).
+- Rationale: some training datasets are research-only or non-commercial, so the released weights cannot be used commercially.

config.json

@@ -0,0 +1,3 @@
+{
+  "library_name": "transformers"
+}

config.toml

@@ -0,0 +1,35 @@
+[model]
+path = "diarizen.models.eend.model_wavlm_conformer.Model"
+
+[model.args]
+wavlm_src = "wavlm_large_s80_md"
+wavlm_layer_num = 25
+wavlm_feat_dim = 1024
+attention_in = 256
+ffn_hidden = 1024
+num_head = 4
+num_layer = 4
+dropout = 0.1
+chunk_size = 16
+use_posi = false
+output_activate_function = false
+selected_channel = 0
+
+[inference.args]
+seg_duration = 16
+segmentation_step = 0.1
+batch_size = 32
+apply_median_filtering = true
+
+[clustering.args]
+method = "VBxClustering"
+min_speakers = 1
+max_speakers = 20
+ahc_criterion = "distance"
+ahc_threshold = 0.6
+Fa = 0.07
+Fb = 0.8
+lda_dim = 128
+max_iters = 20
+
+

plda/plda.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b77bcd840692710dd3496f62ecfeed8d8e5f002fd991b785079b244eab7d255
+size 133852

plda/xvec_transform.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:325f1ce8e48f7e55e9c8aa47e05d2766b7c48c4b25b8de8dd751e7a4cc5fbe8f
+size 134376

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0252b5d926600ad434778a85314f3164601914d36d355cbad3e7e30be968a675
+size 278066342