| --- |
| license: apache-2.0 |
| base_model: openai/whisper-small |
| tags: |
| - whisper-event |
| - generated_from_trainer |
| datasets: |
| - facebook/voxpopuli |
| metrics: |
| - wer |
| model-index: |
| - name: WhisperForSpokenNER |
| results: |
| - task: |
| name: Automatic Speech Recognition |
| type: automatic-speech-recognition |
| dataset: |
| name: facebook/voxpopuli de+es+fr+nl |
| type: facebook/voxpopuli |
| config: de+es+fr+nl |
| split: None |
| metrics: |
| - name: Wer |
| type: wer |
| value: 0.11695951699047914 |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # WhisperForSpokenNER |
|
|
| This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on the facebook/voxpopuli de+es+fr+nl dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 75.5138 |
| - F1 Score: 0.6260 |
| - Label F1: 0.8282 |
| - Wer: 0.1170 |
|
|
| ## Model description |
|
|
| More information needed |
|
|
| ## Intended uses & limitations |
|
|
| More information needed |
|
|
| ## Training and evaluation data |
|
|
| More information needed |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 0.001 |
| - train_batch_size: 32 |
| - eval_batch_size: 32 |
| - seed: 42 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: cosine |
| - lr_scheduler_warmup_steps: 500 |
| - training_steps: 5000 |
| - mixed_precision_training: Native AMP |
| |
| ### Training results |
| |
| | Training Loss | Epoch | Step | Validation Loss | F1 Score | Label F1 | Wer | |
| |:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:------:| |
| | 294.2045 | 0.09 | 200 | 219.4521 | 0.3694 | 0.6423 | 0.1170 | |
| | 172.5491 | 0.18 | 400 | 158.4206 | 0.5112 | 0.7076 | 0.1170 | |
| | 152.1994 | 0.27 | 600 | 148.5779 | 0.5501 | 0.7391 | 0.1170 | |
| | 140.706 | 0.36 | 800 | 151.8108 | 0.5413 | 0.7324 | 0.1170 | |
| | 125.5897 | 0.45 | 1000 | 138.0534 | 0.5601 | 0.7432 | 0.1170 | |
| | 122.0436 | 0.54 | 1200 | 118.2416 | 0.5636 | 0.7724 | 0.1170 | |
| | 117.7194 | 0.63 | 1400 | 116.8705 | 0.5910 | 0.7772 | 0.1170 | |
| | 119.8977 | 0.71 | 1600 | 106.7047 | 0.5905 | 0.7833 | 0.1170 | |
| | 105.5846 | 0.8 | 1800 | 105.5354 | 0.5756 | 0.7774 | 0.1170 | |
| | 106.7833 | 0.89 | 2000 | 101.9971 | 0.5875 | 0.7922 | 0.1170 | |
| | 101.8875 | 0.98 | 2200 | 98.1714 | 0.5945 | 0.8016 | 0.1170 | |
| | 87.7438 | 1.07 | 2400 | 97.7943 | 0.6040 | 0.7967 | 0.1170 | |
| | 86.1916 | 1.16 | 2600 | 93.9310 | 0.6033 | 0.7964 | 0.1170 | |
| | 85.3271 | 1.25 | 2800 | 92.3677 | 0.6188 | 0.8146 | 0.1170 | |
| | 83.1457 | 1.34 | 3000 | 89.3458 | 0.6028 | 0.8116 | 0.1170 | |
| | 79.4126 | 1.43 | 3200 | 86.8935 | 0.6061 | 0.8094 | 0.1170 | |
| | 74.7596 | 1.52 | 3400 | 82.3525 | 0.6147 | 0.8224 | 0.1170 | |
| | 79.5526 | 1.61 | 3600 | 80.6440 | 0.6116 | 0.8153 | 0.1170 | |
| | 76.0212 | 1.7 | 3800 | 80.1555 | 0.6150 | 0.8216 | 0.1170 | |
| | 70.2905 | 1.79 | 4000 | 80.9369 | 0.6152 | 0.8177 | 0.1170 | |
| | 68.0936 | 1.88 | 4200 | 77.4738 | 0.6181 | 0.8206 | 0.1170 | |
| | 72.6116 | 1.97 | 4400 | 75.5524 | 0.6236 | 0.8276 | 0.1170 | |
| | 61.0175 | 2.06 | 4600 | 75.7015 | 0.6242 | 0.8249 | 0.1170 | |
| | 60.3508 | 2.14 | 4800 | 75.5521 | 0.6253 | 0.8270 | 0.1170 | |
| | 57.4103 | 2.23 | 5000 | 75.5138 | 0.6260 | 0.8282 | 0.1170 | |
| |
| |
| ### Framework versions |
| |
| - Transformers 4.37.0.dev0 |
| - Pytorch 2.1.0 |
| - Datasets 2.14.6 |
| - Tokenizers 0.14.1 |
| |
