---
license: apache-2.0
base_model: openai/whisper-small
tags:
- whisper-event
- generated_from_trainer
datasets:
- facebook/voxpopuli
metrics:
- wer
model-index:
- name: WhisperForSpokenNER
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: facebook/voxpopuli de+es+fr+nl
      type: facebook/voxpopuli
      config: de+es+fr+nl
      split: None
    metrics:
    - name: Wer
      type: wer
      value: 0.11695951699047914
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# WhisperForSpokenNER

This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on the facebook/voxpopuli de+es+fr+nl dataset.
It achieves the following results on the evaluation set:
- Loss: 75.5138
- F1 Score: 0.6260
- Label F1: 0.8282
- Wer: 0.1170

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- training_steps: 5000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1 Score | Label F1 | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:------:|
| 294.2045      | 0.09  | 200  | 219.4521        | 0.3694   | 0.6423   | 0.1170 |
| 172.5491      | 0.18  | 400  | 158.4206        | 0.5112   | 0.7076   | 0.1170 |
| 152.1994      | 0.27  | 600  | 148.5779        | 0.5501   | 0.7391   | 0.1170 |
| 140.706       | 0.36  | 800  | 151.8108        | 0.5413   | 0.7324   | 0.1170 |
| 125.5897      | 0.45  | 1000 | 138.0534        | 0.5601   | 0.7432   | 0.1170 |
| 122.0436      | 0.54  | 1200 | 118.2416        | 0.5636   | 0.7724   | 0.1170 |
| 117.7194      | 0.63  | 1400 | 116.8705        | 0.5910   | 0.7772   | 0.1170 |
| 119.8977      | 0.71  | 1600 | 106.7047        | 0.5905   | 0.7833   | 0.1170 |
| 105.5846      | 0.8   | 1800 | 105.5354        | 0.5756   | 0.7774   | 0.1170 |
| 106.7833      | 0.89  | 2000 | 101.9971        | 0.5875   | 0.7922   | 0.1170 |
| 101.8875      | 0.98  | 2200 | 98.1714         | 0.5945   | 0.8016   | 0.1170 |
| 87.7438       | 1.07  | 2400 | 97.7943         | 0.6040   | 0.7967   | 0.1170 |
| 86.1916       | 1.16  | 2600 | 93.9310         | 0.6033   | 0.7964   | 0.1170 |
| 85.3271       | 1.25  | 2800 | 92.3677         | 0.6188   | 0.8146   | 0.1170 |
| 83.1457       | 1.34  | 3000 | 89.3458         | 0.6028   | 0.8116   | 0.1170 |
| 79.4126       | 1.43  | 3200 | 86.8935         | 0.6061   | 0.8094   | 0.1170 |
| 74.7596       | 1.52  | 3400 | 82.3525         | 0.6147   | 0.8224   | 0.1170 |
| 79.5526       | 1.61  | 3600 | 80.6440         | 0.6116   | 0.8153   | 0.1170 |
| 76.0212       | 1.7   | 3800 | 80.1555         | 0.6150   | 0.8216   | 0.1170 |
| 70.2905       | 1.79  | 4000 | 80.9369         | 0.6152   | 0.8177   | 0.1170 |
| 68.0936       | 1.88  | 4200 | 77.4738         | 0.6181   | 0.8206   | 0.1170 |
| 72.6116       | 1.97  | 4400 | 75.5524         | 0.6236   | 0.8276   | 0.1170 |
| 61.0175       | 2.06  | 4600 | 75.7015         | 0.6242   | 0.8249   | 0.1170 |
| 60.3508       | 2.14  | 4800 | 75.5521         | 0.6253   | 0.8270   | 0.1170 |
| 57.4103       | 2.23  | 5000 | 75.5138         | 0.6260   | 0.8282   | 0.1170 |


### Framework versions

- Transformers 4.37.0.dev0
- Pytorch 2.1.0
- Datasets 2.14.6
- Tokenizers 0.14.1