indonesian_to_bataknese_translation_model
This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the nusatranslation_mt dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1479
- Sacrebleu: 31.4299
- Gen Len: 45.1465
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Sacrebleu |
Gen Len |
| 1.6013 |
1.0 |
1650 |
1.3829 |
29.312 |
45.592 |
| 1.2324 |
2.0 |
3300 |
1.2091 |
30.6403 |
45.6005 |
| 1.0217 |
3.0 |
4950 |
1.1552 |
31.428 |
45.0845 |
| 0.8984 |
4.0 |
6600 |
1.1470 |
31.4125 |
45.365 |
| 0.8012 |
5.0 |
8250 |
1.1479 |
31.4299 |
45.1465 |
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.13.1
- Tokenizers 0.14.1
