metadata
library_name: transformers
license: apache-2.0
base_model: google/long-t5-tglobal-xl
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: a50e0479df31a6b42995051c03abb9ef
results: []
a50e0479df31a6b42995051c03abb9ef
This model is a fine-tuned version of google/long-t5-tglobal-xl on the Helsinki-NLP/opus_books [de-pt] dataset. It achieves the following results on the evaluation set:
- Loss: 1.6243
- Data Size: 1.0
- Epoch Runtime: 26.5925
- Bleu: 5.2474
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: constant
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Data Size | Epoch Runtime | Bleu |
|---|---|---|---|---|---|---|
| No log | 0 | 0 | 3.1227 | 0 | 1.8720 | 0.3497 |
| No log | 1 | 27 | 2.8097 | 0.0078 | 2.7344 | 0.4528 |
| No log | 2 | 54 | 2.5536 | 0.0156 | 8.0554 | 0.6786 |
| No log | 3 | 81 | 2.3421 | 0.0312 | 11.1991 | 1.2236 |
| No log | 4 | 108 | 2.1753 | 0.0625 | 15.5740 | 2.1698 |
| No log | 5 | 135 | 2.0890 | 0.125 | 20.8313 | 2.1033 |
| No log | 6 | 162 | 2.0273 | 0.25 | 27.9351 | 1.9696 |
| No log | 7 | 189 | 1.9091 | 0.5 | 27.0546 | 2.1619 |
| 0.5362 | 8.0 | 216 | 1.7893 | 1.0 | 31.7981 | 2.8972 |
| 0.5362 | 9.0 | 243 | 1.7106 | 1.0 | 31.8682 | 3.5717 |
| 2.0484 | 10.0 | 270 | 1.6700 | 1.0 | 34.2794 | 4.1177 |
| 2.0484 | 11.0 | 297 | 1.6465 | 1.0 | 28.6308 | 4.3165 |
| 1.7421 | 12.0 | 324 | 1.6207 | 1.0 | 28.6074 | 4.4859 |
| 1.5418 | 13.0 | 351 | 1.6082 | 1.0 | 31.8599 | 4.6081 |
| 1.5418 | 14.0 | 378 | 1.5989 | 1.0 | 27.9289 | 4.7485 |
| 1.3658 | 15.0 | 405 | 1.6066 | 1.0 | 28.3605 | 5.0321 |
| 1.3658 | 16.0 | 432 | 1.6011 | 1.0 | 31.4451 | 5.1330 |
| 1.2282 | 17.0 | 459 | 1.6088 | 1.0 | 33.0436 | 5.1568 |
| 1.2282 | 18.0 | 486 | 1.6243 | 1.0 | 26.5925 | 5.2474 |
Framework versions
- Transformers 4.57.0
- Pytorch 2.8.0+cu128
- Datasets 4.2.0
- Tokenizers 0.22.1