---
library_name: transformers
license: apache-2.0
base_model: google/long-t5-tglobal-xl
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: b681866e14ce1df397f72335e0e0272e
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# b681866e14ce1df397f72335e0e0272e

This model is a fine-tuned version of [google/long-t5-tglobal-xl](https://huggingface.co/google/long-t5-tglobal-xl) on the Helsinki-NLP/opus_books [de-nl] dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5306
- Data Size: 1.0
- Epoch Runtime: 216.8754
- Bleu: 6.5167

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: constant
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Data Size | Epoch Runtime | Bleu   |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:-------------:|:------:|
| No log        | 0     | 0    | 3.4089          | 0         | 16.6455       | 0.2551 |
| No log        | 1     | 390  | 2.6357          | 0.0078    | 18.5474       | 1.1720 |
| No log        | 2     | 780  | 2.4830          | 0.0156    | 24.8775       | 1.2194 |
| No log        | 3     | 1170 | 2.3850          | 0.0312    | 31.0326       | 1.4041 |
| No log        | 4     | 1560 | 2.2682          | 0.0625    | 40.1484       | 1.8537 |
| 0.1567        | 5     | 1950 | 2.1291          | 0.125     | 52.7841       | 2.3315 |
| 0.3635        | 6     | 2340 | 1.9819          | 0.25      | 73.9581       | 3.1243 |
| 2.1355        | 7     | 2730 | 1.8350          | 0.5       | 123.9935      | 3.7571 |
| 1.885         | 8.0   | 3120 | 1.6774          | 1.0       | 217.7282      | 4.5244 |
| 1.6935        | 9.0   | 3510 | 1.5903          | 1.0       | 219.2729      | 5.0980 |
| 1.5639        | 10.0  | 3900 | 1.5375          | 1.0       | 217.0207      | 5.5020 |
| 1.4526        | 11.0  | 4290 | 1.4975          | 1.0       | 215.9094      | 5.7430 |
| 1.345         | 12.0  | 4680 | 1.4803          | 1.0       | 218.5554      | 5.9550 |
| 1.2678        | 13.0  | 5070 | 1.4667          | 1.0       | 220.3127      | 6.1438 |
| 1.1842        | 14.0  | 5460 | 1.4640          | 1.0       | 216.3124      | 6.2584 |
| 1.082         | 15.0  | 5850 | 1.4679          | 1.0       | 215.7120      | 6.3342 |
| 1.0072        | 16.0  | 6240 | 1.4903          | 1.0       | 216.9731      | 6.4530 |
| 0.9402        | 17.0  | 6630 | 1.4902          | 1.0       | 217.8165      | 6.4267 |
| 0.8681        | 18.0  | 7020 | 1.5306          | 1.0       | 216.8754      | 6.5167 |


### Framework versions

- Transformers 4.57.0
- Pytorch 2.8.0+cu128
- Datasets 4.2.0
- Tokenizers 0.22.1