| license: apache-2.0 | |
| tags: | |
| - generated_from_trainer | |
| datasets: | |
| - mlsum | |
| metrics: | |
| - rouge | |
| base_model: google/mt5-base | |
| model-index: | |
| - name: mt5-base-turkish-sum | |
| results: | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: mlsum tu | |
| type: mlsum | |
| args: tu | |
| metrics: | |
| - type: rouge | |
| value: 47.4222 | |
| name: Rouge1 | |
| # [Mukayese: Turkish NLP Strikes Back](https://arxiv.org/abs/2203.01215) | |
| ## Summarization: mukayese/mbart-large-turkish-sum | |
| This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the mlsum/tu dataset. | |
| It achieves the following results on the evaluation set: | |
| - Rouge1: 47.4222 | |
| - Rouge2: 34.8624 | |
| - Rougel: 42.2487 | |
| - Rougelsum: 43.9494 | |
| Check [this](https://arxiv.org/abs/2203.01215) paper for more details on the model and the dataset. | |
| ### Training hyperparameters | |
| The following hyperparameters were used during training: | |
| - learning_rate: 0.0005 | |
| - train_batch_size: 2 | |
| - eval_batch_size: 4 | |
| - seed: 42 | |
| - distributed_type: multi-GPU | |
| - num_devices: 8 | |
| - gradient_accumulation_steps: 4 | |
| - total_train_batch_size: 64 | |
| - total_eval_batch_size: 32 | |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 | |
| - lr_scheduler_type: linear | |
| - num_epochs: 10.0 | |
| - label_smoothing_factor: 0.1 | |
| ### Framework versions | |
| - Transformers 4.11.3 | |
| - Pytorch 1.8.2+cu111 | |
| - Datasets 1.14.0 | |
| - Tokenizers 0.10.3 | |
| ### Citation | |
| ``` | |
| @misc{safaya-etal-2022-mukayese, | |
| title={Mukayese: Turkish NLP Strikes Back}, | |
| author={Ali Safaya and Emirhan Kurtuluş and Arda Göktoğan and Deniz Yuret}, | |
| year={2022}, | |
| eprint={2203.01215}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.CL} | |
| } | |
| ``` | |