Add evaluation results on the de-en config and test split of wmt14 (#1)

e4c98fd verified over 2 years ago

1.96 kB

language:
  - de
  - en
datasets:
  - wmt14
pipeline_tag: translation
model-index:
  - name: leukas/mt5-large-wmt14-deen
    results:
      - task:
          type: translation
          name: Translation
        dataset:
          name: wmt14
          type: wmt14
          config: de-en
          split: test
        metrics:
          - type: bleu
            value: 15.9193
            name: BLEU
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMTc4MzAyMmM0MzhhOWRkMjdhNjNlYjM2NzYzY2RhN2ZlZTZlMjc2ZTU2NDNiMDViMzBiNGJkZjZkNDVjOGQzMyIsInZlcnNpb24iOjF9.peedpgE2-qr9AnNNkPfS-k3Dym9n84X3IGTYYUlrrdLtGwh1eJa7R5RLoWujQXb8rlGZV25BaIrH5vLtLX20BA
          - type: loss
            value: 1.0981481075286865
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzRhMGVlYTc5ZWU5ODE4MjczYjI3ZmQ3NzVmMWE5NzhmNjUzNjcyNTcxMmE5YjgwMmMwZWJmYzU4ZTIwNWEyZCIsInZlcnNpb24iOjF9.hgJ8jVwR9Et143miqygebY82ui8yUoFAoGj8xAdu3x9H5KUxh-i2rHNYFGV1Ln5Nf6N8oQdKMEUpLP6jYKhQDA
          - type: gen_len
            value: 19.0869
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjIzOGVkMDM3MjU1NjQ4YTYxNzI0OTY3ZDcwM2I2ZWI2YWY4NTQ5YjA3NzIwNTIzNDViNTQxMWJhZGQ5ZjBmNiIsInZlcnNpb24iOjF9.X56ymiUiJQQr8J-h67i8qL3D8_63JqChAwbdiZtGpB2xpst0uacEPl0cmDABDaf78ilQVASFZYkwoG9SnkEhAg

mt5-large-wmt14-deen

This model is released as part of the work from Are Character-level Translations Worth the Wait? Comparing Character- and Subword-level Models for Machine Translation. It is an mT5 model finetuned on German-->English translation the WMT14 dataset.

To use the model correctly, you must prepend the prompt with "translate X to Y: ", where X and Y are your source and target languages (e.g. German, English).

NOTE: The decoder_start_token_id is 259 for byt5 models and 250099 for mt5 models, which is different from the default token from google's byt5 and mt5 models (which is 0).