---
library_name: transformers
tags:
- akkadian
- translation
- byt5
- low-resource-languages
- mesopotamia
- lora
dataset:
- Deep Past
language:
- akk
- en
---

# Model Card: byt5-akkadian-translator

This model is a fine-tuned ByT5-Base architecture specialized in translating Old Assyrian (OA) transliterations into English. It was developed specifically for the **Deep Past** competition.

## Model Details

### Model Description

The model utilizes a Parameter-Efficient Fine-Tuning (PEFT) approach with LoRA to adapt a byte-level Transformer (ByT5) to the specific phonology and semantics of Akkadian cuneiform transliterations.

- **Developed by:** Vinil Patel
- **Model type:** Seq2Seq (Byte-level Transformer)
- **Language(s) (NLP):** Akkadian (transliterated), English
- **Finetuned from model:** google/byt5-base

### Model Sources

- **Repository:** https://huggingface.co/Yoshaaa7/byt5-akkadian-translator

## Uses

### Direct Use

This model is intended for the automated translation of standard Old Assyrian administrative and legal formulas. It excels at translating standardized letter introductions, weights, measures, and seal descriptions. 
(translate Akkadian to English: {akkadin}) is required. 

### Out-of-Scope Use

The model is not suitable for literary or poetic Akkadian (e.g., Epic of Gilgamesh) as it was trained specifically on Old Assyrian dialect datasets. It should not be used for critical historical research without human-in-the-loop verification.

## Bias, Risks, and Limitations

The model exhibits a **complexity bottleneck**. While highly accurate for formulaic text (0–12 tokens), its syntactic coherence degrades as sentence length increases. 

### Recommendations

- **Formulaic Texts:** Highly reliable.
- **Complex Clauses:** May hallucinate high-frequency names (e.g., *Pūšu-kēn*) when encountering rare vocabulary.
- **Preprocessing:** Best results are achieved when splitting multi-clause sentences into smaller segments.

## How to Get Started with the Model

```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("Yoshaaa7/byt5-akkadian-translator")
tokenizer = AutoTokenizer.from_pretrained("Yoshaaa7/byt5-akkadian-translator")

text = "translate Akkadian to English: a-na ma-ar-ti-ia qi2-bi-ma"
inputs = tokenizer(text, return_tensors="pt").input_ids
outputs = model.generate(inputs, max_length=50, num_beams=5, repetition_penalty=2.5)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))