sachin18449/hindi-english-unigram-nmt

Model Card for English-to-Hindi Unigram NMT

This model is a Transformer-based Neural Machine Translation (NMT) model trained to translate English sentences into Hindi using Unigram tokenization.
It was trained from scratch using Hugging Face Transformers.

---

Model Details

Model Description

This sequence-to-sequence model uses a standard Encoder–Decoder Transformer architecture for English→Hindi translation.
It is trained with Unigram tokenization instead of BPE to improve rare word handling and morphological coverage.

• Developed by: Sachin Kumar
• Funded by: Self / Personal Research
• Shared by: Sachin Kumar
• Model type: Transformer Encoder–Decoder (seq2seq)
• Language(s) (NLP): English → Hindi
• License: Apache-2.0
• Finetuned from model: Trained from scratch

Model Sources

• Repository: Hugging Face Model Page
• Paper (architecture): Attention Is All You Need
• Dataset: sachin18449/hindi-english-unigram-nmt

---

Uses

Direct Use

• Translating English text into Hindi.
• Integrating into multilingual apps, chatbots, and translation services.
• Generating Hindi content from English sources.

Downstream Use

• Fine-tuning for domain-specific translation (medical, legal, technical).
• Multi-task setups (e.g., summarization + translation pipelines).

Out-of-Scope Use

• Not designed for Hindi→English translation.
• Not suitable for audio inputs (speech-to-text).
• Should not be used for high-stakes decisions without human review.

---

Bias, Risks, and Limitations

• May reflect gender or cultural biases in training data.
• Rare words, idioms, or domain-specific jargon may be mistranslated.
• Limited ability to handle code-mixed or informal text.

Recommendations

• Post-edit translations for professional use.
• Fine-tune on domain-specific data for critical applications.

---

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "sachin18449/hindi-english-unigram-nmt"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

text = "How are you?"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))