billingsmoore/prototype-tibetan-to-english-translation-v1-4bit

Model Card for tibetan-to-english-translation-4bit

Model Details

This model is a quantized version of billingsmoore/tibetan-to-english-translation. The model is double quanitized to 4bits. The process was performed (and can be replicated) with the following code:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig
import torch

model_id = "billingsmoore/tibetan-to-english-translation"

tokenizer = AutoTokenizer.from_pretrained(model_id)

quantization_config =  BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_use_double_quant=True,
   bnb_4bit_compute_dtype=torch.bfloat16
)

model = AutoModelForSeq2SeqLM.from_pretrained(model_id, device_map="auto", quantization_config=quantization_config)

• Developed by: billingsmoore
• Model type: Seq2Seq
• Language(s) (NLP): Tibetan, English
• License: Apache license 2.0

Model Sources

• Repository: GitHub

Uses

The intended usage of this quantized model is for in-browser usage on edge devices.

Direct Use

This model can be used as part of a web app using Transformers.js as below.

import { pipeline } from '@huggingface/transformers';

// Allocate a pipeline for sentiment-analysis
const pipe = await pipeline('translation', 'billingsmoore/tibetan-to-english-translation-4bit');

const out = await pipe('ན་མོ་མཉྫ་ཤཱི་ཡེ།');

Out-of-Scope Use

This model can also be used in the usual way using the Python transformers library as below.

from transformers import pipeline

# Allocate a pipeline for sentiment-analysis
pipe = pipeline('translation', 'billingsmoore/tibetan-to-english-translation-4bit')

out = pipe('ན་མོ་མཉྫ་ཤཱི་ཡེ།')

More Information

For additional information on training, data, etc. please see the model card for billingsmoore/tibetan-to-english-translation.

Model Card Author

billingsmoore

Model Card Contact

billingsmoore [at] gmail [dot] com