| ---
|
| library_name: ctranslate2
|
| pipeline_tag: translation
|
| license: apache-2.0
|
| base_model: protonx-models/protonx-legal-tc
|
| language:
|
| - vi
|
| tags:
|
| - ctranslate2
|
| - ct2
|
| - text2text-generation
|
| - vietnamese
|
| - ocr-correction
|
| - post-ocr
|
| - distillation
|
| ---
|
|
|
| # Distilled protonx-legal-tc — CTranslate2 (Vietnamese OCR text correction)
|
|
|
| Distilled + CTranslate2-converted version of [`protonx-models/protonx-legal-tc`](https://huggingface.co/protonx-models/protonx-legal-tc) for fast CPU OCR text correction on Vietnamese administrative documents.
|
|
|
| The original protonx-legal-tc is a seq2seq model trained for legal-text correction. This distilled CT2 build is optimized for CPU inference and used by the [ScanIndex](https://github.com/welcomyou/scanindex) pipeline as the correction stage between OCR and PDF/DOCX export.
|
|
|
| ## Performance (ScanIndex internal benchmark)
|
|
|
| | Variant | Time / 13 pages | Accuracy |
|
| |---|---|---|
|
| | protonx-legal-tc (CT2 OPTIMIZE, beam=1) | 14.5s | 99.561% |
|
| | **This distilled CT2 (beam=1)** | **8.3s** | **99.550%** |
|
|
|
| 42% faster, 0.011 pp accuracy drop — recommended trade-off for CPU.
|
|
|
| ## Files
|
|
|
| - `distilled_ct2/model.bin` — CTranslate2 model
|
| - `distilled_ct2/tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `shared_vocabulary.json`, `config.json`
|
|
|
| ## Loading
|
|
|
| ```python
|
| import ctranslate2
|
| from transformers import AutoTokenizer
|
| from huggingface_hub import snapshot_download
|
|
|
| local = snapshot_download("welcomyou/distilled-protonx-vn-correction-ct2", local_dir="models")
|
| sub = f"{local}/distilled_ct2"
|
| translator = ctranslate2.Translator(sub, device="cpu")
|
| tok = AutoTokenizer.from_pretrained(sub)
|
| ```
|
|
|
| ## License
|
|
|
| Apache-2.0, inheriting from the protonx-legal-tc base.
|
|
|
