| --- |
| tags: |
| - model_hub_mixin |
| - pytorch_model_hub_mixin |
| - TrorYongOCR |
| datasets: |
| - SoyVitou/KhmerSynthetic1M |
| - seanghay/khmer-hanuman-100k |
| language: |
| - km |
| - en |
| metrics: |
| - cer |
| pipeline_tag: image-to-text |
| --- |
| <div align="center"> |
| <picture> |
| <img src="https://huggingface.co/KrorngAI/TrorYongASR-tiny/resolve/main/figures/krorngai.png" width="30%" alt="KrorngAI"> |
| </picture> |
| </div> |
| <hr> |
| <!-- |
| <div align="center" style="line-height:1"> |
| <a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K2.6-ff6b6b?color=1783ff&logoColor=white"/></a> |
| <a href="https://www.facebook.com/profile.php?id=61582509385293" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Krorng%20AI-white?logoColor=white"/></a> |
| </div> |
| --> |
| |
| <div align="center" style="line-height: 1;"> |
| <a href="https://huggingface.co/KrorngAI" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Krorng%20AI-ffc107?color=ffc107&logoColor=white"/></a> |
| <a href="https://youtube.com/@krorngai" target="_blank"><img alt="YouTube Channel" src="https://img.shields.io/badge/Youtube-Krorng%20AI-red?logoColor=red"/></a> |
| <a href="https://www.facebook.com/profile.php?id=61582509385293" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Facebook-Krorng%20AI-blue?logoColor=blue"/></a> |
| <a href="https://kimang18.github.io" target="_blank"><img alt="Personal" src="https://img.shields.io/badge/KHUN-white?logoColor=white"/></a> |
| </div> |
| <div align="center" style="line-height: 1;"> |
| <a href="https://huggingface.co/Kimang18/tror-yong-asr-tiny/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a> |
| </div> |
|
|
|
|
| # TrorYongOCR |
|
|
| > [!Note] |
| > This repository contains model weights and configuration files for the pre-trained model compatible with `tror-yong-ocr` version 0.2.6 onwards |
| > |
| |
| TrorYongOCR is a tiny encoder-decoder model for Scene Text Recognition task. |
| It prepends the encoding of image patches to the "begin of sequence" token to condition next character token generation. |
| Using LLM analogy, patch encodings can be simply seen as a prefill prompt. |
| The single text decoder block of TrorYongOCR generates character tokens based on the prefill prompt in an autoregressive manner without cross-attention mechanism. |
| Moreover, TrorYongOCR can process input images of arbitrary aspect ratio. |
| Current pre-trained weight supports 2 languages: Khmer and English. |
|
|
|
|
| ## Model Details |
|
|
| - **Developed by:** KHUN Kimang (Ph.D.) |
| - **Shared by:** KrorngAI |
| - **Model type:** OCR (Optical Character Recognition) |
| - **Language(s) (NLP):** Khmer and English |
|
|
|
|
| ### Model Architecture |
|
|
|  |
|
|
|
|
| ### Model Sources |
|
|
| This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration: |
| - **Code**: https://pypi.org/project/tror-yong-ocr/ |
| - **Blog Post**: https://kimang18.github.io/krorngai-blog/TrorYongOCR/ |
| - **Demo:**: https://krorngai-troryongocr-demo.hf.space |
|
|
|
|
| ### Model Configuration |
|
|
| The choice of model configuration can be found as the following. |
| While preserving aspect ratio, the input image is resized to $min(H, W) = 32$ where $H$ and $W$ are height and width of image respectively. |
| This is to reduce computation cost in the training as images with high resolution and big aspect ratio incur very long sequence of patches. |
| The image patch size is $(8, 4)$ where $8$ is along the width of input image. |
| The context length for character sequence is up to $1024$. |
| Transformer configuration is the following: there are $4$ blocks, each has embedding dimension $d_{model}=384$ and $h=6$ heads. |
| In particular, encoding blocks (block $1$ to $3$) have MLP dimension $d_{MLP}=2*d_{model}=768$ and the decoding block has $d_{MLP}=4*d_{model}=1546$. |
| |
| |
| <div align="center"> |
| |
| |Layer| $d_{model}$ | $h$ | $d_{MLP}$ | Role | |
| |:---:|:-----------:|:---:|:---------:|:-------:| |
| |1 | 384 | 6 | 768 | Encoder | |
| |2 | 384 | 6 | 768 | Encoder | |
| |3 | 384 | 6 | 768 | Encoder | |
| |4 | 384 | 6 | 1546 | Decoder | |
| </div> |
| |
| |
| ## Training Detail |
| |
| TrorYongOCR is implemented as a `PyPI` package and can be installed via |
| |
| ```bash |
| pip install tror-yong-ocr |
| ``` |
| |
| It is obtained by pre-training on [`seanghay/khmer-hanuman-100k`](https://huggingface.co/datasets/seanghay/khmer-hanuman-100k) and [`SoyVitou/KhmerSynthetic1M`](https://huggingface.co/datasets/SoyVitou/KhmerSynthetic1M) datasets and fine-tuning on [Khmer Scene Text](https://arxiv.org/pdf/2410.18277) dataset. |
| |
| |
| ### KhmerSynthetic1M |
| |
| `KhmerSynthetic1M` is a dataset by [`Mr. Soy Vitou`](https://huggingface.co/SoyVitou). |
| This dataset contains images in gray monochromatic color palette (black, white, gray, etc.,). |
| The distribution of the number of tokens, _i.e._ frequency of each number of tokens, is fairly uniform. |
| In particular, the maximum number of tokens is around $120$. |
| This implies that there are images with aspect ratio largely higher than $4$. |
|
|
|
|
| ### khmer-hanuman-100k |
|
|
| This dataset by [`Mr. Yat Seanghay`](https://huggingface.co/seanghay) contains images with a variety of background colors and character colors. |
|
|
| ### KhmerST: A Low-Resource Khmer Scene Text Detection and Recognition Benchmark |
|
|
| KhmerST is the first Khmer scene-text dataset consisting of: |
|
|
| - 1,544 annotated images |
| - 997 indoor scenes |
| - 547 outdoor scenes |
|
|
| It has diverse conditions: |
|
|
| - flat and raised text |
| - low illumination |
| - distant and partially occluded text. |
|
|
| The annotations are done at line-level with polygon bounding boxes. |
|
|
| To fine-tune TrorYongOCR, we cropped the polygon bounding boxes to get only text images. Then, we use warp operation to transform polygon into rectangle. |
|
|
|
|
| ### Weight Initialization |
|
|
| We initialize weights as what SOTA models reguarly do. |
| The code to initialize the weight is given below. |
|
|
| Exceptionally, for position embedding used in the decoding block, I initialized it with $std=1.0$. |
|
|
| ```python |
| def init_weights(self, module: nn.Module, name: str = '', exclude: Sequence[str] = ('')): |
| """Initialize the weights using the typical initialization schemes used in SOTA models.""" |
| if any(map(name.startswith, exclude)): |
| return |
| if isinstance(module, nn.Linear): |
| nn.init.trunc_normal_(module.weight, std=0.02) |
| if module.bias is not None: |
| nn.init.zeros_(module.bias) |
| elif isinstance(module, nn.Embedding): |
| nn.init.trunc_normal_(module.weight, std=0.02) |
| if module.padding_idx is not None: |
| module.weight.data[module.padding_idx].zero_() |
| elif isinstance(module, nn.Conv2d): |
| nn.init.kaiming_normal_(module.weight) |
| if module.bias is not None: |
| nn.init.zeros_(module.bias) |
| elif isinstance(module, (nn.LayerNorm, nn.BatchNorm2d, nn.GroupNorm)): |
| nn.init.ones_(module.weight) |
| nn.init.zeros_(module.bias) |
| ``` |
|
|
| ## Citation |
|
|
| <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
| **BibTeX:** |
|
|
| ```bibtex |
| @online{khun2026, |
| author = {KHUN, Kimang}, |
| title = {TrorYongOCR: {Encoder-Decoder} {Model} for {Scene} {Text} |
| {Recognition}}, |
| date = {2026-02-19}, |
| url = {https://kimang18.github.io/krorngai-blog/TrorYongOCR/}, |
| langid = {en} |
| } |
| ``` |
|
|
|
|
| ## Model Card Author |
|
|
| - ឈ្មោះ: បណ្ឌិត ឃុន គីមអាង |
| - Name: KHUN Kimang (Ph.D.) |
|
|
|
|
| ## Acknowledgement |
|
|
| [`LightningAI`](https://lightning.ai) and `Google Colab` did not specifically sponsor this project. |
| But, both models are be trained thanks to their free credits. |
| So, huge thanks to [`LightningAI`](https://lightning.ai) and `Google Colab`. |
|
|
| Thanks to all the authors of publicly available datasets. |
|
|
|
|
| ## Model Card Contact |
|
|
| If you have any questions, please reach out at [Facebook Page](https://www.facebook.com/profile.php?id=61582509385293). |
