zenosai commited on
Commit
7855189
·
verified ·
1 Parent(s): 90e73e1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +53 -8
README.md CHANGED
@@ -5,7 +5,6 @@ license: apache-2.0
5
  pipeline_tag: image-text-to-text
6
  library_name: transformers
7
  ---
8
-
9
  <div align="center" xmlns="http://www.w3.org/1999/html">
10
  <h2>
11
  <b>MonkeyOCRv2: A Visual-Text Foundation Model for Document AI</b>
@@ -18,15 +17,39 @@ library_name: transformers
18
  [![GitHub closed issues](https://img.shields.io/github/issues-closed/Yuliang-Liu/MonkeyOCRv2?color=success&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aissue+is%3Aclosed)
19
  [![Demo](https://img.shields.io/badge/Demo-white.svg)](http://vlrlabmonkey.xyz:8891/)
20
 
21
- <img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/refs/heads/main/asserts/overview.png" width="600"/>
22
  </div>
23
 
24
  ## News
 
25
  * `2026.07.11` 🚀 We release [MonkeyOCRv2](https://github.com/Yuliang-Liu/MonkeyOCRv2), including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.
26
 
27
- ## Introduction
28
- MonkeyOCRv2 is a text-centric visual foundation model that unifies fine-grained text modeling, cross-task representation learning, and cross-lingual generalization in a single encoder. MonkeyOCRv2 generalizes effectively across a broad range of OCR and document intelligence tasks, including multilingual document parsing, document understanding, text recognition, formula recognition, document tampering detection, scene text detection, and overlapping text segmentation.
 
 
 
 
 
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  ## Model Zoo
32
 
@@ -198,7 +221,6 @@ MonkeyOCRv2 is a text-centric visual foundation model that unifies fine-grained
198
  </tbody>
199
  </table>
200
 
201
-
202
  ## Quick Start
203
  ### Vision Encoder
204
  #### 1. Install
@@ -330,7 +352,7 @@ python infer.py -h
330
  Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
331
 
332
  <p align="center">
333
- <img src="https://github.com/Yuliang-Liu/MonkeyOCRv2/blob/main/asserts/Visualization.gif?raw=true" width="600"/>
334
  </p>
335
 
336
  ## Evaluation Results
@@ -1191,7 +1213,10 @@ Our model supports robust document parsing in real-world scenarios across 17 lan
1191
  </tbody>
1192
  </table>
1193
 
1194
- ## Expert Model Labeling Toolchain
 
 
 
1195
 
1196
  We adopt a multi-expert labeling pipeline to obtain reliable annotations for documents. The pipeline includes the following steps:
1197
  1. **Structure Detection**
@@ -1209,6 +1234,26 @@ This multi-expert agreement strategy reduces model-specific annotation errors an
1209
  - **PaddleOCR-VL**: https://github.com/PaddlePaddle/PaddleOCR
1210
  - **Qwen3-VL**: https://github.com/QwenLM/Qwen3-VL
1211
  - **Qwen3**: https://github.com/QwenLM/Qwen3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1212
 
1213
  ## Copyright
1214
- We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. Note: This model is intended for academic research and non-commercial use only. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.
 
5
  pipeline_tag: image-text-to-text
6
  library_name: transformers
7
  ---
 
8
  <div align="center" xmlns="http://www.w3.org/1999/html">
9
  <h2>
10
  <b>MonkeyOCRv2: A Visual-Text Foundation Model for Document AI</b>
 
17
  [![GitHub closed issues](https://img.shields.io/github/issues-closed/Yuliang-Liu/MonkeyOCRv2?color=success&label=Issues)](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aissue+is%3Aclosed)
18
  [![Demo](https://img.shields.io/badge/Demo-white.svg)](http://vlrlabmonkey.xyz:8891/)
19
 
20
+ <img src="https://github.com/Yuliang-Liu/MonkeyOCRv2/blob/main/asserts/overview.png?raw=true" width="600"/>
21
  </div>
22
 
23
  ## News
24
+ * `2026.07.21` 📦 We release [MonkeyDoc v2](https://modelscope.cn/datasets/zenosai/MonkeyDocv2), an open multilingual corpus for document-oriented pretraining. We hope it can serve as a shared data foundation for more transparent, reproducible, and fair comparisons in Document AI.
25
  * `2026.07.11` 🚀 We release [MonkeyOCRv2](https://github.com/Yuliang-Liu/MonkeyOCRv2), including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.
26
 
27
+ ## Use MonkeyOCRv2 as a Vision Backbone
28
+
29
+ MonkeyOCRv2 is released as a standalone, document-native vision encoder. It can be integrated into different OCR and document AI systems as a visual backbone.
30
+
31
+ The current release has been evaluated on document parsing, document understanding, text recognition, formula recognition, text detection, document tampering detection, and overlapping-text segmentation.
32
+
33
+ Beyond these evaluated tasks, the encoder may also be useful for text-rich scenarios such as scientific papers, historical documents, medical reports, charts and tables, and remote-sensing maps or reports. We welcome community exploration of these directions.
34
 
35
+ ```python
36
+ from transformers import AutoModel
37
+
38
+ encoder = AutoModel.from_pretrained(
39
+ "zenosai/MonkeyOCRv2-B",
40
+ trust_remote_code=True,
41
+ dtype="auto",
42
+ device_map="auto",
43
+ )
44
+ ```
45
+ See the Vision Encoder [Quick Start](https://github.com/Yuliang-Liu/MonkeyOCRv2#vision-encoder) for installation and feature-extraction examples. If you adapt MonkeyOCRv2 to a new task or domain, feel free to open an issue or pull request and share the results.
46
+
47
+ ## MonkeyDoc v2
48
+ MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released all 52 million synthetic samples and 35 million real-world samples derived from FinePDF. Additional real-world data from other sources is currently being organized and will be released progressively. You can download the full datset as follows:
49
+ ```bash
50
+ pip install modelscope
51
+ modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2
52
+ ```
53
 
54
  ## Model Zoo
55
 
 
221
  </tbody>
222
  </table>
223
 
 
224
  ## Quick Start
225
  ### Vision Encoder
226
  #### 1. Install
 
352
  Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
353
 
354
  <p align="center">
355
+ <img src="asserts/Visualization.gif?raw=true" width="600"/>
356
  </p>
357
 
358
  ## Evaluation Results
 
1213
  </tbody>
1214
  </table>
1215
 
1216
+ ## MonkeyDoc v2
1217
+ MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released all 52 million synthetic samples and 41 million real-world samples derived from FinePDF. Additional real-world data from other sources is currently being organized and will be released progressively.
1218
+
1219
+ ### Expert Model Labeling Toolchain
1220
 
1221
  We adopt a multi-expert labeling pipeline to obtain reliable annotations for documents. The pipeline includes the following steps:
1222
  1. **Structure Detection**
 
1234
  - **PaddleOCR-VL**: https://github.com/PaddlePaddle/PaddleOCR
1235
  - **Qwen3-VL**: https://github.com/QwenLM/Qwen3-VL
1236
  - **Qwen3**: https://github.com/QwenLM/Qwen3
1237
+ - **MonkeyOCR**: https://github.com/Yuliang-Liu/MonkeyOCR
1238
+ - **MDPBench**: https://github.com/Yuliang-Liu/MultimodalOCR
1239
+ - **MonkeyDoc**: https://modelscope.cn/datasets/zenosai/MonkeyDoc
1240
+ - **FinePDF**: https://huggingface.co/datasets/HuggingFaceFW/finepdfs
1241
+
1242
+ ## Citation
1243
+ If you use any part of this release — the MonkeyOCRv2 encoders, MonkeyOCRv2-Parsing,
1244
+ MonkeyOCRv2-Und, the MDPBench benchmark, or the MonkeyDoc v2 dataset — please cite:
1245
+
1246
+ ```bibtex
1247
+ @article{liu2026monkeyocrv2,
1248
+ title = {MonkeyOCRv2: A Visual-Text Foundation Model for Document AI},
1249
+ author = {Liu, Yuliang and Li, Zhang and Zhang, Ziyang and Zhang, Shuo and
1250
+ Liu, Qiang and Song, Jiajun and Guo, Zidun and Wang, Xinhan and
1251
+ Zheng, Handong and Liu, Yang and Luo, Dongliang and Ma, Zhiyin and
1252
+ Zhang, Jiarui and Bai, Xiang},
1253
+ journal = {arXiv preprint arXiv:2607.11562},
1254
+ year = {2026}
1255
+ }
1256
+ ```
1257
 
1258
  ## Copyright
1259
+ We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. The models are are released under the [Apache License 2.0](LICENSE) and are free for both research and commercial use. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.