Image-Text-to-Text
Transformers
Safetensors
monkeyocrv2
text-generation
conversational
custom_code
Eval Results
Instructions to use zenosai/MonkeyOCRv2-B-Parsing with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zenosai/MonkeyOCRv2-B-Parsing with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="zenosai/MonkeyOCRv2-B-Parsing", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("zenosai/MonkeyOCRv2-B-Parsing", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use zenosai/MonkeyOCRv2-B-Parsing with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "zenosai/MonkeyOCRv2-B-Parsing" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/zenosai/MonkeyOCRv2-B-Parsing
- SGLang
How to use zenosai/MonkeyOCRv2-B-Parsing with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "zenosai/MonkeyOCRv2-B-Parsing" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "zenosai/MonkeyOCRv2-B-Parsing" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use zenosai/MonkeyOCRv2-B-Parsing with Docker Model Runner:
docker model run hf.co/zenosai/MonkeyOCRv2-B-Parsing
Update README.md
Browse files
README.md
CHANGED
|
@@ -21,8 +21,10 @@ library_name: transformers
|
|
| 21 |
</div>
|
| 22 |
|
| 23 |
## News
|
|
|
|
| 24 |
* `2026.07.21` π¦ We release [MonkeyDoc v2](https://modelscope.cn/datasets/zenosai/MonkeyDocv2), an open multilingual corpus for document-oriented pretraining. We hope it can serve as a shared data foundation for more transparent, reproducible, and fair comparisons in Document AI.
|
| 25 |
-
* `2026.07.
|
|
|
|
| 26 |
|
| 27 |
## Use MonkeyOCRv2 as a Vision Backbone
|
| 28 |
|
|
@@ -45,11 +47,12 @@ encoder = AutoModel.from_pretrained(
|
|
| 45 |
See the Vision Encoder [Quick Start](https://github.com/Yuliang-Liu/MonkeyOCRv2#vision-encoder) for installation and feature-extraction examples. If you adapt MonkeyOCRv2 to a new task or domain, feel free to open an issue or pull request and share the results.
|
| 46 |
|
| 47 |
## MonkeyDoc v2
|
| 48 |
-
MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released
|
| 49 |
```bash
|
| 50 |
pip install modelscope
|
| 51 |
modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2
|
| 52 |
```
|
|
|
|
| 53 |
|
| 54 |
## Model Zoo
|
| 55 |
|
|
@@ -249,9 +252,9 @@ python download_model.py -t modelscope -n MonkeyOCRv2-B # or MonkeyOCRv2-S / Mon
|
|
| 249 |
```bash
|
| 250 |
cd vision
|
| 251 |
# For MonkeyOCRv2-B and MonkeyOCRv2-S
|
| 252 |
-
python extract_feature.py
|
| 253 |
# For MonkeyOCRv2-AS
|
| 254 |
-
python extract_feature_vitae.py
|
| 255 |
```
|
| 256 |
|
| 257 |
### Document Parsing
|
|
@@ -1237,7 +1240,7 @@ This multi-expert agreement strategy reduces model-specific annotation errors an
|
|
| 1237 |
- **MonkeyOCR**: https://github.com/Yuliang-Liu/MonkeyOCR
|
| 1238 |
- **MDPBench**: https://github.com/Yuliang-Liu/MultimodalOCR
|
| 1239 |
- **MonkeyDoc**: https://modelscope.cn/datasets/zenosai/MonkeyDoc
|
| 1240 |
-
- **
|
| 1241 |
|
| 1242 |
## Citation
|
| 1243 |
If you use any part of this release β the MonkeyOCRv2 encoders, MonkeyOCRv2-Parsing,
|
|
|
|
| 21 |
</div>
|
| 22 |
|
| 23 |
## News
|
| 24 |
+
* `2026.07.22` π MonkeyOCRv2-B-Parsing ranks #1 among evaluated open-source models on the official [MDPBench Leaderboard](https://huggingface.co/spaces/Delores-Lin/MDPBench-leaderboard), achieving 83.3 overall across 17 languages, including digital-born and photographed documents.
|
| 25 |
* `2026.07.21` π¦ We release [MonkeyDoc v2](https://modelscope.cn/datasets/zenosai/MonkeyDocv2), an open multilingual corpus for document-oriented pretraining. We hope it can serve as a shared data foundation for more transparent, reproducible, and fair comparisons in Document AI.
|
| 26 |
+
* `2026.07.20` π€ MonkeyOCRv2 was featured in [Hugging Face Daily Papers](https://huggingface.co/papers/2607.11562) and has entered the Weekly Papers list. Thanks to the community for the support, upvotes, and feedback!
|
| 27 |
+
* `2026.07.14` π We release [MonkeyOCRv2](https://arxiv.org/abs/2607.11562), including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.
|
| 28 |
|
| 29 |
## Use MonkeyOCRv2 as a Vision Backbone
|
| 30 |
|
|
|
|
| 47 |
See the Vision Encoder [Quick Start](https://github.com/Yuliang-Liu/MonkeyOCRv2#vision-encoder) for installation and feature-extraction examples. If you adapt MonkeyOCRv2 to a new task or domain, feel free to open an issue or pull request and share the results.
|
| 48 |
|
| 49 |
## MonkeyDoc v2
|
| 50 |
+
MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released 52 million synthetic samples and 41 million real-world samples. You can download the full datset as follows:
|
| 51 |
```bash
|
| 52 |
pip install modelscope
|
| 53 |
modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2
|
| 54 |
```
|
| 55 |
+
After processing and compression, downloading the dataset currently requires approximately 10 TB of disk space. We recommend having at least 11 TB of available storage before starting the download to ensure sufficient space throughout the process.
|
| 56 |
|
| 57 |
## Model Zoo
|
| 58 |
|
|
|
|
| 252 |
```bash
|
| 253 |
cd vision
|
| 254 |
# For MonkeyOCRv2-B and MonkeyOCRv2-S
|
| 255 |
+
python extract_feature.py -m ../model_weight/MonkeyOCRv2-B -i ../images_test/ar.JPEG
|
| 256 |
# For MonkeyOCRv2-AS
|
| 257 |
+
python extract_feature_vitae.py -m ../model_weight/MonkeyOCRv2-AS -i ../images_test/ar.JPEG
|
| 258 |
```
|
| 259 |
|
| 260 |
### Document Parsing
|
|
|
|
| 1240 |
- **MonkeyOCR**: https://github.com/Yuliang-Liu/MonkeyOCR
|
| 1241 |
- **MDPBench**: https://github.com/Yuliang-Liu/MultimodalOCR
|
| 1242 |
- **MonkeyDoc**: https://modelscope.cn/datasets/zenosai/MonkeyDoc
|
| 1243 |
+
- **MonkeyDoc v2**: https://modelscope.cn/datasets/zenosai/MonkeyDocv2
|
| 1244 |
|
| 1245 |
## Citation
|
| 1246 |
If you use any part of this release β the MonkeyOCRv2 encoders, MonkeyOCRv2-Parsing,
|