Upload PP-DocLayoutV3 OpenVINO IR

README.md

@@ -0,0 +1,182 @@
+---
+license: apache-2.0
+pipeline_tag: object-detection
+tags:
+- PaddleOCR
+- PaddleOCR-VL
+- OpenVINO
+- openvino-ir
+- intel
+- AIPC
+- ocr
+- layout
+- layout_detection
+- document-parsing
+language:
+- en
+- zh
+- multilingual
+library_name: openvino
+base_model:
+- PaddlePaddle/PP-DocLayoutV3
+base_model_relation: quantized
+---
+<div align="center">
+
+<h1 align="center">
+
+PP-DocLayoutV3 · OpenVINO IR
+
+</h1>
+
+<p align="center">Layout Analysis Module of PaddleOCR-VL-1.5 — converted to OpenVINO™ IR for local inference on Intel CPU / GPU / NPU</p>
+
+[![OpenVINO](https://img.shields.io/badge/Runtime-OpenVINO-1A73E8)](https://github.com/openvinotoolkit/openvino)
+[![repo](https://img.shields.io/github/stars/PaddlePaddle/PaddleOCR?color=ccf)](https://github.com/PaddlePaddle/PaddleOCR)
+[![Base model](https://img.shields.io/badge/Base-PP--DocLayoutV3-orange)](https://modelscope.cn/models/PaddlePaddle/PP-DocLayoutV3)
+[![License](https://img.shields.io/badge/license-Apache_2.0-green)](./LICENSE)
+
+**🔥 [Official Website](https://www.paddleocr.com)** |
+**📝 [Technical Report](https://arxiv.org/pdf/2601.21957)**
+
+</div>
+
+---
+
+## Introduction · 简介
+
+This repository hosts the **OpenVINO™ IR** build of **PP-DocLayoutV3**, the layout-analysis module of
+**PaddleOCR-VL-1.5**. The original PaddlePaddle weights have been converted to OpenVINO Intermediate
+Representation (`inference.xml` + `inference.bin`) so the model runs **fully locally** on **Intel CPU,
+integrated/discrete GPU, and NPU** via the OpenVINO runtime — no cloud service and no PaddlePaddle
+runtime required.
+
+> 本仓库提供 **PP-DocLayoutV3 的 OpenVINO™ IR 版本**，它是 **PaddleOCR-VL-1.5** 的版面分析（layout）模块。
+> 模型已从 PaddlePaddle 权重转换为 OpenVINO 中间表示（`inference.xml` + `inference.bin`），可在
+> **Intel CPU / 集显 / 独显 / NPU** 上**完全本地**运行，无需联网、无需安装 PaddlePaddle。
+
+**PP-DocLayoutV3 is specifically engineered to handle non-planar document images.** It directly predicts
+multi-point bounding boxes for layout elements (rather than standard two-point boxes) and determines the
+logical reading order for skewed and curved surfaces within a single forward pass, significantly reducing
+cascading errors. It is an essential component of PaddleOCR-VL-1.5, providing the layout analysis that
+drives high-precision parsing of real-world documents.
+
+### Model Architecture
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/PP-DocLayoutV3.png" width="800"/>
+</div>
+
+---
+
+## What's in this repo · 文件说明
+
+| File | Description |
+| --- | --- |
+| `inference.xml` | OpenVINO IR network topology |
+| `inference.bin` | OpenVINO IR weights |
+| `inference.yml` | Preprocessing config (resize 800×800, normalization) + 25-class label list + `draw_threshold` |
+| `config.json` | Model config |
+| `preprocessor_config.json` | Image-processor config |
+
+- **Architecture:** DETR-style detector
+- **Input:** `image` `[1, 3, 800, 800]` (BGR, resized to 800×800, no keep-ratio) + `scale_factor` `[1, 2]`
+- **Default score threshold:** `0.5` (`draw_threshold` in `inference.yml`)
+- **Layout classes (25):** abstract, algorithm, aside_text, chart, content, display_formula, doc_title,
+  figure_title, footer, footer_image, footnote, formula_number, header, header_image, image,
+  inline_formula, number, paragraph_title, reference, reference_content, seal, table, text,
+  vertical_text, vision_footnote
+
+---
+
+## Usage · 使用方法
+
+### 1. Recommended — as part of the PaddleOCR-VL OpenVINO pipeline
+
+This model is the layout stage of an end-to-end document-parsing pipeline. Pair it with
+[`FionaGu1019/PaddleOCR-VL-1.5-ov`](https://modelscope.cn/models/FionaGu1019/PaddleOCR-VL-1.5-ov)
+(the recognition VLM) to get layout detection → reading order → text/table/formula recognition.
+
+```python
+from modelscope import snapshot_download
+
+# Download both stages of the pipeline
+layout_dir = snapshot_download("FionaGu1019/PP-DocLayoutV3-ov")
+vl_dir     = snapshot_download("FionaGu1019/PaddleOCR-VL-1.5-ov")
+```
+
+### 2. Standalone OpenVINO inference
+
+```python
+import cv2
+import numpy as np
+import openvino as ov
+
+model_dir = "PP-DocLayoutV3-ov"          # local path or snapshot_download(...) result
+core = ov.Core()
+compiled = core.compile_model(f"{model_dir}/inference.xml", "GPU")  # "CPU" / "GPU" / "NPU"
+
+# Preprocess: resize to 800x800 (no keep-ratio), CHW, float32
+img = cv2.imread("document.jpg")
+h, w = img.shape[:2]
+resized = cv2.resize(img, (800, 800), interpolation=cv2.INTER_LINEAR)
+blob = resized.astype(np.float32).transpose(2, 0, 1)[None]          # [1,3,800,800]
+scale_factor = np.array([[800 / h, 800 / w]], dtype=np.float32)     # [1,2]
+
+results = compiled({"image": blob, "scale_factor": scale_factor})
+# Outputs are DETR detections (class id, score, box); filter by draw_threshold=0.5
+# and map class ids via the label_list in inference.yml.
+```
+
+> Tip: enable on-disk kernel caching with
+> `core.set_property({"CACHE_DIR": ".ov_cache"})` to avoid re-compiling kernels on every run
+> (a large speedup on GPU).
+
+---
+
+## Visualization
+
+### Light Variation
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/layout_lighting.jpg" width="800"/>
+</div>
+
+### Skewing
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/layout_skew.jpg" width="800"/>
+</div>
+
+### Screen-photo
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/layout_screen.jpg" width="800"/>
+</div>
+
+### Curving
+<div align="center">
+<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr_vl_1_5/layout_curv.jpg" width="800"/>
+</div>
+
+---
+
+## Notes · 说明
+
+- This is a **format conversion** of the official PaddlePaddle model to OpenVINO IR; the network weights
+  and detection behaviour are intended to match the original PP-DocLayoutV3. For the original weights see
+  [PaddlePaddle/PP-DocLayoutV3](https://modelscope.cn/models/PaddlePaddle/PP-DocLayoutV3).
+- For best results across Intel hardware, prefer **GPU** when available and fall back to **CPU**.
+
+## Citation
+
+If you find PP-DocLayoutV3 helpful, feel free to give the original project a star and citation.
+
+```bibtex
+@misc{cui2026paddleocrvl15multitask09bvlm,
+      title={PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing},
+      author={Cheng Cui and Ting Sun and Suyin Liang and Tingquan Gao and Zelun Zhang and Jiaxuan Liu and Xueqing Wang and Changda Zhou and Hongen Liu and Manhui Lin and Yue Zhang and Yubo Zhang and Yi Liu and Dianhai Yu and Yanjun Ma},
+      year={2026},
+      eprint={2601.21957},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2601.21957},
+}
+```

config.json

@@ -0,0 +1,107 @@
+{
+  "activation_dropout": 0.0,
+  "activation_function": "silu",
+  "anchor_image_size": null,
+  "architectures": [
+    "PPDocLayoutV3ForObjectDetection"
+  ],
+  "attention_dropout": 0.0,
+  "backbone": null,
+  "backbone_config": {
+    "model_type": "hgnet_v2",
+    "arch": "L",
+    "return_idx": [0, 1, 2, 3],
+    "freeze_stem_only": true,
+    "freeze_at": 0,
+    "freeze_norm": true,
+    "lr_mult_list": [0, 0.05, 0.05, 0.05, 0.05],
+    "out_features": ["stage1", "stage2", "stage3", "stage4"]
+  },
+  "backbone_kwargs": null,
+  "batch_norm_eps": 1e-05,
+  "box_noise_scale": 1.0,
+  "d_model": 256,
+  "decoder_activation_function": "relu",
+  "decoder_attention_heads": 8,
+  "decoder_ffn_dim": 1024,
+  "decoder_in_channels": [
+    256,
+    256,
+    256
+  ],
+  "decoder_layers": 6,
+  "decoder_n_points": 4,
+  "disable_custom_kernels": true,
+  "dropout": 0.0,
+  "encode_proj_layers": [
+    2
+  ],
+  "encoder_activation_function": "gelu",
+  "encoder_attention_heads": 8,
+  "encoder_ffn_dim": 1024,
+  "encoder_hidden_dim": 256,
+  "encoder_in_channels": [
+    512,
+    1024,
+    2048
+  ],
+  "encoder_layers": 1,
+  "eos_coefficient": 0.0001,
+  "eval_size": null,
+  "feature_strides": [
+    8,
+    16,
+    32
+  ],
+  "hidden_expansion": 1.0,
+  "id2label": {
+    "0": "abstract",
+    "1": "algorithm",
+    "2": "aside_text",
+    "3": "chart",
+    "4": "content",
+    "5": "formula",
+    "6": "doc_title",
+    "7": "figure_title",
+    "8": "footer",
+    "9": "footer",
+    "10": "footnote",
+    "11": "formula_number",
+    "12": "header",
+    "13": "header",
+    "14": "image",
+    "15": "formula",
+    "16": "number",
+    "17": "paragraph_title",
+    "18": "reference",
+    "19": "reference_content",
+    "20": "seal",
+    "21": "table",
+    "22": "text",
+    "23": "text",
+    "24": "vision_footnote"
+  },
+  "initializer_range": 0.01,
+  "is_encoder_decoder": true,
+  "label2id": {},
+  "label_noise_ratio": 0.5,
+  "layer_norm_eps": 1e-05,
+  "learn_initial_query": false,
+  "matcher_alpha": 0.25,
+  "matcher_bbox_cost": 5.0,
+  "matcher_class_cost": 2.0,
+  "matcher_gamma": 2.0,
+  "matcher_giou_cost": 2.0,
+  "model_type": "pp_doclayout_v3",
+  "normalize_before": false,
+  "num_denoising": 100,
+  "num_feature_levels": 3,
+  "num_queries": 300,
+  "positional_encoding_temperature": 10000,
+  "torch_dtype": "float32",
+  "use_pretrained_backbone": false,
+  "use_timm_backbone": false,
+  "global_pointer_head_size": 64,
+  "mask_feature_channels": [64, 64],
+  "x4_feat_dim": 128
+}

inference.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9310c72cb0be540eeb431e1966710a01ae1425b93dd23c0be697f2f0abe0a389
+size 66958738

inference.yml

@@ -0,0 +1,100 @@
+mode: paddle
+draw_threshold: 0.5
+metric: COCO
+use_dynamic_shape: false
+Global:
+  model_name: PP-DocLayoutV3
+arch: DETR
+min_subgraph_size: 3
+Preprocess:
+- interp: 2
+  keep_ratio: false
+  target_size:
+  - 800
+  - 800
+  type: Resize
+- mean:
+  - 0.0
+  - 0.0
+  - 0.0
+  norm_type: none
+  std:
+  - 1.0
+  - 1.0
+  - 1.0
+  type: NormalizeImage
+- type: Permute
+label_list:
+- abstract
+- algorithm
+- aside_text
+- chart
+- content
+- display_formula
+- doc_title
+- figure_title
+- footer
+- footer_image
+- footnote
+- formula_number
+- header
+- header_image
+- image
+- inline_formula
+- number
+- paragraph_title
+- reference
+- reference_content
+- seal
+- table
+- text
+- vertical_text
+- vision_footnote
+Hpi:
+  backend_configs:
+    paddle_infer:
+      trt_dynamic_shapes: &id001
+        image:
+        - - 1
+          - 3
+          - 800
+          - 800
+        - - 1
+          - 3
+          - 800
+          - 800
+        - - 8
+          - 3
+          - 800
+          - 800
+        scale_factor:
+        - - 1
+          - 2
+        - - 1
+          - 2
+        - - 8
+          - 2
+      trt_dynamic_shape_input_data:
+        scale_factor:
+        - - 2
+          - 2
+        - - 1
+          - 1
+        - - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+          - 0.67
+    tensorrt:
+      dynamic_shapes: *id001

preprocessor_config.json

@@ -0,0 +1,36 @@
+{
+  "_valid_processor_keys": [
+    "images",
+    "do_resize",
+    "size",
+    "resample",
+    "do_rescale",
+    "rescale_factor",
+    "do_normalize",
+    "image_mean",
+    "image_std",
+    "return_tensors",
+    "data_format",
+    "input_data_format"
+  ],
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0,
+    0,
+    0
+  ],
+  "image_processor_type": "PPDocLayoutV3ImageProcessor",
+  "image_std": [
+    1,
+    1,
+    1
+  ],
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 800,
+    "width": 800
+  }
+}