---
license: apache-2.0
base_model:
- sionic-ai/pepperocr-1-4b
language:
- en
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- llama-cpp
- text-generation-inference
- document-ai
- document-parsing
- ocr
- multilingual
- safetensors
- vllm
---

# **pepperocr-1-4b-GGUF**

> **[PepperOCR-1-4B](https://huggingface.co/sionic-ai/pepperocr-1-4b)** is a 4.54-billion-parameter multilingual vision-language model for document parsing, fine-tuned from Qwen3.5-4B (architecture `Qwen3_5ForConditionalGeneration`) by sionic-ai, designed to convert document images directly into Markdown output and released as a public pre-release evaluation repository with weights distributed in BF16 safetensors (2 shards, ~9.1 GB). The model ships with a fully reproducible MDPBench evaluation runtime locked via `uv` with stable vLLM 0.24.0, PyTorch 2.11.0+cu130, CUDA 13.0, and Transformers 5.11.0, processing all 17 public MDPBench languages with a CPU-based document-orientation classifier (`PP-LCNet_x1_0_doc_ori`) preceding inference; on locally reproduced Korean and Thai public-set scores it achieves 92.2 and 83.2 overall accuracy respectively (87.7 KO/TH macro average), broken down further into digital versus photographed document subsets, using a greedy-first decoding strategy with a degeneration guard that retried roughly 10% of samples. As a pre-release evaluation preview, no final license has yet been applied to the fine-tuning contribution itself (though the underlying Qwen3.5-4B base remains Apache-2.0, and the final PepperOCR-1-4B release is planned under Apache-2.0 pending evaluation approval), and public downloadability does not currently grant redistribution or reuse rights.

## Model Files

   File Name | Quant Type | File Size | File Link |
 |-----------|------------|-----------|-----------|
 | pepperocr-1-4b.BF16.gguf | BF16 | 8.42 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.BF16.gguf) |
 | pepperocr-1-4b.F16.gguf | F16 | 8.42 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.F16.gguf) |
 | pepperocr-1-4b.Q3_K_L.gguf | Q3_K_L | 2.42 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q3_K_L.gguf) |
 | pepperocr-1-4b.Q3_K_M.gguf | Q3_K_M | 2.26 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q3_K_M.gguf) |
 | pepperocr-1-4b.Q3_K_S.gguf | Q3_K_S | 2.07 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q3_K_S.gguf) |
 | pepperocr-1-4b.Q4_K_M.gguf | Q4_K_M | 2.71 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q4_K_M.gguf) |
 | pepperocr-1-4b.Q4_K_S.gguf | Q4_K_S | 2.56 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q4_K_S.gguf) |
 | pepperocr-1-4b.Q5_K_M.gguf | Q5_K_M | 3.07 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q5_K_M.gguf) |
 | pepperocr-1-4b.Q5_K_S.gguf | Q5_K_S | 2.99 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q5_K_S.gguf) |
 | pepperocr-1-4b.Q8_0.gguf | Q8_0 | 4.48 GB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.Q8_0.gguf) |
 | pepperocr-1-4b.mmproj-bf16.gguf | mmproj-bf16 | 676 MB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.mmproj-bf16.gguf) |
 | pepperocr-1-4b.mmproj-f16.gguf | mmproj-f16 | 676 MB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.mmproj-f16.gguf) |
 | pepperocr-1-4b.mmproj-q8_0.gguf | mmproj-q8_0 | 367 MB | [Download](https://huggingface.co/prithivMLmods/pepperocr-1-4b-GGUF/blob/main/pepperocr-1-4b.mmproj-q8_0.gguf) |

## llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp