README.md

---
language:
- uk
library_name: pytorch
pipeline_tag: image-to-text
license: cc-by-sa-4.0
base_model: Hukyl/parseq-b-cyrillic-handwritten
tags:
- ocr
- htr
- handwritten-text-recognition
- parseq
- ukrainian
- cyrillic
- rukopys
- image-to-text
datasets:
- UkrainianCatholicUniversity/rukopys
- annyhnatiuk/ukrainian-handwritten-text
- constantinwerner/cyrillic-handwriting-dataset
- ai-forever/school_notebooks_RU
- pumb-ai/synthetic-cyrillic-large
- nastyboget/synthetic_cyrillic_large
metrics:
- cer
- wer
model-index:
- name: parseq-b-rukopys
  results:
  - task:
      type: image-to-text
      name: Handwritten Text Recognition
    dataset:
      type: UkrainianCatholicUniversity/rukopys
      name: Rukopys, page-level val
      split: validation
    metrics:
    - type: cer
      value: 0.0953
      name: CER
    - type: wer
      value: 0.2796
      name: WER
---

# parseq-b-rukopys

A PARSeq-B (permuted autoregressive sequence) line recognizer for single-line
handwritten Ukrainian document text, fine-tuned on the
[Rukopys](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) dataset.

The training data included `handwritten`, `printed`, `annotation` and `table` region crops from Rukopys.

## TL;DR

| | value |
|---|---|
| Architecture | PARSeq — ViT-B encoder + 1-layer permutation-LM decoder (ECCV 2022) |
| Parameters | ~96M |
| Init from | [`Hukyl/parseq-b-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten) (mixed pretrain) |
| Handles | `handwritten`, `printed`, `annotation` |
| Decode | AR greedy + 1 refinement iteration |
| Gold-val CER / WER | **0.0953 / 0.2796** |
| Input | a single cropped text line, BGR (numpy) — resized to 48×512 |
| Output | the transcribed string (232-char frozen Cyrillic charset) |

## Intended use

Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream
of a line/region detector or on already-segmented lines.

A companion ViT-S variant (~24M params) fine-tuned with the same recipe is available at
[`Hukyl/parseq-s-rukopys`](https://huggingface.co/Hukyl/parseq-s-rukopys). The two land
in a statistical tie on the same held-out split: the ViT-S is marginally ahead on the
dominant `handwritten` class, this model a little ahead on the low-support
`annotation`/`printed` classes.

## How to use

Note: the checkpoint is a plain `torch.save` archive, not transformers-compatible. The
payload is a dict:

```python
import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("Hukyl/parseq-b-rukopys", "best.pt")
payload = torch.load(path, map_location="cpu", weights_only=True)
# keys: "model_state" (state_dict), "charset" (str), "config" (dict),
#       "metrics" (dict), "epoch" (int)
print(payload["config"])
# {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 768,
#  'enc_num_heads': 12, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12,
#  'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1,
#  'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05}
```

To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation
decoder; see [`baudm/parseq`](https://github.com/baudm/parseq), Apache-2.0) with the
embedded `config`, build a tokenizer over the embedded `charset` (token order is
`[E]` + the charset string + `[B]` + `[P]`; `[E]` is id 0), then
`model.load_state_dict(payload["model_state"])`.

The input geometry ships in the checkpoint's `config`: each crop is resized
unconditionally to 48×512 RGB (no aspect preservation; `INTER_AREA` on downscale — an
`INTER_LINEAR` mismatch aliases on the heavy downscale and degrades accuracy) and
normalized `(x − 0.5) / 0.5`. Output confidence is the mean per-step max-softmax over
the decoded sequence.

## Architecture

PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive
Sequence Models" (ECCV 2022, [arXiv:2207.06966](https://arxiv.org/abs/2207.06966)). Model
code is derived from [`baudm/parseq`](https://github.com/baudm/parseq) (Apache-2.0).
Geometry, decoder, and charset are identical to the ViT-S variant; only the encoder width
and head count are scaled up.

| component | value |
|---|---|
| encoder | ViT-B — 12 layers, dim 768, 12 heads, MLP ratio 4, patch 4×8 |
| decoder | 1-layer two-stream permutation-LM, 12 heads, MLP ratio 4 |
| training | K=6 permutations (permuted AR sequence modeling) |
| decode | autoregressive greedy + 1 refinement iteration |
| dropout / drop-path | 0.1 / 0.05 |
| input | 48×512 RGB, unconditional resize, `(x−0.5)/0.5` |
| max label length | 100 |
| parameters | ~96M |

### Charset

A frozen 232-character Cyrillic charset (`sha256 d1b9161e…3ff976`). Token IDs follow
string order; specials are `[E]` (id 0) first and `[B]`, `[P]` last. Text is not
NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and
the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits,
punctuation, and a small set of Greek/math symbols. Latin lookalikes that are
visually identical to Cyrillic letters (`a c e i o y`; `A B C E H K M O P T X`) are
deliberately excluded — the target output space is Cyrillic-normalized, so those are
folded to their Cyrillic counterparts.

## Training curriculum

Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels
longer than 100 characters before training.

1. **Mixed pretrain** —
   [`Hukyl/parseq-b-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten):
   general Cyrillic handwriting reader trained from scratch on a real + synthetic mix
   (~0.65M crops/epoch, synthetic:real ≈ 1.15:1). See the pretrain card for the full mix
   and recipe.
2. **Rukopys gold fine-tune** (15 ep, LR 5e-5 peak): warm-started from the stage-1
   weights, fine-tuned on the human-labeled Rukopys gold data, with page-level
   validation split — 19,196 train / 3,202 val crops (`handwritten` 18,414 /
   `annotation` 450 / `printed` 240 / `table` 122 in train, before charset/length
   filtering). The published weights are the best-val-CER epoch (13 of 15).

### Stage-2 hyperparameters (as launched)

| hyperparameter | value |
|---|---|
| epochs | 15 |
| effective batch size | 64 (micro-batch 64 × accum 1) |
| learning rate | 5e-5 peak |
| schedule | OneCycleLR, warmup 5% |
| optimizer | AdamW, weight_decay 0.05 |
| label smoothing | 0.1 |
| drop-path rate | 0.05 |
| permutations (K) | 6 |
| gradient clipping | 20 |
| precision | bf16 AMP |
| split | by-page (~185 held-out pages) |
| seed | 42 |

### Online data augmentation

Online augmentation was applied during training (`default` profile). Only training
crops were augmented, and every train crop was augmented (`identity_prob = 0.0`), at
the 48×512 working resolution (downscaled before augmenting).

Each crop was transformed once per epoch by _one geometric_ + _one or two photometric_ operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

| pool | handwritten / annotation | printed |
|---|---|---|
| geometric (pick 1) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–5°, elastic distortion (α25 σ5), baseline warp (amp 2–8 px, freq 0.5–2) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–3° |
| photometric (pick 1–2) | paper-colour shift, Gaussian noise σ5–15, JPEG q30–65, contrast 0.7–1.3 / gamma 0.6–1.5, morphological op (k=2) | paper-colour shift, Gaussian noise σ3–10, JPEG q40–70, contrast 0.8–1.2 / gamma 0.8–1.3, morphological op (k=2) |

## Results

Protocol: AR greedy + 1 refinement iteration, page-level gold validation split
(val crops come from pages the model never saw in training), seed 42.

### Overall

| metric | training-time best | standalone predictor eval (n=3,202) |
|---|---|---|
| CER | 0.0953 | 0.0958 |
| WER | 0.2796 | 0.2802 |
| exact-match accuracy | — | 0.4316 |

### Per class

| class | n | CER | WER | accuracy |
|---|---:|---:|---:|---:|
| handwritten | 3,067 | 0.0849 | 0.2664 | 0.4356 |
| annotation | 73 | 0.4222 | 0.5882 | 0.4932 |
| printed | 48 | 0.3279 | 0.6527 | 0.2083 |
| table | 14 | 1.1667 | 1.4706 | 0.0000 |

We also acknowledge that `printed`/`annotation`/`table` n is quite small, so measuring
CER against them is quite noisy. `table` (multi-line pipe-separated serialization) is
not a target of this model; its CER above 1 is expected.

## Limitations & biases

- Handwritten Ukrainian archival document material only; expect degradation on other
  scripts, languages, or modern born-digital text.
- The character set and normalization are tuned to the document-OCR metric — outputs are
  normalized text, not faithful transcription. Latin lookalikes are folded to their
  Cyrillic counterparts, so the model never emits the Latin forms.
- `printed` and `annotation` have small training support and high CER; `table` and
  `formula` are unsupported.
- Single seed and validation split — no across-run variance estimate.

## Training data & attribution

| dataset | source | license | role |
|---|---|---|---|
| Rukopys | [`UkrainianCatholicUniversity/rukopys`](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) | CC BY 4.0 | gold fine-tune |
| UkrHandwritten | Kaggle [`annyhnatiuk/ukrainian-handwritten-text`](https://www.kaggle.com/datasets/annyhnatiuk/ukrainian-handwritten-text) | CC BY-SA 4.0 | pretrain (lineage) |
| Cyrillic Handwriting Dataset | Kaggle [`constantinwerner/cyrillic-handwriting-dataset`](https://www.kaggle.com/datasets/constantinwerner/cyrillic-handwriting-dataset) | CC0 | pretrain (lineage) |
| school_notebooks_RU | HF [`ai-forever/school_notebooks_RU`](https://huggingface.co/datasets/ai-forever/school_notebooks_RU) | MIT | pretrain (lineage) |
| pumb-ai/synthetic-cyrillic-large | HF [`pumb-ai/synthetic-cyrillic-large`](https://huggingface.co/datasets/pumb-ai/synthetic-cyrillic-large) | Apache-2.0 | pretrain (lineage) |
| nastyboget/synthetic_cyrillic_large | HF [`nastyboget/synthetic_cyrillic_large`](https://huggingface.co/datasets/nastyboget/synthetic_cyrillic_large) | MIT | pretrain (lineage) |

## License & lineage

Model weights are released under CC BY-SA 4.0 — the most restrictive of the training
inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this
requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0
([`baudm/parseq`](https://github.com/baudm/parseq)). The stage-1
[pretrain checkpoint](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten), which
did not train on Rukopys, is released under CC BY-SA 4.0.