Initial release

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,244 @@
+---
+language:
+- uk
+library_name: pytorch
+pipeline_tag: image-to-text
+license: cc-by-nc-sa-4.0
+base_model: Hukyl/parseq-b-cyrillic-handwritten
+tags:
+- ocr
+- htr
+- handwritten-text-recognition
+- parseq
+- ukrainian
+- cyrillic
+- rukopys
+- image-to-text
+datasets:
+- UkrainianCatholicUniversity/rukopys
+- annyhnatiuk/ukrainian-handwritten-text
+- constantinwerner/cyrillic-handwriting-dataset
+- ai-forever/school_notebooks_RU
+- pumb-ai/synthetic-cyrillic-large
+- nastyboget/synthetic_cyrillic_large
+metrics:
+- cer
+- wer
+model-index:
+- name: parseq-b-rukopys
+  results:
+  - task:
+      type: image-to-text
+      name: Handwritten Text Recognition
+    dataset:
+      type: UkrainianCatholicUniversity/rukopys
+      name: Rukopys, page-level val
+      split: validation
+    metrics:
+    - type: cer
+      value: 0.0953
+      name: CER
+    - type: wer
+      value: 0.2796
+      name: WER
+---
+
+# parseq-b-rukopys
+
+A PARSeq-B (permuted autoregressive sequence) line recognizer for single-line
+handwritten Ukrainian document text, fine-tuned on the
+[Rukopys](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) dataset.
+
+The training data included `handwritten`, `printed`, `annotation` and `table` region crops from Rukopys.
+
+## TL;DR
+
+| | value |
+|---|---|
+| Architecture | PARSeq — ViT-B encoder + 1-layer permutation-LM decoder (ECCV 2022) |
+| Parameters | ~96M |
+| Init from | [`Hukyl/parseq-b-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten) (mixed pretrain) |
+| Handles | `handwritten`, `printed`, `annotation` |
+| Decode | AR greedy + 1 refinement iteration |
+| Gold-val CER / WER | **0.0953 / 0.2796** |
+| Input | a single cropped text line, BGR (numpy) — resized to 48×512 |
+| Output | the transcribed string (232-char frozen Cyrillic charset) |
+
+## Intended use
+
+Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream
+of a line/region detector or on already-segmented lines.
+
+A companion ViT-S variant (~24M params) fine-tuned with the same recipe is available at
+[`Hukyl/parseq-s-rukopys`](https://huggingface.co/Hukyl/parseq-s-rukopys). The two land
+in a statistical tie on the same held-out split: the ViT-S is marginally ahead on the
+dominant `handwritten` class, this model a little ahead on the low-support
+`annotation`/`printed` classes.
+
+## How to use
+
+Note: the checkpoint is a plain `torch.save` archive, not transformers-compatible. The
+payload is a dict:
+
+```python
+import torch
+from huggingface_hub import hf_hub_download
+
+path = hf_hub_download("Hukyl/parseq-b-rukopys", "best.pt")
+payload = torch.load(path, map_location="cpu", weights_only=True)
+# keys: "model_state" (state_dict), "charset" (str), "config" (dict),
+#       "metrics" (dict), "epoch" (int)
+print(payload["config"])
+# {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 768,
+#  'enc_num_heads': 12, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12,
+#  'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1,
+#  'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05}
+```
+
+To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation
+decoder; see [`baudm/parseq`](https://github.com/baudm/parseq), Apache-2.0) with the
+embedded `config`, build a tokenizer over the embedded `charset` (token order is
+`[E]` + the charset string + `[B]` + `[P]`; `[E]` is id 0), then
+`model.load_state_dict(payload["model_state"])`.
+
+The input geometry ships in the checkpoint's `config`: each crop is resized
+unconditionally to 48×512 RGB (no aspect preservation; `INTER_AREA` on downscale — an
+`INTER_LINEAR` mismatch aliases on the heavy downscale and degrades accuracy) and
+normalized `(x − 0.5) / 0.5`. Output confidence is the mean per-step max-softmax over
+the decoded sequence.
+
+## Architecture
+
+PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive
+Sequence Models" (ECCV 2022, [arXiv:2207.06966](https://arxiv.org/abs/2207.06966)). Model
+code is derived from [`baudm/parseq`](https://github.com/baudm/parseq) (Apache-2.0).
+Geometry, decoder, and charset are identical to the ViT-S variant; only the encoder width
+and head count are scaled up.
+
+| component | value |
+|---|---|
+| encoder | ViT-B — 12 layers, dim 768, 12 heads, MLP ratio 4, patch 4×8 |
+| decoder | 1-layer two-stream permutation-LM, 12 heads, MLP ratio 4 |
+| training | K=6 permutations (permuted AR sequence modeling) |
+| decode | autoregressive greedy + 1 refinement iteration |
+| dropout / drop-path | 0.1 / 0.05 |
+| input | 48×512 RGB, unconditional resize, `(x−0.5)/0.5` |
+| max label length | 100 |
+| parameters | ~96M |
+
+### Charset
+
+A frozen 232-character Cyrillic charset (`sha256 d1b9161e…3ff976`). Token IDs follow
+string order; specials are `[E]` (id 0) first and `[B]`, `[P]` last. Text is not
+NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and
+the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits,
+punctuation, and a small set of Greek/math symbols. Latin lookalikes that are
+visually identical to Cyrillic letters (`a c e i o y`; `A B C E H K M O P T X`) are
+deliberately excluded — the target output space is Cyrillic-normalized, so those are
+folded to their Cyrillic counterparts.
+
+## Training curriculum
+
+Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels
+longer than 100 characters before training.
+
+1. **Mixed pretrain** —
+   [`Hukyl/parseq-b-cyrillic-handwritten`](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten):
+   general Cyrillic handwriting reader trained from scratch on a real + synthetic mix
+   (~0.65M crops/epoch, synthetic:real ≈ 1.15:1). See the pretrain card for the full mix
+   and recipe.
+2. **Rukopys gold fine-tune** (15 ep, LR 5e-5 peak): warm-started from the stage-1
+   weights, fine-tuned on the human-labeled Rukopys gold data, with page-level
+   validation split — 19,196 train / 3,202 val crops (`handwritten` 18,414 /
+   `annotation` 450 / `printed` 240 / `table` 122 in train, before charset/length
+   filtering). The published weights are the best-val-CER epoch (13 of 15).
+
+### Stage-2 hyperparameters (as launched)
+
+| hyperparameter | value |
+|---|---|
+| epochs | 15 |
+| effective batch size | 64 (micro-batch 64 × accum 1) |
+| learning rate | 5e-5 peak |
+| schedule | OneCycleLR, warmup 5% |
+| optimizer | AdamW, weight_decay 0.05 |
+| label smoothing | 0.1 |
+| drop-path rate | 0.05 |
+| permutations (K) | 6 |
+| gradient clipping | 20 |
+| precision | bf16 AMP |
+| split | by-page (~185 held-out pages) |
+| seed | 42 |
+
+### Online data augmentation
+
+Online augmentation was applied during training (`default` profile). Only training
+crops were augmented, and every train crop was augmented (`identity_prob = 0.0`), at
+the 48×512 working resolution (downscaled before augmenting).
+
+Each crop was transformed once per epoch by _one geometric_ + _one or two photometric_ operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.
+
+| pool | handwritten / annotation | printed |
+|---|---|---|
+| geometric (pick 1) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–5°, elastic distortion (α25 σ5), baseline warp (amp 2–8 px, freq 0.5–2) | margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–3° |
+| photometric (pick 1–2) | paper-colour shift, Gaussian noise σ5–15, JPEG q30–65, contrast 0.7–1.3 / gamma 0.6–1.5, morphological op (k=2) | paper-colour shift, Gaussian noise σ3–10, JPEG q40–70, contrast 0.8–1.2 / gamma 0.8–1.3, morphological op (k=2) |
+
+## Results
+
+Protocol: AR greedy + 1 refinement iteration, page-level gold validation split
+(val crops come from pages the model never saw in training), seed 42.
+
+### Overall
+
+| metric | training-time best | standalone predictor eval (n=3,202) |
+|---|---|---|
+| CER | 0.0953 | 0.0958 |
+| WER | 0.2796 | 0.2802 |
+| exact-match accuracy | — | 0.4316 |
+
+### Per class
+
+| class | n | CER | WER | accuracy |
+|---|---:|---:|---:|---:|
+| handwritten | 3,067 | 0.0849 | 0.2664 | 0.4356 |
+| annotation | 73 | 0.4222 | 0.5882 | 0.4932 |
+| printed | 48 | 0.3279 | 0.6527 | 0.2083 |
+| table | 14 | 1.1667 | 1.4706 | 0.0000 |
+
+We also acknowledge that `printed`/`annotation`/`table` n is quite small, so measuring
+CER against them is quite noisy. `table` (multi-line pipe-separated serialization) is
+not a target of this model; its CER above 1 is expected.
+
+## Limitations & biases
+
+- Handwritten Ukrainian archival document material only; expect degradation on other
+  scripts, languages, or modern born-digital text.
+- The character set and normalization are tuned to the document-OCR metric — outputs are
+  normalized text, not faithful transcription. Latin lookalikes are folded to their
+  Cyrillic counterparts, so the model never emits the Latin forms.
+- `printed` and `annotation` have small training support and high CER; `table` and
+  `formula` are unsupported.
+- Single seed and validation split — no across-run variance estimate.
+
+## Training data & attribution
+
+| dataset | source | license | role |
+|---|---|---|---|
+| Rukopys | [`UkrainianCatholicUniversity/rukopys`](https://huggingface.co/datasets/UkrainianCatholicUniversity/rukopys) | CC BY-NC-SA 4.0 (subject to change, verify with the source repository) | gold fine-tune |
+| UkrHandwritten | Kaggle [`annyhnatiuk/ukrainian-handwritten-text`](https://www.kaggle.com/datasets/annyhnatiuk/ukrainian-handwritten-text) | CC BY-SA 4.0 | pretrain (lineage) |
+| Cyrillic Handwriting Dataset | Kaggle [`constantinwerner/cyrillic-handwriting-dataset`](https://www.kaggle.com/datasets/constantinwerner/cyrillic-handwriting-dataset) | CC0 | pretrain (lineage) |
+| school_notebooks_RU | HF [`ai-forever/school_notebooks_RU`](https://huggingface.co/datasets/ai-forever/school_notebooks_RU) | MIT | pretrain (lineage) |
+| pumb-ai/synthetic-cyrillic-large | HF [`pumb-ai/synthetic-cyrillic-large`](https://huggingface.co/datasets/pumb-ai/synthetic-cyrillic-large) | Apache-2.0 | pretrain (lineage) |
+| nastyboget/synthetic_cyrillic_large | HF [`nastyboget/synthetic_cyrillic_large`](https://huggingface.co/datasets/nastyboget/synthetic_cyrillic_large) | MIT | pretrain (lineage) |
+
+**Attribution is mandatory** for UkrHandwritten (CC BY-SA 4.0, share-alike) — please keep
+this credit if you redistribute or build on this model.
+
+## License & lineage
+
+Model weights are released under CC BY-NC-SA 4.0 — the most restrictive of the training
+inputs (Rukopys is non-commercial as of time of publishing this model; UkrHandwritten is
+share-alike). The PARSeq model code is Apache-2.0
+([`baudm/parseq`](https://github.com/baudm/parseq)). The stage-1
+[pretrain checkpoint](https://huggingface.co/Hukyl/parseq-b-cyrillic-handwritten), which
+did not train on Rukopys, is released under CC BY-SA 4.0.

best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02ff30dc409494afdd5c7b0305243ce2c02ca0dd2502dfb6c48e3ec2c6650afd
+size 382495362

training_meta.jsonl

@@ -0,0 +1,11 @@
+{"event": "run_start", "at": "2026-06-29 17:37:12 UTC", "recipe": {"epochs": 15, "batch_size": 64, "accum_steps": 1, "effective_batch": 64, "learning_rate": 5e-05, "warmup_pct": 0.05, "weight_decay": 0.05, "label_smoothing": 0.1, "drop_path_rate": 0.05, "swa_epoch_start": 0.75, "grad_clip": 20.0, "perm_num": 6, "augment": true, "augment_profile": "default", "augment_identity_prob": 0.0, "seed": 42}, "data": {"n_train": 19196, "n_val": 3202, "dropped_unencodable_train": 30, "dropped_unencodable_val": 7}, "pretrained": "models/parseq_pretrain_b_main/best.pt"}
+{"event": "best", "at": "2026-06-29 17:40:29 UTC", "epoch": 0, "swa": false, "metrics": {"cer": 0.1484086642903845, "wer": 0.3925978281822271, "accuracy": 0.32104934415990005, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 17:42:39 UTC", "epoch": 1, "swa": false, "metrics": {"cer": 0.1254719169847932, "wer": 0.3521172010537814, "accuracy": 0.3644597126795753, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 17:47:13 UTC", "epoch": 3, "swa": false, "metrics": {"cer": 0.11891253453694134, "wer": 0.32243140782625457, "accuracy": 0.40037476577139286, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 17:49:29 UTC", "epoch": 4, "swa": false, "metrics": {"cer": 0.11165714044671293, "wer": 0.3153633618197006, "accuracy": 0.40349781386633354, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 17:58:09 UTC", "epoch": 6, "swa": false, "metrics": {"cer": 0.10672178516440639, "wer": 0.2990426010409304, "accuracy": 0.41817613991255465, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 18:03:26 UTC", "epoch": 7, "swa": false, "metrics": {"cer": 0.10239807647691561, "wer": 0.2926813596350318, "accuracy": 0.42192379762648347, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 18:08:36 UTC", "epoch": 8, "swa": false, "metrics": {"cer": 0.10132242212051547, "wer": 0.28940435648653856, "accuracy": 0.4287945034353529, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 18:13:36 UTC", "epoch": 9, "swa": false, "metrics": {"cer": 0.09831691730116213, "wer": 0.28554905866478186, "accuracy": 0.4306683322923173, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 18:22:27 UTC", "epoch": 11, "swa": false, "metrics": {"cer": 0.09736781051610317, "wer": 0.2812439760971535, "accuracy": 0.4306683322923173, "n_samples": 3202.0}}
+{"event": "best", "at": "2026-06-29 18:32:35 UTC", "epoch": 13, "swa": false, "metrics": {"cer": 0.09525868432708329, "wer": 0.2795733470410589, "accuracy": 0.43191755153029354, "n_samples": 3202.0}}