Replace with private no-Bible SPM8k best checkpoint (20260712-232900)

Browse files

Files changed (10) hide show

.gitattributes +1 -34
README.md +89 -71
added_tokens.json +152 -80
config.json +1 -1
eval/metrics.json +432 -147
experiment_metadata.json +181 -2
model.safetensors +2 -2
sentencepiece.bpe.model +2 -2
special_tokens_map.json +73 -1
tokenizer_config.json +728 -80

.gitattributes CHANGED Viewed

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text


























1	*.safetensors filter=lfs diff=lfs merge=lfs -text
2	+ *.model filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -27,10 +27,8 @@ tags:
 - low-resource
 - endangered-languages
 - formosan-languages
-- seq2seq
-- encoder-decoder
 - sentencepiece
-- traditional-chinese
 metrics:
 - bleu
 - chrf2
@@ -42,37 +40,40 @@ model-index:
       name: Machine Translation
       type: translation
     dataset:
-      name: FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split
       type: custom
     metrics:
     - name: BLEU
       type: bleu
-      value: 7.65
       args:
-        direction: zho_Hant-to-formosan
-        samples: 37435
     - name: chrF2
       type: chrf2
-      value: 32.97
       args:
-        direction: zho_Hant-to-formosan
-        samples: 37435
     - name: TER
       type: ter
-      value: 87.98
       args:
-        direction: zho_Hant-to-formosan
-        samples: 37435
 ---
 # nllb200-zh-formosan-spm8k
-**Repo:** `FormosanBank/nllb200-zh-formosan-spm8k`
 **Base model:** [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M)
 **Direction:** **Traditional Chinese -> Formosan**
-**Companion reverse-direction model:** [`FormosanBank/nllb200-formosan-zh-spm8k`](https://huggingface.co/FormosanBank/nllb200-formosan-zh-spm8k)
-This is a directional NLLB-200 distilled 600M checkpoint for FormosanBank machine translation. It uses an 8k SentencePiece vocabulary extension plus FormosanBank metadata/control tags. Use the companion model for the reverse direction.
 ## Supported Languages
@@ -97,21 +98,21 @@ This is a directional NLLB-200 distilled 600M checkpoint for FormosanBank machin
 ## Input Format
-This model was trained and evaluated with metadata control tags. Prefix the Traditional Chinese source text with:
 `<to_LANG> <src_zh> <dom_BUCKET> <dialect_DIALECT>`
-Example with unknown metadata:
 `<to_ami> <src_zh> <dom_unknown> <dialect_default> 他回家了。`
-If source bucket or dialect is unknown, use `<dom_unknown>` and `<dialect_default>`. If you know the training source bucket or dialect, using the matching tag may improve quality.
 ## Usage
-Tested with `transformers` 4.56.x. Use the slow `NllbTokenizer` or `AutoTokenizer.from_pretrained(model_id, use_fast=False)`. In `transformers` 4.56.x, fast-tokenizer added-token IDs can differ from the slow tokenizer IDs used during training.
-For NLLB generation, keep `decoder_start_token_id=tokenizer.eos_token_id` and set `forced_bos_token_id` to the target language ID.
 ```python
 import torch
@@ -141,91 +142,108 @@ def translate_chinese_to_formosan(text: str, lang_code: str, source_bucket: str
         num_beams=4,
         no_repeat_ngram_size=3,
         repetition_penalty=1.15,
-        length_penalty=1.0,
         early_stopping=True,
     )
     return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
-print(translate_chinese_to_formosan("他回家了。", "ami"))
 ```
 ## Training Setup
 | Setting | Value |
 |---|---|
-| Corpus | FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split |
-| Direction | `zh2f` |
 | Base model | `facebook/nllb-200-distilled-600M` |
-| Tokenizer | 8k Formosan SentencePiece extension |
-| Steps | 300,000 |
-| Batch size | 16 |
-| Gradient accumulation | 4 |
-| Effective batch size | 64 |
-| Max sequence length | 384 |
-| Learning rate | 2e-05 |
-| Warmup steps | 4,000 |
 | Precision | `bf16` |
-| Label smoothing | 0.1 |
 | Easy-source weight | 0.15 |
 | Language sampling alpha | 0.5 |
-| Metadata tags | enabled and validated as single tokenizer IDs |
-This repo publishes the final 300k-step checkpoint because it scored highest on the held-out hard test set among the evaluated final/best checkpoints.
-## Evaluation
-Evaluation used the held-out `in_domain_hard` test split with no normalized source, target, or pair overlap against train. These scores are intentionally lower than leaky or near-duplicate splits and are intended as a harder MT benchmark.
-### Global Metrics
-| Direction | Samples | BLEU | chrF2 | TER |
-|---|---:|---:|---:|---:|
-| Traditional Chinese -> Formosan | 37,435 | 7.65 | 32.97 | 87.98 |
-### Per-Language Metrics
 | Language | Code | Samples | BLEU | chrF2 | TER |
 |---|---:|---:|---:|---:|---:|
-| Amis | `ami_Latn` | 4,866 | 5.85 | 28.07 | 85.97 |
-| Bunun | `bnn_Latn` | 3,223 | 4.85 | 31.73 | 92.54 |
-| Kavalan | `ckv_Latn` | 1,832 | 14.53 | 41.54 | 77.16 |
-| Rukai | `dru_Latn` | 3,846 | 3.25 | 28.74 | 102.38 |
-| Paiwan | `pwn_Latn` | 3,221 | 5.31 | 31.40 | 90.05 |
-| Puyuma | `pyu_Latn` | 2,225 | 13.75 | 40.31 | 80.22 |
-| Thao | `ssf_Latn` | 1,180 | 13.46 | 42.40 | 76.31 |
-| Saaroa | `sxr_Latn` | 1,106 | 3.43 | 39.05 | 89.96 |
-| Sakizaya | `szy_Latn` | 1,601 | 11.26 | 36.01 | 80.32 |
-| Tao / Yami | `tao_Latn` | 1,455 | 5.26 | 27.91 | 91.29 |
-| Atayal | `tay_Latn` | 4,293 | 3.99 | 26.34 | 92.71 |
-| Seediq | `trv_Latn` | 4,573 | 7.65 | 32.00 | 85.36 |
-| Tsou | `tsu_Latn` | 1,250 | 6.72 | 32.61 | 88.12 |
-| Kanakanavu | `xnb_Latn` | 1,552 | 10.14 | 42.70 | 88.79 |
-| Saisiyat | `xsy_Latn` | 1,212 | 19.42 | 47.95 | 76.22 |
-Full source-bucket and length-bin breakdowns are available in [`eval/metrics.json`](eval/metrics.json).
 ## Intended Use
-- Research, teaching, and prototyping for Formosan-language MT.
-- Draft translation assistance where review by knowledgeable speakers is available.
-- Comparative evaluation of low-resource MT methods on leakage-controlled FormosanBank splits.
 ## Limitations
-- Outputs can be incorrect, ungrammatical, incomplete, or culturally inappropriate.
-- Generation into Formosan languages is especially difficult and should be treated as draft-only.
-- This model is not suitable for legal, medical, safety-critical, or authoritative community-facing use without expert review.
-- Evaluation uses a hard split; BLEU should not be compared directly to older leaky or near-duplicate split results.
 ## License
-Released under `cc-by-nc-4.0`. Some underlying corpus sources may carry additional restrictions. Use this model only for non-commercial research and educational purposes unless you have confirmed broader rights for your use case.
 ## Citation
 ```bibtex
-@misc{formosanbank_nllb200_zh_formosan_spm8k,
-  title  = {nllb200-zh-formosan-spm8k: Directional NLLB-200 MT for the FormosanBank Chinese Parallel Corpus, leakage-controlled in-domain hard split},
   author = {FormosanBank contributors},
   year   = {2026},
   url    = {https://huggingface.co/FormosanBank/nllb200-zh-formosan-spm8k}

 - low-resource
 - endangered-languages
 - formosan-languages
 - sentencepiece
+- private-no-bible
 metrics:
 - bleu
 - chrf2
       name: Machine Translation
       type: translation
     dataset:
+      name: FormosanBank Formosan private no-Bible hard split
       type: custom
     metrics:
     - name: BLEU
       type: bleu
+      value: 10.2695
       args:
+        direction: zh2f
+        samples: 61384
+        tokenize: 13a
     - name: chrF2
       type: chrf2
+      value: 34.8012
       args:
+        direction: zh2f
+        samples: 61384
     - name: TER
       type: ter
+      value: 94.5143
       args:
+        direction: zh2f
+        samples: 61384
 ---
 # nllb200-zh-formosan-spm8k
 **Base model:** [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M)
 **Direction:** **Traditional Chinese -> Formosan**
+**Companion model:** [`FormosanBank/nllb200-formosan-zh-spm8k`](https://huggingface.co/FormosanBank/nllb200-formosan-zh-spm8k)
+**Release:** private no-Bible SPM8k flight `20260712-232900`, validation-selected step `280,000`
+This directional checkpoint replaces the earlier release with the strongest `private_no_bible` model from the
+fully rebuilt FormosanBank MT pipeline. It uses an 8,192-piece Formosan-aware SentencePiece extension and explicit
+direction, source-language, source-domain, and dialect control tags.
 ## Supported Languages
 ## Input Format
+Prefix every source with:
 `<to_LANG> <src_zh> <dom_BUCKET> <dialect_DIALECT>`
+Example:
 `<to_ami> <src_zh> <dom_unknown> <dialect_default> 他回家了。`
+Use `<dom_unknown>` and `<dialect_default>` when metadata is unavailable.
 ## Usage
+Use the slow `NllbTokenizer` (`use_fast=False` with `AutoTokenizer`). These checkpoints were trained with
+`transformers==4.56.1`; fast-tokenizer added-token IDs can differ from the slow tokenizer IDs used in training.
+NLLB generation must start with the tokenizer EOS ID and force the target-language BOS ID.
 ```python
 import torch
         num_beams=4,
         no_repeat_ngram_size=3,
         repetition_penalty=1.15,
         early_stopping=True,
     )
     return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
+print(translate_chinese_to_formosan('他回家了。', "ami"))
 ```
+## Checkpoint Selection
+The published checkpoint was selected **only on validation chrF2**, not on the hard test set.
+| Selection step | Validation samples | Validation loss | Perplexity | BLEU | chrF2 | TER |
+|---:|---:|---:|---:|---:|---:|---:|
+| 280,000 | 1,920 | 2.1317 | 8.43 | 24.44 | 48.63 | 72.20 |
+Validation generation sampled 128 rows per Formosan language every 10,000 updates. The full hard test was evaluated
+only after selection.
 ## Training Setup
 | Setting | Value |
 |---|---|
+| Corpus | `private_no_bible` (Formosan) |
 | Base model | `facebook/nllb-200-distilled-600M` |
+| Maximum updates | 300,000 |
+| Published best step | 280,000 |
+| Microbatch / accumulation | 16 / 4 |
+| Effective batch | 64 |
+| Maximum length | 384 |
+| Learning rate | `2e-05` |
 | Precision | `bf16` |
 | Easy-source weight | 0.15 |
 | Language sampling alpha | 0.5 |
+| Metadata control tags | enabled and validated as single tokenizer IDs |
+## Corpus and Split Integrity
+| Total | Train | Test | Validate | Minimum per-language test | Minimum per-language validate |
+|---:|---:|---:|---:|---:|---:|
+| 791,330 | 708,496 | 61,384 | 21,450 | 7.5% | 2.5% |
+The exact `Formosan-Taiwan-Bible-Society-Bibles` repository is excluded. Lexical entries are train-only. Independent
+validation found zero normalized source, target, or pair overlap; zero punctuation/spacing skeleton overlap; and zero
+one-edit source or target conflicts across train and evaluation. Connected similarity groups are assigned as units so
+variants cannot be split independently merely because they are not exact duplicates.
+All hard-test references in this Chinese corpus are original rather than pivot-generated.
+## Hard-Test Results
+SacreBLEU was computed with `13a` tokenization; chrF uses beta 2; TER is lower-is-better.
+| Direction | Samples | BLEU | chrF2 | TER | Exact match | Empty output |
+|---|---:|---:|---:|---:|---:|---:|
+| Traditional Chinese -> Formosan | 61,384 | 10.27 | 34.80 | 94.51 | 1.26% | 0.00% |
+### Per-Language Results
 | Language | Code | Samples | BLEU | chrF2 | TER |
 |---|---:|---:|---:|---:|---:|
+| Amis | `ami_Latn` | 10,929 | 6.91 | 28.42 | 97.95 |
+| Bunun | `bnn_Latn` | 5,450 | 6.49 | 34.18 | 104.26 |
+| Kavalan | `ckv_Latn` | 2,852 | 18.21 | 43.34 | 77.16 |
+| Rukai | `dru_Latn` | 5,524 | 5.69 | 30.35 | 113.71 |
+| Paiwan | `pwn_Latn` | 5,519 | 7.18 | 33.70 | 96.16 |
+| Puyuma | `pyu_Latn` | 4,105 | 15.54 | 41.39 | 79.80 |
+| Thao | `ssf_Latn` | 1,670 | 18.37 | 47.50 | 73.34 |
+| Saaroa | `sxr_Latn` | 1,539 | 6.46 | 40.77 | 99.50 |
+| Sakizaya | `szy_Latn` | 2,147 | 15.04 | 39.50 | 85.09 |
+| Tao / Yami | `tao_Latn` | 1,802 | 9.50 | 32.56 | 88.69 |
+| Atayal | `tay_Latn` | 6,395 | 7.47 | 29.23 | 101.57 |
+| Seediq | `trv_Latn` | 6,689 | 16.10 | 39.05 | 80.46 |
+| Tsou | `tsu_Latn` | 1,889 | 6.82 | 29.62 | 99.82 |
+| Kanakanavu | `xnb_Latn` | 2,927 | 15.02 | 47.16 | 86.58 |
+| Saisiyat | `xsy_Latn` | 1,947 | 16.88 | 43.04 | 84.33 |
+Full source-bucket and length-bin breakdowns are in [`eval/metrics.json`](eval/metrics.json).
 ## Intended Use
+- Research, teaching, and prototyping for Formosan-language machine translation.
+- Draft translation assistance where knowledgeable speakers can review the output.
+- Comparative low-resource MT evaluation on the documented leakage-controlled split.
 ## Limitations
+- Output may be incorrect, ungrammatical, incomplete, or culturally inappropriate.
+- Formosan generation is draft-only and requires speaker review.
+- Aggregate scores across 15 languages conceal substantial per-language variation.
+- This model is unsuitable for legal, medical, safety-critical, or authoritative community-facing use without expert review.
+- Hard-split scores are not directly comparable with earlier evaluations that allowed stronger train-test similarity.
 ## License
+Released under `cc-by-nc-4.0`. Underlying corpus sources may impose additional restrictions. Confirm the rights needed
+for your use case.
 ## Citation
 ```bibtex
+@misc{formosanbank_nllb200_zh_formosan_spm8k_2026,
+  title  = {nllb200-zh-formosan-spm8k: Directional NLLB-200 MT on the FormosanBank private no-Bible corpus},
   author = {FormosanBank contributors},
   year   = {2026},
   url    = {https://huggingface.co/FormosanBank/nllb200-zh-formosan-spm8k}

added_tokens.json CHANGED Viewed

@@ -12,86 +12,158 @@
   "<dialect_hengchun>": 259649,
   "<dialect_jianhe>": 259650,
   "<dialect_junqun>": 259651,
-  "<dialect_kaqun>": 259652,
-  "<dialect_luanqun>": 259653,
-  "<dialect_malan>": 259654,
-  "<dialect_maolin>": 259655,
-  "<dialect_nanwang>": 259656,
-  "<dialect_northern>": 259657,
-  "<dialect_sekolik>": 259658,
-  "<dialect_southern>": 259659,
-  "<dialect_tanqun>": 259660,
-  "<dialect_tegudaya>": 259661,
-  "<dialect_truku>": 259662,
-  "<dialect_unknown>": 259663,
-  "<dialect_wanda>": 259664,
-  "<dialect_wanshan>": 259665,
-  "<dialect_wenshui>": 259666,
-  "<dialect_wutai>": 259667,
-  "<dialect_xiqun>": 259668,
-  "<dialect_xiuguluan>": 259669,
-  "<dialect_yilan_zeaol>": 259670,
-  "<dialect_zeaol>": 259671,
-  "<dialect_zhiben>": 259672,
-  "<dialect_zhuoqun>": 259673,
-  "<dom_classroom_context>": 259674,
-  "<dom_culture>": 259675,
-  "<dom_dictionary>": 259676,
-  "<dom_essays>": 259677,
-  "<dom_formosan_100_paiwan_texts>": 259678,
-  "<dom_formosan_amis_myths_and_customs>": 259679,
-  "<dom_formosan_epark>": 259680,
-  "<dom_formosan_gitbook_translations>": 259681,
-  "<dom_formosan_old_texts>": 259682,
-  "<dom_formosan_paiwanstories>": 259683,
-  "<dom_formosan_rik_bunun>": 259684,
-  "<dom_formosan_seals>": 259685,
-  "<dom_formosan_wilang_yutas_videos>": 259686,
-  "<dom_formosan_yeddas_blog>": 259687,
-  "<dom_formosan_zheng_data>": 259688,
-  "<dom_learning_vocab>": 259689,
-  "<dom_nine_level>": 259690,
-  "<dom_ntu>": 259691,
-  "<dom_picture_book>": 259692,
-  "<dom_picture_story>": 259693,
-  "<dom_presidential_apology>": 259694,
-  "<dom_reading_writing>": 259695,
-  "<dom_unknown>": 259696,
-  "<dom_youtube>": 259697,
-  "<src_ami>": 259698,
-  "<src_bnn>": 259699,
-  "<src_ckv>": 259700,
-  "<src_dru>": 259701,
-  "<src_eng>": 259702,
-  "<src_pwn>": 259703,
-  "<src_pyu>": 259704,
-  "<src_ssf>": 259705,
-  "<src_sxr>": 259706,
-  "<src_szy>": 259707,
-  "<src_tao>": 259708,
-  "<src_tay>": 259709,
-  "<src_trv>": 259710,
-  "<src_tsu>": 259711,
-  "<src_xnb>": 259712,
-  "<src_xsy>": 259713,
-  "<src_zh>": 259714,
-  "<to_ami>": 259715,
-  "<to_bnn>": 259716,
-  "<to_ckv>": 259717,
-  "<to_dru>": 259718,
-  "<to_eng>": 259719,
-  "<to_pwn>": 259720,
-  "<to_pyu>": 259721,
-  "<to_ssf>": 259722,
-  "<to_sxr>": 259723,
-  "<to_szy>": 259724,
-  "<to_tao>": 259725,
-  "<to_tay>": 259726,
-  "<to_trv>": 259727,
-  "<to_tsu>": 259728,
-  "<to_xnb>": 259729,
-  "<to_xsy>": 259730,
-  "<to_zh>": 259731,
   "ami_Latn": 259624,
   "bnn_Latn": 259625,
   "ckv_Latn": 259626,

   "<dialect_hengchun>": 259649,
   "<dialect_jianhe>": 259650,
   "<dialect_junqun>": 259651,
+  "<dialect_kanakanavu>": 259652,
+  "<dialect_kaqun>": 259653,
+  "<dialect_kavalan>": 259654,
+  "<dialect_luanqun>": 259655,
+  "<dialect_malan>": 259656,
+  "<dialect_maolin>": 259657,
+  "<dialect_nanwang>": 259658,
+  "<dialect_northern>": 259659,
+  "<dialect_saaroa>": 259660,
+  "<dialect_saisiyat>": 259661,
+  "<dialect_sakizaya>": 259662,
+  "<dialect_sekolik>": 259663,
+  "<dialect_southern>": 259664,
+  "<dialect_tanqun>": 259665,
+  "<dialect_tegudaya>": 259666,
+  "<dialect_thao>": 259667,
+  "<dialect_truku>": 259668,
+  "<dialect_tsou>": 259669,
+  "<dialect_unknown>": 259670,
+  "<dialect_wanda>": 259671,
+  "<dialect_wanshan>": 259672,
+  "<dialect_wenshui>": 259673,
+  "<dialect_wutai>": 259674,
+  "<dialect_xiqun>": 259675,
+  "<dialect_xiuguluan>": 259676,
+  "<dialect_yami>": 259677,
+  "<dialect_yilanzeaol>": 259678,
+  "<dialect_zeaol>": 259679,
+  "<dialect_zhiben>": 259680,
+  "<dialect_zhuoqun>": 259681,
+  "<dom_classroom_context>": 259682,
+  "<dom_culture>": 259683,
+  "<dom_dictionary>": 259684,
+  "<dom_essays>": 259685,
+  "<dom_ethnography_journal_ritual_song>": 259686,
+  "<dom_formosan_100_paiwan_texts>": 259687,
+  "<dom_formosan_academia_sinica_oral_legends>": 259688,
+  "<dom_formosan_amis_adversative_constructions>": 259689,
+  "<dom_formosan_amis_kavalan_lin_interrogative_verbs>": 259690,
+  "<dom_formosan_amis_kuo_sung_comparative_constructions>": 259691,
+  "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>": 259692,
+  "<dom_formosan_amis_myths_and_customs>": 259693,
+  "<dom_formosan_amis_pa_verbs>": 259694,
+  "<dom_formosan_amis_serial_verb_constructions>": 259695,
+  "<dom_formosan_amis_tung_chiou>": 259696,
+  "<dom_formosan_asai_sedik_language>": 259697,
+  "<dom_formosan_bunun_debusser_dissertation>": 259698,
+  "<dom_formosan_bunun_moriguchi_northern_texts>": 259699,
+  "<dom_formosan_bunun_topic_focus>": 259700,
+  "<dom_formosan_chen_fukuda_relabeling_ergative>": 259701,
+  "<dom_formosan_chen_fukuda_three_ways_steal>": 259702,
+  "<dom_formosan_cip_atayal_grammar_overview>": 259703,
+  "<dom_formosan_dean_johnson_year_clouds_smangus>": 259704,
+  "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>": 259705,
+  "<dom_formosan_egerod_origin_headhunting_atayal_text_v>": 259706,
+  "<dom_formosan_egerod_soren_word_order_word_classes_at>": 259707,
+  "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>": 259708,
+  "<dom_formosan_epark>": 259709,
+  "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>": 259710,
+  "<dom_formosan_gitbook_translations>": 259711,
+  "<dom_formosan_glosbe>": 259712,
+  "<dom_formosan_hala_saku_la_videos>": 259713,
+  "<dom_formosan_holmer_seediq>": 259714,
+  "<dom_formosan_huang_grammaticalization_squliq_atayal>": 259715,
+  "<dom_formosan_huang_mei_chin_atayal_reference_grammar>": 259716,
+  "<dom_formosan_huteson_rukai_survey>": 259717,
+  "<dom_formosan_ilrdf_42_language_practice_word_lists>": 259718,
+  "<dom_formosan_ilrdf_tousvusvutu_kita>": 259719,
+  "<dom_formosan_indigenous_language_literary_awards>": 259720,
+  "<dom_formosan_kanakanavu_texts>": 259721,
+  "<dom_formosan_kavalan_zhang_reference_grammar>": 259722,
+  "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>": 259723,
+  "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>": 259724,
+  "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>": 259725,
+  "<dom_formosan_nowbucyang_truku_thesis>": 259726,
+  "<dom_formosan_old_texts>": 259727,
+  "<dom_formosan_paiwan_collart_zeitoun_time_reference>": 259728,
+  "<dom_formosan_paiwan_ho_five_dialects>": 259729,
+  "<dom_formosan_paiwanstories>": 259730,
+  "<dom_formosan_puyuma_chen_raising_to_object>": 259731,
+  "<dom_formosan_puyuma_katipol_kopfjagdriten>": 259732,
+  "<dom_formosan_puyuma_teng_pronominal_systems>": 259733,
+  "<dom_formosan_puyuma_teng_reference_grammar>": 259734,
+  "<dom_formosan_rik_bunun>": 259735,
+  "<dom_formosan_robert_blust_austronesian_homeland_ling>": 259736,
+  "<dom_formosan_rukai_mantauran_stories>": 259737,
+  "<dom_formosan_rukai_texts>": 259738,
+  "<dom_formosan_rukai_zeitoun_saying>": 259739,
+  "<dom_formosan_saaroa_pan_grammar>": 259740,
+  "<dom_formosan_sakizaya_affixes>": 259741,
+  "<dom_formosan_seals>": 259742,
+  "<dom_formosan_seediq_zhang_reference_grammar>": 259743,
+  "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>": 259744,
+  "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>": 259745,
+  "<dom_formosan_shih_rukai_adverbial>": 259746,
+  "<dom_formosan_song_limei_introduction_kanakanavu_gram>": 259747,
+  "<dom_formosan_song_limei_introduction_seediq_grammar>": 259748,
+  "<dom_formosan_sung_kuo_descriptive_comparative_constr>": 259749,
+  "<dom_formosan_taoshan_elementary_school_atelier_hui_k>": 259750,
+  "<dom_formosan_tsukida_naomi_verb_classification_amis>": 259751,
+  "<dom_formosan_wei_matri_clan_lineage_system_ami>": 259752,
+  "<dom_formosan_wilang_yutas_videos>": 259753,
+  "<dom_formosan_wu_jinglan_introduction_amis_grammar>": 259754,
+  "<dom_formosan_yedda_palemeq_blog>": 259755,
+  "<dom_formosan_yeddas_blog>": 259756,
+  "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>": 259757,
+  "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>": 259758,
+  "<dom_formosan_zheng_acl_2024>": 259759,
+  "<dom_formosan_zheng_data>": 259760,
+  "<dom_learning_vocab>": 259761,
+  "<dom_nine_level>": 259762,
+  "<dom_ntu>": 259763,
+  "<dom_picture_book>": 259764,
+  "<dom_picture_story>": 259765,
+  "<dom_presidential_apology>": 259766,
+  "<dom_reading_writing>": 259767,
+  "<dom_unknown>": 259768,
+  "<dom_youtube>": 259769,
+  "<src_ami>": 259770,
+  "<src_bnn>": 259771,
+  "<src_ckv>": 259772,
+  "<src_dru>": 259773,
+  "<src_eng>": 259774,
+  "<src_pwn>": 259775,
+  "<src_pyu>": 259776,
+  "<src_ssf>": 259777,
+  "<src_sxr>": 259778,
+  "<src_szy>": 259779,
+  "<src_tao>": 259780,
+  "<src_tay>": 259781,
+  "<src_trv>": 259782,
+  "<src_tsu>": 259783,
+  "<src_xnb>": 259784,
+  "<src_xsy>": 259785,
+  "<src_zh>": 259786,
+  "<to_ami>": 259787,
+  "<to_bnn>": 259788,
+  "<to_ckv>": 259789,
+  "<to_dru>": 259790,
+  "<to_eng>": 259791,
+  "<to_pwn>": 259792,
+  "<to_pyu>": 259793,
+  "<to_ssf>": 259794,
+  "<to_sxr>": 259795,
+  "<to_szy>": 259796,
+  "<to_tao>": 259797,
+  "<to_tay>": 259798,
+  "<to_trv>": 259799,
+  "<to_tsu>": 259800,
+  "<to_xnb>": 259801,
+  "<to_xsy>": 259802,
+  "<to_zh>": 259803,
   "ami_Latn": 259624,
   "bnn_Latn": 259625,
   "ckv_Latn": 259626,

config.json CHANGED Viewed

@@ -30,5 +30,5 @@
   "tokenizer_class": "NllbTokenizer",
   "transformers_version": "4.56.1",
   "use_cache": true,
-  "vocab_size": 259732
 }

   "tokenizer_class": "NllbTokenizer",
   "transformers_version": "4.56.1",
   "use_cache": true,
+  "vocab_size": 259804
 }

eval/metrics.json CHANGED Viewed

@@ -1,226 +1,511 @@
 {
-  "input": "/projects/prudlab/formosan_parallel_corpora/splits_zh_v1/big_corpus_zh_in_domain_hard.csv",
-  "model": "/scratch/scheppat/formosan_mt_experiments/runs/E3_zh_spm8192_zh2f_zh_e3_20260510-155741/final",
-  "tokenizer": "/scratch/scheppat/formosan_mt_experiments/runs/E3_zh_spm8192_zh2f_zh_e3_20260510-155741/final",
   "direction": "zh2f",
   "target_lang": "chinese",
   "bleu_tokenize": "13a",
   "split": "test",
-  "samples": 37435,
   "global": {
-    "samples": 37435,
-    "BLEU": 7.652193453569897,
-    "chrF2": 32.97297712876464,
-    "TER": 87.98405562530415
   },
   "by_language": {
     "ami": {
-      "samples": 4866,
-      "BLEU": 5.84540680774625,
-      "chrF2": 28.072578419194517,
-      "TER": 85.97331869802001
     },
     "bnn": {
-      "samples": 3223,
-      "BLEU": 4.850985489907292,
-      "chrF2": 31.730366666066452,
-      "TER": 92.54404079946845
     },
     "ckv": {
-      "samples": 1832,
-      "BLEU": 14.52547298960598,
-      "chrF2": 41.53672271796304,
-      "TER": 77.16032969954799
     },
     "dru": {
-      "samples": 3846,
-      "BLEU": 3.2491162079573686,
-      "chrF2": 28.74023026757287,
-      "TER": 102.37862089839228
     },
     "pwn": {
-      "samples": 3221,
-      "BLEU": 5.305268641610943,
-      "chrF2": 31.395820634893433,
-      "TER": 90.05029836485244
     },
     "pyu": {
-      "samples": 2225,
-      "BLEU": 13.751020013857485,
-      "chrF2": 40.30967111418056,
-      "TER": 80.21649071228117
     },
     "ssf": {
-      "samples": 1180,
-      "BLEU": 13.460302756430439,
-      "chrF2": 42.398830379459845,
-      "TER": 76.3105721794458
     },
     "sxr": {
-      "samples": 1106,
-      "BLEU": 3.433529257944264,
-      "chrF2": 39.045847228092555,
-      "TER": 89.96400885935769
     },
     "szy": {
-      "samples": 1601,
-      "BLEU": 11.260487828269696,
-      "chrF2": 36.00751672151733,
-      "TER": 80.32247840900088
     },
     "tao": {
-      "samples": 1455,
-      "BLEU": 5.258605253466159,
-      "chrF2": 27.91242470897236,
-      "TER": 91.28852886661633
     },
     "tay": {
-      "samples": 4293,
-      "BLEU": 3.9893651763565776,
-      "chrF2": 26.340099651164067,
-      "TER": 92.70688604509445
     },
     "trv": {
-      "samples": 4573,
-      "BLEU": 7.647862205076352,
-      "chrF2": 31.998910277431598,
-      "TER": 85.3630845323741
     },
     "tsu": {
-      "samples": 1250,
-      "BLEU": 6.720204057896507,
-      "chrF2": 32.608312845920054,
-      "TER": 88.1207400194742
     },
     "xnb": {
-      "samples": 1552,
-      "BLEU": 10.136067985712732,
-      "chrF2": 42.69617349413197,
-      "TER": 88.78644186863364
     },
     "xsy": {
-      "samples": 1212,
-      "BLEU": 19.424245721904754,
-      "chrF2": 47.94818066374329,
-      "TER": 76.21970412338685
     }
   },
   "by_source_bucket": {
-    "Formosan-100_Paiwan_Texts": {
-      "samples": 422,
-      "BLEU": 6.852722936501872,
-      "chrF2": 32.5173799179018,
-      "TER": 84.23453869550524
     },
     "Formosan-Amis_myths_and_customs": {
-      "samples": 134,
-      "BLEU": 0.030783257342743113,
-      "chrF2": 7.438008292582795,
-      "TER": 96.29357412851743
     },
-    "Formosan-Old_Texts": {
-      "samples": 8,
-      "BLEU": 2.3789349974698735,
-      "chrF2": 16.73099213437003,
-      "TER": 92.10526315789474
     },
     "Formosan-PaiwanStories": {
-      "samples": 25,
-      "BLEU": 6.364032562359571,
-      "chrF2": 36.3650204213955,
-      "TER": 88.18443804034582
     },
-    "Formosan-Rik-Bunun": {
-      "samples": 225,
-      "BLEU": 1.1611959557367952,
-      "chrF2": 24.856224153617756,
-      "TER": 126.25641025641026
     },
     "Formosan-ePark": {
-      "samples": 241,
-      "BLEU": 17.8326893294227,
-      "chrF2": 48.26834614454279,
-      "TER": 71.1353711790393
     },
     "Formosan-gitbook_translations": {
-      "samples": 93,
-      "BLEU": 1.482864943095601,
-      "chrF2": 23.1313626286465,
-      "TER": 99.78165938864629
     },
     "culture": {
-      "samples": 9919,
-      "BLEU": 4.451461322333989,
-      "chrF2": 28.409535627028248,
-      "TER": 94.71638117846915
     },
     "essays": {
-      "samples": 818,
-      "BLEU": 10.370413512682743,
-      "chrF2": 37.06987087738533,
-      "TER": 82.39480198019803
     },
     "ntu": {
-      "samples": 1950,
-      "BLEU": 6.433802979228347,
-      "chrF2": 34.27186862144693,
-      "TER": 88.42186001917545
     },
     "picture_book": {
-      "samples": 17068,
-      "BLEU": 10.403096509345584,
-      "chrF2": 37.304410333200934,
-      "TER": 82.29113411371016
     },
     "picture_story": {
-      "samples": 558,
-      "BLEU": 8.056200691853272,
-      "chrF2": 33.65109267847052,
-      "TER": 93.69682042095835
     },
     "presidential_apology": {
-      "samples": 8,
-      "BLEU": 1.2132889961495774,
-      "chrF2": 19.02163887475657,
-      "TER": 93.62549800796812
     },
     "reading_writing": {
-      "samples": 3711,
-      "BLEU": 4.920772609247823,
-      "chrF2": 29.180093695384617,
-      "TER": 91.65965197952171
     },
     "youtube": {
-      "samples": 2255,
-      "BLEU": 4.4758437455597635,
-      "chrF2": 29.503154176368263,
-      "TER": 91.7412225984673
     }
   },
   "by_length_bin": {
     "004_008": {
-      "samples": 34,
-      "BLEU": 23.42926032850921,
-      "chrF2": 30.392525327220852,
-      "TER": 75.0
     },
     "009_016": {
-      "samples": 4159,
-      "BLEU": 14.085925983912663,
-      "chrF2": 38.77465206586661,
-      "TER": 80.80641771176347
     },
     "017_032": {
-      "samples": 18195,
-      "BLEU": 9.486185839623138,
-      "chrF2": 35.44435282339664,
-      "TER": 85.63883109115568
     },
     "033_plus": {
-      "samples": 15047,
-      "BLEU": 6.117776190120631,
-      "chrF2": 31.12252110558518,
-      "TER": 89.94558308787086
     }
   }
 }

 {
+  "input": "/projects/prudlab/formosan_parallel_corpora/private_no_bible/big_corpus_zh_in_domain_hard.csv",
+  "model": "/scratch/scheppat/projects/mt/formosan_mt_experiments/runs/private_no_bible/v1_spm8192_zh2f_20260712-232900/best",
+  "tokenizer": "/scratch/scheppat/projects/mt/formosan_mt_experiments/runs/private_no_bible/v1_spm8192_zh2f_20260712-232900/best",
   "direction": "zh2f",
   "target_lang": "chinese",
   "bleu_tokenize": "13a",
   "split": "test",
+  "samples": 61384,
   "global": {
+    "samples": 61384,
+    "BLEU": 10.269549124268107,
+    "chrF2": 34.80124484040179,
+    "TER": 94.5142968923905,
+    "exact_match_rate": 0.012560276293496677,
+    "empty_output_rate": 0.0,
+    "character_length_ratio": 0.9653470961019327
   },
   "by_language": {
     "ami": {
+      "samples": 10929,
+      "BLEU": 6.914105723821887,
+      "chrF2": 28.422028983681123,
+      "TER": 97.95431020783315,
+      "exact_match_rate": 0.006587976942080703,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8677828251859125
     },
     "bnn": {
+      "samples": 5450,
+      "BLEU": 6.4919986563896686,
+      "chrF2": 34.17883492772349,
+      "TER": 104.25526132214785,
+      "exact_match_rate": 0.011926605504587157,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.0408969931468297
     },
     "ckv": {
+      "samples": 2852,
+      "BLEU": 18.205512848950875,
+      "chrF2": 43.33952537429594,
+      "TER": 77.16231925437971,
+      "exact_match_rate": 0.01367461430575035,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9093549956425719
     },
     "dru": {
+      "samples": 5524,
+      "BLEU": 5.692925020608219,
+      "chrF2": 30.35012001876045,
+      "TER": 113.70911555016845,
+      "exact_match_rate": 0.00832729905865315,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.103744308481323
     },
     "pwn": {
+      "samples": 5519,
+      "BLEU": 7.180838288745003,
+      "chrF2": 33.700085898393795,
+      "TER": 96.1613289434301,
+      "exact_match_rate": 0.006885305308932777,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9690585598979807
     },
     "pyu": {
+      "samples": 4105,
+      "BLEU": 15.53892230178134,
+      "chrF2": 41.387939333762176,
+      "TER": 79.79833144761218,
+      "exact_match_rate": 0.025091352009744213,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8805288894009868
     },
     "ssf": {
+      "samples": 1670,
+      "BLEU": 18.366449370165274,
+      "chrF2": 47.497924920922635,
+      "TER": 73.34358799065687,
+      "exact_match_rate": 0.010778443113772455,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.944087575611584
     },
     "sxr": {
+      "samples": 1539,
+      "BLEU": 6.458556035844864,
+      "chrF2": 40.76560285082243,
+      "TER": 99.49654145871641,
+      "exact_match_rate": 0.007147498375568551,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9759231536926147
     },
     "szy": {
+      "samples": 2147,
+      "BLEU": 15.040878188244656,
+      "chrF2": 39.495927333260504,
+      "TER": 85.09006784331055,
+      "exact_match_rate": 0.014904517931998137,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9423002163284534
     },
     "tao": {
+      "samples": 1802,
+      "BLEU": 9.499002338764912,
+      "chrF2": 32.563121155209934,
+      "TER": 88.68606978370428,
+      "exact_match_rate": 0.013873473917869035,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9169190007570023
     },
     "tay": {
+      "samples": 6395,
+      "BLEU": 7.472319192536347,
+      "chrF2": 29.227466942314855,
+      "TER": 101.56598631049232,
+      "exact_match_rate": 0.01751368256450352,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.999633488664328
     },
     "trv": {
+      "samples": 6689,
+      "BLEU": 16.103440471350194,
+      "chrF2": 39.05221955096421,
+      "TER": 80.45892092740272,
+      "exact_match_rate": 0.02167738077440574,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9643189020950927
     },
     "tsu": {
+      "samples": 1889,
+      "BLEU": 6.817369914830053,
+      "chrF2": 29.61931373534734,
+      "TER": 99.81588140874327,
+      "exact_match_rate": 0.005293806246691371,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9630728921458253
     },
     "xnb": {
+      "samples": 2927,
+      "BLEU": 15.018793112717322,
+      "chrF2": 47.157035146575886,
+      "TER": 86.5811048436822,
+      "exact_match_rate": 0.009224461906388794,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.0146939099700165
     },
     "xsy": {
+      "samples": 1947,
+      "BLEU": 16.882563257190064,
+      "chrF2": 43.04271422508787,
+      "TER": 84.33496301867869,
+      "exact_match_rate": 0.014381099126861838,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.993871421314939
     }
   },
   "by_source_bucket": {
+    "Formosan-Academia-Sinica-Oral-Legends": {
+      "samples": 4,
+      "BLEU": 1.2801994542197097,
+      "chrF2": 16.29668103738638,
+      "TER": 89.47368421052632,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9149560117302052
+    },
+    "Formosan-Amis-Lifok-Dongi-Saopo-Kimakimad": {
+      "samples": 1604,
+      "BLEU": 1.3784710187917888,
+      "chrF2": 20.884469552418732,
+      "TER": 141.77648279363396,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.2087005939314248
     },
     "Formosan-Amis_myths_and_customs": {
+      "samples": 7,
+      "BLEU": 1.343710991599318,
+      "chrF2": 13.90105068795303,
+      "TER": 98.60869565217392,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7148619298789948
+    },
+    "Formosan-CIP-Atayal-Grammar-Overview": {
+      "samples": 56,
+      "BLEU": 15.39533996941775,
+      "chrF2": 39.44618944280046,
+      "TER": 72.28381374722838,
+      "exact_match_rate": 0.017857142857142856,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8308300395256917
+    },
+    "Formosan-Glosbe": {
+      "samples": 21,
+      "BLEU": 9.675968025856529,
+      "chrF2": 31.718065989698136,
+      "TER": 81.08882521489971,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.6264501160092807
+    },
+    "Formosan-Hala-Saku-La-Videos": {
+      "samples": 24,
+      "BLEU": 1.0282372177324197,
+      "chrF2": 27.178522016860168,
+      "TER": 138.53881278538813,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.4759275237273513
+    },
+    "Formosan-Huang-Mei-Chin-Atayal-Reference-Grammar": {
+      "samples": 2,
+      "BLEU": 21.7860828319701,
+      "chrF2": 52.47414724044989,
+      "TER": 89.65517241379311,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.092783505154639
+    },
+    "Formosan-ILRDF-Tousvusvutu-Kita": {
+      "samples": 738,
+      "BLEU": 2.3231388539110887,
+      "chrF2": 19.667599166799192,
+      "TER": 115.64985820017984,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9953549408431928
     },
+    "Formosan-Indigenous-Language-Literary-Awards": {
+      "samples": 3206,
+      "BLEU": 1.983167905621269,
+      "chrF2": 19.884952088031383,
+      "TER": 107.73831810693426,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8384160333333859
+    },
+    "Formosan-Kavalan-Zhang-Reference-Grammar": {
+      "samples": 2,
+      "BLEU": 8.130850857597448,
+      "chrF2": 31.031495303049144,
+      "TER": 100.0,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8721804511278195
+    },
+    "Formosan-Li-Peirong-Xu-Weisheng-Introduction-Truku-Grammar": {
+      "samples": 122,
+      "BLEU": 28.112629844538947,
+      "chrF2": 48.126316885271926,
+      "TER": 58.2441113490364,
+      "exact_match_rate": 0.04918032786885246,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.890506329113924
+    },
+    "Formosan-Li-Peizhen-Wu-Xingyu-Shouhu-Xing": {
+      "samples": 6,
+      "BLEU": 3.3789807625348294,
+      "chrF2": 32.44045982819115,
+      "TER": 89.28571428571429,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9090909090909091
+    },
+    "Formosan-Nowbucyang-Truku-Thesis": {
+      "samples": 64,
+      "BLEU": 42.62571991441162,
+      "chrF2": 58.13053672376524,
+      "TER": 43.91025641025641,
+      "exact_match_rate": 0.125,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9693063228974831
     },
     "Formosan-PaiwanStories": {
+      "samples": 18,
+      "BLEU": 12.146582094620106,
+      "chrF2": 40.0512658462955,
+      "TER": 80.69620253164557,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8833833833833834
+    },
+    "Formosan-SEALS": {
+      "samples": 16,
+      "BLEU": 8.415910137810393,
+      "chrF2": 28.26571271281777,
+      "TER": 94.46640316205533,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9056399132321041
     },
+    "Formosan-Seediq-Zhang-Reference-Grammar": {
+      "samples": 15,
+      "BLEU": 21.64207242929072,
+      "chrF2": 41.25886173203152,
+      "TER": 60.273972602739725,
+      "exact_match_rate": 0.06666666666666667,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8900804289544236
+    },
+    "Formosan-Song-Limei-Introduction-Kanakanavu-Grammar": {
+      "samples": 104,
+      "BLEU": 20.018421319303723,
+      "chrF2": 50.65882123564126,
+      "TER": 68.50828729281768,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9215198863636364
+    },
+    "Formosan-Song-Limei-Introduction-Seediq-Grammar": {
+      "samples": 107,
+      "BLEU": 28.438670039370407,
+      "chrF2": 51.16975909850582,
+      "TER": 58.333333333333336,
+      "exact_match_rate": 0.018691588785046728,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9173922276491824
+    },
+    "Formosan-Wilang-Yutas-Videos": {
+      "samples": 10,
+      "BLEU": 4.995748309281416,
+      "chrF2": 32.644127699351905,
+      "TER": 82.91139240506328,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8839634941329857
+    },
+    "Formosan-Wu-Jinglan-Introduction-Amis-Grammar": {
+      "samples": 34,
+      "BLEU": 15.285400748361376,
+      "chrF2": 39.77005031327309,
+      "TER": 75.98116169544741,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7380820399113082
+    },
+    "Formosan-Yeh-Meili-Introduction-Saisiyat-Grammar": {
+      "samples": 4,
+      "BLEU": 0.877898977318005,
+      "chrF2": 22.47130237887143,
+      "TER": 102.38095238095238,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7777777777777778
+    },
+    "Formosan-Zhang-Yongli-Pan-Jiarong-Introduction-Tsou-Grammar": {
+      "samples": 37,
+      "BLEU": 6.111045446859278,
+      "chrF2": 29.522344112831167,
+      "TER": 89.0909090909091,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8224852071005917
+    },
+    "Formosan-Zheng-ACL-2024": {
+      "samples": 4947,
+      "BLEU": 23.163784254220417,
+      "chrF2": 48.483492242805234,
+      "TER": 66.79567779960706,
+      "exact_match_rate": 0.005255710531635335,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9584476447850032
+    },
+    "Formosan-amis_tung-chiou": {
+      "samples": 4,
+      "BLEU": 1.1215459083717743,
+      "chrF2": 16.02261289031762,
+      "TER": 90.64102564102564,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.5716259788116076
     },
     "Formosan-ePark": {
+      "samples": 5491,
+      "BLEU": 32.47106791414506,
+      "chrF2": 58.39562052063364,
+      "TER": 57.561408700799056,
+      "exact_match_rate": 0.11018029502822801,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9401507204770054
     },
     "Formosan-gitbook_translations": {
+      "samples": 62,
+      "BLEU": 1.5895915408872159,
+      "chrF2": 31.81322281402857,
+      "TER": 99.625468164794,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.0295886689949554
     },
     "culture": {
+      "samples": 11335,
+      "BLEU": 5.64921902638139,
+      "chrF2": 30.383705325120484,
+      "TER": 107.6751912434003,
+      "exact_match_rate": 8.82223202470225e-05,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.0670105445747935
     },
     "essays": {
+      "samples": 3176,
+      "BLEU": 12.877681630343817,
+      "chrF2": 41.98631899635104,
+      "TER": 77.95772921812252,
+      "exact_match_rate": 0.001889168765743073,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9003324918416353
+    },
+    "nine_level": {
+      "samples": 1933,
+      "BLEU": 23.220082610030552,
+      "chrF2": 49.667606043905195,
+      "TER": 67.21720586233495,
+      "exact_match_rate": 0.03362648732540093,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9305996044923697
     },
     "ntu": {
+      "samples": 2884,
+      "BLEU": 6.929226380572684,
+      "chrF2": 37.1771599765339,
+      "TER": 109.3261774270575,
+      "exact_match_rate": 0.0013869625520110957,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.001182910515598
     },
     "picture_book": {
+      "samples": 13023,
+      "BLEU": 13.724197275501723,
+      "chrF2": 41.82467517188842,
+      "TER": 78.72949078056818,
+      "exact_match_rate": 0.0026875527912155416,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8899614244283685
     },
     "picture_story": {
+      "samples": 729,
+      "BLEU": 6.2739916591958655,
+      "chrF2": 32.04857787789198,
+      "TER": 105.21101644541311,
+      "exact_match_rate": 0.0013717421124828531,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.0036009031454376
     },
     "presidential_apology": {
+      "samples": 482,
+      "BLEU": 2.9841897823113888,
+      "chrF2": 25.89628040282464,
+      "TER": 110.05302078973071,
+      "exact_match_rate": 0.0,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 1.009524562096668
     },
     "reading_writing": {
+      "samples": 4793,
+      "BLEU": 5.929647858822887,
+      "chrF2": 31.108854216943634,
+      "TER": 101.48612562405665,
+      "exact_match_rate": 0.00020863759649488838,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9925077107233663
     },
     "youtube": {
+      "samples": 6324,
+      "BLEU": 4.914640291500347,
+      "chrF2": 29.507051949569323,
+      "TER": 101.41015310233684,
+      "exact_match_rate": 0.0014231499051233396,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9698602852268398
     }
   },
   "by_length_bin": {
     "004_008": {
+      "samples": 50,
+      "BLEU": 35.03760315459789,
+      "chrF2": 47.55915870255983,
+      "TER": 59.43775100401606,
+      "exact_match_rate": 0.16,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.7219343696027634
     },
     "009_016": {
+      "samples": 5361,
+      "BLEU": 29.913254287347616,
+      "chrF2": 50.441436043101604,
+      "TER": 62.52098489087857,
+      "exact_match_rate": 0.11639619473978735,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.8506984387838948
     },
     "017_032": {
+      "samples": 30498,
+      "BLEU": 13.925419993339279,
+      "chrF2": 38.9375221907002,
+      "TER": 85.30979526456771,
+      "exact_match_rate": 0.004459308807134894,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9186720120687659
     },
     "033_plus": {
+      "samples": 25475,
+      "BLEU": 6.913351604601034,
+      "chrF2": 31.754674128650123,
+      "TER": 101.97612519293382,
+      "exact_match_rate": 0.00011776251226692835,
+      "empty_output_rate": 0.0,
+      "character_length_ratio": 0.9989336873138698
     }
   }
 }

experiment_metadata.json CHANGED Viewed

@@ -1,4 +1,183 @@
 {
-  "step": 300000,
-  "direction": "zh2f"
 }

 {
+  "step": 280000,
+  "best_metric": "chrF2",
+  "best_value": 48.63318542236297,
+  "direction": "zh2f",
+  "validation": {
+    "mean_token_loss": 2.1317479789684692,
+    "ppl": 8.42958868466291,
+    "by_language": {
+      "ami": 2.269358691668898,
+      "bnn": 2.104894006520871,
+      "ckv": 1.8107160835177238,
+      "dru": 2.2205682444297405,
+      "pwn": 2.2337666226988415,
+      "pyu": 1.551884051591749,
+      "ssf": 1.7170341627330012,
+      "sxr": 2.331028452326917,
+      "szy": 2.1995507054022494,
+      "tao": 2.877571459555958,
+      "tay": 1.7397732444987097,
+      "trv": 2.778555995799422,
+      "tsu": 2.2668690758533803,
+      "xnb": 1.363018649051961,
+      "xsy": 1.9556310469665925
+    },
+    "generation": {
+      "samples": 1920,
+      "bleu_tokenize": "13a",
+      "global": {
+        "BLEU": 24.44267743637224,
+        "chrF2": 48.63318542236297,
+        "TER": 72.19686356035612,
+        "exact_match_rate": 0.0328125,
+        "empty_output_rate": 0.0,
+        "character_length_ratio": 0.9440213020983477
+      },
+      "by_language": {
+        "ami": {
+          "samples": 128,
+          "BLEU": 17.85512263123292,
+          "chrF2": 39.98571203917058,
+          "TER": 72.768192048012,
+          "exact_match_rate": 0.015625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.8103996680956991
+        },
+        "bnn": {
+          "samples": 128,
+          "BLEU": 22.645666096272024,
+          "chrF2": 51.080349342786036,
+          "TER": 72.78688524590164,
+          "exact_match_rate": 0.0390625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9546792984252976
+        },
+        "ckv": {
+          "samples": 128,
+          "BLEU": 33.66880755635196,
+          "chrF2": 56.68776294516569,
+          "TER": 58.133971291866025,
+          "exact_match_rate": 0.046875,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9225160829163689
+        },
+        "dru": {
+          "samples": 128,
+          "BLEU": 22.3007660121,
+          "chrF2": 48.207123404656315,
+          "TER": 86.4500792393027,
+          "exact_match_rate": 0.0390625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 1.0123957367933272
+        },
+        "pwn": {
+          "samples": 128,
+          "BLEU": 14.708541368128696,
+          "chrF2": 41.18716454434891,
+          "TER": 83.75254928619987,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9558323895809739
+        },
+        "pyu": {
+          "samples": 128,
+          "BLEU": 37.34728596407374,
+          "chrF2": 57.47543271792994,
+          "TER": 54.82130815913688,
+          "exact_match_rate": 0.125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9251677076104557
+        },
+        "ssf": {
+          "samples": 128,
+          "BLEU": 31.010144775751396,
+          "chrF2": 55.935953143945284,
+          "TER": 60.111576011157595,
+          "exact_match_rate": 0.0390625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9231113097898972
+        },
+        "sxr": {
+          "samples": 128,
+          "BLEU": 14.363923538208141,
+          "chrF2": 51.044451393604305,
+          "TER": 83.9382940108893,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9517633186836746
+        },
+        "szy": {
+          "samples": 128,
+          "BLEU": 25.298077472587014,
+          "chrF2": 49.53596050102026,
+          "TER": 67.36059479553903,
+          "exact_match_rate": 0.0234375,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9305140059831384
+        },
+        "tao": {
+          "samples": 128,
+          "BLEU": 24.445549580912907,
+          "chrF2": 43.47864106568798,
+          "TER": 71.18119975262832,
+          "exact_match_rate": 0.03125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9397546459370825
+        },
+        "tay": {
+          "samples": 128,
+          "BLEU": 27.905381873235783,
+          "chrF2": 50.36756388999408,
+          "TER": 62.61620185922975,
+          "exact_match_rate": 0.0390625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9146141215106732
+        },
+        "trv": {
+          "samples": 128,
+          "BLEU": 16.827752065519654,
+          "chrF2": 38.42296089938853,
+          "TER": 80.38574697613599,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9287296898079763
+        },
+        "tsu": {
+          "samples": 128,
+          "BLEU": 15.659400453652273,
+          "chrF2": 41.39653446689282,
+          "TER": 83.96846254927726,
+          "exact_match_rate": 0.0078125,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9500116522955022
+        },
+        "xnb": {
+          "samples": 128,
+          "BLEU": 31.783554264342545,
+          "chrF2": 60.21115612912942,
+          "TER": 60.189165950128974,
+          "exact_match_rate": 0.0390625,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 0.9772643024836278
+        },
+        "xsy": {
+          "samples": 128,
+          "BLEU": 28.905155126822713,
+          "chrF2": 52.77532696227354,
+          "TER": 75.81120943952803,
+          "exact_match_rate": 0.0234375,
+          "empty_output_rate": 0.0,
+          "character_length_ratio": 1.0457130114840005
+        }
+      }
+    },
+    "step": 280000,
+    "selection": {
+      "metric": "chrF2",
+      "value": 48.63318542236297,
+      "improved": true,
+      "best_value_before_eval": 48.441275571052344
+    }
+  }
 }

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1828c36daebb01b7f4aa431daa0e71725864b432814e691425e7a6d2411c3739
-size 2474797408

 version https://git-lfs.github.com/spec/v1
+oid sha256:a3cffe37ae7ee3aa89ed883bcb5ee85cf8aa4ebb69a74e119461643c78c32753
+size 2475092320

sentencepiece.bpe.model CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5450fbcd0f7a01bc2a8d6f39edcb102752660081bfd686b9e46d76a399a31b3f
-size 4900258

 version https://git-lfs.github.com/spec/v1
+oid sha256:0571a653d72d4d5a892f302084363ae86de93f9da19521a9bd3498b60819b960
+size 4898297

special_tokens_map.json CHANGED Viewed

@@ -231,17 +231,24 @@
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
     "<dialect_kaqun>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
     "<dialect_truku>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
@@ -249,7 +256,8 @@
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
-    "<dialect_yilan_zeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
@@ -257,16 +265,80 @@
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
     "<dom_formosan_100_paiwan_texts>",
     "<dom_formosan_amis_myths_and_customs>",
     "<dom_formosan_epark>",
     "<dom_formosan_gitbook_translations>",
     "<dom_formosan_old_texts>",
     "<dom_formosan_paiwanstories>",
     "<dom_formosan_rik_bunun>",
     "<dom_formosan_seals>",
     "<dom_formosan_wilang_yutas_videos>",
     "<dom_formosan_yeddas_blog>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
+    "<dialect_kanakanavu>",
     "<dialect_kaqun>",
+    "<dialect_kavalan>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
+    "<dialect_saaroa>",
+    "<dialect_saisiyat>",
+    "<dialect_sakizaya>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
+    "<dialect_thao>",
     "<dialect_truku>",
+    "<dialect_tsou>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
+    "<dialect_yami>",
+    "<dialect_yilanzeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
+    "<dom_ethnography_journal_ritual_song>",
     "<dom_formosan_100_paiwan_texts>",
+    "<dom_formosan_academia_sinica_oral_legends>",
+    "<dom_formosan_amis_adversative_constructions>",
+    "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
+    "<dom_formosan_amis_kuo_sung_comparative_constructions>",
+    "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
     "<dom_formosan_amis_myths_and_customs>",
+    "<dom_formosan_amis_pa_verbs>",
+    "<dom_formosan_amis_serial_verb_constructions>",
+    "<dom_formosan_amis_tung_chiou>",
+    "<dom_formosan_asai_sedik_language>",
+    "<dom_formosan_bunun_debusser_dissertation>",
+    "<dom_formosan_bunun_moriguchi_northern_texts>",
+    "<dom_formosan_bunun_topic_focus>",
+    "<dom_formosan_chen_fukuda_relabeling_ergative>",
+    "<dom_formosan_chen_fukuda_three_ways_steal>",
+    "<dom_formosan_cip_atayal_grammar_overview>",
+    "<dom_formosan_dean_johnson_year_clouds_smangus>",
+    "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
+    "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
+    "<dom_formosan_egerod_soren_word_order_word_classes_at>",
+    "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
     "<dom_formosan_epark>",
+    "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
     "<dom_formosan_gitbook_translations>",
+    "<dom_formosan_glosbe>",
+    "<dom_formosan_hala_saku_la_videos>",
+    "<dom_formosan_holmer_seediq>",
+    "<dom_formosan_huang_grammaticalization_squliq_atayal>",
+    "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
+    "<dom_formosan_huteson_rukai_survey>",
+    "<dom_formosan_ilrdf_42_language_practice_word_lists>",
+    "<dom_formosan_ilrdf_tousvusvutu_kita>",
+    "<dom_formosan_indigenous_language_literary_awards>",
+    "<dom_formosan_kanakanavu_texts>",
+    "<dom_formosan_kavalan_zhang_reference_grammar>",
+    "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
+    "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
+    "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
+    "<dom_formosan_nowbucyang_truku_thesis>",
     "<dom_formosan_old_texts>",
+    "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
+    "<dom_formosan_paiwan_ho_five_dialects>",
     "<dom_formosan_paiwanstories>",
+    "<dom_formosan_puyuma_chen_raising_to_object>",
+    "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+    "<dom_formosan_puyuma_teng_pronominal_systems>",
+    "<dom_formosan_puyuma_teng_reference_grammar>",
     "<dom_formosan_rik_bunun>",
+    "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+    "<dom_formosan_rukai_mantauran_stories>",
+    "<dom_formosan_rukai_texts>",
+    "<dom_formosan_rukai_zeitoun_saying>",
+    "<dom_formosan_saaroa_pan_grammar>",
+    "<dom_formosan_sakizaya_affixes>",
     "<dom_formosan_seals>",
+    "<dom_formosan_seediq_zhang_reference_grammar>",
+    "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+    "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+    "<dom_formosan_shih_rukai_adverbial>",
+    "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+    "<dom_formosan_song_limei_introduction_seediq_grammar>",
+    "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+    "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+    "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+    "<dom_formosan_wei_matri_clan_lineage_system_ami>",
     "<dom_formosan_wilang_yutas_videos>",
+    "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+    "<dom_formosan_yedda_palemeq_blog>",
     "<dom_formosan_yeddas_blog>",
+    "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+    "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+    "<dom_formosan_zheng_acl_2024>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

tokenizer_config.json CHANGED Viewed

@@ -1881,7 +1881,7 @@
       "special": true
     },
     "259652": {
-      "content": "<dialect_kaqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1889,7 +1889,7 @@
       "special": true
     },
     "259653": {
-      "content": "<dialect_luanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1897,7 +1897,7 @@
       "special": true
     },
     "259654": {
-      "content": "<dialect_malan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1905,7 +1905,7 @@
       "special": true
     },
     "259655": {
-      "content": "<dialect_maolin>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1913,7 +1913,7 @@
       "special": true
     },
     "259656": {
-      "content": "<dialect_nanwang>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1921,7 +1921,7 @@
       "special": true
     },
     "259657": {
-      "content": "<dialect_northern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1929,7 +1929,7 @@
       "special": true
     },
     "259658": {
-      "content": "<dialect_sekolik>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1937,7 +1937,7 @@
       "special": true
     },
     "259659": {
-      "content": "<dialect_southern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1945,7 +1945,7 @@
       "special": true
     },
     "259660": {
-      "content": "<dialect_tanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1953,7 +1953,7 @@
       "special": true
     },
     "259661": {
-      "content": "<dialect_tegudaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1961,7 +1961,7 @@
       "special": true
     },
     "259662": {
-      "content": "<dialect_truku>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1969,7 +1969,7 @@
       "special": true
     },
     "259663": {
-      "content": "<dialect_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1977,7 +1977,7 @@
       "special": true
     },
     "259664": {
-      "content": "<dialect_wanda>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1985,7 +1985,7 @@
       "special": true
     },
     "259665": {
-      "content": "<dialect_wanshan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -1993,7 +1993,7 @@
       "special": true
     },
     "259666": {
-      "content": "<dialect_wenshui>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2001,7 +2001,7 @@
       "special": true
     },
     "259667": {
-      "content": "<dialect_wutai>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2009,7 +2009,7 @@
       "special": true
     },
     "259668": {
-      "content": "<dialect_xiqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2017,7 +2017,7 @@
       "special": true
     },
     "259669": {
-      "content": "<dialect_xiuguluan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2025,7 +2025,7 @@
       "special": true
     },
     "259670": {
-      "content": "<dialect_yilan_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2033,7 +2033,7 @@
       "special": true
     },
     "259671": {
-      "content": "<dialect_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2041,7 +2041,7 @@
       "special": true
     },
     "259672": {
-      "content": "<dialect_zhiben>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2049,7 +2049,7 @@
       "special": true
     },
     "259673": {
-      "content": "<dialect_zhuoqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2057,7 +2057,7 @@
       "special": true
     },
     "259674": {
-      "content": "<dom_classroom_context>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2065,7 +2065,7 @@
       "special": true
     },
     "259675": {
-      "content": "<dom_culture>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2073,7 +2073,7 @@
       "special": true
     },
     "259676": {
-      "content": "<dom_dictionary>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2081,7 +2081,7 @@
       "special": true
     },
     "259677": {
-      "content": "<dom_essays>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2089,7 +2089,7 @@
       "special": true
     },
     "259678": {
-      "content": "<dom_formosan_100_paiwan_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2097,7 +2097,7 @@
       "special": true
     },
     "259679": {
-      "content": "<dom_formosan_amis_myths_and_customs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2105,7 +2105,7 @@
       "special": true
     },
     "259680": {
-      "content": "<dom_formosan_epark>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2113,7 +2113,7 @@
       "special": true
     },
     "259681": {
-      "content": "<dom_formosan_gitbook_translations>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2121,7 +2121,7 @@
       "special": true
     },
     "259682": {
-      "content": "<dom_formosan_old_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2129,7 +2129,7 @@
       "special": true
     },
     "259683": {
-      "content": "<dom_formosan_paiwanstories>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2137,7 +2137,7 @@
       "special": true
     },
     "259684": {
-      "content": "<dom_formosan_rik_bunun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2145,7 +2145,7 @@
       "special": true
     },
     "259685": {
-      "content": "<dom_formosan_seals>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2153,7 +2153,7 @@
       "special": true
     },
     "259686": {
-      "content": "<dom_formosan_wilang_yutas_videos>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2161,7 +2161,7 @@
       "special": true
     },
     "259687": {
-      "content": "<dom_formosan_yeddas_blog>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2169,7 +2169,7 @@
       "special": true
     },
     "259688": {
-      "content": "<dom_formosan_zheng_data>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2177,7 +2177,7 @@
       "special": true
     },
     "259689": {
-      "content": "<dom_learning_vocab>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2185,7 +2185,7 @@
       "special": true
     },
     "259690": {
-      "content": "<dom_nine_level>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2193,7 +2193,7 @@
       "special": true
     },
     "259691": {
-      "content": "<dom_ntu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2201,7 +2201,7 @@
       "special": true
     },
     "259692": {
-      "content": "<dom_picture_book>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2209,7 +2209,7 @@
       "special": true
     },
     "259693": {
-      "content": "<dom_picture_story>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2217,7 +2217,7 @@
       "special": true
     },
     "259694": {
-      "content": "<dom_presidential_apology>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2225,7 +2225,7 @@
       "special": true
     },
     "259695": {
-      "content": "<dom_reading_writing>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2233,7 +2233,7 @@
       "special": true
     },
     "259696": {
-      "content": "<dom_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2241,7 +2241,7 @@
       "special": true
     },
     "259697": {
-      "content": "<dom_youtube>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2249,7 +2249,7 @@
       "special": true
     },
     "259698": {
-      "content": "<src_ami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2257,7 +2257,7 @@
       "special": true
     },
     "259699": {
-      "content": "<src_bnn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2265,7 +2265,7 @@
       "special": true
     },
     "259700": {
-      "content": "<src_ckv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2273,7 +2273,7 @@
       "special": true
     },
     "259701": {
-      "content": "<src_dru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2281,7 +2281,7 @@
       "special": true
     },
     "259702": {
-      "content": "<src_eng>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2289,7 +2289,7 @@
       "special": true
     },
     "259703": {
-      "content": "<src_pwn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2297,7 +2297,7 @@
       "special": true
     },
     "259704": {
-      "content": "<src_pyu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2305,7 +2305,7 @@
       "special": true
     },
     "259705": {
-      "content": "<src_ssf>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2313,7 +2313,7 @@
       "special": true
     },
     "259706": {
-      "content": "<src_sxr>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2321,7 +2321,7 @@
       "special": true
     },
     "259707": {
-      "content": "<src_szy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2329,7 +2329,7 @@
       "special": true
     },
     "259708": {
-      "content": "<src_tao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2337,7 +2337,7 @@
       "special": true
     },
     "259709": {
-      "content": "<src_tay>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2345,7 +2345,7 @@
       "special": true
     },
     "259710": {
-      "content": "<src_trv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2353,7 +2353,7 @@
       "special": true
     },
     "259711": {
-      "content": "<src_tsu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2361,7 +2361,7 @@
       "special": true
     },
     "259712": {
-      "content": "<src_xnb>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2369,7 +2369,7 @@
       "special": true
     },
     "259713": {
-      "content": "<src_xsy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2377,7 +2377,7 @@
       "special": true
     },
     "259714": {
-      "content": "<src_zh>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2385,7 +2385,7 @@
       "special": true
     },
     "259715": {
-      "content": "<to_ami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2393,7 +2393,7 @@
       "special": true
     },
     "259716": {
-      "content": "<to_bnn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2401,7 +2401,7 @@
       "special": true
     },
     "259717": {
-      "content": "<to_ckv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2409,7 +2409,7 @@
       "special": true
     },
     "259718": {
-      "content": "<to_dru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2417,7 +2417,7 @@
       "special": true
     },
     "259719": {
-      "content": "<to_eng>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2425,7 +2425,7 @@
       "special": true
     },
     "259720": {
-      "content": "<to_pwn>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2433,7 +2433,7 @@
       "special": true
     },
     "259721": {
-      "content": "<to_pyu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2441,7 +2441,7 @@
       "special": true
     },
     "259722": {
-      "content": "<to_ssf>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2449,7 +2449,7 @@
       "special": true
     },
     "259723": {
-      "content": "<to_sxr>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2457,7 +2457,7 @@
       "special": true
     },
     "259724": {
-      "content": "<to_szy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2465,7 +2465,7 @@
       "special": true
     },
     "259725": {
-      "content": "<to_tao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2473,7 +2473,7 @@
       "special": true
     },
     "259726": {
-      "content": "<to_tay>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2481,7 +2481,7 @@
       "special": true
     },
     "259727": {
-      "content": "<to_trv>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2489,7 +2489,7 @@
       "special": true
     },
     "259728": {
-      "content": "<to_tsu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2497,7 +2497,7 @@
       "special": true
     },
     "259729": {
-      "content": "<to_xnb>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2505,7 +2505,7 @@
       "special": true
     },
     "259730": {
-      "content": "<to_xsy>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
@@ -2513,6 +2513,582 @@
       "special": true
     },
     "259731": {
       "content": "<to_zh>",
       "lstrip": false,
       "normalized": false,
@@ -2753,17 +3329,24 @@
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
     "<dialect_kaqun>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
     "<dialect_truku>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
@@ -2771,7 +3354,8 @@
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
-    "<dialect_yilan_zeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
@@ -2779,16 +3363,80 @@
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
     "<dom_formosan_100_paiwan_texts>",
     "<dom_formosan_amis_myths_and_customs>",
     "<dom_formosan_epark>",
     "<dom_formosan_gitbook_translations>",
     "<dom_formosan_old_texts>",
     "<dom_formosan_paiwanstories>",
     "<dom_formosan_rik_bunun>",
     "<dom_formosan_seals>",
     "<dom_formosan_wilang_yutas_videos>",
     "<dom_formosan_yeddas_blog>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",

       "special": true
     },
     "259652": {
+      "content": "<dialect_kanakanavu>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259653": {
+      "content": "<dialect_kaqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259654": {
+      "content": "<dialect_kavalan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259655": {
+      "content": "<dialect_luanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259656": {
+      "content": "<dialect_malan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259657": {
+      "content": "<dialect_maolin>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259658": {
+      "content": "<dialect_nanwang>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259659": {
+      "content": "<dialect_northern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259660": {
+      "content": "<dialect_saaroa>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259661": {
+      "content": "<dialect_saisiyat>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259662": {
+      "content": "<dialect_sakizaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259663": {
+      "content": "<dialect_sekolik>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259664": {
+      "content": "<dialect_southern>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259665": {
+      "content": "<dialect_tanqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259666": {
+      "content": "<dialect_tegudaya>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259667": {
+      "content": "<dialect_thao>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259668": {
+      "content": "<dialect_truku>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259669": {
+      "content": "<dialect_tsou>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259670": {
+      "content": "<dialect_unknown>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259671": {
+      "content": "<dialect_wanda>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259672": {
+      "content": "<dialect_wanshan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259673": {
+      "content": "<dialect_wenshui>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259674": {
+      "content": "<dialect_wutai>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259675": {
+      "content": "<dialect_xiqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259676": {
+      "content": "<dialect_xiuguluan>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259677": {
+      "content": "<dialect_yami>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259678": {
+      "content": "<dialect_yilanzeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259679": {
+      "content": "<dialect_zeaol>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259680": {
+      "content": "<dialect_zhiben>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259681": {
+      "content": "<dialect_zhuoqun>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259682": {
+      "content": "<dom_classroom_context>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259683": {
+      "content": "<dom_culture>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259684": {
+      "content": "<dom_dictionary>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259685": {
+      "content": "<dom_essays>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259686": {
+      "content": "<dom_ethnography_journal_ritual_song>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259687": {
+      "content": "<dom_formosan_100_paiwan_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259688": {
+      "content": "<dom_formosan_academia_sinica_oral_legends>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259689": {
+      "content": "<dom_formosan_amis_adversative_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259690": {
+      "content": "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259691": {
+      "content": "<dom_formosan_amis_kuo_sung_comparative_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259692": {
+      "content": "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259693": {
+      "content": "<dom_formosan_amis_myths_and_customs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259694": {
+      "content": "<dom_formosan_amis_pa_verbs>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259695": {
+      "content": "<dom_formosan_amis_serial_verb_constructions>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259696": {
+      "content": "<dom_formosan_amis_tung_chiou>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259697": {
+      "content": "<dom_formosan_asai_sedik_language>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259698": {
+      "content": "<dom_formosan_bunun_debusser_dissertation>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259699": {
+      "content": "<dom_formosan_bunun_moriguchi_northern_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259700": {
+      "content": "<dom_formosan_bunun_topic_focus>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259701": {
+      "content": "<dom_formosan_chen_fukuda_relabeling_ergative>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259702": {
+      "content": "<dom_formosan_chen_fukuda_three_ways_steal>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259703": {
+      "content": "<dom_formosan_cip_atayal_grammar_overview>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259704": {
+      "content": "<dom_formosan_dean_johnson_year_clouds_smangus>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259705": {
+      "content": "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259706": {
+      "content": "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259707": {
+      "content": "<dom_formosan_egerod_soren_word_order_word_classes_at>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259708": {
+      "content": "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259709": {
+      "content": "<dom_formosan_epark>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259710": {
+      "content": "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259711": {
+      "content": "<dom_formosan_gitbook_translations>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259712": {
+      "content": "<dom_formosan_glosbe>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259713": {
+      "content": "<dom_formosan_hala_saku_la_videos>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259714": {
+      "content": "<dom_formosan_holmer_seediq>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259715": {
+      "content": "<dom_formosan_huang_grammaticalization_squliq_atayal>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259716": {
+      "content": "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259717": {
+      "content": "<dom_formosan_huteson_rukai_survey>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259718": {
+      "content": "<dom_formosan_ilrdf_42_language_practice_word_lists>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259719": {
+      "content": "<dom_formosan_ilrdf_tousvusvutu_kita>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259720": {
+      "content": "<dom_formosan_indigenous_language_literary_awards>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259721": {
+      "content": "<dom_formosan_kanakanavu_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259722": {
+      "content": "<dom_formosan_kavalan_zhang_reference_grammar>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259723": {
+      "content": "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259724": {
+      "content": "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259725": {
+      "content": "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259726": {
+      "content": "<dom_formosan_nowbucyang_truku_thesis>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259727": {
+      "content": "<dom_formosan_old_texts>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259728": {
+      "content": "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259729": {
+      "content": "<dom_formosan_paiwan_ho_five_dialects>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259730": {
+      "content": "<dom_formosan_paiwanstories>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "special": true
     },
     "259731": {
+      "content": "<dom_formosan_puyuma_chen_raising_to_object>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259732": {
+      "content": "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259733": {
+      "content": "<dom_formosan_puyuma_teng_pronominal_systems>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259734": {
+      "content": "<dom_formosan_puyuma_teng_reference_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259735": {
+      "content": "<dom_formosan_rik_bunun>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259736": {
+      "content": "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259737": {
+      "content": "<dom_formosan_rukai_mantauran_stories>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259738": {
+      "content": "<dom_formosan_rukai_texts>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259739": {
+      "content": "<dom_formosan_rukai_zeitoun_saying>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259740": {
+      "content": "<dom_formosan_saaroa_pan_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259741": {
+      "content": "<dom_formosan_sakizaya_affixes>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259742": {
+      "content": "<dom_formosan_seals>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259743": {
+      "content": "<dom_formosan_seediq_zhang_reference_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259744": {
+      "content": "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259745": {
+      "content": "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259746": {
+      "content": "<dom_formosan_shih_rukai_adverbial>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259747": {
+      "content": "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259748": {
+      "content": "<dom_formosan_song_limei_introduction_seediq_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259749": {
+      "content": "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259750": {
+      "content": "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259751": {
+      "content": "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259752": {
+      "content": "<dom_formosan_wei_matri_clan_lineage_system_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259753": {
+      "content": "<dom_formosan_wilang_yutas_videos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259754": {
+      "content": "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259755": {
+      "content": "<dom_formosan_yedda_palemeq_blog>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259756": {
+      "content": "<dom_formosan_yeddas_blog>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259757": {
+      "content": "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259758": {
+      "content": "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259759": {
+      "content": "<dom_formosan_zheng_acl_2024>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259760": {
+      "content": "<dom_formosan_zheng_data>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259761": {
+      "content": "<dom_learning_vocab>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259762": {
+      "content": "<dom_nine_level>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259763": {
+      "content": "<dom_ntu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259764": {
+      "content": "<dom_picture_book>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259765": {
+      "content": "<dom_picture_story>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259766": {
+      "content": "<dom_presidential_apology>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259767": {
+      "content": "<dom_reading_writing>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259768": {
+      "content": "<dom_unknown>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259769": {
+      "content": "<dom_youtube>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259770": {
+      "content": "<src_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259771": {
+      "content": "<src_bnn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259772": {
+      "content": "<src_ckv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259773": {
+      "content": "<src_dru>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259774": {
+      "content": "<src_eng>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259775": {
+      "content": "<src_pwn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259776": {
+      "content": "<src_pyu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259777": {
+      "content": "<src_ssf>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259778": {
+      "content": "<src_sxr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259779": {
+      "content": "<src_szy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259780": {
+      "content": "<src_tao>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259781": {
+      "content": "<src_tay>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259782": {
+      "content": "<src_trv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259783": {
+      "content": "<src_tsu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259784": {
+      "content": "<src_xnb>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259785": {
+      "content": "<src_xsy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259786": {
+      "content": "<src_zh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259787": {
+      "content": "<to_ami>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259788": {
+      "content": "<to_bnn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259789": {
+      "content": "<to_ckv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259790": {
+      "content": "<to_dru>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259791": {
+      "content": "<to_eng>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259792": {
+      "content": "<to_pwn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259793": {
+      "content": "<to_pyu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259794": {
+      "content": "<to_ssf>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259795": {
+      "content": "<to_sxr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259796": {
+      "content": "<to_szy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259797": {
+      "content": "<to_tao>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259798": {
+      "content": "<to_tay>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259799": {
+      "content": "<to_trv>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259800": {
+      "content": "<to_tsu>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259801": {
+      "content": "<to_xnb>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259802": {
+      "content": "<to_xsy>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259803": {
       "content": "<to_zh>",
       "lstrip": false,
       "normalized": false,
     "<dialect_hengchun>",
     "<dialect_jianhe>",
     "<dialect_junqun>",
+    "<dialect_kanakanavu>",
     "<dialect_kaqun>",
+    "<dialect_kavalan>",
     "<dialect_luanqun>",
     "<dialect_malan>",
     "<dialect_maolin>",
     "<dialect_nanwang>",
     "<dialect_northern>",
+    "<dialect_saaroa>",
+    "<dialect_saisiyat>",
+    "<dialect_sakizaya>",
     "<dialect_sekolik>",
     "<dialect_southern>",
     "<dialect_tanqun>",
     "<dialect_tegudaya>",
+    "<dialect_thao>",
     "<dialect_truku>",
+    "<dialect_tsou>",
     "<dialect_unknown>",
     "<dialect_wanda>",
     "<dialect_wanshan>",
     "<dialect_wutai>",
     "<dialect_xiqun>",
     "<dialect_xiuguluan>",
+    "<dialect_yami>",
+    "<dialect_yilanzeaol>",
     "<dialect_zeaol>",
     "<dialect_zhiben>",
     "<dialect_zhuoqun>",
     "<dom_culture>",
     "<dom_dictionary>",
     "<dom_essays>",
+    "<dom_ethnography_journal_ritual_song>",
     "<dom_formosan_100_paiwan_texts>",
+    "<dom_formosan_academia_sinica_oral_legends>",
+    "<dom_formosan_amis_adversative_constructions>",
+    "<dom_formosan_amis_kavalan_lin_interrogative_verbs>",
+    "<dom_formosan_amis_kuo_sung_comparative_constructions>",
+    "<dom_formosan_amis_lifok_dongi_saopo_kimakimad>",
     "<dom_formosan_amis_myths_and_customs>",
+    "<dom_formosan_amis_pa_verbs>",
+    "<dom_formosan_amis_serial_verb_constructions>",
+    "<dom_formosan_amis_tung_chiou>",
+    "<dom_formosan_asai_sedik_language>",
+    "<dom_formosan_bunun_debusser_dissertation>",
+    "<dom_formosan_bunun_moriguchi_northern_texts>",
+    "<dom_formosan_bunun_topic_focus>",
+    "<dom_formosan_chen_fukuda_relabeling_ergative>",
+    "<dom_formosan_chen_fukuda_three_ways_steal>",
+    "<dom_formosan_cip_atayal_grammar_overview>",
+    "<dom_formosan_dean_johnson_year_clouds_smangus>",
+    "<dom_formosan_dorinda_tsai_hsiu_liu_neutral_imperfect>",
+    "<dom_formosan_egerod_origin_headhunting_atayal_text_v>",
+    "<dom_formosan_egerod_soren_word_order_word_classes_at>",
+    "<dom_formosan_elizabeth_zeitoun_chen_huei_wu_overview>",
     "<dom_formosan_epark>",
+    "<dom_formosan_ferrell_taiwan_aboriginal_groups_proble>",
     "<dom_formosan_gitbook_translations>",
+    "<dom_formosan_glosbe>",
+    "<dom_formosan_hala_saku_la_videos>",
+    "<dom_formosan_holmer_seediq>",
+    "<dom_formosan_huang_grammaticalization_squliq_atayal>",
+    "<dom_formosan_huang_mei_chin_atayal_reference_grammar>",
+    "<dom_formosan_huteson_rukai_survey>",
+    "<dom_formosan_ilrdf_42_language_practice_word_lists>",
+    "<dom_formosan_ilrdf_tousvusvutu_kita>",
+    "<dom_formosan_indigenous_language_literary_awards>",
+    "<dom_formosan_kanakanavu_texts>",
+    "<dom_formosan_kavalan_zhang_reference_grammar>",
+    "<dom_formosan_li_peirong_xu_weisheng_introduction_tru>",
+    "<dom_formosan_li_peizhen_wu_xingyu_shouhu_xing>",
+    "<dom_formosan_naomi_tsukida_correlative_clauses_seedi>",
+    "<dom_formosan_nowbucyang_truku_thesis>",
     "<dom_formosan_old_texts>",
+    "<dom_formosan_paiwan_collart_zeitoun_time_reference>",
+    "<dom_formosan_paiwan_ho_five_dialects>",
     "<dom_formosan_paiwanstories>",
+    "<dom_formosan_puyuma_chen_raising_to_object>",
+    "<dom_formosan_puyuma_katipol_kopfjagdriten>",
+    "<dom_formosan_puyuma_teng_pronominal_systems>",
+    "<dom_formosan_puyuma_teng_reference_grammar>",
     "<dom_formosan_rik_bunun>",
+    "<dom_formosan_robert_blust_austronesian_homeland_ling>",
+    "<dom_formosan_rukai_mantauran_stories>",
+    "<dom_formosan_rukai_texts>",
+    "<dom_formosan_rukai_zeitoun_saying>",
+    "<dom_formosan_saaroa_pan_grammar>",
+    "<dom_formosan_sakizaya_affixes>",
     "<dom_formosan_seals>",
+    "<dom_formosan_seediq_zhang_reference_grammar>",
+    "<dom_formosan_shigeru_tsuchida_kanakanavu_texts>",
+    "<dom_formosan_shigeru_tsuchida_reconstruction_proto_t>",
+    "<dom_formosan_shih_rukai_adverbial>",
+    "<dom_formosan_song_limei_introduction_kanakanavu_gram>",
+    "<dom_formosan_song_limei_introduction_seediq_grammar>",
+    "<dom_formosan_sung_kuo_descriptive_comparative_constr>",
+    "<dom_formosan_taoshan_elementary_school_atelier_hui_k>",
+    "<dom_formosan_tsukida_naomi_verb_classification_amis>",
+    "<dom_formosan_wei_matri_clan_lineage_system_ami>",
     "<dom_formosan_wilang_yutas_videos>",
+    "<dom_formosan_wu_jinglan_introduction_amis_grammar>",
+    "<dom_formosan_yedda_palemeq_blog>",
     "<dom_formosan_yeddas_blog>",
+    "<dom_formosan_yeh_meili_introduction_saisiyat_grammar>",
+    "<dom_formosan_zhang_yongli_pan_jiarong_introduction_t>",
+    "<dom_formosan_zheng_acl_2024>",
     "<dom_formosan_zheng_data>",
     "<dom_learning_vocab>",
     "<dom_nine_level>",