Promote from WindstormLabs/translate-windy-hplt-en-vi

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+source.spm filter=lfs diff=lfs merge=lfs -text
+target.spm filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,54 @@
+---
+license: cc-by-4.0
+base_model: HPLT/translate-en-vi-v2.0-hplt_opus
+library_name: ctranslate2
+pipeline_tag: translation
+tags: [translation, marian, ctranslate2, int8, en, vi]
+---
+
+# translate-windy-hplt-en — en to vi
+
+Bilingual machine translation in **CTranslate2 INT8** for fast CPU inference.
+Fine-tuned by **Windstorm Labs** from [`HPLT/translate-en-vi-v2.0-hplt_opus`](https://huggingface.co/HPLT/translate-en-vi-v2.0-hplt_opus).
+
+## Attribution
+
+Derived from [`HPLT/translate-en-vi-v2.0-hplt_opus`](https://huggingface.co/HPLT/translate-en-vi-v2.0-hplt_opus), from the **HPLT** project, licensed under
+**CC-BY-4.0**. **Modified by Windstorm Labs.** Attribution is retained as the licence requires.
+
+## What we did
+
+Converted from the upstream Marian release, LoRA fine-tuned on parallel data for this
+language pair, merged into the base weights, then quantized to INT8.
+
+| | |
+|---|---|
+| Rank / alpha | 8 / 16 |
+| Learning rate / steps | 2.5e-06 / 50 |
+| Target modules | `q_proj`, `v_proj` |
+| Seed | 42 |
+| Training data | opus100_en-vi (clinic cache), 2,000 sentence pairs |
+| Tensors modified | 36 |
+
+## Provenance
+
+```
+base model.bin  sha256  0c94dc9fcf527ddab4e9df98dee176b401213aa3fbaf9c4feed5623e439b9c99
+this model.bin  sha256  d2108b1d5cf28a080d3ca20b17a17f313ed6f731a24a9bb9e155a0c05d08b1a9
+```
+
+Verified after INT8 quantization, on the published artifact.
+
+## Evaluation
+
+FLORES-200 devtest, 200 sentences, beam 4, CTranslate2 INT8 on CPU.
+
+| | spBLEU | chrF |
+|---|---:|---:|
+| this model | **40.35** | 57.7 |
+
+## Notes
+
+- Single language pair: en to vi.
+- FLORES-200 is news and encyclopedic prose.
+- No human evaluation was performed.

config.json

@@ -0,0 +1,10 @@
+{
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<s>",
+  "decoder_start_token": "</s>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "multi_query_attention": false,
+  "unk_token": "<unk>"
+}

model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2108b1d5cf28a080d3ca20b17a17f313ed6f731a24a9bb9e155a0c05d08b1a9
+size 79047635

source.spm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee5740271c3510149e02e0cd4211a832d967874e695e606a8b7eb78261d1739
+size 1333836

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

target.spm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee5740271c3510149e02e0cd4211a832d967874e695e606a8b7eb78261d1739
+size 1333836

tokenizer_config.json

@@ -0,0 +1,38 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64000": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "separate_vocabs": false,
+  "source_lang": "",
+  "sp_model_kwargs": {},
+  "target_lang": "marian",
+  "tokenizer_class": "MarianTokenizer",
+  "unk_token": "<unk>"
+}