Reduce bundle size after ONNX post-export layout optimization
Browse files- .gitattributes +1 -0
- README.md +5 -6
- categories.png +0 -0
- decoder_model.onnx +2 -2
- decoder_shared.onnx.data +3 -0
- decoder_with_past_model.onnx +2 -2
- encoder_model.onnx +2 -2
- encoder_model.onnx.data +2 -2
- languages.png +0 -0
- overall.png +0 -0
.gitattributes
CHANGED
|
@@ -39,3 +39,4 @@ encoder_model.onnx.data filter=lfs diff=lfs merge=lfs -text
|
|
| 39 |
model.SRC filter=lfs diff=lfs merge=lfs -text
|
| 40 |
model.TGT filter=lfs diff=lfs merge=lfs -text
|
| 41 |
tokenizer_src.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 39 |
model.SRC filter=lfs diff=lfs merge=lfs -text
|
| 40 |
model.TGT filter=lfs diff=lfs merge=lfs -text
|
| 41 |
tokenizer_src.json filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
decoder_shared.onnx.data filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -72,17 +72,16 @@ Compared against the FP32 ONNX oracle on the golden evaluation fixtures.
|
|
| 72 |
|
| 73 |
| Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
|
| 74 |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
| 75 |
-
| FP32 | 4.88 GB | 100.00% | 100.00% | 100.00 |
|
| 76 |
-
| FP16 | 2.44 GB | 99.
|
| 77 |
-
| INT8 | 1.22 GB | 94.
|
| 78 |
-
| Q4F16 | 625.0 MB |
|
| 79 |
|
| 80 |
|
| 81 |
## Files
|
| 82 |
|
| 83 |
- `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
|
| 84 |
-
- `decoder_model.onnx`
|
| 85 |
-
- `decoder_with_past_model.onnx` (and optional `.onnx.data` weights sidecar)
|
| 86 |
- `translate.py` — self-contained Python inference helper (see Usage below)
|
| 87 |
- Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
|
| 88 |
- Model configuration configs (`config.json`, `generation_config.json`)
|
|
|
|
| 72 |
|
| 73 |
| Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
|
| 74 |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
| 75 |
+
| FP32 | 4.88 GB | 100.00% | 100.00% | 100.00 | 49.0 ms | 1.000x |
|
| 76 |
+
| FP16 | 2.44 GB | 99.82% | 99.82% | 99.96 | 49.7 ms | 0.987x |
|
| 77 |
+
| INT8 | 1.22 GB | 94.45% | 94.45% | 98.00 | 25.2 ms | 1.900x |
|
| 78 |
+
| Q4F16 | 625.0 MB | 88.55% | 88.55% | 95.44 | 42.7 ms | 1.080x |
|
| 79 |
|
| 80 |
|
| 81 |
## Files
|
| 82 |
|
| 83 |
- `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
|
| 84 |
+
- `decoder_model.onnx` and `decoder_with_past_model.onnx` (share `decoder_shared.onnx.data` when present)
|
|
|
|
| 85 |
- `translate.py` — self-contained Python inference helper (see Usage below)
|
| 86 |
- Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
|
| 87 |
- Model configuration configs (`config.json`, `generation_config.json`)
|
categories.png
CHANGED
|
|
decoder_model.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:32521de67622944aad448816662fa90d0ca4871f9b06ceee4b1139a1f7608d32
|
| 3 |
+
size 424179
|
decoder_shared.onnx.data
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ab8a30e3e00aedfbc0a68dc544fb1e16ff231103c2a09568b0397c4d772ea21b
|
| 3 |
+
size 2079268864
|
decoder_with_past_model.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:17af04408d8a68775b831538c5f71b207e30c912812acbceac41eef0fce276d4
|
| 3 |
+
size 392700
|
encoder_model.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b8ba4f5af7f811c68741901203e56399a2b03a04316e4dd40498becc540ce7c1
|
| 3 |
+
size 236919
|
encoder_model.onnx.data
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ff9f98a1bf7a9e6a32dce33b2691c448957d634c9b7ed788e9f15ddc6e056acb
|
| 3 |
+
size 2014871552
|
languages.png
CHANGED
|
|
overall.png
CHANGED
|
|