hari31416 commited on
Commit
231becc
·
verified ·
1 Parent(s): 63751ce

Reduce bundle size after ONNX post-export layout optimization

Browse files
.gitattributes CHANGED
@@ -39,3 +39,4 @@ encoder_model.onnx.data filter=lfs diff=lfs merge=lfs -text
39
  model.SRC filter=lfs diff=lfs merge=lfs -text
40
  model.TGT filter=lfs diff=lfs merge=lfs -text
41
  tokenizer_src.json filter=lfs diff=lfs merge=lfs -text
 
 
39
  model.SRC filter=lfs diff=lfs merge=lfs -text
40
  model.TGT filter=lfs diff=lfs merge=lfs -text
41
  tokenizer_src.json filter=lfs diff=lfs merge=lfs -text
42
+ decoder_shared.onnx.data filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -72,17 +72,16 @@ Compared against the FP32 ONNX oracle on the golden evaluation fixtures.
72
 
73
  | Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
74
  | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
75
- | FP32 | 4.88 GB | 100.00% | 100.00% | 100.00 | 171.8 ms | 1.000x |
76
- | FP16 | 2.44 GB | 99.91% | 99.91% | 99.98 | 180.5 ms | 0.952x |
77
- | INT8 | 1.22 GB | 94.18% | 94.18% | 97.94 | 76.4 ms | 2.196x |
78
- | Q4F16 | 625.0 MB | 87.55% | 87.55% | 95.17 | 85.5 ms | 1.962x |
79
 
80
 
81
  ## Files
82
 
83
  - `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
84
- - `decoder_model.onnx` (and optional `.onnx.data` weights sidecar)
85
- - `decoder_with_past_model.onnx` (and optional `.onnx.data` weights sidecar)
86
  - `translate.py` — self-contained Python inference helper (see Usage below)
87
  - Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
88
  - Model configuration configs (`config.json`, `generation_config.json`)
 
72
 
73
  | Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
74
  | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
75
+ | FP32 | 4.88 GB | 100.00% | 100.00% | 100.00 | 49.0 ms | 1.000x |
76
+ | FP16 | 2.44 GB | 99.82% | 99.82% | 99.96 | 49.7 ms | 0.987x |
77
+ | INT8 | 1.22 GB | 94.45% | 94.45% | 98.00 | 25.2 ms | 1.900x |
78
+ | Q4F16 | 625.0 MB | 88.55% | 88.55% | 95.44 | 42.7 ms | 1.080x |
79
 
80
 
81
  ## Files
82
 
83
  - `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
84
+ - `decoder_model.onnx` and `decoder_with_past_model.onnx` (share `decoder_shared.onnx.data` when present)
 
85
  - `translate.py` — self-contained Python inference helper (see Usage below)
86
  - Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
87
  - Model configuration configs (`config.json`, `generation_config.json`)
categories.png CHANGED
decoder_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1402f93d1d02cb3624089fc85e3fa1fe93532d3ee4bfa4760b36ffcd6fd66702
3
- size 1750892
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32521de67622944aad448816662fa90d0ca4871f9b06ceee4b1139a1f7608d32
3
+ size 424179
decoder_shared.onnx.data ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab8a30e3e00aedfbc0a68dc544fb1e16ff231103c2a09568b0397c4d772ea21b
3
+ size 2079268864
decoder_with_past_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:99fb1dd0167139301d7a0350b538e3e452f756d1118f03778884d606a46ddbf4
3
- size 1693322
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17af04408d8a68775b831538c5f71b207e30c912812acbceac41eef0fce276d4
3
+ size 392700
encoder_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4a780d6188a20c3f8292b36cec48aa405c147a72cc8978b603a3d7c4503ab675
3
- size 1452471
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8ba4f5af7f811c68741901203e56399a2b03a04316e4dd40498becc540ce7c1
3
+ size 236919
encoder_model.onnx.data CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f106d51bc1c2edc508542e2bdbacbd89ac90f75e5945872366fa4d1ca1409c14
3
- size 2013814784
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff9f98a1bf7a9e6a32dce33b2691c448957d634c9b7ed788e9f15ddc6e056acb
3
+ size 2014871552
languages.png CHANGED
overall.png CHANGED