hari31416 commited on
Commit
dab8707
·
verified ·
1 Parent(s): 6033a5d

Reduce bundle size after ONNX post-export layout optimization

Browse files
.gitattributes CHANGED
@@ -40,3 +40,4 @@ languages.png filter=lfs diff=lfs merge=lfs -text
40
  model.SRC filter=lfs diff=lfs merge=lfs -text
41
  model.TGT filter=lfs diff=lfs merge=lfs -text
42
  tokenizer_tgt.json filter=lfs diff=lfs merge=lfs -text
 
 
40
  model.SRC filter=lfs diff=lfs merge=lfs -text
41
  model.TGT filter=lfs diff=lfs merge=lfs -text
42
  tokenizer_tgt.json filter=lfs diff=lfs merge=lfs -text
43
+ decoder_shared.onnx.data filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -72,17 +72,16 @@ Compared against the FP32 ONNX oracle on the golden evaluation fixtures.
72
 
73
  | Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
74
  | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
75
- | FP32 | 6.64 GB | 100.00% | 100.00% | 100.00 | 244.4 ms | 1.000x |
76
- | FP16 | 3.32 GB | 99.73% | 99.73% | 100.00 | 259.8 ms | 0.941x |
77
- | INT8 | 1.66 GB | 89.64% | 89.73% | 95.69 | 112.6 ms | 2.228x |
78
- | Q4F16 | 850.5 MB | 82.27% | 82.36% | 92.55 | 143.3 ms | 1.673x |
79
 
80
 
81
  ## Files
82
 
83
  - `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
84
- - `decoder_model.onnx` (and optional `.onnx.data` weights sidecar)
85
- - `decoder_with_past_model.onnx` (and optional `.onnx.data` weights sidecar)
86
  - `translate.py` — self-contained Python inference helper (see Usage below)
87
  - Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
88
  - Model configuration configs (`config.json`, `generation_config.json`)
 
72
 
73
  | Format | Model Size | Exact Match (Token) | Exact Match (Text) | SacreBLEU (Raw) | Latency (Mean) | Speedup vs. FP32 |
74
  | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
75
+ | FP32 | 6.64 GB | 100.00% | 100.00% | 100.00 | 69.5 ms | 1.000x |
76
+ | FP16 | 3.32 GB | 99.73% | 99.73% | 100.00 | 74.3 ms | 0.935x |
77
+ | INT8 | 1.66 GB | 89.55% | 89.55% | 96.27 | 31.4 ms | 2.125x |
78
+ | Q4F16 | 850.5 MB | 82.45% | 82.55% | 91.99 | 58.4 ms | 1.186x |
79
 
80
 
81
  ## Files
82
 
83
  - `encoder_model.onnx` (and optional `.onnx.data` weights sidecar)
84
+ - `decoder_model.onnx` and `decoder_with_past_model.onnx` (share `decoder_shared.onnx.data` when present)
 
85
  - `translate.py` — self-contained Python inference helper (see Usage below)
86
  - Fast tokenizer config files (`tokenizer_src.json`, `tokenizer_tgt.json`, `tokenizer_meta.json`)
87
  - Model configuration configs (`config.json`, `generation_config.json`)
categories.png CHANGED
decoder_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:27177ad88e0ace28bb0931400a75a74a1e7e7db2218fdb69d6433d503bc2f1a7
3
- size 1811875
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3c69727dc4d5b4510bb4dda10d616ada571d43ee665232015e76877af36288f
3
+ size 424244
decoder_with_past_model.onnx.data → decoder_shared.onnx.data RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:24ba50c8f3dfbac032cf0c5b5517e1ebd423108668f8b6fc6c34cbd52c3bd9fe
3
- size 2667429888
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f525dc7de9ccb2e11608accc530dec4ac7332739f2fb3ba7e5f56a233bdba69
3
+ size 2819629056
decoder_with_past_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5a60ae0313a87dcab5b3aaf37425bdfce49e2d3e5fc7866ba26c435e5bcb62e4
3
- size 1747287
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e68d1d33b71656f71a7a0bdc301459ac7f94f556d2c8461de69a2a9fe7a7b9b2
3
+ size 392757
encoder_model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5049976b746d0f49bb831fcf437bc1f461883bae26ef11c8d8d4154b044ec73c
3
- size 1452442
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84b706d878bd5fc2680be90e89ebe0e923b098d7a1ef2dc28dc0cab8ebe99932
3
+ size 236890
encoder_model.onnx.data CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7356a14c26e92a2118a4b7ac78d245934a705f8802b6c91f29d665a408c2915c
3
- size 1643601920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7877cb26cd20178fb1288bc6d16d1bb245fc99c1437d00e626389e92971818e
3
+ size 1644658688
languages.png CHANGED

Git LFS Details

  • SHA256: 0364a35505569c68dc2a4bddf22bcdb5ebfa3b2374fc92e1f448c39d43d930a8
  • Pointer size: 131 Bytes
  • Size of remote file: 187 kB

Git LFS Details

  • SHA256: e1fb65b1553f16c6b14ef6840b17d1df75503acf7b99cda4df7928012cc395d1
  • Pointer size: 131 Bytes
  • Size of remote file: 185 kB
overall.png CHANGED