File size: 950 Bytes
77d19a8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Granite Speech 4.1 2B NAR Portable INT4 for Taurscribe
This artifact is derived from IBM Granite Speech 4.1 2B NAR:
https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar
Original model provider:
IBM Granite Team
Original license:
Apache License 2.0
Taurscribe modifications:
- Copied the Granite ONNX model bundle and tokenizer assets.
- Applied INT4 weight-only MatMulNBits quantization to selected MatMul weights.
- Patched editor.onnx to output token_ids through in-graph ArgMax.
- Flattened rank-5 windowed-attention MatMul operations to equivalent rank-3 operations for DirectML.
- Baked fixed-bucket shape calculations as ONNX initializers and pruned dead shape nodes.
- Replaced two-output GLU Split operations with equivalent Slice pairs for DirectML correctness.
- Packaged the artifact for Taurscribe's ONNX Runtime DirectML and CPU paths.
No additional training or fine-tuning was performed by Taurscribe.
|