Abdullahu5mani's picture
Upload Taurscribe Granite portable INT4 DirectML and CPU artifact
77d19a8 verified
Raw
History Blame Contribute Delete
950 Bytes
Granite Speech 4.1 2B NAR Portable INT4 for Taurscribe
This artifact is derived from IBM Granite Speech 4.1 2B NAR:
https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar
Original model provider:
IBM Granite Team
Original license:
Apache License 2.0
Taurscribe modifications:
- Copied the Granite ONNX model bundle and tokenizer assets.
- Applied INT4 weight-only MatMulNBits quantization to selected MatMul weights.
- Patched editor.onnx to output token_ids through in-graph ArgMax.
- Flattened rank-5 windowed-attention MatMul operations to equivalent rank-3 operations for DirectML.
- Baked fixed-bucket shape calculations as ONNX initializers and pruned dead shape nodes.
- Replaced two-output GLU Split operations with equivalent Slice pairs for DirectML correctness.
- Packaged the artifact for Taurscribe's ONNX Runtime DirectML and CPU paths.
No additional training or fine-tuning was performed by Taurscribe.