Instructions to use cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Can I request a higher quantization, such as 3-bit or 2-bit awq?
#2
by win10 - opened
Can I request a higher quantization, such as 3-bit or 2-bit awq?
This comment has been hidden (marked as Graphic Content)
win10 changed discussion title from Can I request a smaller quantization such as 3bit, or 2bit? to Can I request a higher quantization, such as 3-bit or 2-bit awq?