How to use cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit", dtype="auto")
Nice work with this model. Can we get a GPTQ version in 4 bits? I tried to create it in GPTQModel but got OOM killed.
· Sign up or log in to comment
