Instructions to use cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit", dtype="auto") - Notebooks
- Google Colab
- Kaggle
wrong ouput
Hi, thanks for sharing this quantized model, but my output maybe incorrect.
I use the example below:
conversation = [
{
"role": "user",
"content": [
{"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"},
{"type": "audio", "audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cough.wav"},
{"type": "text", "text": "What can you see and hear? Answer in one short sentence."}
],
},
]
I'm running this on my jetson orin AGX, and here is the output:
"The image is a collage of repeating text that says "I'm a little teapot," and the audio is a series of short, percussive, clicking sounds."
It seems to be a wrong answer
Are you using Docker, If so, could you share the image you’re using? I was wondering if this might be related to the environment.
Are you using Docker, If so, could you share the image you’re using? I was wondering if this might be related to the environment.
No, I didn't use docker
Can you tell me how to compile vllm on Jetson Orin AGX? I haven't been successful no matter how I try. Thank you.
Can you tell me how to compile vllm on Jetson Orin AGX? I haven't been successful no matter how I try. Thank you.
I did it in a standard way, perhaps you can post your issues here.