metadata
base_model:
- mispeech/dasheng-0.6B
- Qwen/Qwen2.5-Omni-7B
language:
- en
- zh
- th
- id
- vi
license: apache-2.0
pipeline_tag: audio-text-to-text
tags:
- multimodal
- audio-language-model
- audio
base_model_relation: finetune
library_name: transformers
MiDashengLM-7B-0804 (4bit, GPTQ quantized)
The 4bit (w4a16) weights for mispeech/midashenglm-7b-0804-fp32, quantized by GPTQ.
This model is described in the paper MiDashengLM: Efficient Audio Understanding with General Audio Captions. For more details, visit the project page: https://xiaomi-research.github.io/dasheng-lm/
An ideal choice for resource-constrained environments. It offers broad GPU compatibility and a smaller memory footprint, making it suitable for deployment where VRAM, memory, or storage is limited, provided that a slight trade-off in quality is acceptable.
Usage
Load Model
from transformers import AutoModelForCausalLM, AutoProcessor, AutoTokenizer
model_id = "mispeech/midashenglm-7b-0804-w4a16-gptq"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
Construct Prompt
user_prompt = "Caption the audio." # You may try any other prompt
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": user_prompt},
{
"type": "audio",
"path": "/path/to/example.wav",
# or "url": "https://example.com/example.wav"
# or "audio": np.random.randn(16000)
},
],
},
]
Generate Output
import torch
with torch.no_grad():
model_inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
add_special_tokens=True,
return_dict=True,
).to(device=model.device, dtype=model.dtype)
generation = model.generate(**model_inputs)
output = tokenizer.batch_decode(generation, skip_special_tokens=True) # ["An engine is idling."]
Citation
MiDashengLM is under the Apache License 2.0, and we encourage its use in both research and business applications.
If you find MiDashengLM useful in your research, please consider citing our work:
@techreport{midashenglm7b,
title = {MiDashengLM: Efficient Audio Understanding with General Audio Captions},
author = {{Horizon Team, MiLM Plus}},
institution= {Xiaomi Inc.},
year = {2025},
note = {Contributors: Heinrich Dinkel et al. (listed alphabetically in Appendix B)},
url = {https://arxiv.org/abs/2508.03983},
eprint = {2508.03983},
}