Instructions to use fixie-ai/ultravox-v0_5-glm-4_5-355b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use fixie-ai/ultravox-v0_5-glm-4_5-355b with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("fixie-ai/ultravox-v0_5-glm-4_5-355b", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -107,9 +107,9 @@ pipe({'audio': audio, 'turns': turns, 'sampling_rate': sr}, max_new_tokens=30)
|
|
| 107 |
|
| 108 |
The model uses a pre-trained [GLM-4.5](https://huggingface.co/zai-org/GLM-4.5) backbone as well as the encoder part of [whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo).
|
| 109 |
|
| 110 |
-
The multi-modal adapter is trained, the Whisper encoder is fine-tuned, and the
|
| 111 |
|
| 112 |
-
We use a knowledge-distillation loss where Ultravox is trying to match the logits of the text-based
|
| 113 |
|
| 114 |
### Training Data
|
| 115 |
|
|
|
|
| 107 |
|
| 108 |
The model uses a pre-trained [GLM-4.5](https://huggingface.co/zai-org/GLM-4.5) backbone as well as the encoder part of [whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo).
|
| 109 |
|
| 110 |
+
The multi-modal adapter is trained, the Whisper encoder is fine-tuned, and the GLM model is kept frozen.
|
| 111 |
|
| 112 |
+
We use a knowledge-distillation loss where Ultravox is trying to match the logits of the text-based GLM backbone.
|
| 113 |
|
| 114 |
### Training Data
|
| 115 |
|