| --- |
| language: |
| - en |
| - ar |
| - bn |
| - zh |
| - da |
| - nl |
| - de |
| - fi |
| - fr |
| - hi |
| - id |
| - it |
| - ja |
| - ko |
| - mr |
| - 'no' |
| - pl |
| - pt |
| - ru |
| - es |
| - tr |
| - uk |
| - vi |
| license: apache-2.0 |
| library_name: onnxruntime |
| pipeline_tag: voice-activity-detection |
| tags: |
| - turn-detection |
| - end-of-utterance |
| - mmbert |
| - onnx |
| - quantized |
| - conversational-ai |
| - voice-assistant |
| - real-time |
| - voice-activity-detection |
| base_model: jhu-clsp/mmBERT-base |
| datasets: |
| - videosdk-live/Namo-Turn-Detector-v1-Train |
| model-index: |
| - name: Namo Turn Detector v1 - Multilingual |
| results: |
| - task: |
| type: text-classification |
| name: Turn Detection |
| dataset: |
| name: Namo Turn Detector v1 Test - Multilingual |
| type: videosdk-live/Namo-Turn-Detector-v1-Test |
| split: train |
| metrics: |
| - type: accuracy |
| value: 0.9025 |
| --- |
| |
| # 🎯 Namo Turn Detector v1 - MultiLingual |
|
|
| <div align="center"> |
|
|
| [](https://opensource.org/licenses/Apache-2.0) |
| [](https://onnx.ai/) |
| [](https://huggingface.co/videosdk-live/Namo-Turn-Detector-v1-Multilingual) |
| []() |
|
|
| **🚀 Namo Turn Detection Model for Multiple Languages** |
|
|
| 🇸🇦 Arabic, 🇮🇳 Bengali, 🇨🇳 Chinese, 🇩🇰 Danish, 🇳🇱 Dutch, 🇩🇪 German, 🇬🇧🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, and 🇻🇳 Vietnamese |
|
|
| </div> |
|
|
| --- |
|
|
| ## 📋 Overview |
|
|
| The **Namo Turn Detector** is a specialized AI model designed to solve one of the most challenging problems in conversational AI: **knowing when a user has finished speaking**. |
|
|
| This Multilingual model uses advanced natural language understanding to distinguish between: |
| - ✅ **Complete utterances** (user is done speaking) |
| - 🔄 **Incomplete utterances** (user will continue speaking) |
|
|
| Built on **mmBERT** architecture and optimized with quantized ONNX format, it delivers enterprise-grade performance with minimal latency. |
|
|
| ## 🔑 Key Features |
|
|
| - **Turn Detection Specialist**: Detects end-of-turn vs. continuation in multilingual speech transcripts. |
| - **Low Latency**: Optimized with **quantized ONNX** for <29ms inference. |
| - **Robust Performance**: Average 90.25% accuracy on multilingual utterances. |
| - **Easy Integration**: Compatible with Python, ONNX Runtime, and VideoSDK Agents SDK. |
| - **Enterprise Ready**: Supports real-time conversational AI and voice assistants. |
|
|
| ## 📊 Performance Metrics |
| <div> |
|
|
| | Metric | Score | |
| |--------|-------| |
| | **⚡ Latency** | **<29ms** | |
| | **💾 Model Size** | **~295MB** | |
|
|
| | Language | Accuracy | Precision | Recall | F1 Score | Samples | |
| | --------------- | -------- | --------- | ------ | -------- | ------- | |
| | 🇹🇷 Turkish | 0.9731 | 0.9611 | 0.9853 | 0.9730 | 966 | |
| | 🇰🇷 Korean | 0.9685 | 0.9541 | 0.9842 | 0.9690 | 890 | |
| | 🇩🇪 German | 0.9425 | 0.9135 | 0.9772 | 0.9443 | 1322 | |
| | 🇯🇵 Japanese | 0.9436 | 0.9099 | 0.9857 | 0.9463 | 834 | |
| | 🇮🇳 Hindi | 0.9398 | 0.9276 | 0.9603 | 0.9436 | 1295 | |
| | 🇳🇱 Dutch | 0.9279 | 0.8959 | 0.9738 | 0.9332 | 1401 | |
| | 🇳🇴 Norwegian | 0.9165 | 0.8717 | 0.9801 | 0.9227 | 1976 | |
| | 🇨🇳 Chinese | 0.9164 | 0.8859 | 0.9608 | 0.9219 | 945 | |
| | 🇫🇮 Finnish | 0.9158 | 0.8746 | 0.9702 | 0.9199 | 1010 | |
| | 🇬🇧 English | 0.9086 | 0.8507 | 0.9801 | 0.9108 | 2845 | |
| | 🇮🇩 Indonesian | 0.9022 | 0.8514 | 0.9707 | 0.9071 | 971 | |
| | 🇮🇹 Italian | 0.9015 | 0.8562 | 0.9640 | 0.9069 | 782 | |
| | 🇵🇱 Polish | 0.9068 | 0.8619 | 0.9568 | 0.9069 | 976 | |
| | 🇵🇹 Portuguese | 0.8956 | 0.8410 | 0.9676 | 0.8999 | 1398 | |
| | 🇩🇰 Danish | 0.8973 | 0.8517 | 0.9644 | 0.9045 | 779 | |
| | 🇪🇸 Spanish | 0.8888 | 0.8304 | 0.9681 | 0.8940 | 1295 | |
| | 🇮🇳 Marathi | 0.8850 | 0.8762 | 0.9008 | 0.8883 | 774 | |
| | 🇷🇺 Russian | 0.8748 | 0.8318 | 0.9547 | 0.8890 | 1470 | |
| | 🇺🇦 Ukrainian | 0.8794 | 0.8164 | 0.9587 | 0.8819 | 929 | |
| | 🇻🇳 Vietnamese | 0.8645 | 0.8135 | 0.9439 | 0.8738 | 1004 | |
| | 🇸🇦 Arabic | 0.8490 | 0.7965 | 0.9439 | 0.8639 | 947 | |
| | 🇮🇳 Bengali | 0.7940 | 0.7874 | 0.7939 | 0.7907 | 1000 | |
|
|
|
|
| > 📊 *Evaluated on 25,000+ Multilingual utterances from diverse conversational contexts* |
|
|
| ## ⚡️ Speed Analysis |
|
|
| <img src="./performance_analysis.png" alt="Alt text" width="600" height="400"/> |
|
|
| ## 🔧 Train & Test Scripts |
|
|
| <div align="center"> |
|
|
| [](https://colab.research.google.com/drive/1WEVVAzu1WHiucPRabnyPiWWc-OYvBMNj) [](https://colab.research.google.com/drive/19ZOlNoHS2WLX2V4r5r492tsCUnYLXnQR) |
|
|
| </div> |
|
|
| ## 🛠️ Installation |
|
|
| To use this model, you will need to install the following libraries. |
|
|
| ```bash |
| pip install onnxruntime transformers huggingface_hub |
| ``` |
|
|
| ## 🚀 Quick Start |
|
|
| You can run inference directly from Hugging Face repository. |
|
|
| ```python |
| import numpy as np |
| import onnxruntime as ort |
| from transformers import AutoTokenizer |
| from huggingface_hub import hf_hub_download |
| |
| class TurnDetector: |
| def __init__(self, repo_id="videosdk-live/Namo-Turn-Detector-v1-Multilingual"): |
| """ |
| Initializes the detector by downloading the model and tokenizer |
| from the Hugging Face Hub. |
| """ |
| print(f"Loading model from repo: {repo_id}") |
| |
| # Download the model and tokenizer from the Hub |
| # Authentication is handled automatically if you are logged in |
| model_path = hf_hub_download(repo_id=repo_id, filename="model_quant.onnx") |
| self.tokenizer = AutoTokenizer.from_pretrained(repo_id) |
| |
| # Set up the ONNX Runtime inference session |
| self.session = ort.InferenceSession(model_path) |
| self.max_length = 8192 |
| print("✅ Model and tokenizer loaded successfully.") |
| |
| def predict(self, text: str) -> tuple: |
| """ |
| Predicts if a given text utterance is the end of a turn. |
| Returns (predicted_label, confidence) where: |
| - predicted_label: 0 for "Not End of Turn", 1 for "End of Turn" |
| - confidence: confidence score between 0 and 1 |
| """ |
| # Tokenize the input text |
| inputs = self.tokenizer( |
| text, |
| truncation=True, |
| max_length=self.max_length, |
| return_tensors="np" |
| ) |
| |
| # Prepare the feed dictionary for the ONNX model |
| feed_dict = { |
| "input_ids": inputs["input_ids"], |
| "attention_mask": inputs["attention_mask"] |
| } |
| |
| # Run inference |
| outputs = self.session.run(None, feed_dict) |
| logits = outputs[0] |
| |
| probabilities = self._softmax(logits[0]) |
| predicted_label = np.argmax(probabilities) |
| confidence = float(np.max(probabilities)) |
| |
| return predicted_label, confidence |
| |
| def _softmax(self, x, axis=None): |
| if axis is None: |
| axis = -1 |
| exp_x = np.exp(x - np.max(x, axis=axis, keepdims=True)) |
| return exp_x / np.sum(exp_x, axis=axis, keepdims=True) |
| |
| # --- Example Usage --- |
| if __name__ == "__main__": |
| detector = TurnDetector() |
| |
| sentences = [ |
| "They're often made with oil or sugar.", # Expected: End of Turn |
| "I think the next logical step is to", # Expected: Not End of Turn |
| "What are you doing tonight?", # Expected: End of Turn |
| "The Revenue Act of 1862 adopted rates that increased with", # Expected: Not End of Turn |
| ] |
| |
| for sentence in sentences: |
| predicted_label, confidence = detector.predict(sentence) |
| result = "End of Turn" if predicted_label == 1 else "Not End of Turn" |
| print(f"'{sentence}' -> {result} (confidence: {confidence:.3f})") |
| print("-" * 50) |
| ``` |
|
|
|
|
| ## 🤖 VideoSDK Agents Integration |
|
|
| Integrate this turn detector directly with VideoSDK Agents for production-ready conversational AI applications. |
|
|
| ```python |
| from videosdk_agents import NamoTurnDetectorV1, pre_download_namo_turn_v1_model |
| |
| #download model |
| pre_download_namo_turn_v1_model() |
| |
| # Initialize Multilingual turn detector for VideoSDK Agents |
| turn_detector = NamoTurnDetectorV1() |
| ``` |
|
|
| > 📚 [**Complete Integration Guide**](https://docs.videosdk.live/ai_agents/plugins/namo-turn-detector) - Learn how to use `NamoTurnDetectorV1` with VideoSDK Agents |
|
|
| ## 📖 Citation |
|
|
| ```bibtex |
| @model{namo_turn_detector_en_2025, |
| title={Namo Turn Detector v1: Multilingual}, |
| author={VideoSDK Team}, |
| year={2025}, |
| publisher={Hugging Face}, |
| url={https://huggingface.co/videosdk-live/Namo-Turn-Detector-v1-Multilingual}, |
| note={ONNX-optimized mmBERT for turn detection in 23 Languages} |
| } |
| ``` |
|
|
| ## 📄 License |
|
|
| This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. |
|
|
| <div align="center"> |
|
|
| **Made with ❤️ by the VideoSDK Team** |
|
|
| [](https://videosdk.live) |
|
|
| </div> |
