zylin12 commited on
Commit
2c33139
·
verified ·
1 Parent(s): a487b1f

Revert README to 7f2c1bd

Browse files
Files changed (1) hide show
  1. README.md +24 -102
README.md CHANGED
@@ -39,11 +39,10 @@ Given an audio or video file, the model generates a compact speaker-aware transc
39
  - [Model Architecture](#model-architecture)
40
  - [Evaluation](#evaluation)
41
  - [Quickstart](#quickstart)
42
- - [Serve with SGLang Omni](#serve-with-sglang-omni)
43
- - [Serving with Hugging Face](#serving-with-native-hugging-face-transformers)
44
  - [Python Usage](#python-usage)
45
  - [Custom Prompt and Hotwords](#custom-prompt-and-hotwords)
46
- - [Serve with vLLM](#serve-with-vllm)
47
  - [Subtitle Web App](#subtitle-web-app)
48
  - [Output Format](#output-format)
49
  - [More Information](#more-information)
@@ -163,102 +162,7 @@ We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Err
163
 
164
  ## Quickstart
165
 
166
- ### Serve with SGLang Omni
167
-
168
- The recommended way to serve MOSS-Transcribe-Diarize is [SGLang Omni](https://github.com/sgl-project/sglang-omni) through the OpenAI-compatible `/v1/audio/transcriptions` endpoint. Install `sglang-omni` by following the [Installation guide](https://github.com/sgl-project/sglang-omni/blob/main/docs/get_started/installation.md), then download the model:
169
-
170
- ```bash
171
- hf download OpenMOSS-Team/MOSS-Transcribe-Diarize
172
- ```
173
-
174
- Serve the model:
175
-
176
- ```bash
177
- sgl-omni serve \
178
- --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \
179
- --port 8000 \
180
- --max-running-requests 16 \
181
- --cuda-graph-max-bs 16 \
182
- --mem-fraction-static 0.80
183
- ```
184
-
185
- Use `response_format=verbose_json` when you need parsed speaker segments. `json` returns the raw transcript text only.
186
-
187
- ```bash
188
- curl -X POST http://localhost:8000/v1/audio/transcriptions \
189
- -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
190
- -F file=@audio.wav \
191
- -F response_format=verbose_json
192
- ```
193
-
194
- ```python
195
- import requests
196
-
197
- with open("audio.wav", "rb") as f:
198
- resp = requests.post(
199
- "http://localhost:8000/v1/audio/transcriptions",
200
- data={
201
- "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
202
- "response_format": "verbose_json",
203
- },
204
- files={"file": ("audio.wav", f, "audio/wav")},
205
- timeout=300,
206
- )
207
-
208
- resp.raise_for_status()
209
- payload = resp.json()
210
- print(payload["text"])
211
- for segment in payload.get("segments", []):
212
- print(
213
- f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}"
214
- )
215
- ```
216
-
217
- For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript:
218
-
219
- ```bash
220
- curl -X POST http://localhost:8000/v1/audio/transcriptions \
221
- -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
222
- -F file=@audio.wav \
223
- -F response_format=verbose_json \
224
- -F max_new_tokens=65536
225
- ```
226
-
227
- | Parameter | Type | Default | Description |
228
- |---|---|---|---|
229
- | `file` | file | required | Audio file uploaded as multipart form data |
230
- | `model` | string | server default | Model identifier |
231
- | `language` | string | unset | Optional language hint |
232
- | `response_format` | string | `json` | `json`, `verbose_json`, or `text` |
233
- | `temperature` | float | model default (`0.0`) | Sampling temperature |
234
- | `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio (e.g. `65536`) |
235
- | `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |
236
-
237
- For benchmarking, performance numbers, and more details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md), here we list the performance of short/long-sequence mutli-speaker ASR tasks on single H100:
238
-
239
-
240
- 1. `movies800times` for short sequence ASR:
241
-
242
- | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
243
- |---:|---:|---:|---:|---:|
244
- | 1 | 2.57 | 0.388 | 0.0612 | 29.76 |
245
- | 2 | 4.89 | 0.409 | 0.0659 | 56.55 |
246
- | 4 | 6.62 | 0.513 | 0.0790 | 76.64 |
247
- | 8 | 6.80 | 0.533 | 0.0810 | 78.70 |
248
- | 16 | 7.08 | 0.659 | 0.0922 | 81.98 |
249
-
250
- 2. `aishell4_long` for long sequence ASR:
251
-
252
- | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
253
- |---:|---:|---:|---:|---:|
254
- | 1 | 0.022 | 45.2 | 0.0197 | 50.64 |
255
- | 2 | 0.032 | 60.7 | 0.0265 | 74.25 |
256
- | 4 | 0.036 | 105.6 | 0.0461 | 81.64 |
257
- | 8 | 0.040 | 172.6 | 0.0754 | 90.62 |
258
- | 16 | 0.043 | 282.8 | 0.1237 | 98.83 |
259
-
260
-
261
- ### Serving with Native Hugging Face Transformers
262
 
263
  Use a clean Python environment. The model uses custom Transformers code, so load the model and processor with `trust_remote_code=True`.
264
 
@@ -345,10 +249,9 @@ To add hotwords, append a short hint to the default prompt:
345
 
346
  More prompt recipes are available in the GitHub repository: <https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/main/examples/prompts.md>
347
 
 
348
 
349
- ### Serve with vLLM
350
-
351
- MOSS-Transcribe-Diarize also supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.
352
 
353
  ```bash
354
  uv pip install -U vllm \
@@ -376,6 +279,25 @@ curl http://localhost:8000/v1/audio/transcriptions \
376
  -F temperature="0"
377
  ```
378
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
379
  ### Subtitle Web App
380
 
381
  The source package includes a local subtitle workflow for upload, review, subtitle export, and optional FFmpeg burn-in:
 
39
  - [Model Architecture](#model-architecture)
40
  - [Evaluation](#evaluation)
41
  - [Quickstart](#quickstart)
42
+ - [Environment Setup](#environment-setup)
 
43
  - [Python Usage](#python-usage)
44
  - [Custom Prompt and Hotwords](#custom-prompt-and-hotwords)
45
+ - [Serve with vLLM and SGLang](#serve-with-vllm-and-sglang)
46
  - [Subtitle Web App](#subtitle-web-app)
47
  - [Output Format](#output-format)
48
  - [More Information](#more-information)
 
162
 
163
  ## Quickstart
164
 
165
+ ### Environment Setup
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
 
167
  Use a clean Python environment. The model uses custom Transformers code, so load the model and processor with `trust_remote_code=True`.
168
 
 
249
 
250
  More prompt recipes are available in the GitHub repository: <https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/main/examples/prompts.md>
251
 
252
+ ### Serve with vLLM and SGLang
253
 
254
+ MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.
 
 
255
 
256
  ```bash
257
  uv pip install -U vllm \
 
279
  -F temperature="0"
280
  ```
281
 
282
+ The same request format can be used with an SGLang OpenAI-compatible server:
283
+
284
+ ```bash
285
+ python -m sglang.launch_server \
286
+ --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \
287
+ --served-model-name OpenMOSS-Team/MOSS-Transcribe-Diarize \
288
+ --trust-remote-code \
289
+ --host 0.0.0.0 \
290
+ --port 30000
291
+ ```
292
+
293
+ ```bash
294
+ curl http://localhost:30000/v1/audio/transcriptions \
295
+ -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
296
+ -F file=@"audio.wav" \
297
+ -F response_format="json" \
298
+ -F temperature="0"
299
+ ```
300
+
301
  ### Subtitle Web App
302
 
303
  The source package includes a local subtitle workflow for upload, review, subtitle export, and optional FFmpeg burn-in: