Instructions to use Arsh9210/Nemotron-Labs-Audex-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Arsh9210/Nemotron-Labs-Audex-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Arsh9210/Nemotron-Labs-Audex-2B")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Arsh9210/Nemotron-Labs-Audex-2B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Arsh9210/Nemotron-Labs-Audex-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Arsh9210/Nemotron-Labs-Audex-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arsh9210/Nemotron-Labs-Audex-2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Arsh9210/Nemotron-Labs-Audex-2B

SGLang

How to use Arsh9210/Nemotron-Labs-Audex-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Arsh9210/Nemotron-Labs-Audex-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arsh9210/Nemotron-Labs-Audex-2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Arsh9210/Nemotron-Labs-Audex-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arsh9210/Nemotron-Labs-Audex-2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Arsh9210/Nemotron-Labs-Audex-2B with Docker Model Runner:
```
docker model run hf.co/Arsh9210/Nemotron-Labs-Audex-2B
```

Arsh9210 commited on 10 days ago

Commit

93b6955

verified ·

1 Parent(s): 1d67081

Added inference_scripts_vllm/unified_s2s_scripts/web/index.html

Browse files

Files changed (1) hide show

inference_scripts_vllm/unified_s2s_scripts/web/index.html +88 -0

inference_scripts_vllm/unified_s2s_scripts/web/index.html ADDED Viewed

	@@ -0,0 +1,88 @@

+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>Cascaded Speech-to-Speech Demo</title>
+    <link rel="stylesheet" href="/styles.css" />
+  </head>
+  <body>
+    <main class="app">
+      <header class="hero">
+        <h1>Cascaded Speech-to-Speech Demo</h1>
+        <p>Upload or record speech, then use the Audex vLLM model for ASR, text reasoning, and TTS. The bundled Audex causal speech decoder streams the final speech.</p>
+      </header>
+      <section class="model-row panel">
+        <label class="field">
+          <span>Model</span>
+          <select id="model-select"></select>
+        </label>
+        <div id="model-state" class="status-pill">Starting...</div>
+      </section>
+      <section class="grid">
+        <section class="panel controls">
+          <label class="checkbox">
+            <input id="auto-e2e" type="checkbox" checked/>
+            <span>Auto full end-to-end after audio input</span>
+          </label>
+          <label class="checkbox">
+            <input id="enable-reasoning" type="checkbox" checked/>
+            <span>Enable reasoning</span>
+          </label>
+          <div class="field">
+            <span>Record or Upload Audio</span>
+            <div class="audio-inputs">
+              <input id="audio-file" type="file" accept="audio/*" />
+              <button id="record-btn" type="button" class="secondary">Record Audio</button>
+            </div>
+            <div id="audio-name" class="hint">No audio selected.</div>
+          </div>
+          <label class="field">
+            <span>Text Prompt</span>
+            <textarea id="prompt-input" rows="4"></textarea>
+          </label>
+          <label class="field">
+            <span>Text Input</span>
+            <textarea id="text-input" rows="5" placeholder="Type text here, or transcribe audio."></textarea>
+          </label>
+          <div class="button-stack">
+            <button id="transcribe-btn" type="button" class="secondary">Transcribe Audio</button>
+            <button id="generate-response-btn" type="button" class="secondary">Generate Response</button>
+            <button id="generate-audio-btn" type="button" class="secondary">Generate Audio</button>
+            <button id="clear-btn" type="button">Clear</button>
+          </div>
+        </section>
+        <section class="panel outputs">
+          <label class="field">
+            <span>Status</span>
+            <input id="status-box" type="text" readonly />
+          </label>
+          <label class="field">
+            <span>Text Output</span>
+            <textarea id="text-output" rows="16" readonly></textarea>
+          </label>
+          <div class="field">
+            <span>Speech Output</span>
+            <div class="speech-card">
+              <canvas id="visualizer" width="720" height="180"></canvas>
+              <div id="audio-state" class="hint">Idle</div>
+              <audio id="audio-output" controls></audio>
+            </div>
+          </div>
+        </section>
+      </section>
+    </main>
+    <script type="module" src="/app.js"></script>
+  </body>
+</html>