Audio Classification
Transformers
Safetensors
Russian
emotion-recognition
speech-emotion-recognition
speech
russian
quantized
compressed-tensors
int8
fp8
int4
Eval Results (legacy)
Instructions to use Aniemore/wavlm-bert-base-s-emotion-russian-resd-quantized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Aniemore/wavlm-bert-base-s-emotion-russian-resd-quantized with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("audio-classification", model="Aniemore/wavlm-bert-base-s-emotion-russian-resd-quantized")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Aniemore/wavlm-bert-base-s-emotion-russian-resd-quantized", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add torchaudio, torchcodec and datasets ways to load the audio
Browse files
README.md
CHANGED
|
@@ -102,10 +102,9 @@ model = AutoModelForAudioClassification.from_pretrained(
|
|
| 102 |
repo, subfolder="int8").eval() # or "fp8", "int4"
|
| 103 |
fe = AutoFeatureExtractor.from_pretrained(repo, subfolder="int8")
|
| 104 |
|
| 105 |
-
#
|
| 106 |
-
#
|
| 107 |
-
#
|
| 108 |
-
# answer rather than raising.
|
| 109 |
wav, _ = librosa.load("clip.wav", sr=16000, mono=True)
|
| 110 |
x = fe(wav, sampling_rate=16000, return_tensors="pt", padding=True)
|
| 111 |
with torch.no_grad():
|
|
@@ -113,6 +112,27 @@ with torch.no_grad():
|
|
| 113 |
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})
|
| 114 |
```
|
| 115 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
## What is quantized
|
| 117 |
|
| 118 |
Only `nn.Linear` inside the encoder blocks — that is where 98.6% of the weight mass sits. Left in fp32: the convolutional feature extractor, the positional convolution, the layer norms, the projector and the classifier. The classifier is seven rows wide, so quantizing it would save nothing and put rounding error straight onto the logits.
|
|
|
|
| 102 |
repo, subfolder="int8").eval() # or "fp8", "int4"
|
| 103 |
fe = AutoFeatureExtractor.from_pretrained(repo, subfolder="int8")
|
| 104 |
|
| 105 |
+
# Resample to 16 kHz. Do not skip it: RESD itself ships at 44.1 kHz,
|
| 106 |
+
# and handing the model 44.1 kHz audio while telling the extractor it
|
| 107 |
+
# is 16 kHz stretches time 2.8x and silently changes the answer.
|
|
|
|
| 108 |
wav, _ = librosa.load("clip.wav", sr=16000, mono=True)
|
| 109 |
x = fe(wav, sampling_rate=16000, return_tensors="pt", padding=True)
|
| 110 |
with torch.no_grad():
|
|
|
|
| 112 |
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})
|
| 113 |
```
|
| 114 |
|
| 115 |
+
<details><summary>Loading the audio without librosa</summary>
|
| 116 |
+
|
| 117 |
+
```python
|
| 118 |
+
# torchaudio
|
| 119 |
+
import torchaudio
|
| 120 |
+
wav, sr = torchaudio.load("clip.wav")
|
| 121 |
+
wav = torchaudio.functional.resample(wav, sr, 16000).mean(0).numpy()
|
| 122 |
+
|
| 123 |
+
# torchcodec, the newer decoder
|
| 124 |
+
from torchcodec.decoders import AudioDecoder
|
| 125 |
+
wav = AudioDecoder("clip.wav", sample_rate=16000).get_all_samples().data.mean(0).numpy()
|
| 126 |
+
|
| 127 |
+
# straight from the dataset, which resamples on the column
|
| 128 |
+
from datasets import load_dataset, Audio
|
| 129 |
+
ds = load_dataset("Aniemore/resd", split="test")
|
| 130 |
+
ds = ds.cast_column("speech", Audio(sampling_rate=16000))
|
| 131 |
+
wav = ds[0]["speech"]["array"]
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
</details>
|
| 135 |
+
|
| 136 |
## What is quantized
|
| 137 |
|
| 138 |
Only `nn.Linear` inside the encoder blocks — that is where 98.6% of the weight mass sits. Left in fp32: the convolutional feature extractor, the positional convolution, the layer norms, the projector and the classifier. The classifier is seven rows wide, so quantizing it would save nothing and put rounding error straight onto the logits.
|