Automatic Speech Recognition
Transformers
Safetensors
Russian
wav2vec2
speech
russian
children
ctc
forced-alignment
pronunciation
phonetics
Instructions to use dysata/Wav2Vec2-Ru-Child with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dysata/Wav2Vec2-Ru-Child with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="dysata/Wav2Vec2-Ru-Child")# Load model directly from transformers import AutoProcessor, AutoModelForCTC processor = AutoProcessor.from_pretrained("dysata/Wav2Vec2-Ru-Child") model = AutoModelForCTC.from_pretrained("dysata/Wav2Vec2-Ru-Child", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,130 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- ru
|
| 4 |
+
license: mit
|
| 5 |
+
library_name: transformers
|
| 6 |
+
pipeline_tag: automatic-speech-recognition
|
| 7 |
+
tags:
|
| 8 |
+
- wav2vec2
|
| 9 |
+
- speech
|
| 10 |
+
- russian
|
| 11 |
+
- children
|
| 12 |
+
- ctc
|
| 13 |
+
- forced-alignment
|
| 14 |
+
- pronunciation
|
| 15 |
+
- phonetics
|
| 16 |
+
datasets:
|
| 17 |
+
- dysata/rwords
|
| 18 |
+
model-index:
|
| 19 |
+
- name: Wav2Vec2-Ru-Child
|
| 20 |
+
results: []
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
# Wav2Vec2-Ru-Child
|
| 24 |
+
|
| 25 |
+
Модель автоматического распознавания речи (ASR) для русского языка, дообученная на записях детского чтения.
|
| 26 |
+
|
| 27 |
+
## Model Details
|
| 28 |
+
|
| 29 |
+
### Architecture
|
| 30 |
+
|
| 31 |
+
- **Base model:** wav2vec2-large
|
| 32 |
+
- **Architecture:** `Wav2Vec2ForCTC`
|
| 33 |
+
- **Hidden size:** 1024
|
| 34 |
+
- **Layers:** 24 transformer layers
|
| 35 |
+
- **Attention heads:** 16
|
| 36 |
+
- **Parameters:** ~317M
|
| 37 |
+
- **Vocabulary:** 37 токенов (33 буквы русского алфавита + 4 служебных)
|
| 38 |
+
- **CTC loss:** mean reduction
|
| 39 |
+
|
| 40 |
+
### Intended Use
|
| 41 |
+
|
| 42 |
+
Модель предназначена для:
|
| 43 |
+
- Распознавания русской детской речи
|
| 44 |
+
- Forced alignment (выравнивание текста по аудио на уровне букв)
|
| 45 |
+
- Анализа произношения — выявление ошибок в детском чтении
|
| 46 |
+
- Классификации качества произношения отдельных звуков (например, звука "Р")
|
| 47 |
+
|
| 48 |
+
## How to Use
|
| 49 |
+
|
| 50 |
+
### Speech Recognition
|
| 51 |
+
|
| 52 |
+
```python
|
| 53 |
+
import torch
|
| 54 |
+
import librosa
|
| 55 |
+
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
|
| 56 |
+
|
| 57 |
+
processor = Wav2Vec2Processor.from_pretrained("dysata/Wav2Vec2-Ru-Child")
|
| 58 |
+
model = Wav2Vec2ForCTC.from_pretrained("dysata/Wav2Vec2-Ru-Child")
|
| 59 |
+
|
| 60 |
+
audio, sr = librosa.load("audio.wav", sr=16000)
|
| 61 |
+
|
| 62 |
+
processed = processor([audio], sampling_rate=16000,
|
| 63 |
+
return_tensors="pt", padding="longest")
|
| 64 |
+
|
| 65 |
+
with torch.no_grad():
|
| 66 |
+
logits = model(processed.input_values,
|
| 67 |
+
attention_mask=processed.attention_mask).logits
|
| 68 |
+
|
| 69 |
+
predicted_ids = torch.argmax(logits, dim=-1)
|
| 70 |
+
transcription = processor.decode(predicted_ids[0])
|
| 71 |
+
print(transcription)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
### Forced Alignment
|
| 75 |
+
|
| 76 |
+
Модель может использоваться для побуквенного выравнивания эталонного текста по аудио через CTC forced alignment (trellis + backtrack + merge_repeats). Это позволяет определить временные границы каждой буквы в записи.
|
| 77 |
+
|
| 78 |
+
### Hidden States для классификации
|
| 79 |
+
|
| 80 |
+
```python
|
| 81 |
+
with torch.no_grad():
|
| 82 |
+
outputs = model(processed.input_values,
|
| 83 |
+
attention_mask=processed.attention_mask,
|
| 84 |
+
output_hidden_states=True, return_dict=True)
|
| 85 |
+
last_hidden_state = outputs.hidden_states[-1] # [batch, frames, 1024]
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
Вектора последнего скрытого слоя (1024-мерные) могут быть использованы как признаки для классификации качества произношения отдельных звуков.
|
| 89 |
+
|
| 90 |
+
## Training
|
| 91 |
+
|
| 92 |
+
Модель дообучена на записях детского чтения на русском языке. Аудиозаписи преобразованы в формат WAV 16 кГц и вручную оттранскрибированы.
|
| 93 |
+
|
| 94 |
+
## Technical Specifications
|
| 95 |
+
|
| 96 |
+
| Parameter | Value |
|
| 97 |
+
|---|---|
|
| 98 |
+
| Sample rate | 16 kHz |
|
| 99 |
+
| Feature extractor | 7-layer CNN |
|
| 100 |
+
| Transformer layers | 24 |
|
| 101 |
+
| Hidden size | 1024 |
|
| 102 |
+
| Vocab size | 37 |
|
| 103 |
+
| Precision | float32 |
|
| 104 |
+
| Format | Safetensors |
|
| 105 |
+
|
| 106 |
+
## Vocabulary
|
| 107 |
+
|
| 108 |
+
Алфавит модели: `<pad>`, `<s>`, `</s>`, `<unk>`, `|` (разделитель слов), а-я (33 буквы русского алфавита).
|
| 109 |
+
|
| 110 |
+
## Limitations
|
| 111 |
+
|
| 112 |
+
- Модель обучена на детской речи и может показывать худшие результаты на взрослой речи
|
| 113 |
+
- Только русский язык
|
| 114 |
+
- Оптимальное качество на записях в формате WAV 16 кГц
|
| 115 |
+
|
| 116 |
+
## Citation
|
| 117 |
+
|
| 118 |
+
```bibtex
|
| 119 |
+
@misc{wav2vec2-ru-child,
|
| 120 |
+
author = {Павел Рудич},
|
| 121 |
+
title = {Wav2Vec2-Ru-Child: Russian Children's Speech Recognition Model},
|
| 122 |
+
year = {2025},
|
| 123 |
+
publisher = {Hugging Face},
|
| 124 |
+
url = {https://huggingface.co/dysata/Wav2Vec2-Ru-Child}
|
| 125 |
+
}
|
| 126 |
+
```
|
| 127 |
+
|
| 128 |
+
## Funding
|
| 129 |
+
|
| 130 |
+
Фонд содействия инновациям (fasie).
|