raniczan commited on
Commit
3bb93c0
·
verified ·
1 Parent(s): 56d8a85

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -11
README.md CHANGED
@@ -12,16 +12,13 @@ tags:
12
  - korean
13
  - english
14
  - code-switching
15
- - forced-alignment
16
  ---
17
 
18
  # wav2vec2-xls-r-300m-korean-english
19
 
20
- A **jamo + Latin CTC acoustic model** fine-tuned from `facebook/wav2vec2-xls-r-300m`
21
  for **Korean / English code-switched** speech. It emits per-frame character
22
- probabilities, so it works both as a character recognizer and (its validated use)
23
- as a **streaming CTC forced aligner** — pair the emissions with known text + a
24
- Viterbi pass to get per-character timestamps (~20 ms granularity).
25
 
26
  ## Training
27
 
@@ -34,7 +31,7 @@ Viterbi pass to get per-character timestamps (~20 ms granularity).
34
  ## Important: jamo preprocessing
35
 
36
  The vocab is **jamo (decomposed Hangul) + Latin**, not raw syllables. You MUST
37
- decompose reference/target text the same way, and recompose model output:
38
 
39
  ```python
40
  import unicodedata
@@ -44,10 +41,10 @@ def from_jamo(t): # model output -> readable text
44
  return unicodedata.normalize("NFC", t)
45
  ```
46
 
47
- ## Usage (recognition)
48
 
49
  ```python
50
- import torch, soundfile as sf, unicodedata
51
  from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
52
 
53
  proc = Wav2Vec2Processor.from_pretrained("raniczan/wav2vec2-xls-r-300m-korean-english")
@@ -58,6 +55,3 @@ x = proc(wav, sampling_rate=16000, return_tensors="pt").input_values
58
  ids = model(x).logits.argmax(-1)[0]
59
  print(unicodedata.normalize("NFC", proc.decode(ids)))
60
  ```
61
-
62
- For **forced alignment**, run the same logits through `torchaudio.functional.forced_align`
63
- (or a custom streaming Viterbi) against your jamo-decomposed known text.
 
12
  - korean
13
  - english
14
  - code-switching
 
15
  ---
16
 
17
  # wav2vec2-xls-r-300m-korean-english
18
 
19
+ A **jamo + Latin CTC** acoustic model fine-tuned from `facebook/wav2vec2-xls-r-300m`
20
  for **Korean / English code-switched** speech. It emits per-frame character
21
+ probabilities over a compact jamo + Latin vocabulary.
 
 
22
 
23
  ## Training
24
 
 
31
  ## Important: jamo preprocessing
32
 
33
  The vocab is **jamo (decomposed Hangul) + Latin**, not raw syllables. You MUST
34
+ decompose target text the same way, and recompose model output:
35
 
36
  ```python
37
  import unicodedata
 
41
  return unicodedata.normalize("NFC", t)
42
  ```
43
 
44
+ ## Usage
45
 
46
  ```python
47
+ import soundfile as sf, unicodedata
48
  from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
49
 
50
  proc = Wav2Vec2Processor.from_pretrained("raniczan/wav2vec2-xls-r-300m-korean-english")
 
55
  ids = model(x).logits.argmax(-1)[0]
56
  print(unicodedata.normalize("NFC", proc.decode(ids)))
57
  ```