Update README.md
Browse files
README.md
CHANGED
|
@@ -35,10 +35,10 @@ tags:
|
|
| 35 |
This model is designed to support **phoneme-based text-to-speech (TTS) systems**, including but not limited to [StyleTTS2](https://github.com/yl4579/StyleTTS2). Thanks to its Catalan-specific phoneme vocabulary and contextual embedding capabilities, it can serve as a phoneme encoder for any TTS architecture requiring phoneme-level features.
|
| 36 |
|
| 37 |
Features of our PL-BERT:
|
| 38 |
-
- It is trained **exclusively on Catalan** phonemized text
|
| 39 |
-
- It uses a reduced **phoneme vocabulary of 178 tokens**
|
| 40 |
-
- It uses a simple tokenizer for words
|
| 41 |
-
- It includes a custom `token_maps.pkl` and adapted `util.py`
|
| 42 |
|
| 43 |
---
|
| 44 |
|
|
@@ -46,15 +46,15 @@ Features of our PL-BERT:
|
|
| 46 |
|
| 47 |
### Intended uses
|
| 48 |
|
| 49 |
-
- Integration into phoneme-based TTS pipelines such as StyleTTS2, Matxa-TTS, or custom diffusion-based synthesizers
|
| 50 |
-
- Accent-aware synthesis and phoneme embedding extraction for Catalan
|
| 51 |
-
- Research on phoneme-level language modeling in low-resource or multi-accent settings
|
| 52 |
-
|
| 53 |
### Limitations
|
| 54 |
|
| 55 |
-
- Not designed for general NLP tasks like classification or sentiment analysis
|
| 56 |
-
- Only supports Catalan phoneme tokens
|
| 57 |
-
- Some accents may be underrepresented in the training data
|
| 58 |
|
| 59 |
---
|
| 60 |
|
|
@@ -63,7 +63,7 @@ Features of our PL-BERT:
|
|
| 63 |
Here is an example of how to use this model within the StyleTTS2 framework:
|
| 64 |
|
| 65 |
1. Clone the StyleTTS2 repository: https://github.com/yl4579/StyleTTS2
|
| 66 |
-
2. Inside the `Utils` directory, create a new folder, for example: `PLBERT_cat_multiaccent`
|
| 67 |
3. Copy the following files into that folder:
|
| 68 |
- `config.yml` (training configuration)
|
| 69 |
- `step_1000000.t7` (trained checkpoint)
|
|
@@ -78,7 +78,7 @@ Here is an example of how to use this model within the StyleTTS2 framework:
|
|
| 78 |
|
| 79 |
`from Utils.PLBERT_cat_multiaccent.util import load_plbert`
|
| 80 |
|
| 81 |
-
6.
|
| 82 |
|
| 83 |
Note: Although this example uses StyleTTS2, the model is compatible with other TTS architectures that operate on phoneme sequences. You can use the contextualized phoneme embeddings from PL-BERT in any compatible synthesis system.
|
| 84 |
|
|
@@ -88,7 +88,7 @@ Note: Although this example uses StyleTTS2, the model is compatible with other T
|
|
| 88 |
|
| 89 |
### Training data
|
| 90 |
|
| 91 |
-
The model was trained on a Catalan corpus
|
| 92 |
|
| 93 |
Tokenizer: custom (split using whitespaces)
|
| 94 |
Phoneme masking strategy: word-level and phoneme-level masking and replacement
|
|
|
|
| 35 |
This model is designed to support **phoneme-based text-to-speech (TTS) systems**, including but not limited to [StyleTTS2](https://github.com/yl4579/StyleTTS2). Thanks to its Catalan-specific phoneme vocabulary and contextual embedding capabilities, it can serve as a phoneme encoder for any TTS architecture requiring phoneme-level features.
|
| 36 |
|
| 37 |
Features of our PL-BERT:
|
| 38 |
+
- It is trained **exclusively on Catalan** phonemized text
|
| 39 |
+
- It uses a reduced **phoneme vocabulary of 178 tokens**
|
| 40 |
+
- It uses a simple tokenizer for words
|
| 41 |
+
- It includes a custom `token_maps.pkl` and adapted `util.py`
|
| 42 |
|
| 43 |
---
|
| 44 |
|
|
|
|
| 46 |
|
| 47 |
### Intended uses
|
| 48 |
|
| 49 |
+
- Integration into phoneme-based TTS pipelines such as StyleTTS2, Matxa-TTS, or custom diffusion-based synthesizers
|
| 50 |
+
- Accent-aware synthesis and phoneme embedding extraction for Catalan
|
| 51 |
+
- Research on phoneme-level language modeling in low-resource or multi-accent settings
|
| 52 |
+
-
|
| 53 |
### Limitations
|
| 54 |
|
| 55 |
+
- Not designed for general NLP tasks like classification or sentiment analysis
|
| 56 |
+
- Only supports Catalan phoneme tokens
|
| 57 |
+
- Some accents may be underrepresented in the training data
|
| 58 |
|
| 59 |
---
|
| 60 |
|
|
|
|
| 63 |
Here is an example of how to use this model within the StyleTTS2 framework:
|
| 64 |
|
| 65 |
1. Clone the StyleTTS2 repository: https://github.com/yl4579/StyleTTS2
|
| 66 |
+
2. Inside the `Utils` directory, create a new folder, for example: `PLBERT_cat_multiaccent`
|
| 67 |
3. Copy the following files into that folder:
|
| 68 |
- `config.yml` (training configuration)
|
| 69 |
- `step_1000000.t7` (trained checkpoint)
|
|
|
|
| 78 |
|
| 79 |
`from Utils.PLBERT_cat_multiaccent.util import load_plbert`
|
| 80 |
|
| 81 |
+
6. Phonemize your Catalan text files for training and validation (if you use `espeak-ng` use the language code `ca`)
|
| 82 |
|
| 83 |
Note: Although this example uses StyleTTS2, the model is compatible with other TTS architectures that operate on phoneme sequences. You can use the contextualized phoneme embeddings from PL-BERT in any compatible synthesis system.
|
| 84 |
|
|
|
|
| 88 |
|
| 89 |
### Training data
|
| 90 |
|
| 91 |
+
The model was trained on a phonemized Catalan corpus (any phonemizer can be used) extracted from the [CATalog](https://huggingface.co/datasets/projecte-aina/CATalog) corpus. The dataset includes sentences from speakers across Catalonia, Balearic Islands, and Valencia. It uses a consistent phoneme token set with boundary markers and masking tokens.
|
| 92 |
|
| 93 |
Tokenizer: custom (split using whitespaces)
|
| 94 |
Phoneme masking strategy: word-level and phoneme-level masking and replacement
|