proxectonos/Nos_TTS-icia-matcha-phonemes-ljspeech

Icía: Nos Project's Galician TTS Model

Model description

Icía is a Galician TTS model developed by the Nós project. It was fine-tuned on the pre-trained LJSpeech model using the Matcha-TTS Python library. The Icía corpus of the dataset CRPIH_UVigo-GL-Voices was used for fine-tuning, which comprises a total of 2,950 sentences recorded by an amateur voice talent.

The model was trained on phonemes, so it needs a phonetic transcription of the input text to synthesize speech. The Cotovía tool is required to generate this transcription. The Cotovía linguistic module also performs text normalization as a preliminary step to phonetic transcription.

Intended uses and limitations

You can use this model to generate synthetic speech in Galician.

Installation

Cotovía

For phonetic transcription, the Cotovía front-end must be used. This software is available for download on the SourceForge website. The required Debian packages are cotovia_0.5_amd64.deb and cotovia-lang-gl_0.5_all.deb, which can be installed using the following commands:

sudo dpkg -i cotovia_0.5_amd64.deb
sudo dpkg -i cotovia-lang-gl_0.5_all.deb

TTS library

To synthesize speech, you need to install the Matcha-TTS library:

pip install matcha-tts

How to use

Python usage

Normalization can be performed within Python:

import argparse
import random
import re
import string
import subprocess

PUNCLIST = [';', '?', '¿', ',', ':', '.', '!', '¡']

def sanitize_filename(filename):
    """Remove or replace any characters that are not allowed in file names."""
    return ''.join(c for c in filename if c.isalnum() or c in (' ', '_', '-')).rstrip()

def canBeNumber(n):
    try:
        int(n)
        return True
    except ValueError:
        # Not a number
        return False

def is_number(index, text):
    if index == 0:
        return False
    elif index == len(text) - 1:
        return False
    else:
        return canBeNumber(text[index - 1]) and canBeNumber(text[index + 1])

def split_punc(text):
    segments = []
    puncs = []
    curr_seg = ""
    previous_punc = False

    for i, c in enumerate(text):        
        if c in PUNCLIST and not previous_punc and not is_number(i, text):
            segments.append(curr_seg.strip())
            puncs.append(c)
            curr_seg = ""
            previous_punc = True
        elif c in PUNCLIST and previous_punc:
            puncs[-1] += c
        else:
            curr_seg += c
            previous_punc = False

    segments.append(curr_seg.strip())

    #Remove empty segments in the list
    segments = filter(None, segments)

    # store segments as a list
    segments = list(segments)

    return segments, puncs

def remove_tra3_tags(phontrans):
    s = re.sub(r'#(.+?)#', r'', phontrans)
    s = re.sub(r'%(.+?)%', r'', s)
    s = re.sub(' +',' ',s)
    s = re.sub('-','',s)
    return s.strip()

def to_cotovia(text_segments):
    # Input and output Cotovía files
    res = ''.join(random.choices(string.ascii_lowercase + string.digits, k=5))
    COTOVIA_IN_TXT_PATH = res + '.txt'
    COTOVIA_IN_TXT_PATH_ISO = 'iso8859-1' + res + '.txt'
    COTOVIA_OUT_PRE_PATH = 'iso8859-1' + res + '.tra'
    COTOVIA_OUT_PRE_PATH_UTF8 = 'utf8' + res + '.tra'
    
    with open(COTOVIA_IN_TXT_PATH, 'w') as f:
        for seg in text_segments:
            if seg:
                f.write(seg + '\n')
            else:
                f.write(',' + '\n')

    # utf-8 to iso8859-1
    subprocess.run(["iconv", "-f", "utf-8", "-t", "iso8859-1", COTOVIA_IN_TXT_PATH, "-o", COTOVIA_IN_TXT_PATH_ISO], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
    # call cotovia with -t3 option
    subprocess.run(["cotovia", "-i", COTOVIA_IN_TXT_PATH_ISO, "-t3", "-n"], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
    # iso8859-1 to utf-8
    subprocess.run(["iconv", "-f", "iso8859-1", "-t", "utf-8", COTOVIA_OUT_PRE_PATH, "-o", COTOVIA_OUT_PRE_PATH_UTF8], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)

    segs = []
    try:
        with open(COTOVIA_OUT_PRE_PATH_UTF8, 'r') as f:
            segs = [line.rstrip() for line in f]
            segs = [remove_tra3_tags(line) for line in segs]
    except:
        print("ERROR: Couldn't read cotovia output")

    subprocess.run(["rm", COTOVIA_IN_TXT_PATH, COTOVIA_IN_TXT_PATH_ISO, COTOVIA_OUT_PRE_PATH, COTOVIA_OUT_PRE_PATH_UTF8], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)

    return segs

def merge_punc(text_segs, puncs):
    merged_str = ""

    for i, seg in enumerate(text_segs):
        merged_str += seg + " "
        if i < len(puncs):
            merged_str += puncs[i] + " "

    # remove spaces before , . ! ? ; : ) ] of the merged string
    merged_str = re.sub(r"\s+([.,!?;:)\]])", r"\1", merged_str)

    # remove spaces after ( [ ¡ ¿ of the merged string
    merged_str = re.sub(r"([\(\[¡¿])\s+", r"\1", merged_str)

    return merged_str.strip()


def accent_convert(phontrans):
    transcript = re.sub('a\^','á',phontrans)
    transcript = re.sub('e\^','é',transcript)
    transcript = re.sub('i\^','í',transcript)
    transcript = re.sub('o\^','ó',transcript)
    transcript = re.sub('u\^','ú',transcript)
    transcript = re.sub('E\^','É',transcript)
    transcript = re.sub('O\^','Ó',transcript)
    return transcript


def text_preprocess(text):
    #Split from punc
    text_segments, puncs = split_punc(text)
    cotovia_phon_segs = to_cotovia(text_segments)
    cotovia_phon_str = merge_punc(cotovia_phon_segs, puncs)
    phon_str = accent_convert(cotovia_phon_str)
    return phon_str


def main():
    parser = argparse.ArgumentParser(description='Cotovia phoneme transcription.')
    parser.add_argument('text', type=str, help='Text to synthetize')
    parser.add_argument('model_path', type=str, help='Absolute path to the model checkpoint.pth')
    parser.add_argument('config_path', type=str, help='Absolute path to the model config.json')

    args = parser.parse_args()

    print("Text before preprocessing: ", args.text)
    text = text_preprocess(args.text)
    print("Text after preprocessing: ", text)

if __name__ == "__main__":
    main()

This Python code takes a text input, preprocesses it using Cotovía's front-end, and returns the preprocessed text.

A more advanced version, including additional text preprocessing, can be found in the script normalize.py, avaliable in this repository. You can use this script to obtain the preprocessed text from an input text as follows:

python normalize.py text

Training

Hyperparameter

The model is based on Matcha-TTS proposed by Mehta et al. The following hyperparameters were set in the matcha-tts framework.

Hyperparameter	Value
Model	matcha
Batch Size	32
Mixed Precision	true
Hop Length	256
Optimizer	adam
Learning Rate Gen	0.0001

The model was trained for 6,500 epochs.

Additional information

Authors

Antonio Moscoso, Carmen Magariños and Alberto Bugarín.

Contact information

For further information, send an email to proxecto.nos@usc.gal

Funding

This research was produced within the framework of the Proxecto Nós, funded by the Ministry for Digital Transformation and Public Administration and the Recovery, Transformation, and Resilience Plan – Funded by the European Union – NextGenerationEU, as part of the Ilenia Project with reference 2022/TL22/00215336, and previously “The Nós project: Galician in the society and economy of Artificial Intelligence”, resulting from the agreement 2021-CP080 between the Xunta de Galicia and the University of Santiago de Compostela, and thanks to the Investigo program, within the National Recovery, Transformation and Resilience Plan, within the framework of the European Recovery Fund (NextGenerationEU).

Citation information

If you use this model, please cite as follows:

Moscoso, Antonio; Magariños, Carmen; Bugarín-Diz, Alberto. 2025. Nos_TTS-icia-matcha-phonemes-ljspeech. URL: https://huggingface.co/proxectonos/Nos_TTS-icia-matcha-phonemes-ljspeech