| --- |
| license: bsd-3-clause-clear |
| language: |
| - en |
| - zh |
| base_model: |
| - myshell-ai/MeloTTS-Chinese |
| pipeline_tag: text-to-speech |
| --- |
| |
| # melotts.axera |
|
|
| - MeloTTS DEMO on Axera AX650、AX630C |
| - 目前模型分成了 encoder、decoder 两部分，encoder 部分尚未转成 axmodel（目前通过 onnxruntime 运行） |
| - Github: https://github.com/ml-inory/melotts.axera |
|
|
|
|
| ## Support Platform |
|
|
| - AX650 |
| - [M4N-Dock(爱芯派Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html) |
| - [M.2 Accelerator card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html) |
| - AX630C |
| - [爱芯派2](https://axera-pi-2-docs-cn.readthedocs.io/zh-cn/latest/index.html) |
| - [Module-LLM](https://docs.m5stack.com/zh_CN/module/Module-LLM) |
| - [LLM630 Compute Kit](https://docs.m5stack.com/zh_CN/core/LLM630%20Compute%20Kit) |
|
|
| |Chips|output wav | cost time | RTF | |
| |--|--|--|--| |
| |AX650| 12s | 1.5s | 0.125 | |
| |AX630C| 12s | | | |
|
|
| ## Requirements |
|
|
| ### 添加中文输入支持 |
|
|
| 执行以下命令，正确安装中文输入法之后，重启终端登录 |
|
|
| ``` |
| locale-gen C.utf8 |
| update-locale LANG=C.utf8 |
| ``` |
|
|
| ### Python Requirements |
|
|
| #### Requirements |
|
|
| ``` |
| cp -rf nltk_data ~/ |
| apt-get install libsndfile1-dev libmecab-dev |
| cd python |
| pip3 install -r requirements.txt |
| ``` |
|
|
| #### pyaxengine |
|
|
| pyaxengine 是 npu 的 python api，详细安装请参考 |
|
|
| - https://github.com/AXERA-TECH/pyaxengine |
|
|
| ## How to use |
|
|
| ``` |
| root@ax650:/mnt/qtang/melotts.axera/python# python3 melotts.py --help |
| [INFO] Available providers: ['AxEngineExecutionProvider'] |
| usage: melotts [-h] [--sentence SENTENCE] [--wav WAV] [--encoder ENCODER] [--decoder DECODER] [--dec_len DEC_LEN] [--sample_rate SAMPLE_RATE] [--speed SPEED] |
| [--language {ZH,ZH_MIX_EN,JP,EN,KR,ES,SP,FR}] |
| |
| Run TTS on input sentence |
| |
| options: |
| -h, --help show this help message and exit |
| --sentence SENTENCE, -s SENTENCE |
| --wav WAV, -w WAV |
| --encoder ENCODER, -e ENCODER |
| --decoder DECODER, -d DECODER |
| --dec_len DEC_LEN |
| --sample_rate SAMPLE_RATE, -sr SAMPLE_RATE |
| --speed SPEED |
| --language {ZH,ZH_MIX_EN,JP,EN,KR,ES,SP,FR}, -l {ZH,ZH_MIX_EN,JP,EN,KR,ES,SP,FR} |
| |
| ``` |
|
|
| 输入命令 |
|
|
| ``` |
| python3 melotts.py -s 爱芯元智半导体股份有限公司，致力于打造世界领先的人工智能感知与边缘计算芯片。服务智慧城市、智能驾驶、机器人的海量普惠的应用 \ |
| -e encoder-onnx/encoder-zh.onnx \ |
| -d decoder-ax650/decoder-zh.axmodel \ |
| ``` |
|
|
| ``` |
| root@ax650:/mnt/qtang/melotts.axera/python# python3 melotts.py \ |
| --wav output.wav \ |
| --encoder ../models/encoder-onnx/encoder-zh.onnx \ |
| --decoder ../models/ax650/decoder-zh.axmodel \ |
| --language ZH \ |
| --speed 0.9 |
| |
| [INFO] Available providers: ['AxEngineExecutionProvider'] |
| sentence: 爱芯元智半导体股份有限公司，致力于打造世界领先的人工智能感知与边缘计算芯片。服务智慧城市、智能驾驶、机器人的海量普惠的应用 |
| sample_rate: 44100 |
| encoder: ../models/encoder-onnx/encoder-zh.onnx |
| decoder: ../models/ax650/decoder-zh.axmodel |
| language: ZH_MIX_EN |
| > Text split to sentences. |
| 爱芯元智半导体股份有限公司, |
| 致力于打造世界领先的人工智能感知与边缘计算芯片. |
| 服务智慧城市、智能驾驶、机器人的海量普惠的应用 |
| > =========================== |
| split_sentences_into_pieces take 3.1397342681884766ms |
| [INFO] Using provider: AxEngineExecutionProvider |
| [INFO] Chip type: ChipType.MC50 |
| [INFO] VNPU type: VNPUType.DISABLED |
| [INFO] Engine version: 2.10.1s |
| [INFO] Model type: 0 (single core) |
| [INFO] Compiler version: 3.3 3251425d |
| load models take 7986.6042137146ms |
| |
| Sentence[0]: 爱芯元智半导体股份有限公司, |
| Load language module take 33348.33884239197ms |
| Building prefix dict from the default dictionary ... |
| Loading model from cache /tmp/jieba.cache |
| Loading model cost 3.227 seconds. |
| Prefix dict has been built successfully. |
| encoder run take 89.70ms |
| Decode slice[0]: decoder run take 108.08ms |
| Decode slice[1]: decoder run take 92.15ms |
| Decode slice[2]: decoder run take 92.17ms |
| |
| Sentence[1]: 致力于打造世界领先的人工智能感知与边缘计算芯片. |
| Load language module take 0.042438507080078125ms |
| encoder run take 122.83ms |
| Decode slice[0]: decoder run take 92.24ms |
| Decode slice[1]: decoder run take 92.34ms |
| Decode slice[2]: decoder run take 92.16ms |
| Decode slice[3]: decoder run take 92.16ms |
| Decode slice[4]: decoder run take 92.22ms |
| |
| Sentence[2]: 服务智慧城市、智能驾驶、机器人的海量普惠的应用 |
| Load language module take 0.046253204345703125ms |
| encoder run take 112.59ms |
| Decode slice[0]: decoder run take 92.26ms |
| Decode slice[1]: decoder run take 92.16ms |
| Decode slice[2]: decoder run take 92.13ms |
| Decode slice[3]: decoder run take 92.13ms |
| Decode slice[4]: decoder run take 92.10ms |
| Save to output.wav |
| root@ax650:/mnt/qtang/melotts.axera/python# |
| ``` |
|
|
| 输出音频 |
|
|
| https://github.com/user-attachments/assets/eda5c10c-7d30-46e5-a56a-f6edcf7813af |
|
|
|
|
| 详细的运行参数： |
| | 参数名称 | 说明 | 默认值 | |
| | --- | --- | --- | |
| | -s/--sentence | 输入句子 | | |
| | -w/--wav | 输出音频路径，wav格式 | output.wav | |
| | -e/--encoder | encoder模型路径 | ../models/encoder.onnx | |
| | -d/--decoder | decoder模型路径 | ../models/decoder.axmodel | |
| | -sr/--sample_rate | 采样率 | 44100 | |
| | --speed | 语速，越大表示越快 | 0.8 | |
| | --language | 从"ZH", "ZH_MIX_EN", "JP", "EN", 'KR', "SP", "FR"选择，分别对应中文、中英混合、日语、英语、韩语、西班牙语，法语 | ZH_MIX_EN |
| |
