Compatible with ZipVoice AXCL version
Browse files- .gitattributes +7 -0
- README.md +39 -23
- cpp/CMakeLists.txt +17 -13
- cpp/README.md +47 -249
- cpp/build_ax650.sh +6 -5
- cpp/build_axcl.sh +34 -0
- cpp/cmake/msp_dependencies.cmake +64 -19
- cpp/download_bsp.sh +16 -1
- cpp/install/ax650/zipvoice +3 -0
- cpp/install/axcl/zipvoice +3 -0
- cpp/src/EngineWrapper.cpp +9 -278
- cpp/src/EngineWrapper.hpp +30 -11
- cpp/src/EngineWrapper_axcl.cpp +217 -0
- cpp/src/EngineWrapper_axera.cpp +283 -0
- cpp/src/middleware/ax_npu_runner.cc +131 -0
- cpp/src/middleware/ax_npu_runner.hpp +65 -0
- cpp/src/middleware/ax_npu_runner_impl.hpp +640 -0
- cpp/src/middleware/axcl_base.cc +241 -0
- cpp/src/middleware/axcl_base.hpp +42 -0
- cpp/src/middleware/axcl_native_runner.cc +105 -0
- cpp/src/middleware/axcl_native_runner.hpp +57 -0
- cpp/src/middleware/axcl_native_runner_impl.hpp +504 -0
- cpp/src/middleware/axcl_runtime_runner.cc +105 -0
- cpp/src/middleware/axcl_runtime_runner.hpp +56 -0
- cpp/src/middleware/axcl_runtime_runner_impl.hpp +482 -0
- cpp/src/middleware/runner.cc +88 -0
- cpp/src/middleware/runner.hpp +67 -0
- cpp/src/utilities/file.hpp +251 -0
- cpp/src/utilities/file_mapper.hpp +86 -0
- cpp/src/utilities/log.hpp +64 -0
- cpp/src/utilities/scalar_guard.hpp +60 -0
- cpp/src/utilities/timer.hpp +84 -0
- cpp/src/utilities/vector_guard.hpp +64 -0
- cpp/src/vocoder.cpp +1 -1
- cpp/src/vocoder.hpp +1 -0
- cpp/src/zipvoice_engine.cpp +5 -5
- cpp/src/zipvoice_engine.hpp +5 -7
- cpp/toolchains/aarch64-none-linux-gnu.toolchain.cmake +4 -15
- cpp/zipvoice.cpp +13 -2
- run_ax650.sh +57 -0
- run_axcl.sh +57 -0
- scripts/__pycache__/__init__.cpython-310.pyc +0 -0
- scripts/__pycache__/local_tokenizer.cpython-310.pyc +0 -0
- scripts/__pycache__/text_processing.cpython-310.pyc +0 -0
.gitattributes
CHANGED
|
@@ -35,8 +35,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/moss_prompts/en_4_4p5s.wav filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/moss_prompts/zh_1_4p5s.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 38 |
cpp/install/ax650/zipvoice_axera filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 39 |
cpp/vocoder/vocos_full.axmodel filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
models/zipvoice_ax650/decoder_part0.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 41 |
models/zipvoice_ax650/decoder_part1.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 42 |
models/zipvoice_ax650/decoder_part2.axmodel filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/moss_prompts/en_4_4p5s.wav filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/moss_prompts/zh_1_4p5s.wav filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
cpp/install/ax650/zipvoice filter=lfs diff=lfs merge=lfs -text
|
| 39 |
cpp/install/ax650/zipvoice_axera filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
cpp/install/axcl/zipvoice filter=lfs diff=lfs merge=lfs -text
|
| 41 |
cpp/vocoder/vocos_full.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
models/axmodels/decoder_part0.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
models/axmodels/decoder_part1.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
models/axmodels/decoder_part2.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
models/axmodels/decoder_part3.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
models/axmodels/encoder.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 47 |
models/zipvoice_ax650/decoder_part0.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 48 |
models/zipvoice_ax650/decoder_part1.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 49 |
models/zipvoice_ax650/decoder_part2.axmodel filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,27 +1,49 @@
|
|
| 1 |
# ZipVoice.AXERA
|
| 2 |
|
| 3 |
-
[ZipVoice]((https://github.com/k2-fsa/ZipVoice)) AXERA 板端推理 demo。
|
| 4 |
|
| 5 |
## 功能
|
| 6 |
|
| 7 |
- 支持中文和英文语音生成。
|
| 8 |
- 支持语音克隆。
|
| 9 |
- 支持 ZipVoice、ZipVoice Distill
|
|
|
|
|
|
|
| 10 |
|
| 11 |
## 模型说明
|
| 12 |
|
| 13 |
-
ZipVoice Distill 是 ZipVoice 的蒸馏版本，主要优势是在较小性能损失下提升推理速度。
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
## 模型
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
模型量化参考:
|
| 20 |
|
| 21 |
- [Pulsar2 Docs — How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)
|
| 22 |
- [export / quantization Project](https://github.com/AXERA-TECH/ZipVoice.AXERA)
|
| 23 |
|
| 24 |
-
## 支持平台
|
| 25 |
|
| 26 |
- AX650
|
| 27 |
- AX650 demo 板
|
|
@@ -29,7 +51,7 @@ AX630C 版本当前推理结果差，RTF 约为 1.5 左右，需要继续调优
|
|
| 29 |
- [M.2 Accelerator Card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)
|
| 30 |
|
| 31 |
|
| 32 |
-
## 目录结构
|
| 33 |
|
| 34 |
```text
|
| 35 |
ZipVoice.AXERA/
|
|
@@ -49,7 +71,7 @@ ZipVoice.AXERA/
|
|
| 49 |
└── README.md
|
| 50 |
```
|
| 51 |
|
| 52 |
-
## 环境
|
| 53 |
|
| 54 |
安装 pyaxengine：
|
| 55 |
|
|
@@ -65,7 +87,7 @@ conda activate ZipVoice
|
|
| 65 |
pip3 install -r requirements.txt
|
| 66 |
```
|
| 67 |
|
| 68 |
-
## 推理命令
|
| 69 |
|
| 70 |
进入目录：
|
| 71 |
|
|
@@ -73,7 +95,7 @@ pip3 install -r requirements.txt
|
|
| 73 |
cd ZipVoice.AXERA
|
| 74 |
```
|
| 75 |
|
| 76 |
-
### AX650 ZipVoice
|
| 77 |
|
| 78 |
中文句子：
|
| 79 |
|
|
@@ -163,7 +185,7 @@ RTF: 0.9600
|
|
| 163 |
音频：[outputs/en_long_paragraph_ax650.wav](outputs/en_long_paragraph_ax650.wav)
|
| 164 |
提示音：[assets/moss_prompts/en_4_4p5s.wav](assets/moss_prompts/en_4_4p5s.wav)
|
| 165 |
|
| 166 |
-
### AX650 ZipVoice Distill
|
| 167 |
|
| 168 |
中文句子：
|
| 169 |
|
|
@@ -254,7 +276,7 @@ RTF: 0.3045
|
|
| 254 |
提示音：[assets/moss_prompts/en_4_4p5s.wav](assets/moss_prompts/en_4_4p5s.wav)
|
| 255 |
|
| 256 |
|
| 257 |
-
## 参数说明
|
| 258 |
|
| 259 |
- `--model-name`：选择模型目录。可选 `zipvoice_ax650`、`zipvoice_distill_ax650`、`zipvoice_distill_ax630C`。
|
| 260 |
- `--prompt-wav`：参考音频，用于控制音色，建议 3-5s。
|
|
@@ -262,19 +284,13 @@ RTF: 0.3045
|
|
| 262 |
- `--num-step`：采样步数。默认从模型目录的 `runtime_config.json` 读取。
|
| 263 |
- `--max-feat-len`：decoder 固定 feature 长度，当前模型均为 1024。
|
| 264 |
|
| 265 |
-
##
|
| 266 |
|
| 267 |
-
|
| 268 |
-
|
| 269 |
-
## C++ 版本
|
| 270 |
-
|
| 271 |
-
纯 C++ 推理实现，全链路 axmodel，性能更高。详见 [/data/shared/huyuan/TTS/ZipVoice.AXERA/cpp/README.md](cpp/README.md)。
|
| 272 |
|
| 273 |
-
### 性能对比 (AX650 Distill, 中文段落, 44.97s 音频)
|
| 274 |
|
| 275 |
-
|
| 276 |
-
|------|---------|-----|
|
| 277 |
-
| Python | 13.457s | 0.30 |
|
| 278 |
-
| C++ | 7.357s | 0.16 |
|
| 279 |
|
| 280 |
-
|
|
|
|
|
|
|
|
|
| 1 |
# ZipVoice.AXERA
|
| 2 |
|
| 3 |
+
[ZipVoice]((https://github.com/k2-fsa/ZipVoice)) AXERA 板端 & AXCL 算力卡推理 demo。
|
| 4 |
|
| 5 |
## 功能
|
| 6 |
|
| 7 |
- 支持中文和英文语音生成。
|
| 8 |
- 支持语音克隆。
|
| 9 |
- 支持 ZipVoice、ZipVoice Distill
|
| 10 |
+
- 支持 C++ 和 Python 推理
|
| 11 |
+
- 支持 AXERA 和 AXCL platform
|
| 12 |
|
| 13 |
## 模型说明
|
| 14 |
|
| 15 |
+
ZipVoice Distill 是 ZipVoice 的蒸馏版本，主要优势是在较小性能损失下提升推理速度。
|
| 16 |
|
| 17 |
+
## 性能对比
|
| 18 |
|
| 19 |
+
### Distill 模型 (num_step=4)
|
| 20 |
+
|
| 21 |
+
| 场景 | 音频 | Python AX650 | C++ AX650 | C++ AXCL |
|
| 22 |
+
|------|------|-------------|-----------|----------|
|
| 23 |
+
| 中文句子 | 6.41s | 1.992s / 0.311 | 1.057s / 0.165 | 0.918s / **0.143** |
|
| 24 |
+
| 中文段落 | 44.97s | 13.457s / 0.301 | 7.357s / 0.164 | 6.443s / **0.143** |
|
| 25 |
+
| 英文句子 | 6.41s | 2.045s / 0.319 | 1.067s / 0.166 | 0.919s / **0.143** |
|
| 26 |
+
| 英文段落 | 59.16s | 19.715s / 0.305 | 10.529s / 0.178 | 9.207s / **0.156** |
|
| 27 |
+
|
| 28 |
+
### 普通模型 (num_step=10)
|
| 29 |
+
|
| 30 |
+
| 场景 | 音频 | Python AX650 | C++ AX650 | C++ AXCL |
|
| 31 |
+
|------|------|-------------|-----------|----------|
|
| 32 |
+
| 中文句子 | 6.41s | 5.781s / 0.902 | 5.709s / 0.891 | 5.452s / **0.850** |
|
| 33 |
+
| 中文段落 | 44.97s | 40.292s / 0.901 | 39.714s / 0.883 | 38.138s / **0.848** |
|
| 34 |
+
| 英文句子 | 6.41s | 5.711s / 0.891 | 5.693s / 0.888 | 5.447s / **0.850** |
|
| 35 |
+
| 英文段落 | 59.16s | 62.161s / 0.960 | 56.759s / 0.957 | 54.451s / **0.920** |
|
| 36 |
+
|
| 37 |
+
## Python
|
| 38 |
+
|
| 39 |
+
### 模型转换
|
| 40 |
|
| 41 |
模型量化参考:
|
| 42 |
|
| 43 |
- [Pulsar2 Docs — How to Convert ONNX to axmodel](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)
|
| 44 |
- [export / quantization Project](https://github.com/AXERA-TECH/ZipVoice.AXERA)
|
| 45 |
|
| 46 |
+
### 支持平台
|
| 47 |
|
| 48 |
- AX650
|
| 49 |
- AX650 demo 板
|
|
|
|
| 51 |
- [M.2 Accelerator Card](https://docs.m5stack.com/zh_CN/ai_hardware/LLM-8850_Card)
|
| 52 |
|
| 53 |
|
| 54 |
+
### 目录结构
|
| 55 |
|
| 56 |
```text
|
| 57 |
ZipVoice.AXERA/
|
|
|
|
| 71 |
└── README.md
|
| 72 |
```
|
| 73 |
|
| 74 |
+
### 环境
|
| 75 |
|
| 76 |
安装 pyaxengine：
|
| 77 |
|
|
|
|
| 87 |
pip3 install -r requirements.txt
|
| 88 |
```
|
| 89 |
|
| 90 |
+
### 推理命令
|
| 91 |
|
| 92 |
进入目录：
|
| 93 |
|
|
|
|
| 95 |
cd ZipVoice.AXERA
|
| 96 |
```
|
| 97 |
|
| 98 |
+
#### AX650 ZipVoice
|
| 99 |
|
| 100 |
中文句子：
|
| 101 |
|
|
|
|
| 185 |
音频：[outputs/en_long_paragraph_ax650.wav](outputs/en_long_paragraph_ax650.wav)
|
| 186 |
提示音：[assets/moss_prompts/en_4_4p5s.wav](assets/moss_prompts/en_4_4p5s.wav)
|
| 187 |
|
| 188 |
+
#### AX650 ZipVoice Distill
|
| 189 |
|
| 190 |
中文句子：
|
| 191 |
|
|
|
|
| 276 |
提示音：[assets/moss_prompts/en_4_4p5s.wav](assets/moss_prompts/en_4_4p5s.wav)
|
| 277 |
|
| 278 |
|
| 279 |
+
### 参数说明
|
| 280 |
|
| 281 |
- `--model-name`：选择模型目录。可选 `zipvoice_ax650`、`zipvoice_distill_ax650`、`zipvoice_distill_ax630C`。
|
| 282 |
- `--prompt-wav`：参考音频，用于控制音色，建议 3-5s。
|
|
|
|
| 284 |
- `--num-step`：采样步数。默认从模型目录的 `runtime_config.json` 读取。
|
| 285 |
- `--max-feat-len`：decoder 固定 feature 长度，当前模型均为 1024。
|
| 286 |
|
| 287 |
+
## C++
|
| 288 |
|
| 289 |
+
C++ 推理实现。详见 [cpp/README.md](cpp/README.md)。
|
|
|
|
|
|
|
|
|
|
|
|
|
| 290 |
|
|
|
|
| 291 |
|
| 292 |
+
## 参考
|
|
|
|
|
|
|
|
|
|
| 293 |
|
| 294 |
+
- [ZipVoice](https://github.com/k2-fsa/ZipVoice)
|
| 295 |
+
- [Pulsar2 Docs](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)
|
| 296 |
+
- [模型量化工程](https://github.com/AXERA-TECH/ZipVoice.AXERA)
|
cpp/CMakeLists.txt
CHANGED
|
@@ -1,7 +1,9 @@
|
|
| 1 |
cmake_minimum_required(VERSION 3.13 FATAL_ERROR)
|
| 2 |
-
project(
|
| 3 |
|
| 4 |
-
|
|
|
|
|
|
|
| 5 |
|
| 6 |
if(CMAKE_BUILD_TYPE MATCHES Debug)
|
| 7 |
set(CMAKE_CXX_FLAGS "-fvisibility=hidden -g -O0")
|
|
@@ -9,17 +11,16 @@ elseif(CMAKE_BUILD_TYPE MATCHES Release)
|
|
| 9 |
set(CMAKE_CXX_FLAGS "-fvisibility=hidden -O2 -fdata-sections -ffunction-sections")
|
| 10 |
endif()
|
| 11 |
|
| 12 |
-
#
|
| 13 |
-
# Adjust these paths to match your SDK installation
|
| 14 |
include(cmake/msp_dependencies.cmake)
|
| 15 |
|
| 16 |
include_directories(${MSP_INC_DIR})
|
| 17 |
-
link_directories(${MSP_LIB_DIR})
|
| 18 |
|
| 19 |
-
# Project includes
|
| 20 |
include_directories(${CMAKE_SOURCE_DIR})
|
|
|
|
| 21 |
|
| 22 |
-
#
|
| 23 |
set(SRC
|
| 24 |
src/EngineWrapper.cpp
|
| 25 |
src/tokenizer.cpp
|
|
@@ -29,6 +30,14 @@ set(SRC
|
|
| 29 |
third_party/kissfft/kiss_fft.c
|
| 30 |
)
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
add_executable(${PROJECT_NAME} zipvoice.cpp ${SRC})
|
| 33 |
target_include_directories(${PROJECT_NAME} PRIVATE third_party/kissfft)
|
| 34 |
target_link_libraries(${PROJECT_NAME} ${MSP_LIBS})
|
|
@@ -39,11 +48,6 @@ set_target_properties(${PROJECT_NAME}
|
|
| 39 |
PROPERTIES
|
| 40 |
INSTALL_RPATH "$ORIGIN/")
|
| 41 |
|
| 42 |
-
# Print configuration summary
|
| 43 |
-
message(STATUS "========================================")
|
| 44 |
-
message(STATUS "ZipVoice AXERA C++ Build Configuration")
|
| 45 |
message(STATUS "========================================")
|
| 46 |
-
message(STATUS "
|
| 47 |
-
message(STATUS "MSP_LIB_DIR: ${MSP_LIB_DIR}")
|
| 48 |
-
message(STATUS "MSP_LIBS: ${MSP_LIBS}")
|
| 49 |
message(STATUS "========================================")
|
|
|
|
| 1 |
cmake_minimum_required(VERSION 3.13 FATAL_ERROR)
|
| 2 |
+
project(zipvoice)
|
| 3 |
|
| 4 |
+
if (NOT DEFINED CMAKE_CXX_STANDARD)
|
| 5 |
+
set(CMAKE_CXX_STANDARD 14)
|
| 6 |
+
endif()
|
| 7 |
|
| 8 |
if(CMAKE_BUILD_TYPE MATCHES Debug)
|
| 9 |
set(CMAKE_CXX_FLAGS "-fvisibility=hidden -g -O0")
|
|
|
|
| 11 |
set(CMAKE_CXX_FLAGS "-fvisibility=hidden -O2 -fdata-sections -ffunction-sections")
|
| 12 |
endif()
|
| 13 |
|
| 14 |
+
# Platform dependencies (AXERA or AXCL)
|
|
|
|
| 15 |
include(cmake/msp_dependencies.cmake)
|
| 16 |
|
| 17 |
include_directories(${MSP_INC_DIR})
|
|
|
|
| 18 |
|
| 19 |
+
# Project includes
|
| 20 |
include_directories(${CMAKE_SOURCE_DIR})
|
| 21 |
+
include_directories(${CMAKE_SOURCE_DIR}/src/)
|
| 22 |
|
| 23 |
+
# Core source files (common to AXERA and AXCL)
|
| 24 |
set(SRC
|
| 25 |
src/EngineWrapper.cpp
|
| 26 |
src/tokenizer.cpp
|
|
|
|
| 30 |
third_party/kissfft/kiss_fft.c
|
| 31 |
)
|
| 32 |
|
| 33 |
+
# AXCL-only source files (middleware + utilities)
|
| 34 |
+
if(AXCL)
|
| 35 |
+
aux_source_directory(${CMAKE_SOURCE_DIR}/src/middleware AXCL_SRCS)
|
| 36 |
+
aux_source_directory(${CMAKE_SOURCE_DIR}/src/utilities AXCL_SRCS)
|
| 37 |
+
list(APPEND SRC ${AXCL_SRCS})
|
| 38 |
+
message(STATUS "AXCL middleware: ${AXCL_SRCS}")
|
| 39 |
+
endif()
|
| 40 |
+
|
| 41 |
add_executable(${PROJECT_NAME} zipvoice.cpp ${SRC})
|
| 42 |
target_include_directories(${PROJECT_NAME} PRIVATE third_party/kissfft)
|
| 43 |
target_link_libraries(${PROJECT_NAME} ${MSP_LIBS})
|
|
|
|
| 48 |
PROPERTIES
|
| 49 |
INSTALL_RPATH "$ORIGIN/")
|
| 50 |
|
|
|
|
|
|
|
|
|
|
| 51 |
message(STATUS "========================================")
|
| 52 |
+
message(STATUS "ZipVoice C++ Build Configuration")
|
|
|
|
|
|
|
| 53 |
message(STATUS "========================================")
|
cpp/README.md
CHANGED
|
@@ -1,286 +1,84 @@
|
|
| 1 |
-
# ZipVoice
|
| 2 |
|
| 3 |
-
|
| 4 |
|
| 5 |
-
##
|
| 6 |
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
-
|
| 10 |
-
|------|------|------|-----|
|
| 11 |
-
| 中文句子 | 6.41s | 1.057s | 0.165 |
|
| 12 |
-
| 中文段落 | 44.97s | 7.357s | 0.164 |
|
| 13 |
-
| 英文句子 | 6.41s | 1.067s | 0.166 |
|
| 14 |
-
| 英文段落 | 59.31s | 10.529s | 0.178 |
|
| 15 |
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
|
| 22 |
-
| 英文句子 | 6.41s | 5.693s | 0.888 |
|
| 23 |
-
| 英文段落 | 59.31s | 56.759s | 0.957 |
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
## 目录结构
|
| 28 |
-
|
| 29 |
-
```
|
| 30 |
-
├── CMakeLists.txt # 构建配置
|
| 31 |
-
├── zipvoice.cpp # 主程序
|
| 32 |
-
├── build_ax650.sh # 交叉编译脚本
|
| 33 |
-
├── download_bsp.sh # 下载 BSP SDK
|
| 34 |
-
├── cmake/
|
| 35 |
-
│ └── msp_dependencies.cmake # MSP SDK 配置
|
| 36 |
-
├── toolchains/
|
| 37 |
-
│ └── aarch64-none-linux-gnu.toolchain.cmake # 工具链
|
| 38 |
-
├── third_party/kissfft/ # kissfft (vocoder IRFFT)
|
| 39 |
-
├── utils/ # 日志/IO/检查
|
| 40 |
-
├── src/ # 核心源码
|
| 41 |
-
│ ├── EngineWrapper.* # AX引擎封装
|
| 42 |
-
│ ├── tokenizer.* # 中文分词 (41K汉字pinyin映射)
|
| 43 |
-
│ ├── pinyin_table.hpp # 汉字→拼音表 (自动生成)
|
| 44 |
-
│ ├── fbank.* # Mel滤波器 (FFT)
|
| 45 |
-
│ ├── zipvoice_engine.* # 推理引擎 (encoder+decoder4)
|
| 46 |
-
│ ├── vocoder.* # 声码器 (axmodel+kissfft IRFFT)
|
| 47 |
-
│ └── wav_writer.hpp # WAV写入
|
| 48 |
-
├── vocoder/
|
| 49 |
-
│ └── vocos_full.axmodel # 量化声码器 (15MB)
|
| 50 |
-
├── scripts/ # Python辅助脚本
|
| 51 |
-
│ ├── gen_cat_tokens.py # 预计算token
|
| 52 |
-
│ ├── gen_pinyin_table.py # 生成汉字映射表
|
| 53 |
-
│ ├── export_vocos_onnx.py # 导出ONNX
|
| 54 |
-
│ ├── generate_vocoder_calib.py # 校准数据
|
| 55 |
-
│ └── quantize_vocoder.sh # pulsar2量化
|
| 56 |
-
└── README.md
|
| 57 |
-
```
|
| 58 |
|
| 59 |
## 环境准备
|
| 60 |
|
|
|
|
|
|
|
| 61 |
```bash
|
| 62 |
-
|
| 63 |
-
bash download_bsp.sh
|
| 64 |
|
| 65 |
-
#
|
|
|
|
| 66 |
wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 67 |
tar -xf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
## 编译
|
| 72 |
-
|
| 73 |
-
```bash
|
| 74 |
-
mkdir build_ax650 && cd build_ax650
|
| 75 |
-
cmake .. \
|
| 76 |
-
-DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-none-linux-gnu.toolchain.cmake \
|
| 77 |
-
-DTOOLCHAIN_DIR=/path/to/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu \
|
| 78 |
-
-DCMAKE_INSTALL_PREFIX=../install/ax650 \
|
| 79 |
-
-DCMAKE_BUILD_TYPE=Release
|
| 80 |
-
make -j$(nproc) && make install
|
| 81 |
-
```
|
| 82 |
-
|
| 83 |
-
## 板端推理
|
| 84 |
-
|
| 85 |
-
所有命令在 repo 根目录（`ZipVoice.AXERA/`）下执行。
|
| 86 |
-
|
| 87 |
-
### 中文
|
| 88 |
-
|
| 89 |
-
```bash
|
| 90 |
-
# 普通模型 — 句子
|
| 91 |
-
cpp/install/ax650/zipvoice_axera \
|
| 92 |
-
--model-dir ./models/zipvoice_ax650 \
|
| 93 |
-
--token-file ./resources/zipvoice_hf/zipvoice/tokens.txt \
|
| 94 |
-
--prompt-wav ./assets/moss_prompts/zh_1_4p5s.wav \
|
| 95 |
-
--prompt-text "不管怎么样我和汤姆还是要感谢贝尔卡金的援手" \
|
| 96 |
-
--text "今天午后天气很好，我打开窗户，听见远处有人聊天，水杯也轻轻晃了一下。" \
|
| 97 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 98 |
-
--output-wav output.wav --seed 42
|
| 99 |
-
```
|
| 100 |
-
|
| 101 |
-
```text
|
| 102 |
-
推理结果:
|
| 103 |
-
Segments: 1
|
| 104 |
-
Audio duration: 6.41 s
|
| 105 |
-
Total time: 5.71 s
|
| 106 |
-
RTF (端到端): 0.891
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
-
```bash
|
| 110 |
-
# 蒸馏模型 — 句子 (约3倍速)
|
| 111 |
-
cpp/install/ax650/zipvoice_axera \
|
| 112 |
-
--model-dir ./models/zipvoice_distill_ax650 \
|
| 113 |
-
--token-file ./resources/zipvoice_hf/zipvoice/tokens.txt \
|
| 114 |
-
--prompt-wav ./assets/moss_prompts/zh_1_4p5s.wav \
|
| 115 |
-
--prompt-text "不管怎么样我和汤姆还是要感谢贝尔卡金的援手" \
|
| 116 |
-
--text "今天午后天气很好，我打开窗户，听见远处有人聊天，水杯也轻轻晃了一下。" \
|
| 117 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 118 |
-
--output-wav output.wav --seed 42
|
| 119 |
-
```
|
| 120 |
-
|
| 121 |
-
```text
|
| 122 |
-
推理结果:
|
| 123 |
-
Segments: 1
|
| 124 |
-
Audio duration: 6.41 s
|
| 125 |
-
Total time: 1.06 s
|
| 126 |
-
RTF (端到端): 0.165
|
| 127 |
```
|
| 128 |
|
| 129 |
-
```
|
| 130 |
-
# 普通模型 — 段落
|
| 131 |
-
cpp/install/ax650/zipvoice_axera \
|
| 132 |
-
--model-dir ./models/zipvoice_ax650 \
|
| 133 |
-
--token-file ./resources/zipvoice_hf/zipvoice/tokens.txt \
|
| 134 |
-
--prompt-wav ./assets/moss_prompts/zh_1_4p5s.wav \
|
| 135 |
-
--prompt-text "不管怎么样我和汤姆还是要感谢贝尔卡金的援手" \
|
| 136 |
-
--text-file ./assets/paragraphs/zh_ginkgo.txt \
|
| 137 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 138 |
-
--output-wav output.wav --seed 42
|
| 139 |
-
```
|
| 140 |
-
|
| 141 |
-
```text
|
| 142 |
-
推理结果:
|
| 143 |
-
Segments: 7
|
| 144 |
-
Audio duration: 44.97 s
|
| 145 |
-
Total time: 39.71 s
|
| 146 |
-
RTF (端到端): 0.883
|
| 147 |
-
```
|
| 148 |
|
|
|
|
| 149 |
```bash
|
| 150 |
-
|
| 151 |
-
cpp/install/ax650/zipvoice_axera \
|
| 152 |
-
--model-dir ./models/zipvoice_distill_ax650 \
|
| 153 |
-
--token-file ./resources/zipvoice_hf/zipvoice/tokens.txt \
|
| 154 |
-
--prompt-wav ./assets/moss_prompts/zh_1_4p5s.wav \
|
| 155 |
-
--prompt-text "不管怎么样我和汤姆还是要感谢贝尔卡金的援手" \
|
| 156 |
-
--text-file ./assets/paragraphs/zh_ginkgo.txt \
|
| 157 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 158 |
-
--output-wav output.wav --seed 42
|
| 159 |
```
|
| 160 |
|
| 161 |
-
|
| 162 |
-
推理结果:
|
| 163 |
-
Segments: 7
|
| 164 |
-
Audio duration: 44.97 s
|
| 165 |
-
Total time: 7.38 s
|
| 166 |
-
RTF (端到端): 0.164
|
| 167 |
-
```
|
| 168 |
-
|
| 169 |
-
### 英文 (加 `--repo-dir .` 自动调 Python 分词器)
|
| 170 |
-
|
| 171 |
-
```bash
|
| 172 |
-
# 普通模型 — 句子
|
| 173 |
-
cpp/install/ax650/zipvoice_axera \
|
| 174 |
-
--model-dir ./models/zipvoice_ax650 \
|
| 175 |
-
--prompt-wav ./assets/moss_prompts/en_4_4p5s.wav \
|
| 176 |
-
--prompt-text "This is almost twice the current industry production level per train." \
|
| 177 |
-
--text "This morning, a small train left the station, carrying sleepy passengers toward a bright coastal town." \
|
| 178 |
-
--repo-dir . \
|
| 179 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 180 |
-
--output-wav output_en.wav --seed 42
|
| 181 |
-
```
|
| 182 |
-
|
| 183 |
-
```text
|
| 184 |
-
推理结果:
|
| 185 |
-
Segments: 1
|
| 186 |
-
Audio duration: 6.41 s
|
| 187 |
-
Total time: 5.69 s
|
| 188 |
-
RTF (端到端): 0.888
|
| 189 |
-
```
|
| 190 |
|
| 191 |
```bash
|
| 192 |
-
#
|
| 193 |
-
|
| 194 |
-
--model-dir ./models/zipvoice_distill_ax650 \
|
| 195 |
-
--prompt-wav ./assets/moss_prompts/en_4_4p5s.wav \
|
| 196 |
-
--prompt-text "This is almost twice the current industry production level per train." \
|
| 197 |
-
--text "This morning, a small train left the station, carrying sleepy passengers toward a bright coastal town." \
|
| 198 |
-
--repo-dir . \
|
| 199 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 200 |
-
--output-wav output_en.wav --seed 42
|
| 201 |
-
```
|
| 202 |
-
|
| 203 |
-
```text
|
| 204 |
-
推理结果:
|
| 205 |
-
Segments: 1
|
| 206 |
-
Audio duration: 6.41 s
|
| 207 |
-
Total time: 1.06 s
|
| 208 |
-
RTF (端到端): 0.166
|
| 209 |
```
|
| 210 |
|
| 211 |
-
|
| 212 |
-
# 普通模型 — 段落
|
| 213 |
-
cpp/install/ax650/zipvoice_axera \
|
| 214 |
-
--model-dir ./models/zipvoice_ax650 \
|
| 215 |
-
--prompt-wav ./assets/moss_prompts/en_4_4p5s.wav \
|
| 216 |
-
--prompt-text "This is almost twice the current industry production level per train." \
|
| 217 |
-
--text-file ./assets/paragraphs/en_scavenger.txt \
|
| 218 |
-
--repo-dir . \
|
| 219 |
-
--vocoder-model ./cpp/vocoder/vocos_full.axmodel \
|
| 220 |
-
--output-wav output_en.wav --seed 42
|
| 221 |
-
```
|
| 222 |
|
| 223 |
-
``
|
| 224 |
-
推理结果:
|
| 225 |
-
Segments: 10
|
| 226 |
-
Audio duration: 59.31 s
|
| 227 |
-
Total time: 56.76 s
|
| 228 |
-
RTF (端到端): 0.957
|
| 229 |
-
```
|
| 230 |
|
| 231 |
```bash
|
| 232 |
-
#
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
```
|
| 242 |
|
| 243 |
-
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
Total time: 10.56 s
|
| 248 |
-
RTF (端到端): 0.178
|
| 249 |
```
|
| 250 |
|
| 251 |
-
##
|
| 252 |
|
| 253 |
| 参数 | 说明 | 默认值 |
|
| 254 |
|------|------|--------|
|
| 255 |
| `--model-dir` | 模型目录 | 必填 |
|
| 256 |
-
| `--token-file` | tokens.txt (中文) | 可选 |
|
| 257 |
| `--prompt-wav` | 提示音频 | 必填 |
|
| 258 |
| `--prompt-text` | 提示文本 | 必填 |
|
| 259 |
| `--text` / `--text-file` | 合成文本 | 二选一 |
|
| 260 |
-
| `--
|
| 261 |
-
| `--
|
| 262 |
-
| `--output-wav` | 输出WAV | output.wav |
|
| 263 |
| `--num-step` | 采样步数 | 10 (distill: 4) |
|
| 264 |
-
| `--speed` | 语速 | 1.0 |
|
| 265 |
| `--seed` | 随机种子 | 42 |
|
| 266 |
-
|
| 267 |
-
## 重新量化 vocoder
|
| 268 |
-
|
| 269 |
-
```bash
|
| 270 |
-
cd cpp/scripts
|
| 271 |
-
python3 export_vocos_onnx.py # 导出ONNX
|
| 272 |
-
python3 generate_vocoder_calib.py # 生成校准数据
|
| 273 |
-
source /path/to/npu_dev && bash quantize_vocoder.sh # 量化
|
| 274 |
-
```
|
| 275 |
-
|
| 276 |
-
## 实现要点
|
| 277 |
-
|
| 278 |
-
- **中文分词**: C++ 内置, 基于 pypinyin 的 41K 汉字→拼音映射表, 0ms 开销
|
| 279 |
-
- **英文分词**: Python daemon 持久进程, 首次 import ~8s, 后续每次 ~10ms
|
| 280 |
-
- **长文本分段**: 对齐 Python `text_processing.build_segments`, 自动按 token/帧数约束切段
|
| 281 |
-
- **声码器**: vocos axmodel (NPU) + kissfft IRFFT (CPU), 比手写逐点版快 ~30x
|
| 282 |
-
- **RTF 口径**: 仅计入增量推理 (tokenizer + encoder + decoder + vocoder), 不含一次性初始化
|
| 283 |
-
|
| 284 |
-
## 许可
|
| 285 |
-
|
| 286 |
-
MIT License
|
|
|
|
| 1 |
+
# ZipVoice C++ — AXERA & AXCL
|
| 2 |
|
| 3 |
+
## 性能
|
| 4 |
|
| 5 |
+
### Distill 模型 (num_step=4)
|
| 6 |
|
| 7 |
+
| 场景 | 音频 | Python AX650 | C++ AX650 | C++ AXCL |
|
| 8 |
+
|------|------|-------------|-----------|----------|
|
| 9 |
+
| 中文句子 | 6.41s | 1.992s / 0.311 | 1.057s / 0.165 | 0.918s / **0.143** |
|
| 10 |
+
| 中文段落 | 44.97s | 13.457s / 0.301 | 7.357s / 0.164 | 6.443s / **0.143** |
|
| 11 |
+
| 英文句子 | 6.41s | 2.045s / 0.319 | 1.067s / 0.166 | 0.919s / **0.143** |
|
| 12 |
+
| 英文段落 | 59.16s | 19.715s / 0.305 | 10.529s / 0.178 | 9.207s / **0.156** |
|
| 13 |
|
| 14 |
+
### 普通模型 (num_step=10)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
| 场景 | 音频 | Python AX650 | C++ AX650 | C++ AXCL |
|
| 17 |
+
|------|------|-------------|-----------|----------|
|
| 18 |
+
| 中文句子 | 6.41s | 5.781s / 0.902 | 5.709s / 0.891 | 5.452s / **0.850** |
|
| 19 |
+
| 中文段落 | 44.97s | 40.292s / 0.901 | 39.714s / 0.883 | 38.138s / **0.848** |
|
| 20 |
+
| 英文句子 | 6.41s | 5.711s / 0.891 | 5.693s / 0.888 | 5.447s / **0.850** |
|
| 21 |
+
| 英文段落 | 59.16s | 62.161s / 0.960 | 56.759s / 0.957 | 54.451s / **0.920** |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
## 环境准备
|
| 24 |
|
| 25 |
+
下载交叉编译器、AXERA SDK、AXCL SDK 到指定目录（记为 `<TOOLCHAIN_DIR>`）：
|
| 26 |
+
|
| 27 |
```bash
|
| 28 |
+
bash download_bsp.sh
|
|
|
|
| 29 |
|
| 30 |
+
# 或手动:
|
| 31 |
+
mkdir -p <TOOLCHAIN_DIR> && cd <TOOLCHAIN_DIR>
|
| 32 |
wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 33 |
tar -xf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 34 |
+
git clone https://github.com/AXERA-TECH/ax650n_bsp_sdk.git --depth=1
|
| 35 |
+
git clone https://github.com/Abandon-ht/axcl_bsp_sdk.git --depth=1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
```
|
| 37 |
|
| 38 |
+
修改 `cmake/msp_dependencies.cmake` 和 `toolchains/aarch64-none-linux-gnu.toolchain.cmake` 中的路径。
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
+
英文分词需 Python 依赖（运行设备上安装）：
|
| 41 |
```bash
|
| 42 |
+
pip install jieba numpy pypinyin piper_phonemize
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
```
|
| 44 |
|
| 45 |
+
## 编译
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
```bash
|
| 48 |
+
bash build_ax650.sh # → install/ax650/zipvoice
|
| 49 |
+
bash build_axcl.sh # → install/axcl/zipvoice
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
```
|
| 51 |
|
| 52 |
+
## 运行
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
+
在 repo 根目录（`ZipVoice.AXERA/`）执行：
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
```bash
|
| 57 |
+
# 板端
|
| 58 |
+
bash run_ax650.sh distill zh sentence
|
| 59 |
+
bash run_ax650.sh distill zh paragraph
|
| 60 |
+
bash run_ax650.sh standard zh sentence
|
| 61 |
+
bash run_ax650.sh standard zh paragraph
|
| 62 |
+
bash run_ax650.sh distill en sentence
|
| 63 |
+
bash run_ax650.sh distill en paragraph
|
| 64 |
+
bash run_ax650.sh standard en sentence
|
| 65 |
+
bash run_ax650.sh standard en paragraph
|
|
|
|
| 66 |
|
| 67 |
+
# 算力卡
|
| 68 |
+
bash run_axcl.sh distill zh paragraph
|
| 69 |
+
bash run_axcl.sh standard en sentence
|
| 70 |
+
...
|
|
|
|
|
|
|
| 71 |
```
|
| 72 |
|
| 73 |
+
## 参数
|
| 74 |
|
| 75 |
| 参数 | 说明 | 默认值 |
|
| 76 |
|------|------|--------|
|
| 77 |
| `--model-dir` | 模型目录 | 必填 |
|
|
|
|
| 78 |
| `--prompt-wav` | 提示音频 | 必填 |
|
| 79 |
| `--prompt-text` | 提示文本 | 必填 |
|
| 80 |
| `--text` / `--text-file` | 合成文本 | 二选一 |
|
| 81 |
+
| `--vocoder-model` | vocos_full.axmodel 路径 | 必填 |
|
| 82 |
+
| `--output-wav` | 输出 WAV | output.wav |
|
|
|
|
| 83 |
| `--num-step` | 采样步数 | 10 (distill: 4) |
|
|
|
|
| 84 |
| `--seed` | 随机种子 | 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
cpp/build_ax650.sh
CHANGED
|
@@ -1,8 +1,9 @@
|
|
| 1 |
#!/bin/bash
|
| 2 |
-
# Cross-compile ZipVoice
|
| 3 |
#
|
| 4 |
-
#
|
| 5 |
-
#
|
|
|
|
| 6 |
|
| 7 |
set -e
|
| 8 |
|
|
@@ -13,8 +14,8 @@ rm -rf build_ax650
|
|
| 13 |
mkdir -p build_ax650 && cd build_ax650
|
| 14 |
|
| 15 |
cmake .. \
|
|
|
|
| 16 |
-DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-none-linux-gnu.toolchain.cmake \
|
| 17 |
-
-DTOOLCHAIN_DIR=/data/shared/huyuan/toolchains/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu \
|
| 18 |
-DCMAKE_INSTALL_PREFIX=../install/ax650 \
|
| 19 |
-DCMAKE_BUILD_TYPE=Release \
|
| 20 |
$@
|
|
@@ -25,5 +26,5 @@ make install
|
|
| 25 |
echo ""
|
| 26 |
echo "========================================"
|
| 27 |
echo "Build complete!"
|
| 28 |
-
echo "Executable: $(pwd)/../install/ax650/
|
| 29 |
echo "========================================"
|
|
|
|
| 1 |
#!/bin/bash
|
| 2 |
+
# Cross-compile ZipVoice for AXERA (AX650 demo board)
|
| 3 |
#
|
| 4 |
+
# Prerequisites:
|
| 5 |
+
# - AX650 BSP SDK: cpp/ax650n_bsp_sdk/ (run download_bsp.sh first)
|
| 6 |
+
# - Cross compiler: /data/shared/huyuan/toolchains/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/
|
| 7 |
|
| 8 |
set -e
|
| 9 |
|
|
|
|
| 14 |
mkdir -p build_ax650 && cd build_ax650
|
| 15 |
|
| 16 |
cmake .. \
|
| 17 |
+
-DAX650=ON \
|
| 18 |
-DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-none-linux-gnu.toolchain.cmake \
|
|
|
|
| 19 |
-DCMAKE_INSTALL_PREFIX=../install/ax650 \
|
| 20 |
-DCMAKE_BUILD_TYPE=Release \
|
| 21 |
$@
|
|
|
|
| 26 |
echo ""
|
| 27 |
echo "========================================"
|
| 28 |
echo "Build complete!"
|
| 29 |
+
echo "Executable: $(pwd)/../install/ax650/zipvoice"
|
| 30 |
echo "========================================"
|
cpp/build_axcl.sh
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Cross-compile ZipVoice for AXCL (AXCL compute card)
|
| 3 |
+
#
|
| 4 |
+
# Prerequisites:
|
| 5 |
+
# - AXCL BSP SDK: /data/shared/huyuan/toolchains/axcl_bsp_sdk/
|
| 6 |
+
# - Cross compiler: /data/shared/huyuan/toolchains/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/
|
| 7 |
+
#
|
| 8 |
+
# Clone SDK (one-time):
|
| 9 |
+
# cd /data/shared/huyuan/toolchains
|
| 10 |
+
# git clone https://github.com/Abandon-ht/axcl_bsp_sdk.git --depth=1
|
| 11 |
+
|
| 12 |
+
set -e
|
| 13 |
+
|
| 14 |
+
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
| 15 |
+
cd "$SCRIPT_DIR"
|
| 16 |
+
|
| 17 |
+
rm -rf build_axcl
|
| 18 |
+
mkdir -p build_axcl && cd build_axcl
|
| 19 |
+
|
| 20 |
+
cmake .. \
|
| 21 |
+
-DAXCL=ON \
|
| 22 |
+
-DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-none-linux-gnu.toolchain.cmake \
|
| 23 |
+
-DCMAKE_INSTALL_PREFIX=../install/axcl \
|
| 24 |
+
-DCMAKE_BUILD_TYPE=Release \
|
| 25 |
+
$@
|
| 26 |
+
|
| 27 |
+
make -j$(nproc)
|
| 28 |
+
make install
|
| 29 |
+
|
| 30 |
+
echo ""
|
| 31 |
+
echo "========================================"
|
| 32 |
+
echo "Build complete!"
|
| 33 |
+
echo "Executable: $(pwd)/../install/axcl/zipvoice"
|
| 34 |
+
echo "========================================"
|
cpp/cmake/msp_dependencies.cmake
CHANGED
|
@@ -1,26 +1,71 @@
|
|
| 1 |
-
# BSP
|
| 2 |
-
#
|
| 3 |
#
|
| 4 |
-
#
|
|
|
|
| 5 |
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
# Allow the linker to find transitive dependencies in the BSP lib dir
|
| 26 |
link_directories(${MSP_LIB_DIR})
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# BSP / SDK Dependencies Configuration
|
| 2 |
+
# Supports both AXERA (demo board) and AXCL (compute card)
|
| 3 |
#
|
| 4 |
+
# AXERA: AX650 / AX630C / AX620Q
|
| 5 |
+
# AXCL: AXCL
|
| 6 |
|
| 7 |
+
if (AX650)
|
| 8 |
+
add_definitions(-DAX650)
|
| 9 |
+
if(NOT BSP_MSP_DIR)
|
| 10 |
+
set(BSP_MSP_DIR /data/shared/huyuan/toolchains/ax650n_bsp_sdk/msp/out)
|
| 11 |
+
endif()
|
| 12 |
+
if(NOT EXISTS ${BSP_MSP_DIR})
|
| 13 |
+
message(FATAL_ERROR "BSP_MSP_DIR ${BSP_MSP_DIR} not exist. Run download_bsp.sh first.")
|
| 14 |
+
endif()
|
| 15 |
+
set(MSP_INC_DIR ${BSP_MSP_DIR}/include)
|
| 16 |
+
set(MSP_LIB_DIR ${BSP_MSP_DIR}/lib)
|
| 17 |
+
list(APPEND MSP_LIBS ax_sys ax_engine ax_interpreter)
|
| 18 |
|
| 19 |
+
elseif(AX630C)
|
| 20 |
+
add_definitions(-DAX630C)
|
| 21 |
+
if(NOT BSP_MSP_DIR)
|
| 22 |
+
set(BSP_MSP_DIR ${CMAKE_SOURCE_DIR}/ax620e_bsp_sdk/msp/out/arm64_glibc)
|
| 23 |
+
endif()
|
| 24 |
+
if(NOT EXISTS ${BSP_MSP_DIR})
|
| 25 |
+
message(FATAL_ERROR "BSP_MSP_DIR ${BSP_MSP_DIR} not exist.")
|
| 26 |
+
endif()
|
| 27 |
+
set(MSP_INC_DIR ${BSP_MSP_DIR}/include)
|
| 28 |
+
set(MSP_LIB_DIR ${BSP_MSP_DIR}/lib)
|
| 29 |
+
list(APPEND MSP_LIBS ax_sys ax_engine ax_interpreter)
|
| 30 |
+
|
| 31 |
+
elseif(AX620Q)
|
| 32 |
+
add_definitions(-DAX620Q)
|
| 33 |
+
if(NOT BSP_MSP_DIR)
|
| 34 |
+
set(BSP_MSP_DIR ${CMAKE_SOURCE_DIR}/ax620e_bsp_sdk/msp/out/arm_uclibc)
|
| 35 |
+
endif()
|
| 36 |
+
if(NOT EXISTS ${BSP_MSP_DIR})
|
| 37 |
+
message(FATAL_ERROR "BSP_MSP_DIR ${BSP_MSP_DIR} not exist.")
|
| 38 |
+
endif()
|
| 39 |
+
set(MSP_INC_DIR ${BSP_MSP_DIR}/include)
|
| 40 |
+
set(MSP_LIB_DIR ${BSP_MSP_DIR}/lib)
|
| 41 |
+
list(APPEND MSP_LIBS ax_sys ax_engine ax_interpreter)
|
| 42 |
|
| 43 |
+
elseif(AXCL)
|
| 44 |
+
add_definitions(-DAXCL)
|
| 45 |
+
add_definitions(-DENV_AXCL_RUNTIME_API_ENABLE)
|
| 46 |
+
add_definitions(-DENV_AXCL_NATIVE_API_ENABLE)
|
| 47 |
+
add_definitions(-DENV_HAS_STD_FILESYSTEM)
|
| 48 |
+
add_definitions(-DENV_HAS_POSIX_FILE_STAT)
|
| 49 |
|
| 50 |
+
# AXCL SDK at /data/shared/huyuan/toolchains/axcl_bsp_sdk/out/
|
| 51 |
+
if(NOT AXCL_SDK_DIR)
|
| 52 |
+
set(AXCL_SDK_DIR /data/shared/huyuan/toolchains/axcl_bsp_sdk/out)
|
| 53 |
+
endif()
|
| 54 |
+
if(NOT EXISTS ${AXCL_SDK_DIR})
|
| 55 |
+
message(FATAL_ERROR "AXCL_SDK_DIR ${AXCL_SDK_DIR} not exist. Run download_bsp.sh first.")
|
| 56 |
+
endif()
|
| 57 |
+
set(MSP_INC_DIR ${AXCL_SDK_DIR}/include ${AXCL_SDK_DIR}/bsp)
|
| 58 |
+
set(MSP_LIB_DIR ${AXCL_SDK_DIR}/lib)
|
| 59 |
+
list(APPEND MSP_LIBS
|
| 60 |
+
axcl_rt axcl_pkg axcl_comm axcl_npu spdlog
|
| 61 |
+
axcl_token axcl_pcie_msg axcl_pcie_dma)
|
| 62 |
+
set(CMAKE_CXX_STANDARD 17)
|
| 63 |
+
|
| 64 |
+
else()
|
| 65 |
+
message(FATAL_ERROR "Unknown platform. Set -DAX650, -DAX630C, -DAX620Q, or -DAXCL.")
|
| 66 |
+
endif()
|
| 67 |
|
|
|
|
| 68 |
link_directories(${MSP_LIB_DIR})
|
| 69 |
+
message(STATUS "MSP_INC_DIR: ${MSP_INC_DIR}")
|
| 70 |
+
message(STATUS "MSP_LIB_DIR: ${MSP_LIB_DIR}")
|
| 71 |
+
message(STATUS "MSP_LIBS: ${MSP_LIBS}")
|
cpp/download_bsp.sh
CHANGED
|
@@ -1,4 +1,19 @@
|
|
| 1 |
#!/bin/bash
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
if [ ! -d ax650n_bsp_sdk ]; then
|
| 3 |
-
|
|
|
|
| 4 |
fi
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
#!/bin/bash
|
| 2 |
+
# Download BSP SDKs to /data/shared/huyuan/toolchains/
|
| 3 |
+
# AXERA: ax650n_bsp_sdk | AXCL: axcl_bsp_sdk
|
| 4 |
+
|
| 5 |
+
TARGET=/data/shared/huyuan/toolchains
|
| 6 |
+
mkdir -p $TARGET && cd $TARGET
|
| 7 |
+
|
| 8 |
if [ ! -d ax650n_bsp_sdk ]; then
|
| 9 |
+
echo "Downloading ax650n_bsp_sdk..."
|
| 10 |
+
git clone https://github.com/AXERA-TECH/ax650n_bsp_sdk.git --depth=1
|
| 11 |
fi
|
| 12 |
+
|
| 13 |
+
if [ ! -d axcl_bsp_sdk ]; then
|
| 14 |
+
echo "Downloading axcl_bsp_sdk..."
|
| 15 |
+
git clone https://github.com/Abandon-ht/axcl_bsp_sdk.git --depth=1
|
| 16 |
+
fi
|
| 17 |
+
|
| 18 |
+
echo "Done. SDKs at $TARGET/"
|
| 19 |
+
ls -d ax650n_bsp_sdk axcl_bsp_sdk 2>/dev/null
|
cpp/install/ax650/zipvoice
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bb407d306e74de864ee7966937552da945db72207ccdea7c7b17950805642978
|
| 3 |
+
size 582632
|
cpp/install/axcl/zipvoice
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f70b9a838f6505d99d2c1cc3131c89565aa25550109336387fbc8a3d49205165
|
| 3 |
+
size 658128
|
cpp/src/EngineWrapper.cpp
CHANGED
|
@@ -1,282 +1,13 @@
|
|
| 1 |
/**************************************************************************************************
|
| 2 |
-
* ZipVoice
|
| 3 |
*
|
| 4 |
-
*
|
| 5 |
**************************************************************************************************/
|
| 6 |
|
| 7 |
-
#
|
| 8 |
-
#include "
|
| 9 |
-
|
| 10 |
-
#include
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
// NPU type enum
|
| 15 |
-
typedef enum axNPU_TYPE_E {
|
| 16 |
-
AX_NPU_DEFAULT = 0,
|
| 17 |
-
AX_STD_VNPU_1 = (1 << 0),
|
| 18 |
-
AX_STD_VNPU_2 = (1 << 1),
|
| 19 |
-
AX_STD_VNPU_3 = (1 << 2),
|
| 20 |
-
AX_BL_VNPU_1 = (1 << 3),
|
| 21 |
-
AX_BL_VNPU_2 = (1 << 4)
|
| 22 |
-
} AX_NPU_TYPE_E;
|
| 23 |
-
|
| 24 |
-
static AX_S32 CheckModelVNpu(const std::string &strModel,
|
| 25 |
-
const AX_ENGINE_MODEL_TYPE_T &eModelType,
|
| 26 |
-
const AX_S32 &nNpuType, AX_U32 &nNpuSet) {
|
| 27 |
-
AX_ENGINE_NPU_ATTR_T stNpuAttr;
|
| 28 |
-
memset(&stNpuAttr, 0x00, sizeof(stNpuAttr));
|
| 29 |
-
|
| 30 |
-
auto ret = AX_ENGINE_GetVNPUAttr(&stNpuAttr);
|
| 31 |
-
if (ret == 0) {
|
| 32 |
-
if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_DISABLE) {
|
| 33 |
-
nNpuSet = 0x01;
|
| 34 |
-
} else if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_STD) {
|
| 35 |
-
if (eModelType == AX_ENGINE_MODEL_TYPE1 || eModelType == AX_ENGINE_MODEL_TYPE2)
|
| 36 |
-
return -1;
|
| 37 |
-
if (nNpuType == 0) nNpuSet = 0x02;
|
| 38 |
-
else {
|
| 39 |
-
if (nNpuType & AX_STD_VNPU_1) nNpuSet |= 0x01;
|
| 40 |
-
if (nNpuType & AX_STD_VNPU_2) nNpuSet |= 0x02;
|
| 41 |
-
if (nNpuType & AX_STD_VNPU_3) nNpuSet |= 0x04;
|
| 42 |
-
}
|
| 43 |
-
} else if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_BIG_LITTLE) {
|
| 44 |
-
if (eModelType == AX_ENGINE_MODEL_TYPE2) return -1;
|
| 45 |
-
if (nNpuType == 0) {
|
| 46 |
-
nNpuSet = (eModelType == AX_ENGINE_MODEL_TYPE1) ? 0x01 : 0x02;
|
| 47 |
-
} else {
|
| 48 |
-
if (eModelType == AX_ENGINE_MODEL_TYPE1) {
|
| 49 |
-
if (nNpuType & AX_BL_VNPU_2) return -1;
|
| 50 |
-
if (nNpuType & AX_BL_VNPU_1) nNpuSet |= 0x01;
|
| 51 |
-
} else {
|
| 52 |
-
if (nNpuType & AX_BL_VNPU_1) nNpuSet |= 0x01;
|
| 53 |
-
if (nNpuType & AX_BL_VNPU_2) nNpuSet |= 0x02;
|
| 54 |
-
}
|
| 55 |
-
}
|
| 56 |
-
}
|
| 57 |
-
}
|
| 58 |
-
return ret;
|
| 59 |
-
}
|
| 60 |
-
|
| 61 |
-
EngineWrapper::EngineWrapper()
|
| 62 |
-
: m_hasInit(false), m_handle(nullptr), m_io_info(nullptr),
|
| 63 |
-
m_input_num(0), m_output_num(0) {
|
| 64 |
-
memset(&m_io, 0, sizeof(m_io));
|
| 65 |
-
}
|
| 66 |
-
|
| 67 |
-
EngineWrapper::~EngineWrapper() {
|
| 68 |
-
Release();
|
| 69 |
-
}
|
| 70 |
-
|
| 71 |
-
int EngineWrapper::Init(const char* strModelPath, uint32_t nNpuType) {
|
| 72 |
-
// 1. Load model
|
| 73 |
-
AX_BOOL bLoadModelUseCmm = AX_TRUE;
|
| 74 |
-
AX_CHAR *pModelBufferVirAddr = nullptr;
|
| 75 |
-
AX_U64 u64ModelBufferPhyAddr = 0;
|
| 76 |
-
AX_U32 nModelBufferSize = 0;
|
| 77 |
-
|
| 78 |
-
if (bLoadModelUseCmm) {
|
| 79 |
-
if (!utils::read_file(strModelPath, (AX_VOID **)&pModelBufferVirAddr,
|
| 80 |
-
u64ModelBufferPhyAddr, nModelBufferSize)) {
|
| 81 |
-
printf("Read model(%s) fail\n", strModelPath);
|
| 82 |
-
return -1;
|
| 83 |
-
}
|
| 84 |
-
} else {
|
| 85 |
-
std::vector<char> model_buffer;
|
| 86 |
-
if (!utils::read_file(strModelPath, model_buffer)) {
|
| 87 |
-
printf("Read model(%s) fail\n", strModelPath);
|
| 88 |
-
return -1;
|
| 89 |
-
}
|
| 90 |
-
pModelBufferVirAddr = model_buffer.data();
|
| 91 |
-
nModelBufferSize = model_buffer.size();
|
| 92 |
-
}
|
| 93 |
-
|
| 94 |
-
auto freeModelBuffer = [&]() {
|
| 95 |
-
if (bLoadModelUseCmm) {
|
| 96 |
-
if (u64ModelBufferPhyAddr != 0)
|
| 97 |
-
AX_SYS_MemFree(u64ModelBufferPhyAddr, &pModelBufferVirAddr);
|
| 98 |
-
}
|
| 99 |
-
};
|
| 100 |
-
|
| 101 |
-
// 1.1 Get Model Type
|
| 102 |
-
AX_ENGINE_MODEL_TYPE_T eModelType = AX_ENGINE_MODEL_TYPE0;
|
| 103 |
-
AX_S32 ret = AX_ENGINE_GetModelType(pModelBufferVirAddr, nModelBufferSize, &eModelType);
|
| 104 |
-
if (0 != ret || eModelType >= AX_ENGINE_MODEL_TYPE_BUTT) {
|
| 105 |
-
printf("%s AX_ENGINE_GetModelType fail ret=%x\n", strModelPath, ret);
|
| 106 |
-
freeModelBuffer();
|
| 107 |
-
return -1;
|
| 108 |
-
}
|
| 109 |
-
|
| 110 |
-
// 1.2 Check VNPU
|
| 111 |
-
AX_U32 nNpuSet = 0;
|
| 112 |
-
ret = CheckModelVNpu(strModelPath, eModelType, nNpuType, nNpuSet);
|
| 113 |
-
if (0 != ret) {
|
| 114 |
-
printf("CheckModelVNpu fail\n");
|
| 115 |
-
freeModelBuffer();
|
| 116 |
-
return -1;
|
| 117 |
-
}
|
| 118 |
-
|
| 119 |
-
// 2. Create handle
|
| 120 |
-
AX_ENGINE_HANDLE handle = nullptr;
|
| 121 |
-
ret = AX_ENGINE_CreateHandle(&handle, pModelBufferVirAddr, nModelBufferSize);
|
| 122 |
-
freeModelBuffer();
|
| 123 |
-
|
| 124 |
-
auto deinit_handle = [&handle]() {
|
| 125 |
-
if (handle) { AX_ENGINE_DestroyHandle(handle); }
|
| 126 |
-
return -1;
|
| 127 |
-
};
|
| 128 |
-
|
| 129 |
-
if (0 != ret || !handle) {
|
| 130 |
-
printf("Create model(%s) handle fail\n", strModelPath);
|
| 131 |
-
return deinit_handle();
|
| 132 |
-
}
|
| 133 |
-
|
| 134 |
-
// 3. Create context
|
| 135 |
-
ret = AX_ENGINE_CreateContext(handle);
|
| 136 |
-
if (0 != ret) return deinit_handle();
|
| 137 |
-
|
| 138 |
-
// 4. Get IO info
|
| 139 |
-
m_io_info = nullptr;
|
| 140 |
-
ret = AX_ENGINE_GetIOInfo(handle, &m_io_info);
|
| 141 |
-
if (0 != ret) return deinit_handle();
|
| 142 |
-
|
| 143 |
-
m_input_num = m_io_info->nInputSize;
|
| 144 |
-
m_output_num = m_io_info->nOutputSize;
|
| 145 |
-
|
| 146 |
-
// Build name-to-index maps
|
| 147 |
-
m_input_name_to_idx.clear();
|
| 148 |
-
for (int i = 0; i < m_input_num; ++i) {
|
| 149 |
-
m_input_name_to_idx[std::string(m_io_info->pInputs[i].pName)] = i;
|
| 150 |
-
}
|
| 151 |
-
m_output_name_to_idx.clear();
|
| 152 |
-
for (int i = 0; i < m_output_num; ++i) {
|
| 153 |
-
m_output_name_to_idx[std::string(m_io_info->pOutputs[i].pName)] = i;
|
| 154 |
-
}
|
| 155 |
-
|
| 156 |
-
// 5. Prepare IO buffers
|
| 157 |
-
ret = utils::prepare_io("enc", m_io_info, m_io, utils::IO_BUFFER_STRATEGY_DEFAULT);
|
| 158 |
-
if (0 != ret) {
|
| 159 |
-
printf("prepare io failed!\n");
|
| 160 |
-
utils::free_io(m_io);
|
| 161 |
-
return deinit_handle();
|
| 162 |
-
}
|
| 163 |
-
|
| 164 |
-
m_handle = handle;
|
| 165 |
-
m_hasInit = true;
|
| 166 |
-
return 0;
|
| 167 |
-
}
|
| 168 |
-
|
| 169 |
-
int EngineWrapper::SetInput(void* pInput, int index) {
|
| 170 |
-
if (!m_hasInit || index < 0 || index >= m_input_num) return -1;
|
| 171 |
-
return utils::push_io_input(pInput, index, m_io);
|
| 172 |
-
}
|
| 173 |
-
|
| 174 |
-
int EngineWrapper::RunSync() {
|
| 175 |
-
if (!m_hasInit) return -1;
|
| 176 |
-
auto ret = AX_ENGINE_RunSync(m_handle, &m_io);
|
| 177 |
-
if (0 != ret) {
|
| 178 |
-
printf("AX_ENGINE_RunSync failed. ret=0x%x\n", ret);
|
| 179 |
-
}
|
| 180 |
-
return ret;
|
| 181 |
-
}
|
| 182 |
-
|
| 183 |
-
int EngineWrapper::GetOutput(void* pOutput, int index) {
|
| 184 |
-
if (!m_hasInit || index < 0 || index >= m_output_num) return -1;
|
| 185 |
-
return utils::push_io_output(pOutput, index, m_io);
|
| 186 |
-
}
|
| 187 |
-
|
| 188 |
-
int EngineWrapper::SetInputByName(const char* name, void* pInput) {
|
| 189 |
-
auto it = m_input_name_to_idx.find(std::string(name));
|
| 190 |
-
if (it == m_input_name_to_idx.end()) {
|
| 191 |
-
printf("Input '%s' not found in model\n", name);
|
| 192 |
-
return -1;
|
| 193 |
-
}
|
| 194 |
-
return SetInput(pInput, it->second);
|
| 195 |
-
}
|
| 196 |
-
|
| 197 |
-
int EngineWrapper::GetOutputByName(const char* name, void* pOutput) {
|
| 198 |
-
auto it = m_output_name_to_idx.find(std::string(name));
|
| 199 |
-
if (it == m_output_name_to_idx.end()) {
|
| 200 |
-
printf("Output '%s' not found in model\n", name);
|
| 201 |
-
return -1;
|
| 202 |
-
}
|
| 203 |
-
return GetOutput(pOutput, it->second);
|
| 204 |
-
}
|
| 205 |
-
|
| 206 |
-
int EngineWrapper::GetInputSize(int index) {
|
| 207 |
-
if (index < 0 || index >= m_input_num) return -1;
|
| 208 |
-
return m_io.pInputs[index].nSize;
|
| 209 |
-
}
|
| 210 |
-
|
| 211 |
-
int EngineWrapper::GetOutputSize(int index) {
|
| 212 |
-
if (index < 0 || index >= m_output_num) return -1;
|
| 213 |
-
return m_io.pOutputs[index].nSize;
|
| 214 |
-
}
|
| 215 |
-
|
| 216 |
-
int EngineWrapper::GetInputSizeByName(const char* name) {
|
| 217 |
-
int idx = GetInputIndex(name);
|
| 218 |
-
if (idx < 0) return -1;
|
| 219 |
-
return GetInputSize(idx);
|
| 220 |
-
}
|
| 221 |
-
|
| 222 |
-
int EngineWrapper::GetOutputSizeByName(const char* name) {
|
| 223 |
-
int idx = GetOutputIndex(name);
|
| 224 |
-
if (idx < 0) return -1;
|
| 225 |
-
return GetOutputSize(idx);
|
| 226 |
-
}
|
| 227 |
-
|
| 228 |
-
int EngineWrapper::GetInputIndex(const char* name) {
|
| 229 |
-
auto it = m_input_name_to_idx.find(std::string(name));
|
| 230 |
-
return (it != m_input_name_to_idx.end()) ? it->second : -1;
|
| 231 |
-
}
|
| 232 |
-
|
| 233 |
-
int EngineWrapper::GetOutputIndex(const char* name) {
|
| 234 |
-
auto it = m_output_name_to_idx.find(std::string(name));
|
| 235 |
-
return (it != m_output_name_to_idx.end()) ? it->second : -1;
|
| 236 |
-
}
|
| 237 |
-
|
| 238 |
-
const char* EngineWrapper::GetInputName(int index) const {
|
| 239 |
-
if (index < 0 || index >= m_input_num) return nullptr;
|
| 240 |
-
return m_io_info->pInputs[index].pName;
|
| 241 |
-
}
|
| 242 |
-
|
| 243 |
-
const char* EngineWrapper::GetOutputName(int index) const {
|
| 244 |
-
if (index < 0 || index >= m_output_num) return nullptr;
|
| 245 |
-
return m_io_info->pOutputs[index].pName;
|
| 246 |
-
}
|
| 247 |
-
|
| 248 |
-
static const char* dtype_str(AX_ENGINE_DATA_TYPE_T t) {
|
| 249 |
-
switch (t) {
|
| 250 |
-
case AX_ENGINE_DT_FLOAT32: return "float32";
|
| 251 |
-
case AX_ENGINE_DT_FLOAT64: return "float64";
|
| 252 |
-
case AX_ENGINE_DT_SINT8: return "sint8";
|
| 253 |
-
case AX_ENGINE_DT_UINT8: return "uint8";
|
| 254 |
-
case AX_ENGINE_DT_SINT16: return "sint16";
|
| 255 |
-
case AX_ENGINE_DT_UINT16: return "uint16";
|
| 256 |
-
case AX_ENGINE_DT_SINT32: return "sint32";
|
| 257 |
-
case AX_ENGINE_DT_UINT32: return "uint32";
|
| 258 |
-
default: return "unknown";
|
| 259 |
-
}
|
| 260 |
-
}
|
| 261 |
-
|
| 262 |
-
const char* EngineWrapper::GetInputDtypeStr(int index) const {
|
| 263 |
-
if (index < 0 || index >= m_input_num) return "?";
|
| 264 |
-
return dtype_str(m_io_info->pInputs[index].eDataType);
|
| 265 |
-
}
|
| 266 |
-
|
| 267 |
-
const char* EngineWrapper::GetOutputDtypeStr(int index) const {
|
| 268 |
-
if (index < 0 || index >= m_output_num) return "?";
|
| 269 |
-
return dtype_str(m_io_info->pOutputs[index].eDataType);
|
| 270 |
-
}
|
| 271 |
-
|
| 272 |
-
int EngineWrapper::Release() {
|
| 273 |
-
if (m_handle) {
|
| 274 |
-
utils::free_io(m_io);
|
| 275 |
-
AX_ENGINE_DestroyHandle(m_handle);
|
| 276 |
-
m_handle = nullptr;
|
| 277 |
-
}
|
| 278 |
-
m_hasInit = false;
|
| 279 |
-
m_input_name_to_idx.clear();
|
| 280 |
-
m_output_name_to_idx.clear();
|
| 281 |
-
return 0;
|
| 282 |
-
}
|
|
|
|
| 1 |
/**************************************************************************************************
|
| 2 |
+
* ZipVoice C++ — Unified EngineWrapper implementation (AXERA & AXCL)
|
| 3 |
*
|
| 4 |
+
* Conditionally includes the platform-specific implementation.
|
| 5 |
**************************************************************************************************/
|
| 6 |
|
| 7 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 8 |
+
#include "EngineWrapper_axera.cpp"
|
| 9 |
+
#elif defined(AXCL)
|
| 10 |
+
#include "EngineWrapper_axcl.cpp"
|
| 11 |
+
#else
|
| 12 |
+
#error "Unknown platform. Define AX650, AX630C, AX620Q, or AXCL."
|
| 13 |
+
#endif
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
cpp/src/EngineWrapper.hpp
CHANGED
|
@@ -1,10 +1,11 @@
|
|
| 1 |
/**************************************************************************************************
|
| 2 |
-
* ZipVoice
|
| 3 |
*
|
| 4 |
-
*
|
| 5 |
-
*
|
|
|
|
| 6 |
*
|
| 7 |
-
*
|
| 8 |
**************************************************************************************************/
|
| 9 |
|
| 10 |
#pragma once
|
|
@@ -14,22 +15,33 @@
|
|
| 14 |
#include <unordered_map>
|
| 15 |
#include <cstring>
|
| 16 |
#include <cstdint>
|
|
|
|
| 17 |
|
| 18 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
class EngineWrapper {
|
| 21 |
public:
|
| 22 |
EngineWrapper();
|
| 23 |
~EngineWrapper();
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
// Index-based I/O
|
| 28 |
int SetInput(void* pInput, int index);
|
| 29 |
int RunSync();
|
| 30 |
int GetOutput(void* pOutput, int index);
|
| 31 |
|
| 32 |
-
// Name-based I/O
|
| 33 |
int SetInputByName(const char* name, void* pInput);
|
| 34 |
int GetOutputByName(const char* name, void* pOutput);
|
| 35 |
|
|
@@ -44,19 +56,26 @@ public:
|
|
| 44 |
int GetOutputCount() const { return m_output_num; }
|
| 45 |
const char* GetInputName(int index) const;
|
| 46 |
const char* GetOutputName(int index) const;
|
|
|
|
| 47 |
const char* GetInputDtypeStr(int index) const;
|
| 48 |
const char* GetOutputDtypeStr(int index) const;
|
|
|
|
| 49 |
|
| 50 |
bool HasInit() const { return m_hasInit; }
|
| 51 |
int Release();
|
| 52 |
|
| 53 |
private:
|
| 54 |
bool m_hasInit;
|
| 55 |
-
AX_ENGINE_HANDLE m_handle;
|
| 56 |
-
AX_ENGINE_IO_INFO_T *m_io_info;
|
| 57 |
-
AX_ENGINE_IO_T m_io;
|
| 58 |
int m_input_num, m_output_num;
|
| 59 |
|
| 60 |
std::unordered_map<std::string, int> m_input_name_to_idx;
|
| 61 |
std::unordered_map<std::string, int> m_output_name_to_idx;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
};
|
|
|
|
| 1 |
/**************************************************************************************************
|
| 2 |
+
* ZipVoice C++ — Unified EngineWrapper (AXERA & AXCL)
|
| 3 |
*
|
| 4 |
+
* Supports both AXERA (demo board) and AXCL (compute card) via conditional compilation.
|
| 5 |
+
* AX650 / AX630C / AX620Q → AXERA path (ax_engine_api)
|
| 6 |
+
* AXCL → AXCL path (middleware::runner)
|
| 7 |
*
|
| 8 |
+
* Interface is identical — callers don't need to know which backend is active.
|
| 9 |
**************************************************************************************************/
|
| 10 |
|
| 11 |
#pragma once
|
|
|
|
| 15 |
#include <unordered_map>
|
| 16 |
#include <cstring>
|
| 17 |
#include <cstdint>
|
| 18 |
+
#include <memory>
|
| 19 |
|
| 20 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 21 |
+
#include "ax_engine_api.h"
|
| 22 |
+
#elif defined(AXCL)
|
| 23 |
+
#include "middleware/axcl_runtime_runner.hpp"
|
| 24 |
+
#endif
|
| 25 |
|
| 26 |
class EngineWrapper {
|
| 27 |
public:
|
| 28 |
EngineWrapper();
|
| 29 |
~EngineWrapper();
|
| 30 |
|
| 31 |
+
/**
|
| 32 |
+
* Initialize and load a model.
|
| 33 |
+
* @param strModelPath Path to .axmodel file
|
| 34 |
+
* @param nNpuType AXERA: VNPU type (0=default); AXCL: device_index (0=first card)
|
| 35 |
+
* @param axclConfig AXCL: path to axcl.json (nullptr → /usr/local/axcl/axcl.json). Ignored on AXERA.
|
| 36 |
+
*/
|
| 37 |
+
int Init(const char* strModelPath, uint32_t nNpuType = 0, const char* axclConfig = nullptr);
|
| 38 |
|
| 39 |
+
// Index-based I/O
|
| 40 |
int SetInput(void* pInput, int index);
|
| 41 |
int RunSync();
|
| 42 |
int GetOutput(void* pOutput, int index);
|
| 43 |
|
| 44 |
+
// Name-based I/O
|
| 45 |
int SetInputByName(const char* name, void* pInput);
|
| 46 |
int GetOutputByName(const char* name, void* pOutput);
|
| 47 |
|
|
|
|
| 56 |
int GetOutputCount() const { return m_output_num; }
|
| 57 |
const char* GetInputName(int index) const;
|
| 58 |
const char* GetOutputName(int index) const;
|
| 59 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 60 |
const char* GetInputDtypeStr(int index) const;
|
| 61 |
const char* GetOutputDtypeStr(int index) const;
|
| 62 |
+
#endif
|
| 63 |
|
| 64 |
bool HasInit() const { return m_hasInit; }
|
| 65 |
int Release();
|
| 66 |
|
| 67 |
private:
|
| 68 |
bool m_hasInit;
|
|
|
|
|
|
|
|
|
|
| 69 |
int m_input_num, m_output_num;
|
| 70 |
|
| 71 |
std::unordered_map<std::string, int> m_input_name_to_idx;
|
| 72 |
std::unordered_map<std::string, int> m_output_name_to_idx;
|
| 73 |
+
|
| 74 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 75 |
+
AX_ENGINE_HANDLE m_handle;
|
| 76 |
+
AX_ENGINE_IO_INFO_T *m_io_info;
|
| 77 |
+
AX_ENGINE_IO_T m_io;
|
| 78 |
+
#elif defined(AXCL)
|
| 79 |
+
std::unique_ptr<middleware::runner> m_runner;
|
| 80 |
+
#endif
|
| 81 |
};
|
cpp/src/EngineWrapper_axcl.cpp
ADDED
|
@@ -0,0 +1,217 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
* ZipVoice AXCL C++ Port
|
| 3 |
+
*
|
| 4 |
+
* EngineWrapper implementation using middleware::runner (AXCL Runtime API).
|
| 5 |
+
*
|
| 6 |
+
* Key differences from AXERA:
|
| 7 |
+
* - Uses axclrtMemcpy for host↔device data transfer
|
| 8 |
+
* - No CMM (contiguous memory) allocation
|
| 9 |
+
* - Runner manages device-side memory and NPU execution
|
| 10 |
+
**************************************************************************************************/
|
| 11 |
+
|
| 12 |
+
#include "EngineWrapper.hpp"
|
| 13 |
+
|
| 14 |
+
#include <cstdio>
|
| 15 |
+
#include <cstdlib>
|
| 16 |
+
#include <cstring>
|
| 17 |
+
|
| 18 |
+
#include <axcl.h>
|
| 19 |
+
|
| 20 |
+
EngineWrapper::EngineWrapper()
|
| 21 |
+
: m_hasInit(false), m_input_num(0), m_output_num(0) {}
|
| 22 |
+
|
| 23 |
+
EngineWrapper::~EngineWrapper() {
|
| 24 |
+
Release();
|
| 25 |
+
}
|
| 26 |
+
|
| 27 |
+
int EngineWrapper::Init(const char* strModelPath,
|
| 28 |
+
uint32_t nNpuType,
|
| 29 |
+
const char* axclConfig) {
|
| 30 |
+
const char* config = axclConfig ? axclConfig : "/usr/local/axcl/axcl.json";
|
| 31 |
+
uint32_t device_index = nNpuType; // nNpuType = device_index on AXCL
|
| 32 |
+
uint32_t npu_kind = 0; // AXCL_VNPU_DISABLE (default)
|
| 33 |
+
|
| 34 |
+
// 1. Create runtime runner
|
| 35 |
+
m_runner = std::make_unique<middleware::runtime_runner>();
|
| 36 |
+
|
| 37 |
+
// 2. Initialize runner (handles axclrtEngineInit internally)
|
| 38 |
+
if (!m_runner->init(config, device_index, npu_kind)) {
|
| 39 |
+
fprintf(stderr, "[ERROR] AXCL runner init failed for model: %s\n", strModelPath);
|
| 40 |
+
return -1;
|
| 41 |
+
}
|
| 42 |
+
|
| 43 |
+
// 3. Load model from file
|
| 44 |
+
if (!m_runner->load(strModelPath)) {
|
| 45 |
+
fprintf(stderr, "[ERROR] AXCL runner load failed for model: %s\n", strModelPath);
|
| 46 |
+
return -1;
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
// 4. Prepare IO buffers (allocates device memory, sets up IO)
|
| 50 |
+
if (!m_runner->prepare(true, true, 0, 0)) {
|
| 51 |
+
fprintf(stderr, "[ERROR] AXCL runner prepare failed for model: %s\n", strModelPath);
|
| 52 |
+
return -1;
|
| 53 |
+
}
|
| 54 |
+
|
| 55 |
+
// 5. Build name-to-index maps
|
| 56 |
+
m_input_num = static_cast<int>(m_runner->get_input_count());
|
| 57 |
+
m_output_num = static_cast<int>(m_runner->get_output_count());
|
| 58 |
+
|
| 59 |
+
m_input_name_to_idx.clear();
|
| 60 |
+
for (int i = 0; i < m_input_num; ++i) {
|
| 61 |
+
std::string name = m_runner->get_input_name(i);
|
| 62 |
+
m_input_name_to_idx[name] = i;
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
m_output_name_to_idx.clear();
|
| 66 |
+
for (int i = 0; i < m_output_num; ++i) {
|
| 67 |
+
std::string name = m_runner->get_output_name(i);
|
| 68 |
+
m_output_name_to_idx[name] = i;
|
| 69 |
+
}
|
| 70 |
+
|
| 71 |
+
m_hasInit = true;
|
| 72 |
+
|
| 73 |
+
printf("AXCL EngineWrapper loaded: %s (inputs=%d, outputs=%d)\n",
|
| 74 |
+
strModelPath, m_input_num, m_output_num);
|
| 75 |
+
for (int i = 0; i < m_input_num; ++i) {
|
| 76 |
+
printf(" input[%d]: %s (%zu bytes)\n", i,
|
| 77 |
+
m_runner->get_input_name(i).c_str(),
|
| 78 |
+
(size_t)m_runner->get_input_size(i));
|
| 79 |
+
}
|
| 80 |
+
for (int i = 0; i < m_output_num; ++i) {
|
| 81 |
+
printf(" output[%d]: %s (%zu bytes)\n", i,
|
| 82 |
+
m_runner->get_output_name(i).c_str(),
|
| 83 |
+
(size_t)m_runner->get_output_size(i));
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
return 0;
|
| 87 |
+
}
|
| 88 |
+
|
| 89 |
+
int EngineWrapper::SetInput(void* pInput, int index) {
|
| 90 |
+
if (!m_hasInit || index < 0 || index >= m_input_num) return -1;
|
| 91 |
+
|
| 92 |
+
uintmax_t size = m_runner->get_input_size(index);
|
| 93 |
+
void* dev_ptr = m_runner->get_input_pointer(index);
|
| 94 |
+
|
| 95 |
+
if (!dev_ptr || size == 0) {
|
| 96 |
+
fprintf(stderr, "[ERROR] Invalid input pointer/size for index %d\n", index);
|
| 97 |
+
return -1;
|
| 98 |
+
}
|
| 99 |
+
|
| 100 |
+
// Copy host data to device memory
|
| 101 |
+
axclError ret = axclrtMemcpy(dev_ptr, pInput, size, AXCL_MEMCPY_HOST_TO_DEVICE);
|
| 102 |
+
if (ret != 0) {
|
| 103 |
+
fprintf(stderr, "[ERROR] axclrtMemcpy H2D failed for input %d: 0x%x\n", index, ret);
|
| 104 |
+
return -1;
|
| 105 |
+
}
|
| 106 |
+
|
| 107 |
+
return 0;
|
| 108 |
+
}
|
| 109 |
+
|
| 110 |
+
int EngineWrapper::RunSync() {
|
| 111 |
+
if (!m_hasInit) return -1;
|
| 112 |
+
|
| 113 |
+
if (!m_runner->run(false)) {
|
| 114 |
+
fprintf(stderr, "[ERROR] AXCL runner run failed\n");
|
| 115 |
+
return -1;
|
| 116 |
+
}
|
| 117 |
+
|
| 118 |
+
return 0;
|
| 119 |
+
}
|
| 120 |
+
|
| 121 |
+
int EngineWrapper::GetOutput(void* pOutput, int index) {
|
| 122 |
+
if (!m_hasInit || index < 0 || index >= m_output_num) return -1;
|
| 123 |
+
|
| 124 |
+
uintmax_t size = m_runner->get_output_size(index);
|
| 125 |
+
void* dev_ptr = m_runner->get_output_pointer(index);
|
| 126 |
+
|
| 127 |
+
if (!dev_ptr || size == 0) {
|
| 128 |
+
fprintf(stderr, "[ERROR] Invalid output pointer/size for index %d\n", index);
|
| 129 |
+
return -1;
|
| 130 |
+
}
|
| 131 |
+
|
| 132 |
+
// Copy device data to host memory
|
| 133 |
+
axclError ret = axclrtMemcpy(pOutput, dev_ptr, size, AXCL_MEMCPY_DEVICE_TO_HOST);
|
| 134 |
+
if (ret != 0) {
|
| 135 |
+
fprintf(stderr, "[ERROR] axclrtMemcpy D2H failed for output %d: 0x%x\n", index, ret);
|
| 136 |
+
return -1;
|
| 137 |
+
}
|
| 138 |
+
|
| 139 |
+
return 0;
|
| 140 |
+
}
|
| 141 |
+
|
| 142 |
+
int EngineWrapper::SetInputByName(const char* name, void* pInput) {
|
| 143 |
+
auto it = m_input_name_to_idx.find(std::string(name));
|
| 144 |
+
if (it == m_input_name_to_idx.end()) {
|
| 145 |
+
fprintf(stderr, "[ERROR] Input '%s' not found in model\n", name);
|
| 146 |
+
return -1;
|
| 147 |
+
}
|
| 148 |
+
return SetInput(pInput, it->second);
|
| 149 |
+
}
|
| 150 |
+
|
| 151 |
+
int EngineWrapper::GetOutputByName(const char* name, void* pOutput) {
|
| 152 |
+
auto it = m_output_name_to_idx.find(std::string(name));
|
| 153 |
+
if (it == m_output_name_to_idx.end()) {
|
| 154 |
+
fprintf(stderr, "[ERROR] Output '%s' not found in model\n", name);
|
| 155 |
+
return -1;
|
| 156 |
+
}
|
| 157 |
+
return GetOutput(pOutput, it->second);
|
| 158 |
+
}
|
| 159 |
+
|
| 160 |
+
int EngineWrapper::GetInputSize(int index) {
|
| 161 |
+
if (!m_hasInit || index < 0 || index >= m_input_num) return -1;
|
| 162 |
+
return static_cast<int>(m_runner->get_input_size(index));
|
| 163 |
+
}
|
| 164 |
+
|
| 165 |
+
int EngineWrapper::GetOutputSize(int index) {
|
| 166 |
+
if (!m_hasInit || index < 0 || index >= m_output_num) return -1;
|
| 167 |
+
return static_cast<int>(m_runner->get_output_size(index));
|
| 168 |
+
}
|
| 169 |
+
|
| 170 |
+
int EngineWrapper::GetInputSizeByName(const char* name) {
|
| 171 |
+
int idx = GetInputIndex(name);
|
| 172 |
+
if (idx < 0) return -1;
|
| 173 |
+
return GetInputSize(idx);
|
| 174 |
+
}
|
| 175 |
+
|
| 176 |
+
int EngineWrapper::GetOutputSizeByName(const char* name) {
|
| 177 |
+
int idx = GetOutputIndex(name);
|
| 178 |
+
if (idx < 0) return -1;
|
| 179 |
+
return GetOutputSize(idx);
|
| 180 |
+
}
|
| 181 |
+
|
| 182 |
+
int EngineWrapper::GetInputIndex(const char* name) {
|
| 183 |
+
auto it = m_input_name_to_idx.find(std::string(name));
|
| 184 |
+
return (it != m_input_name_to_idx.end()) ? it->second : -1;
|
| 185 |
+
}
|
| 186 |
+
|
| 187 |
+
int EngineWrapper::GetOutputIndex(const char* name) {
|
| 188 |
+
auto it = m_output_name_to_idx.find(std::string(name));
|
| 189 |
+
return (it != m_output_name_to_idx.end()) ? it->second : -1;
|
| 190 |
+
}
|
| 191 |
+
|
| 192 |
+
int EngineWrapper::Release() {
|
| 193 |
+
if (m_runner) {
|
| 194 |
+
m_runner->final();
|
| 195 |
+
m_runner.reset();
|
| 196 |
+
}
|
| 197 |
+
m_hasInit = false;
|
| 198 |
+
m_input_name_to_idx.clear();
|
| 199 |
+
m_output_name_to_idx.clear();
|
| 200 |
+
m_input_num = 0;
|
| 201 |
+
m_output_num = 0;
|
| 202 |
+
return 0;
|
| 203 |
+
}
|
| 204 |
+
|
| 205 |
+
const char* EngineWrapper::GetInputName(int index) const {
|
| 206 |
+
static thread_local std::string cached_name;
|
| 207 |
+
if (!m_hasInit || index < 0 || index >= m_input_num) return nullptr;
|
| 208 |
+
cached_name = m_runner->get_input_name(index);
|
| 209 |
+
return cached_name.c_str();
|
| 210 |
+
}
|
| 211 |
+
|
| 212 |
+
const char* EngineWrapper::GetOutputName(int index) const {
|
| 213 |
+
static thread_local std::string cached_name;
|
| 214 |
+
if (!m_hasInit || index < 0 || index >= m_output_num) return nullptr;
|
| 215 |
+
cached_name = m_runner->get_output_name(index);
|
| 216 |
+
return cached_name.c_str();
|
| 217 |
+
}
|
cpp/src/EngineWrapper_axera.cpp
ADDED
|
@@ -0,0 +1,283 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
* ZipVoice AXERA C++ Port
|
| 3 |
+
*
|
| 4 |
+
* EngineWrapper implementation
|
| 5 |
+
**************************************************************************************************/
|
| 6 |
+
|
| 7 |
+
#include "EngineWrapper.hpp"
|
| 8 |
+
#include "utils/io.hpp"
|
| 9 |
+
|
| 10 |
+
#include <cstdlib>
|
| 11 |
+
|
| 12 |
+
static const char *strAlgoModelType[AX_ENGINE_MODEL_TYPE_BUTT] = {"3.6T", "7.2T", "10.8T"};
|
| 13 |
+
|
| 14 |
+
// NPU type enum
|
| 15 |
+
typedef enum axNPU_TYPE_E {
|
| 16 |
+
AX_NPU_DEFAULT = 0,
|
| 17 |
+
AX_STD_VNPU_1 = (1 << 0),
|
| 18 |
+
AX_STD_VNPU_2 = (1 << 1),
|
| 19 |
+
AX_STD_VNPU_3 = (1 << 2),
|
| 20 |
+
AX_BL_VNPU_1 = (1 << 3),
|
| 21 |
+
AX_BL_VNPU_2 = (1 << 4)
|
| 22 |
+
} AX_NPU_TYPE_E;
|
| 23 |
+
|
| 24 |
+
static AX_S32 CheckModelVNpu(const std::string &strModel,
|
| 25 |
+
const AX_ENGINE_MODEL_TYPE_T &eModelType,
|
| 26 |
+
const AX_S32 &nNpuType, AX_U32 &nNpuSet) {
|
| 27 |
+
AX_ENGINE_NPU_ATTR_T stNpuAttr;
|
| 28 |
+
memset(&stNpuAttr, 0x00, sizeof(stNpuAttr));
|
| 29 |
+
|
| 30 |
+
auto ret = AX_ENGINE_GetVNPUAttr(&stNpuAttr);
|
| 31 |
+
if (ret == 0) {
|
| 32 |
+
if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_DISABLE) {
|
| 33 |
+
nNpuSet = 0x01;
|
| 34 |
+
} else if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_STD) {
|
| 35 |
+
if (eModelType == AX_ENGINE_MODEL_TYPE1 || eModelType == AX_ENGINE_MODEL_TYPE2)
|
| 36 |
+
return -1;
|
| 37 |
+
if (nNpuType == 0) nNpuSet = 0x02;
|
| 38 |
+
else {
|
| 39 |
+
if (nNpuType & AX_STD_VNPU_1) nNpuSet |= 0x01;
|
| 40 |
+
if (nNpuType & AX_STD_VNPU_2) nNpuSet |= 0x02;
|
| 41 |
+
if (nNpuType & AX_STD_VNPU_3) nNpuSet |= 0x04;
|
| 42 |
+
}
|
| 43 |
+
} else if (stNpuAttr.eHardMode == AX_ENGINE_VIRTUAL_NPU_BIG_LITTLE) {
|
| 44 |
+
if (eModelType == AX_ENGINE_MODEL_TYPE2) return -1;
|
| 45 |
+
if (nNpuType == 0) {
|
| 46 |
+
nNpuSet = (eModelType == AX_ENGINE_MODEL_TYPE1) ? 0x01 : 0x02;
|
| 47 |
+
} else {
|
| 48 |
+
if (eModelType == AX_ENGINE_MODEL_TYPE1) {
|
| 49 |
+
if (nNpuType & AX_BL_VNPU_2) return -1;
|
| 50 |
+
if (nNpuType & AX_BL_VNPU_1) nNpuSet |= 0x01;
|
| 51 |
+
} else {
|
| 52 |
+
if (nNpuType & AX_BL_VNPU_1) nNpuSet |= 0x01;
|
| 53 |
+
if (nNpuType & AX_BL_VNPU_2) nNpuSet |= 0x02;
|
| 54 |
+
}
|
| 55 |
+
}
|
| 56 |
+
}
|
| 57 |
+
}
|
| 58 |
+
return ret;
|
| 59 |
+
}
|
| 60 |
+
|
| 61 |
+
EngineWrapper::EngineWrapper()
|
| 62 |
+
: m_hasInit(false), m_handle(nullptr), m_io_info(nullptr),
|
| 63 |
+
m_input_num(0), m_output_num(0) {
|
| 64 |
+
memset(&m_io, 0, sizeof(m_io));
|
| 65 |
+
}
|
| 66 |
+
|
| 67 |
+
EngineWrapper::~EngineWrapper() {
|
| 68 |
+
Release();
|
| 69 |
+
}
|
| 70 |
+
|
| 71 |
+
int EngineWrapper::Init(const char* strModelPath, uint32_t nNpuType, const char* axclConfig) {
|
| 72 |
+
(void)axclConfig; // unused on AXERA
|
| 73 |
+
// 1. Load model
|
| 74 |
+
AX_BOOL bLoadModelUseCmm = AX_TRUE;
|
| 75 |
+
AX_CHAR *pModelBufferVirAddr = nullptr;
|
| 76 |
+
AX_U64 u64ModelBufferPhyAddr = 0;
|
| 77 |
+
AX_U32 nModelBufferSize = 0;
|
| 78 |
+
|
| 79 |
+
if (bLoadModelUseCmm) {
|
| 80 |
+
if (!utils::read_file(strModelPath, (AX_VOID **)&pModelBufferVirAddr,
|
| 81 |
+
u64ModelBufferPhyAddr, nModelBufferSize)) {
|
| 82 |
+
printf("Read model(%s) fail\n", strModelPath);
|
| 83 |
+
return -1;
|
| 84 |
+
}
|
| 85 |
+
} else {
|
| 86 |
+
std::vector<char> model_buffer;
|
| 87 |
+
if (!utils::read_file(strModelPath, model_buffer)) {
|
| 88 |
+
printf("Read model(%s) fail\n", strModelPath);
|
| 89 |
+
return -1;
|
| 90 |
+
}
|
| 91 |
+
pModelBufferVirAddr = model_buffer.data();
|
| 92 |
+
nModelBufferSize = model_buffer.size();
|
| 93 |
+
}
|
| 94 |
+
|
| 95 |
+
auto freeModelBuffer = [&]() {
|
| 96 |
+
if (bLoadModelUseCmm) {
|
| 97 |
+
if (u64ModelBufferPhyAddr != 0)
|
| 98 |
+
AX_SYS_MemFree(u64ModelBufferPhyAddr, &pModelBufferVirAddr);
|
| 99 |
+
}
|
| 100 |
+
};
|
| 101 |
+
|
| 102 |
+
// 1.1 Get Model Type
|
| 103 |
+
AX_ENGINE_MODEL_TYPE_T eModelType = AX_ENGINE_MODEL_TYPE0;
|
| 104 |
+
AX_S32 ret = AX_ENGINE_GetModelType(pModelBufferVirAddr, nModelBufferSize, &eModelType);
|
| 105 |
+
if (0 != ret || eModelType >= AX_ENGINE_MODEL_TYPE_BUTT) {
|
| 106 |
+
printf("%s AX_ENGINE_GetModelType fail ret=%x\n", strModelPath, ret);
|
| 107 |
+
freeModelBuffer();
|
| 108 |
+
return -1;
|
| 109 |
+
}
|
| 110 |
+
|
| 111 |
+
// 1.2 Check VNPU
|
| 112 |
+
AX_U32 nNpuSet = 0;
|
| 113 |
+
ret = CheckModelVNpu(strModelPath, eModelType, nNpuType, nNpuSet);
|
| 114 |
+
if (0 != ret) {
|
| 115 |
+
printf("CheckModelVNpu fail\n");
|
| 116 |
+
freeModelBuffer();
|
| 117 |
+
return -1;
|
| 118 |
+
}
|
| 119 |
+
|
| 120 |
+
// 2. Create handle
|
| 121 |
+
AX_ENGINE_HANDLE handle = nullptr;
|
| 122 |
+
ret = AX_ENGINE_CreateHandle(&handle, pModelBufferVirAddr, nModelBufferSize);
|
| 123 |
+
freeModelBuffer();
|
| 124 |
+
|
| 125 |
+
auto deinit_handle = [&handle]() {
|
| 126 |
+
if (handle) { AX_ENGINE_DestroyHandle(handle); }
|
| 127 |
+
return -1;
|
| 128 |
+
};
|
| 129 |
+
|
| 130 |
+
if (0 != ret || !handle) {
|
| 131 |
+
printf("Create model(%s) handle fail\n", strModelPath);
|
| 132 |
+
return deinit_handle();
|
| 133 |
+
}
|
| 134 |
+
|
| 135 |
+
// 3. Create context
|
| 136 |
+
ret = AX_ENGINE_CreateContext(handle);
|
| 137 |
+
if (0 != ret) return deinit_handle();
|
| 138 |
+
|
| 139 |
+
// 4. Get IO info
|
| 140 |
+
m_io_info = nullptr;
|
| 141 |
+
ret = AX_ENGINE_GetIOInfo(handle, &m_io_info);
|
| 142 |
+
if (0 != ret) return deinit_handle();
|
| 143 |
+
|
| 144 |
+
m_input_num = m_io_info->nInputSize;
|
| 145 |
+
m_output_num = m_io_info->nOutputSize;
|
| 146 |
+
|
| 147 |
+
// Build name-to-index maps
|
| 148 |
+
m_input_name_to_idx.clear();
|
| 149 |
+
for (int i = 0; i < m_input_num; ++i) {
|
| 150 |
+
m_input_name_to_idx[std::string(m_io_info->pInputs[i].pName)] = i;
|
| 151 |
+
}
|
| 152 |
+
m_output_name_to_idx.clear();
|
| 153 |
+
for (int i = 0; i < m_output_num; ++i) {
|
| 154 |
+
m_output_name_to_idx[std::string(m_io_info->pOutputs[i].pName)] = i;
|
| 155 |
+
}
|
| 156 |
+
|
| 157 |
+
// 5. Prepare IO buffers
|
| 158 |
+
ret = utils::prepare_io("enc", m_io_info, m_io, utils::IO_BUFFER_STRATEGY_DEFAULT);
|
| 159 |
+
if (0 != ret) {
|
| 160 |
+
printf("prepare io failed!\n");
|
| 161 |
+
utils::free_io(m_io);
|
| 162 |
+
return deinit_handle();
|
| 163 |
+
}
|
| 164 |
+
|
| 165 |
+
m_handle = handle;
|
| 166 |
+
m_hasInit = true;
|
| 167 |
+
return 0;
|
| 168 |
+
}
|
| 169 |
+
|
| 170 |
+
int EngineWrapper::SetInput(void* pInput, int index) {
|
| 171 |
+
if (!m_hasInit || index < 0 || index >= m_input_num) return -1;
|
| 172 |
+
return utils::push_io_input(pInput, index, m_io);
|
| 173 |
+
}
|
| 174 |
+
|
| 175 |
+
int EngineWrapper::RunSync() {
|
| 176 |
+
if (!m_hasInit) return -1;
|
| 177 |
+
auto ret = AX_ENGINE_RunSync(m_handle, &m_io);
|
| 178 |
+
if (0 != ret) {
|
| 179 |
+
printf("AX_ENGINE_RunSync failed. ret=0x%x\n", ret);
|
| 180 |
+
}
|
| 181 |
+
return ret;
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
int EngineWrapper::GetOutput(void* pOutput, int index) {
|
| 185 |
+
if (!m_hasInit || index < 0 || index >= m_output_num) return -1;
|
| 186 |
+
return utils::push_io_output(pOutput, index, m_io);
|
| 187 |
+
}
|
| 188 |
+
|
| 189 |
+
int EngineWrapper::SetInputByName(const char* name, void* pInput) {
|
| 190 |
+
auto it = m_input_name_to_idx.find(std::string(name));
|
| 191 |
+
if (it == m_input_name_to_idx.end()) {
|
| 192 |
+
printf("Input '%s' not found in model\n", name);
|
| 193 |
+
return -1;
|
| 194 |
+
}
|
| 195 |
+
return SetInput(pInput, it->second);
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
int EngineWrapper::GetOutputByName(const char* name, void* pOutput) {
|
| 199 |
+
auto it = m_output_name_to_idx.find(std::string(name));
|
| 200 |
+
if (it == m_output_name_to_idx.end()) {
|
| 201 |
+
printf("Output '%s' not found in model\n", name);
|
| 202 |
+
return -1;
|
| 203 |
+
}
|
| 204 |
+
return GetOutput(pOutput, it->second);
|
| 205 |
+
}
|
| 206 |
+
|
| 207 |
+
int EngineWrapper::GetInputSize(int index) {
|
| 208 |
+
if (index < 0 || index >= m_input_num) return -1;
|
| 209 |
+
return m_io.pInputs[index].nSize;
|
| 210 |
+
}
|
| 211 |
+
|
| 212 |
+
int EngineWrapper::GetOutputSize(int index) {
|
| 213 |
+
if (index < 0 || index >= m_output_num) return -1;
|
| 214 |
+
return m_io.pOutputs[index].nSize;
|
| 215 |
+
}
|
| 216 |
+
|
| 217 |
+
int EngineWrapper::GetInputSizeByName(const char* name) {
|
| 218 |
+
int idx = GetInputIndex(name);
|
| 219 |
+
if (idx < 0) return -1;
|
| 220 |
+
return GetInputSize(idx);
|
| 221 |
+
}
|
| 222 |
+
|
| 223 |
+
int EngineWrapper::GetOutputSizeByName(const char* name) {
|
| 224 |
+
int idx = GetOutputIndex(name);
|
| 225 |
+
if (idx < 0) return -1;
|
| 226 |
+
return GetOutputSize(idx);
|
| 227 |
+
}
|
| 228 |
+
|
| 229 |
+
int EngineWrapper::GetInputIndex(const char* name) {
|
| 230 |
+
auto it = m_input_name_to_idx.find(std::string(name));
|
| 231 |
+
return (it != m_input_name_to_idx.end()) ? it->second : -1;
|
| 232 |
+
}
|
| 233 |
+
|
| 234 |
+
int EngineWrapper::GetOutputIndex(const char* name) {
|
| 235 |
+
auto it = m_output_name_to_idx.find(std::string(name));
|
| 236 |
+
return (it != m_output_name_to_idx.end()) ? it->second : -1;
|
| 237 |
+
}
|
| 238 |
+
|
| 239 |
+
const char* EngineWrapper::GetInputName(int index) const {
|
| 240 |
+
if (index < 0 || index >= m_input_num) return nullptr;
|
| 241 |
+
return m_io_info->pInputs[index].pName;
|
| 242 |
+
}
|
| 243 |
+
|
| 244 |
+
const char* EngineWrapper::GetOutputName(int index) const {
|
| 245 |
+
if (index < 0 || index >= m_output_num) return nullptr;
|
| 246 |
+
return m_io_info->pOutputs[index].pName;
|
| 247 |
+
}
|
| 248 |
+
|
| 249 |
+
static const char* dtype_str(AX_ENGINE_DATA_TYPE_T t) {
|
| 250 |
+
switch (t) {
|
| 251 |
+
case AX_ENGINE_DT_FLOAT32: return "float32";
|
| 252 |
+
case AX_ENGINE_DT_FLOAT64: return "float64";
|
| 253 |
+
case AX_ENGINE_DT_SINT8: return "sint8";
|
| 254 |
+
case AX_ENGINE_DT_UINT8: return "uint8";
|
| 255 |
+
case AX_ENGINE_DT_SINT16: return "sint16";
|
| 256 |
+
case AX_ENGINE_DT_UINT16: return "uint16";
|
| 257 |
+
case AX_ENGINE_DT_SINT32: return "sint32";
|
| 258 |
+
case AX_ENGINE_DT_UINT32: return "uint32";
|
| 259 |
+
default: return "unknown";
|
| 260 |
+
}
|
| 261 |
+
}
|
| 262 |
+
|
| 263 |
+
const char* EngineWrapper::GetInputDtypeStr(int index) const {
|
| 264 |
+
if (index < 0 || index >= m_input_num) return "?";
|
| 265 |
+
return dtype_str(m_io_info->pInputs[index].eDataType);
|
| 266 |
+
}
|
| 267 |
+
|
| 268 |
+
const char* EngineWrapper::GetOutputDtypeStr(int index) const {
|
| 269 |
+
if (index < 0 || index >= m_output_num) return "?";
|
| 270 |
+
return dtype_str(m_io_info->pOutputs[index].eDataType);
|
| 271 |
+
}
|
| 272 |
+
|
| 273 |
+
int EngineWrapper::Release() {
|
| 274 |
+
if (m_handle) {
|
| 275 |
+
utils::free_io(m_io);
|
| 276 |
+
AX_ENGINE_DestroyHandle(m_handle);
|
| 277 |
+
m_handle = nullptr;
|
| 278 |
+
}
|
| 279 |
+
m_hasInit = false;
|
| 280 |
+
m_input_name_to_idx.clear();
|
| 281 |
+
m_output_name_to_idx.clear();
|
| 282 |
+
return 0;
|
| 283 |
+
}
|
cpp/src/middleware/ax_npu_runner.cc
ADDED
|
@@ -0,0 +1,131 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#include "middleware/ax_npu_runner_impl.hpp"
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AX_LOCAL_API_ENABLE)
|
| 14 |
+
middleware::npu_runner::npu_runner() {
|
| 15 |
+
this->impl_ = new impl();
|
| 16 |
+
}
|
| 17 |
+
|
| 18 |
+
middleware::npu_runner::~npu_runner() {
|
| 19 |
+
delete this->impl_;
|
| 20 |
+
}
|
| 21 |
+
|
| 22 |
+
bool middleware::npu_runner::init(const std::string& config_file, const uint32_t& index, const uint32_t& kind) {
|
| 23 |
+
std::ignore = config_file;
|
| 24 |
+
std::ignore = index;
|
| 25 |
+
return this->impl_->init(kind);
|
| 26 |
+
}
|
| 27 |
+
|
| 28 |
+
bool middleware::npu_runner::final() {
|
| 29 |
+
return this->impl_->final();
|
| 30 |
+
}
|
| 31 |
+
|
| 32 |
+
bool middleware::npu_runner::load(const std::string& model_path) {
|
| 33 |
+
return this->impl_->load(model_path);
|
| 34 |
+
}
|
| 35 |
+
|
| 36 |
+
bool middleware::npu_runner::prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 37 |
+
return this->impl_->prepare(input_cached, output_cached, group, batch);
|
| 38 |
+
}
|
| 39 |
+
|
| 40 |
+
bool middleware::npu_runner::run(const bool& parallel) {
|
| 41 |
+
return this->impl_->run(parallel);
|
| 42 |
+
}
|
| 43 |
+
|
| 44 |
+
uint32_t middleware::npu_runner::get_input_count() const {
|
| 45 |
+
return this->impl_->get_input_count();
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
uint32_t middleware::npu_runner::get_output_count() const {
|
| 49 |
+
return this->impl_->get_output_count();
|
| 50 |
+
}
|
| 51 |
+
|
| 52 |
+
std::string middleware::npu_runner::get_input_name(const uint32_t& index) const {
|
| 53 |
+
return this->impl_->get_input_name(index);
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
std::string middleware::npu_runner::get_output_name(const uint32_t& index) const {
|
| 57 |
+
return this->impl_->get_output_name(index);
|
| 58 |
+
}
|
| 59 |
+
|
| 60 |
+
void *middleware::npu_runner::get_input_pointer(const uint32_t& index) const {
|
| 61 |
+
return this->impl_->get_input_pointer(index);
|
| 62 |
+
}
|
| 63 |
+
|
| 64 |
+
void *middleware::npu_runner::get_output_pointer(const uint32_t& index) const {
|
| 65 |
+
return this->impl_->get_output_pointer(index);
|
| 66 |
+
}
|
| 67 |
+
|
| 68 |
+
uintmax_t middleware::npu_runner::get_input_size(const uint32_t& index) const {
|
| 69 |
+
return this->impl_->get_input_size(index);
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
uintmax_t middleware::npu_runner::get_output_size(const uint32_t& index) const {
|
| 73 |
+
return this->impl_->get_output_size(index);
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
uintmax_t middleware::npu_runner::get_shape_group_count() const {
|
| 77 |
+
return this->impl_->get_shape_group_count();
|
| 78 |
+
}
|
| 79 |
+
|
| 80 |
+
bool middleware::npu_runner::flush_input() const {
|
| 81 |
+
return this->impl_->flush_input();
|
| 82 |
+
}
|
| 83 |
+
|
| 84 |
+
bool middleware::npu_runner::invalidate_output() const {
|
| 85 |
+
return this->impl_->invalidate_output();
|
| 86 |
+
}
|
| 87 |
+
|
| 88 |
+
bool middleware::npu_runner::feed(const std::string& input_folder, const std::string& stimulus_name) const {
|
| 89 |
+
return this->impl_->feed(input_folder, stimulus_name);
|
| 90 |
+
}
|
| 91 |
+
|
| 92 |
+
bool middleware::npu_runner::verify(const std::string& output_folder, const std::string& stimulus_name) const {
|
| 93 |
+
return this->impl_->verify(output_folder, stimulus_name);
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
bool middleware::npu_runner::save(const std::string& output_folder, const std::string& stimulus_name) const {
|
| 97 |
+
return this->impl_->save(output_folder, stimulus_name);
|
| 98 |
+
}
|
| 99 |
+
|
| 100 |
+
std::string middleware::npu_runner::get_library_version() const {
|
| 101 |
+
return this->impl_->get_library_version();
|
| 102 |
+
}
|
| 103 |
+
|
| 104 |
+
std::string middleware::npu_runner::get_model_version() const {
|
| 105 |
+
return this->impl_->get_model_version();
|
| 106 |
+
}
|
| 107 |
+
|
| 108 |
+
int32_t middleware::npu_runner::get_model_type() const {
|
| 109 |
+
return this->impl_->get_model_type();
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
int32_t middleware::npu_runner::get_npu_type() const {
|
| 113 |
+
return this->impl_->get_npu_type();
|
| 114 |
+
}
|
| 115 |
+
|
| 116 |
+
int32_t middleware::npu_runner::get_batch_size() const {
|
| 117 |
+
return this->impl_->get_batch_size();
|
| 118 |
+
}
|
| 119 |
+
|
| 120 |
+
intmax_t middleware::npu_runner::get_sys_usage() const {
|
| 121 |
+
return this->impl_->get_sys_usage();
|
| 122 |
+
}
|
| 123 |
+
|
| 124 |
+
intmax_t middleware::npu_runner::get_cmm_usage() const {
|
| 125 |
+
return this->impl_->get_cmm_usage();
|
| 126 |
+
}
|
| 127 |
+
|
| 128 |
+
void middleware::npu_runner::sleep_for(const uint32_t sleep_duration) const {
|
| 129 |
+
impl::sleep_for(sleep_duration);
|
| 130 |
+
}
|
| 131 |
+
#endif
|
cpp/src/middleware/ax_npu_runner.hpp
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AX_LOCAL_API_ENABLE)
|
| 14 |
+
#include "middleware/runner.hpp"
|
| 15 |
+
|
| 16 |
+
namespace middleware {
|
| 17 |
+
|
| 18 |
+
class npu_runner final : public runner {
|
| 19 |
+
public:
|
| 20 |
+
npu_runner();
|
| 21 |
+
~npu_runner() override;
|
| 22 |
+
|
| 23 |
+
[[nodiscard]] bool init(const std::string& config_file, const uint32_t& index, const uint32_t& kind) override;
|
| 24 |
+
[[nodiscard]] bool final() override;
|
| 25 |
+
|
| 26 |
+
[[nodiscard]] bool load(const std::string& model_path) override;
|
| 27 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) override;
|
| 28 |
+
|
| 29 |
+
[[nodiscard]] bool run(const bool& parallel) override;
|
| 30 |
+
|
| 31 |
+
[[nodiscard]] uint32_t get_input_count() const override;
|
| 32 |
+
[[nodiscard]] uint32_t get_output_count() const override;
|
| 33 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) const override;
|
| 34 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) const override;
|
| 35 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) const override;
|
| 36 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) const override;
|
| 37 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) const override;
|
| 38 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) const override;
|
| 39 |
+
[[nodiscard]] uintmax_t get_shape_group_count() const override;
|
| 40 |
+
|
| 41 |
+
[[nodiscard]] bool flush_input() const override;
|
| 42 |
+
[[nodiscard]] bool invalidate_output() const override;
|
| 43 |
+
|
| 44 |
+
[[nodiscard]] bool feed(const std::string& input_folder, const std::string& stimulus_name) const override;
|
| 45 |
+
[[nodiscard]] bool verify(const std::string& output_folder, const std::string& stimulus_name) const override;
|
| 46 |
+
[[nodiscard]] bool save(const std::string& output_folder, const std::string& stimulus_name) const override;
|
| 47 |
+
|
| 48 |
+
[[nodiscard]] std::string get_library_version() const override;
|
| 49 |
+
[[nodiscard]] std::string get_model_version() const override;
|
| 50 |
+
|
| 51 |
+
[[nodiscard]] int32_t get_model_type() const override;
|
| 52 |
+
[[nodiscard]] int32_t get_npu_type() const override;
|
| 53 |
+
[[nodiscard]] int32_t get_batch_size() const override;
|
| 54 |
+
[[nodiscard]] intmax_t get_sys_usage() const override;
|
| 55 |
+
[[nodiscard]] intmax_t get_cmm_usage() const override;
|
| 56 |
+
|
| 57 |
+
void sleep_for(uint32_t sleep_duration) const override;
|
| 58 |
+
|
| 59 |
+
private:
|
| 60 |
+
struct impl;
|
| 61 |
+
impl *impl_;
|
| 62 |
+
};
|
| 63 |
+
#endif
|
| 64 |
+
|
| 65 |
+
}
|
cpp/src/middleware/ax_npu_runner_impl.hpp
ADDED
|
@@ -0,0 +1,640 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AX_LOCAL_API_ENABLE)
|
| 14 |
+
#include "middleware/ax_npu_runner.hpp"
|
| 15 |
+
|
| 16 |
+
#include "utilities/scalar_guard.hpp"
|
| 17 |
+
#include "utilities/file.hpp"
|
| 18 |
+
#include "utilities/file_mapper.hpp"
|
| 19 |
+
#include "utilities/log.hpp"
|
| 20 |
+
|
| 21 |
+
#include <ax_hrtimer.h>
|
| 22 |
+
#include <ax_sys_api.h>
|
| 23 |
+
#include <ax_engine_api.h>
|
| 24 |
+
|
| 25 |
+
#include <mutex>
|
| 26 |
+
|
| 27 |
+
constexpr ::AX_U32 align_size = 128;
|
| 28 |
+
constexpr ::AX_S8 token_name[] = "toolkit";
|
| 29 |
+
|
| 30 |
+
struct middleware::npu_runner::impl {
|
| 31 |
+
impl() = default;
|
| 32 |
+
|
| 33 |
+
~impl() {
|
| 34 |
+
std::ignore = this->final();
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
[[nodiscard]] bool init(const uint32_t& kind) {
|
| 38 |
+
std::lock_guard guard(this->mutex_);
|
| 39 |
+
|
| 40 |
+
// 0. check the NPU kind
|
| 41 |
+
if (kind >= ::AX_ENGINE_VIRTUAL_NPU_BUTT) {
|
| 42 |
+
utilities::glog.print(utilities::log::type::error, "Specified NPU kind{%d} is out of range{total %d}.\n", kind, ::AX_ENGINE_VIRTUAL_NPU_BUTT);
|
| 43 |
+
return false;
|
| 44 |
+
}
|
| 45 |
+
|
| 46 |
+
// 1. init ax system, using scalar_guard to ensure the finalization
|
| 47 |
+
auto sys_guard = utilities::scalar_guard<::AX_S32>(
|
| 48 |
+
::AX_SYS_Init(),
|
| 49 |
+
[](const ::AX_S32& state) {
|
| 50 |
+
if (0 == state) {
|
| 51 |
+
std::ignore = ::AX_SYS_Deinit();
|
| 52 |
+
}
|
| 53 |
+
});
|
| 54 |
+
|
| 55 |
+
// 2. check the initialization result
|
| 56 |
+
if (0 != sys_guard.get()) {
|
| 57 |
+
utilities::glog.print(utilities::log::type::error, "AX SYS init failed{0x%08X}.\n", sys_guard.get());
|
| 58 |
+
return false;
|
| 59 |
+
}
|
| 60 |
+
utilities::glog.print(utilities::log::type::info, "AX SYS inited.\n");
|
| 61 |
+
|
| 62 |
+
// 3. init ax engine
|
| 63 |
+
::AX_ENGINE_NPU_ATTR_T attr{};
|
| 64 |
+
attr.eHardMode = static_cast<::AX_ENGINE_NPU_MODE_T>(kind);
|
| 65 |
+
if (const int ret = ::AX_ENGINE_Init(&attr); 0 != ret) {
|
| 66 |
+
utilities::glog.print(utilities::log::type::error, "Init AX ENGINE as kind{%d} failed{0x%08X}.\n", kind, ret);
|
| 67 |
+
return false;
|
| 68 |
+
}
|
| 69 |
+
utilities::glog.print(utilities::log::type::info, "AX ENGINE inited.\n");
|
| 70 |
+
|
| 71 |
+
// 4. disable guard of env
|
| 72 |
+
sys_guard.get() = -1;
|
| 73 |
+
|
| 74 |
+
// 5. set the flag
|
| 75 |
+
this->is_initialized_ = true;
|
| 76 |
+
return true;
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
+
[[nodiscard]] bool final() {
|
| 80 |
+
auto flag = true;
|
| 81 |
+
|
| 82 |
+
std::lock_guard guard(this->mutex_);
|
| 83 |
+
|
| 84 |
+
if (this->is_initialized_) {
|
| 85 |
+
if (nullptr != this->handle_) {
|
| 86 |
+
for (auto& input : inputs_) {
|
| 87 |
+
if (0 != input.phyAddr) {
|
| 88 |
+
flag &= 0 == ::AX_SYS_MemFree(input.phyAddr, input.pVirAddr);
|
| 89 |
+
input.phyAddr = 0;
|
| 90 |
+
}
|
| 91 |
+
}
|
| 92 |
+
for (auto& output : outputs_) {
|
| 93 |
+
if (0 != output.phyAddr) {
|
| 94 |
+
flag &= 0 == ::AX_SYS_MemFree(output.phyAddr, output.pVirAddr);
|
| 95 |
+
output.phyAddr = 0;
|
| 96 |
+
}
|
| 97 |
+
}
|
| 98 |
+
|
| 99 |
+
flag &= 0 == ::AX_ENGINE_DestroyHandle(this->handle_);
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
flag &= 0 == ::AX_ENGINE_Deinit();
|
| 103 |
+
flag &= 0 == ::AX_SYS_Deinit();
|
| 104 |
+
this->is_initialized_ = false;
|
| 105 |
+
}
|
| 106 |
+
|
| 107 |
+
return flag;
|
| 108 |
+
}
|
| 109 |
+
|
| 110 |
+
[[nodiscard]] bool load(const std::string& model_path) {
|
| 111 |
+
std::lock_guard guard(this->mutex_);
|
| 112 |
+
|
| 113 |
+
if (!this->is_initialized_) {
|
| 114 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 115 |
+
return false;
|
| 116 |
+
}
|
| 117 |
+
|
| 118 |
+
if (nullptr != this->handle_) {
|
| 119 |
+
utilities::glog.print(utilities::log::type::error, "Model is already loaded.\n");
|
| 120 |
+
return false;
|
| 121 |
+
}
|
| 122 |
+
|
| 123 |
+
if (!utilities::exists(model_path)
|
| 124 |
+
|| !utilities::is_regular_file(model_path)
|
| 125 |
+
|| 0 == utilities::file_size(model_path)
|
| 126 |
+
|| utilities::error_size == utilities::file_size(model_path)) {
|
| 127 |
+
utilities::glog.print(utilities::log::type::error, "Model file{%s} error, please check it.\n", model_path.c_str());
|
| 128 |
+
return false;
|
| 129 |
+
}
|
| 130 |
+
|
| 131 |
+
const auto model_map = utilities::file_mapper(model_path);
|
| 132 |
+
if (nullptr == model_map.get()) {
|
| 133 |
+
utilities::glog.print(utilities::log::type::error, "Model file{%s} mapping failed.\n", model_path.c_str());
|
| 134 |
+
return false;
|
| 135 |
+
}
|
| 136 |
+
|
| 137 |
+
const auto model_name = utilities::get_file_name(model_path);
|
| 138 |
+
::AX_ENGINE_HANDLE_EXTRA_T extra_param{};
|
| 139 |
+
extra_param.pName = reinterpret_cast<::AX_S8*>(const_cast<char*>(model_name.c_str()));
|
| 140 |
+
if (const auto ret = ::AX_ENGINE_CreateHandleV2(&this->handle_, model_map.get(), model_map.size(), &extra_param); 0 != ret) {
|
| 141 |
+
utilities::glog.print(utilities::log::type::error, "Create model{%s} handle failed.\n", model_path.c_str());
|
| 142 |
+
return false;
|
| 143 |
+
}
|
| 144 |
+
|
| 145 |
+
if (const auto ret = ::AX_ENGINE_CreateContextV2(this->handle_, &this->context_); 0 != ret) {
|
| 146 |
+
utilities::glog.print(utilities::log::type::error, "Create model{%s} context failed.\n", model_path.c_str());
|
| 147 |
+
return false;
|
| 148 |
+
}
|
| 149 |
+
|
| 150 |
+
return true;
|
| 151 |
+
}
|
| 152 |
+
|
| 153 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 154 |
+
std::lock_guard guard(this->mutex_);
|
| 155 |
+
|
| 156 |
+
// 0. check the handle
|
| 157 |
+
if (nullptr == this->handle_) {
|
| 158 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is null.\n");
|
| 159 |
+
return false;
|
| 160 |
+
}
|
| 161 |
+
|
| 162 |
+
// 1. get the count of shape group
|
| 163 |
+
if (const auto ret = ::AX_ENGINE_GetGroupIOInfoCount(this->handle_, &this->group_); 0 != ret) {
|
| 164 |
+
utilities::glog.print(utilities::log::type::error, "Get model shape group count failed{0x%08X}.\n", ret);
|
| 165 |
+
return false;
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
// 2. check the group index
|
| 169 |
+
if (group >= this->group_) {
|
| 170 |
+
utilities::glog.print(utilities::log::type::error, "Model group{%d} is out of range{total %d}.\n", group, this->group_);
|
| 171 |
+
return false;
|
| 172 |
+
}
|
| 173 |
+
|
| 174 |
+
// 3. get the IO info
|
| 175 |
+
if (1 == this->group_) {
|
| 176 |
+
if (const auto ret = ::AX_ENGINE_GetIOInfo(this->handle_, &info_); 0 != ret) {
|
| 177 |
+
utilities::glog.print(utilities::log::type::error, "Get model IO info failed{0x%08X}.\n", ret);
|
| 178 |
+
return false;
|
| 179 |
+
}
|
| 180 |
+
} else {
|
| 181 |
+
if (const auto ret = ::AX_ENGINE_GetGroupIOInfo(this->handle_, group, &info_); 0 != ret) {
|
| 182 |
+
utilities::glog.print(utilities::log::type::error, "Get model group{index: %d} IO info failed{0x%08X}.\n", group, ret);
|
| 183 |
+
return false;
|
| 184 |
+
}
|
| 185 |
+
}
|
| 186 |
+
|
| 187 |
+
// 4. check the batch size
|
| 188 |
+
this->batch = (0 == batch ? 1 : batch);
|
| 189 |
+
this->io_.nBatchSize = this->batch;
|
| 190 |
+
|
| 191 |
+
// 5. prepare the input and output
|
| 192 |
+
this->inputs_.resize(this->info_->nInputSize, ::AX_ENGINE_IO_BUFFER_T{});
|
| 193 |
+
this->io_.pInputs = this->inputs_.data();
|
| 194 |
+
this->io_.nInputSize = this->info_->nInputSize;
|
| 195 |
+
|
| 196 |
+
this->outputs_.resize(this->info_->nOutputSize, ::AX_ENGINE_IO_BUFFER_T{});
|
| 197 |
+
this->io_.pOutputs = this->outputs_.data();
|
| 198 |
+
this->io_.nOutputSize = this->info_->nOutputSize;
|
| 199 |
+
|
| 200 |
+
this->input_cached_.resize(inputs_.size(), input_cached);
|
| 201 |
+
this->output_cached_.resize(outputs_.size(), output_cached);
|
| 202 |
+
|
| 203 |
+
// 6. prepare the memory, inputs
|
| 204 |
+
for (::AX_U32 i = 0; i < this->info_->nInputSize; i++) {
|
| 205 |
+
const auto& meta = this->info_->pInputs[i];
|
| 206 |
+
auto& io = this->io_.pInputs[i];
|
| 207 |
+
io.nSize = meta.nSize * this->batch;
|
| 208 |
+
|
| 209 |
+
if (this->input_cached_[i]) {
|
| 210 |
+
if (const auto ret = ::AX_SYS_MemAllocCached(&io.phyAddr, &io.pVirAddr, io.nSize, align_size, token_name); 0 != ret) {
|
| 211 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 212 |
+
return false;
|
| 213 |
+
}
|
| 214 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta.pName);
|
| 215 |
+
} else {
|
| 216 |
+
if (const auto ret = ::AX_SYS_MemAlloc(&io.phyAddr, &io.pVirAddr, io.nSize, align_size, token_name); 0 != ret) {
|
| 217 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 218 |
+
return false;
|
| 219 |
+
}
|
| 220 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta.pName);
|
| 221 |
+
}
|
| 222 |
+
|
| 223 |
+
// clean memory, some cases model may need to clean memory
|
| 224 |
+
memset(io.pVirAddr, 0, io.nSize);
|
| 225 |
+
|
| 226 |
+
if (this->output_cached_[i]) {
|
| 227 |
+
if (const auto ret = ::AX_SYS_MflushCache(io.phyAddr, io.pVirAddr, io.nSize); 0 != ret) {
|
| 228 |
+
utilities::glog.print(utilities::log::type::error, "Memory flush cache for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 229 |
+
return false;
|
| 230 |
+
}
|
| 231 |
+
}
|
| 232 |
+
|
| 233 |
+
utilities::glog.print(utilities::log::type::info, "Set tensor { name: %s, phy: %p, vir: %p, size: %u Bytes }.\n",
|
| 234 |
+
meta.pName, reinterpret_cast<void*>(io.phyAddr), io.pVirAddr, io.nSize);
|
| 235 |
+
}
|
| 236 |
+
|
| 237 |
+
// 7. prepare the memory, outputs
|
| 238 |
+
for (::AX_U32 i = 0; i < this->info_->nOutputSize; i++) {
|
| 239 |
+
const auto& meta = this->info_->pOutputs[i];
|
| 240 |
+
auto& io = this->io_.pOutputs[i];
|
| 241 |
+
io.nSize = meta.nSize * this->batch;
|
| 242 |
+
|
| 243 |
+
if (this->output_cached_[i]) {
|
| 244 |
+
if (const auto ret = ::AX_SYS_MemAllocCached(&io.phyAddr, &io.pVirAddr, io.nSize, align_size, token_name); 0 != ret) {
|
| 245 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 246 |
+
return false;
|
| 247 |
+
}
|
| 248 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta.pName);
|
| 249 |
+
} else {
|
| 250 |
+
if (const auto ret = ::AX_SYS_MemAlloc(&io.phyAddr, &io.pVirAddr, io.nSize, align_size, token_name); 0 != ret) {
|
| 251 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 252 |
+
return false;
|
| 253 |
+
}
|
| 254 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta.pName);
|
| 255 |
+
}
|
| 256 |
+
|
| 257 |
+
// clean memory, some cases model may need to clean memory
|
| 258 |
+
memset(io.pVirAddr, 0, io.nSize);
|
| 259 |
+
|
| 260 |
+
if (this->output_cached_[i]) {
|
| 261 |
+
if (const auto ret = ::AX_SYS_MflushCache(io.phyAddr, io.pVirAddr, io.nSize); 0 != ret) {
|
| 262 |
+
utilities::glog.print(utilities::log::type::error, "Memory flush cache for tensor {%s} failed{0x%08X}.\n", meta.pName, ret);
|
| 263 |
+
return false;
|
| 264 |
+
}
|
| 265 |
+
}
|
| 266 |
+
|
| 267 |
+
utilities::glog.print(utilities::log::type::info, "Set tensor { name: %s, phy: %p, vir: %p, size: %u Bytes }.\n",
|
| 268 |
+
meta.pName, reinterpret_cast<void*>(io.phyAddr), io.pVirAddr, io.nSize);
|
| 269 |
+
}
|
| 270 |
+
|
| 271 |
+
return true;
|
| 272 |
+
}
|
| 273 |
+
|
| 274 |
+
[[nodiscard]] bool run(const bool& parallel) {
|
| 275 |
+
std::lock_guard guard(this->mutex_);
|
| 276 |
+
|
| 277 |
+
this->io_.nParallelRun = (parallel ? 1 : 0);
|
| 278 |
+
|
| 279 |
+
if (nullptr == this->handle_ || nullptr == this->context_) {
|
| 280 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 281 |
+
return false;
|
| 282 |
+
}
|
| 283 |
+
|
| 284 |
+
if (0 == this->io_.nInputSize || 0 == this->io_.nOutputSize) {
|
| 285 |
+
utilities::glog.print(utilities::log::type::error, "Model io is not set, prepare first.\n");
|
| 286 |
+
return false;
|
| 287 |
+
}
|
| 288 |
+
|
| 289 |
+
::AX_S32 ret = 0;
|
| 290 |
+
if (1 == this->group_) {
|
| 291 |
+
ret = ::AX_ENGINE_RunSyncV2(this->handle_, this->context_, &this->io_);
|
| 292 |
+
} else {
|
| 293 |
+
ret = ::AX_ENGINE_RunGroupIOSync(this->handle_, this->context_, this->group_, &this->io_);
|
| 294 |
+
}
|
| 295 |
+
if (0 != ret) {
|
| 296 |
+
utilities::glog.print(utilities::log::type::error, "Run model failed{0x%08X}.\n", ret);
|
| 297 |
+
return false;
|
| 298 |
+
}
|
| 299 |
+
utilities::glog.print(utilities::log::type::info, "Running done.\n");
|
| 300 |
+
|
| 301 |
+
return true;
|
| 302 |
+
}
|
| 303 |
+
|
| 304 |
+
[[nodiscard]] uint32_t get_input_count() {
|
| 305 |
+
std::lock_guard guard(this->mutex_);
|
| 306 |
+
|
| 307 |
+
if (nullptr == this->info_) {
|
| 308 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 309 |
+
return 0;
|
| 310 |
+
}
|
| 311 |
+
|
| 312 |
+
return this->info_->nInputSize;
|
| 313 |
+
}
|
| 314 |
+
|
| 315 |
+
[[nodiscard]] uint32_t get_output_count() {
|
| 316 |
+
std::lock_guard guard(this->mutex_);
|
| 317 |
+
|
| 318 |
+
if (nullptr == this->info_) {
|
| 319 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 320 |
+
return 0;
|
| 321 |
+
}
|
| 322 |
+
|
| 323 |
+
return this->info_->nOutputSize;
|
| 324 |
+
}
|
| 325 |
+
|
| 326 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) {
|
| 327 |
+
std::lock_guard guard(this->mutex_);
|
| 328 |
+
|
| 329 |
+
if (nullptr == this->info_) {
|
| 330 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 331 |
+
return {};
|
| 332 |
+
}
|
| 333 |
+
|
| 334 |
+
if (index >= this->info_->nInputSize) {
|
| 335 |
+
return {};
|
| 336 |
+
}
|
| 337 |
+
return std::string{this->info_->pInputs[index].pName};
|
| 338 |
+
}
|
| 339 |
+
|
| 340 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) {
|
| 341 |
+
std::lock_guard guard(this->mutex_);
|
| 342 |
+
|
| 343 |
+
if (nullptr == this->info_) {
|
| 344 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 345 |
+
return {};
|
| 346 |
+
}
|
| 347 |
+
|
| 348 |
+
if (index >= this->info_->nOutputSize) {
|
| 349 |
+
return {};
|
| 350 |
+
}
|
| 351 |
+
return std::string{this->info_->pOutputs[index].pName};
|
| 352 |
+
}
|
| 353 |
+
|
| 354 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) {
|
| 355 |
+
std::lock_guard guard(this->mutex_);
|
| 356 |
+
|
| 357 |
+
if (nullptr == this->info_) {
|
| 358 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 359 |
+
return nullptr;
|
| 360 |
+
}
|
| 361 |
+
|
| 362 |
+
if (index >= this->info_->nInputSize) {
|
| 363 |
+
return nullptr;
|
| 364 |
+
}
|
| 365 |
+
return reinterpret_cast<void *>(this->io_.pInputs[index].phyAddr);
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) {
|
| 369 |
+
std::lock_guard guard(this->mutex_);
|
| 370 |
+
|
| 371 |
+
if (nullptr == this->info_) {
|
| 372 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 373 |
+
return nullptr;
|
| 374 |
+
}
|
| 375 |
+
|
| 376 |
+
if (index >= this->info_->nOutputSize) {
|
| 377 |
+
return nullptr;
|
| 378 |
+
}
|
| 379 |
+
return reinterpret_cast<void *>(this->io_.pOutputs[index].phyAddr);
|
| 380 |
+
}
|
| 381 |
+
|
| 382 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) {
|
| 383 |
+
std::lock_guard guard(this->mutex_);
|
| 384 |
+
|
| 385 |
+
if (nullptr == this->info_) {
|
| 386 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 387 |
+
return 0;
|
| 388 |
+
}
|
| 389 |
+
|
| 390 |
+
if (index >= this->info_->nInputSize) {
|
| 391 |
+
return 0;
|
| 392 |
+
}
|
| 393 |
+
return this->io_.pInputs[index].nSize;
|
| 394 |
+
}
|
| 395 |
+
|
| 396 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) {
|
| 397 |
+
std::lock_guard guard(this->mutex_);
|
| 398 |
+
|
| 399 |
+
if (nullptr == this->info_) {
|
| 400 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 401 |
+
return 0;
|
| 402 |
+
}
|
| 403 |
+
|
| 404 |
+
if (index >= this->info_->nOutputSize) {
|
| 405 |
+
return 0;
|
| 406 |
+
}
|
| 407 |
+
return this->io_.pOutputs[index].nSize;
|
| 408 |
+
}
|
| 409 |
+
|
| 410 |
+
[[nodiscard]] uintmax_t get_shape_group_count() {
|
| 411 |
+
std::lock_guard guard(this->mutex_);
|
| 412 |
+
|
| 413 |
+
if (nullptr == this->info_) {
|
| 414 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 415 |
+
return 0;
|
| 416 |
+
}
|
| 417 |
+
|
| 418 |
+
return this->group_;
|
| 419 |
+
}
|
| 420 |
+
|
| 421 |
+
[[nodiscard]] bool flush_input() {
|
| 422 |
+
std::lock_guard guard(this->mutex_);
|
| 423 |
+
|
| 424 |
+
for (::AX_U32 i = 0; i < this->io_.nInputSize; i++) {
|
| 425 |
+
const auto buffer = &(this->io_.pInputs[i]);
|
| 426 |
+
if (const auto ret = ::AX_SYS_MflushCache(buffer->phyAddr, buffer->pVirAddr, buffer->nSize); 0 != ret) {
|
| 427 |
+
utilities::glog.print(utilities::log::type::warn, "Flush input{index: %d} failed{0x%08X}.\n", i, ret);
|
| 428 |
+
return false;
|
| 429 |
+
}
|
| 430 |
+
}
|
| 431 |
+
return true;
|
| 432 |
+
}
|
| 433 |
+
|
| 434 |
+
[[nodiscard]] bool invalidate_output() {
|
| 435 |
+
std::lock_guard guard(this->mutex_);
|
| 436 |
+
|
| 437 |
+
for (::AX_U32 i = 0; i < this->io_.nOutputSize; i++) {
|
| 438 |
+
const auto buffer = &(this->io_.pOutputs[i]);
|
| 439 |
+
if (const auto ret = ::AX_SYS_MinvalidateCache(buffer->phyAddr, buffer->pVirAddr, buffer->nSize); 0 != ret) {
|
| 440 |
+
utilities::glog.print(utilities::log::type::warn, "Invalidate output{index: %d} failed{0x%08X}.\n", i, ret);
|
| 441 |
+
return false;
|
| 442 |
+
}
|
| 443 |
+
}
|
| 444 |
+
return true;
|
| 445 |
+
}
|
| 446 |
+
|
| 447 |
+
[[nodiscard]] bool feed(const std::string& input_folder, const std::string& stimulus_name) {
|
| 448 |
+
for (uint32_t i = 0; i < this->get_input_count(); i++) {
|
| 449 |
+
const auto file_name = utilities::get_legal_name(this->get_input_name(i));
|
| 450 |
+
auto file_path = input_folder;
|
| 451 |
+
file_path.append("/").append(stimulus_name).append("/").append(file_name).append(".bin");
|
| 452 |
+
|
| 453 |
+
if (!read(file_path, this->inputs_[i].pVirAddr, this->inputs_[i].nSize)) {
|
| 454 |
+
utilities::glog.print(utilities::log::type::error, "Read tensor {idx: %d, name: %s} file {%s} failed.\n", i, file_name.c_str(), file_path.c_str());
|
| 455 |
+
return false;
|
| 456 |
+
}
|
| 457 |
+
|
| 458 |
+
if (this->input_cached_[i]) {
|
| 459 |
+
if (const auto ret = ::AX_SYS_MflushCache(this->inputs_[i].phyAddr, this->inputs_[i].pVirAddr, this->inputs_[i].nSize); 0 != ret) {
|
| 460 |
+
utilities::glog.print(utilities::log::type::error, "Flush tensor {idx: %d, name: %s} cache failed{%d}.\n", i, file_name.c_str(), ret);
|
| 461 |
+
return false;
|
| 462 |
+
}
|
| 463 |
+
utilities::glog.print(utilities::log::type::info, "Tensor {idx: %d, name: %s} cache flushed.\n", i, file_name.c_str());
|
| 464 |
+
}
|
| 465 |
+
}
|
| 466 |
+
return true;
|
| 467 |
+
}
|
| 468 |
+
|
| 469 |
+
[[nodiscard]] bool verify(const std::string& output_folder, const std::string& stimulus_name) {
|
| 470 |
+
for (uint32_t i = 0; i < this->get_output_count(); i++) {
|
| 471 |
+
const auto file_name = utilities::get_legal_name(this->get_output_name(i));
|
| 472 |
+
auto file_path = output_folder;
|
| 473 |
+
file_path.append("/").append(stimulus_name).append("/").append(file_name).append(".bin");
|
| 474 |
+
|
| 475 |
+
std::vector<uint8_t> file_buffer(this->get_output_size(i));
|
| 476 |
+
if (!read(file_path, file_buffer.data(), file_buffer.size())) {
|
| 477 |
+
utilities::glog.print(utilities::log::type::error, "Read tensor {idx: %d, name: %s} file {%s} failed.\n", i, file_name.c_str(), file_path.c_str());
|
| 478 |
+
return false;
|
| 479 |
+
}
|
| 480 |
+
|
| 481 |
+
if (this->output_cached_[i]) {
|
| 482 |
+
if (const auto ret = ::AX_SYS_MinvalidateCache(this->outputs_[i].phyAddr, this->outputs_[i].pVirAddr, this->outputs_[i].nSize); 0 != ret) {
|
| 483 |
+
utilities::glog.print(utilities::log::type::error, "Invalidate tensor {idx: %d, name: %s} cache failed{%d}.\n", i, file_name.c_str(), ret);
|
| 484 |
+
return false;
|
| 485 |
+
}
|
| 486 |
+
utilities::glog.print(utilities::log::type::info, "Tensor {idx: %d, name: %s} cache invalidated.\n", i, file_name.c_str());
|
| 487 |
+
}
|
| 488 |
+
|
| 489 |
+
if (!runner::verify(file_buffer.data(), this->outputs_[i].pVirAddr, this->outputs_[i].nSize)) {
|
| 490 |
+
utilities::glog.print(utilities::log::type::error, "Verify tensor {idx: %d, name: %s} failed.\n", i, file_name.c_str());
|
| 491 |
+
return false;
|
| 492 |
+
}
|
| 493 |
+
}
|
| 494 |
+
return true;
|
| 495 |
+
}
|
| 496 |
+
|
| 497 |
+
[[nodiscard]] bool save(const std::string& output_folder, const std::string& stimulus_name) {
|
| 498 |
+
auto file_folder = output_folder;
|
| 499 |
+
file_folder.append("/").append(stimulus_name);
|
| 500 |
+
|
| 501 |
+
if (!utilities::exists(file_folder) && !utilities::create_directory(file_folder)) {
|
| 502 |
+
utilities::glog.print(utilities::log::type::error, "Create folder {%s} failed.\n", file_folder.c_str());
|
| 503 |
+
return false;
|
| 504 |
+
}
|
| 505 |
+
|
| 506 |
+
for (uint32_t i = 0; i < this->get_output_count(); i++) {
|
| 507 |
+
const auto file_name = utilities::get_legal_name(this->get_output_name(i));
|
| 508 |
+
auto file_path = file_folder;
|
| 509 |
+
file_path.append("/").append(file_name).append(".bin");
|
| 510 |
+
|
| 511 |
+
if (this->output_cached_[i]) {
|
| 512 |
+
if (const auto ret = ::AX_SYS_MinvalidateCache(this->outputs_[i].phyAddr, this->outputs_[i].pVirAddr, this->outputs_[i].nSize); 0 != ret) {
|
| 513 |
+
utilities::glog.print(utilities::log::type::error, "Invalidate tensor {idx: %d, name: %s} cache failed{%d}.\n", i, file_name.c_str(), ret);
|
| 514 |
+
return false;
|
| 515 |
+
}
|
| 516 |
+
utilities::glog.print(utilities::log::type::info, "Tensor {idx: %d, name: %s} cache invalidated.\n", i, file_name.c_str());
|
| 517 |
+
}
|
| 518 |
+
|
| 519 |
+
if (!write(file_path, this->outputs_[i].pVirAddr, this->outputs_[i].nSize)) {
|
| 520 |
+
utilities::glog.print(utilities::log::type::error, "Write tensor {idx: %d, name: %s} file {%s} failed.\n", i, file_name.c_str(), file_path.c_str());
|
| 521 |
+
return false;
|
| 522 |
+
}
|
| 523 |
+
}
|
| 524 |
+
return true;
|
| 525 |
+
}
|
| 526 |
+
|
| 527 |
+
[[nodiscard]] std::string get_library_version() {
|
| 528 |
+
std::lock_guard guard(this->mutex_);
|
| 529 |
+
|
| 530 |
+
const auto ver = ::AX_ENGINE_GetVersion();
|
| 531 |
+
if (nullptr == ver) {
|
| 532 |
+
utilities::glog.print(utilities::log::type::error, "Get library version failed.\n");
|
| 533 |
+
return "";
|
| 534 |
+
}
|
| 535 |
+
return {ver};
|
| 536 |
+
}
|
| 537 |
+
|
| 538 |
+
[[nodiscard]] std::string get_model_version() {
|
| 539 |
+
std::lock_guard guard(this->mutex_);
|
| 540 |
+
|
| 541 |
+
if (nullptr == this->handle_) {
|
| 542 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 543 |
+
return "";
|
| 544 |
+
}
|
| 545 |
+
return {::AX_ENGINE_GetModelToolsVersion(this->handle_)};
|
| 546 |
+
}
|
| 547 |
+
|
| 548 |
+
[[nodiscard]] int32_t get_model_type() {
|
| 549 |
+
std::lock_guard guard(this->mutex_);
|
| 550 |
+
|
| 551 |
+
if (nullptr == this->handle_) {
|
| 552 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 553 |
+
return -1;
|
| 554 |
+
}
|
| 555 |
+
|
| 556 |
+
::AX_ENGINE_MODEL_TYPE_T type;
|
| 557 |
+
if (const auto ret = ::AX_ENGINE_GetHandleModelType(this->handle_, &type); 0 != ret) {
|
| 558 |
+
utilities::glog.print(utilities::log::type::error, "Get model type failed{0x%08X}.\n", ret);
|
| 559 |
+
return -1;
|
| 560 |
+
}
|
| 561 |
+
return type;
|
| 562 |
+
}
|
| 563 |
+
|
| 564 |
+
[[nodiscard]] int32_t get_npu_type() {
|
| 565 |
+
std::lock_guard guard(this->mutex_);
|
| 566 |
+
|
| 567 |
+
if (!this->is_initialized_) {
|
| 568 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 569 |
+
return false;
|
| 570 |
+
}
|
| 571 |
+
|
| 572 |
+
::AX_ENGINE_NPU_ATTR_T attr{};
|
| 573 |
+
if (const auto ret = ::AX_ENGINE_GetVNPUAttr(&attr); 0 != ret) {
|
| 574 |
+
utilities::glog.print(utilities::log::type::error, "Get visual NPU type failed{0x%08X}.\n", ret);
|
| 575 |
+
return -1;
|
| 576 |
+
}
|
| 577 |
+
return attr.eHardMode;
|
| 578 |
+
}
|
| 579 |
+
|
| 580 |
+
[[nodiscard]] int32_t get_batch_size() {
|
| 581 |
+
std::lock_guard guard(this->mutex_);
|
| 582 |
+
|
| 583 |
+
if (nullptr == this->handle_) {
|
| 584 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 585 |
+
return -1;
|
| 586 |
+
}
|
| 587 |
+
|
| 588 |
+
return static_cast<int32_t>(this->batch);
|
| 589 |
+
}
|
| 590 |
+
|
| 591 |
+
[[nodiscard]] intmax_t get_sys_usage() {
|
| 592 |
+
std::lock_guard guard(this->mutex_);
|
| 593 |
+
|
| 594 |
+
if (nullptr == this->handle_) {
|
| 595 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 596 |
+
return -1;
|
| 597 |
+
}
|
| 598 |
+
|
| 599 |
+
return 0;
|
| 600 |
+
}
|
| 601 |
+
|
| 602 |
+
[[nodiscard]] intmax_t get_cmm_usage() {
|
| 603 |
+
std::lock_guard guard(this->mutex_);
|
| 604 |
+
|
| 605 |
+
if (nullptr == this->handle_) {
|
| 606 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 607 |
+
return -1;
|
| 608 |
+
}
|
| 609 |
+
|
| 610 |
+
::AX_ENGINE_CMM_INFO usage{};
|
| 611 |
+
if (const auto ret = ::AX_ENGINE_GetCMMUsage(this->handle_, &usage); 0 != ret) {
|
| 612 |
+
utilities::glog.print(utilities::log::type::error, "Get model usage failed{0x%08X}.\n", ret);
|
| 613 |
+
return -1;
|
| 614 |
+
}
|
| 615 |
+
|
| 616 |
+
return usage.nCMMSize;
|
| 617 |
+
}
|
| 618 |
+
|
| 619 |
+
static void sleep_for(const uint32_t sleep_duration) {
|
| 620 |
+
::AX_SYS_Msleep(sleep_duration);
|
| 621 |
+
}
|
| 622 |
+
|
| 623 |
+
private:
|
| 624 |
+
uint32_t group_ = 0;
|
| 625 |
+
uint32_t batch = 0;
|
| 626 |
+
|
| 627 |
+
::AX_ENGINE_HANDLE handle_{};
|
| 628 |
+
::AX_ENGINE_CONTEXT_T context_{};
|
| 629 |
+
::AX_ENGINE_IO_INFO_T* info_{};
|
| 630 |
+
::AX_ENGINE_IO_T io_{};
|
| 631 |
+
std::vector<::AX_ENGINE_IO_BUFFER_T> inputs_;
|
| 632 |
+
std::vector<::AX_ENGINE_IO_BUFFER_T> outputs_;
|
| 633 |
+
|
| 634 |
+
std::vector<bool> input_cached_;
|
| 635 |
+
std::vector<bool> output_cached_;
|
| 636 |
+
|
| 637 |
+
bool is_initialized_ = false;
|
| 638 |
+
std::mutex mutex_;
|
| 639 |
+
};
|
| 640 |
+
#endif
|
cpp/src/middleware/axcl_base.cc
ADDED
|
@@ -0,0 +1,241 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#include "middleware/axcl_base.hpp"
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
#if defined(ENV_AXCL_RUNTIME_API_ENABLE) || defined(ENV_AXCL_NATIVE_API_ENABLE)
|
| 15 |
+
|
| 16 |
+
#include "utilities/scalar_guard.hpp"
|
| 17 |
+
#include "utilities/file.hpp"
|
| 18 |
+
#include "utilities/log.hpp"
|
| 19 |
+
|
| 20 |
+
#include <axcl.h>
|
| 21 |
+
|
| 22 |
+
#include <mutex>
|
| 23 |
+
|
| 24 |
+
static int static_module_count{0};
|
| 25 |
+
static std::mutex static_module_mutex;
|
| 26 |
+
|
| 27 |
+
static bool module_init(const std::string& config, const uint32_t& index, const middleware::axcl_base::npu_func& func) {
|
| 28 |
+
// 1. init axcl, using scalar_guard to ensure the finalization
|
| 29 |
+
const auto cfg_flag = utilities::exists(config) && utilities::is_regular_file(config);
|
| 30 |
+
utilities::glog.print(utilities::log::type::info, "axcl initializing...\n");
|
| 31 |
+
auto env_guard = utilities::scalar_guard<int32_t>(
|
| 32 |
+
axclInit(cfg_flag ? config.c_str() : nullptr),
|
| 33 |
+
[](const int32_t& code) {
|
| 34 |
+
if (0 == code) {
|
| 35 |
+
std::ignore = axclFinalize();
|
| 36 |
+
}
|
| 37 |
+
}
|
| 38 |
+
);
|
| 39 |
+
|
| 40 |
+
// 2. check the initialization result
|
| 41 |
+
if (const int ret = env_guard.get(); 0 != ret) {
|
| 42 |
+
utilities::glog.print(utilities::log::type::error, "Init axcl failed{0x%08X}.\n", ret);
|
| 43 |
+
return false;
|
| 44 |
+
}
|
| 45 |
+
utilities::glog.print(utilities::log::type::info, "axcl inited.\n");
|
| 46 |
+
|
| 47 |
+
// 3. get device list
|
| 48 |
+
axclrtDeviceList lst;
|
| 49 |
+
if (const auto ret = axclrtGetDeviceList(&lst); 0 != ret || 0 == lst.num) {
|
| 50 |
+
utilities::glog.print(utilities::log::type::error,
|
| 51 |
+
"Get axcl device failed{0x%08X}, find total %d device.\n", ret, lst.num);
|
| 52 |
+
return false;
|
| 53 |
+
}
|
| 54 |
+
|
| 55 |
+
// 4. check the device index
|
| 56 |
+
if (index >= lst.num) {
|
| 57 |
+
utilities::glog.print(utilities::log::type::error,
|
| 58 |
+
"Specified device index{%d} is out of range{total %d}.\n", index, lst.num);
|
| 59 |
+
return false;
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
// 5. set device
|
| 63 |
+
if (const auto ret = axclrtSetDevice(lst.devices[index]); 0 != ret) {
|
| 64 |
+
utilities::glog.print(utilities::log::type::error,
|
| 65 |
+
"Set axcl device as index{%d} failed{0x%08X}.\n", index, ret);
|
| 66 |
+
return false;
|
| 67 |
+
}
|
| 68 |
+
utilities::glog.print(utilities::log::type::info,
|
| 69 |
+
"Select axcl device{index: %d} as {%d}.\n", index, lst.devices[index]);
|
| 70 |
+
|
| 71 |
+
// 6. init NPU
|
| 72 |
+
if (!func()) {
|
| 73 |
+
utilities::glog.print(utilities::log::type::error, "Init NPU failed.\n");
|
| 74 |
+
return false;
|
| 75 |
+
}
|
| 76 |
+
|
| 77 |
+
// 7. disengage guard of env
|
| 78 |
+
env_guard.get() = -1;
|
| 79 |
+
return true;
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
static bool module_final(const middleware::axcl_base::npu_func& func) {
|
| 83 |
+
const auto flag = func();
|
| 84 |
+
const auto ret = axclFinalize();
|
| 85 |
+
return flag && (0 == ret);
|
| 86 |
+
}
|
| 87 |
+
|
| 88 |
+
bool middleware::axcl_base::init(const std::string &config, const uint32_t &index, const npu_func &func) {
|
| 89 |
+
bool flag = false;
|
| 90 |
+
{
|
| 91 |
+
std::lock_guard lock(static_module_mutex);
|
| 92 |
+
if (static_module_count == 0) {
|
| 93 |
+
flag = module_init(config, index, func);
|
| 94 |
+
if (flag) {
|
| 95 |
+
static_module_count++;
|
| 96 |
+
}
|
| 97 |
+
} else {
|
| 98 |
+
static_module_count++;
|
| 99 |
+
flag = true;
|
| 100 |
+
}
|
| 101 |
+
}
|
| 102 |
+
|
| 103 |
+
return flag;
|
| 104 |
+
}
|
| 105 |
+
|
| 106 |
+
bool middleware::axcl_base::final(const npu_func& func) {
|
| 107 |
+
bool flag = false;
|
| 108 |
+
{
|
| 109 |
+
std::lock_guard lock(static_module_mutex);
|
| 110 |
+
if (static_module_count == 1) {
|
| 111 |
+
flag = module_final(func);
|
| 112 |
+
if (flag) {
|
| 113 |
+
static_module_count--;
|
| 114 |
+
}
|
| 115 |
+
} else if (static_module_count > 1) {
|
| 116 |
+
static_module_count--;
|
| 117 |
+
flag = true;
|
| 118 |
+
}
|
| 119 |
+
}
|
| 120 |
+
|
| 121 |
+
return flag;
|
| 122 |
+
}
|
| 123 |
+
|
| 124 |
+
bool middleware::axcl_base::flush_input() const {
|
| 125 |
+
return true;
|
| 126 |
+
}
|
| 127 |
+
|
| 128 |
+
bool middleware::axcl_base::invalidate_output() const {
|
| 129 |
+
return true;
|
| 130 |
+
}
|
| 131 |
+
|
| 132 |
+
[[nodiscard]] bool middleware::axcl_base::feed(const std::string& input_folder, const std::string& stimulus_name) const {
|
| 133 |
+
for (uint32_t i = 0; i < this->get_input_count(); i++) {
|
| 134 |
+
if (const auto ret = feed(input_folder, stimulus_name, this->get_input_name(i), this->get_input_pointer(i), this->get_input_size(i) * this->get_batch_size()); !ret) {
|
| 135 |
+
utilities::glog.print(utilities::log::type::error, "Feed tensor {idx: %d, name: %s} failed.\n", i, this->get_input_name(i).c_str());
|
| 136 |
+
return false;
|
| 137 |
+
}
|
| 138 |
+
}
|
| 139 |
+
return true;
|
| 140 |
+
}
|
| 141 |
+
|
| 142 |
+
[[nodiscard]] bool middleware::axcl_base::verify(const std::string& output_folder, const std::string& stimulus_name) const {
|
| 143 |
+
for (uint32_t i = 0; i < this->get_output_count(); i++) {
|
| 144 |
+
if (const auto ret = verify(output_folder, stimulus_name, this->get_output_name(i),this->get_output_pointer(i), this->get_output_size(i) * this->get_batch_size()); !ret) {
|
| 145 |
+
utilities::glog.print(utilities::log::type::error, "Verify tensor {idx: %d, name: %s} failed.\n", i, this->get_output_name(i).c_str());
|
| 146 |
+
return false;
|
| 147 |
+
}
|
| 148 |
+
}
|
| 149 |
+
return true;
|
| 150 |
+
}
|
| 151 |
+
|
| 152 |
+
[[nodiscard]] bool middleware::axcl_base::save(const std::string& output_folder, const std::string& stimulus_name) const {
|
| 153 |
+
auto file_folder = output_folder;
|
| 154 |
+
file_folder.append("/").append(stimulus_name);
|
| 155 |
+
|
| 156 |
+
if (!utilities::exists(file_folder) && !utilities::create_directory(file_folder)) {
|
| 157 |
+
utilities::glog.print(utilities::log::type::error, "Create folder {%s} failed.\n", file_folder.c_str());
|
| 158 |
+
return false;
|
| 159 |
+
}
|
| 160 |
+
|
| 161 |
+
for (uint32_t i = 0; i < this->get_output_count(); i++) {
|
| 162 |
+
if (const auto ret = save(output_folder, stimulus_name, this->get_output_name(i), this->get_output_pointer(i), this->get_output_size(i) * this->get_batch_size()); !ret) {
|
| 163 |
+
utilities::glog.print(utilities::log::type::error, "Save tensor {idx: %d, name: %s} failed.\n", i, this->get_output_name(i).c_str());
|
| 164 |
+
return false;
|
| 165 |
+
}
|
| 166 |
+
}
|
| 167 |
+
return true;
|
| 168 |
+
}
|
| 169 |
+
|
| 170 |
+
bool middleware::axcl_base::feed(const std::string& folder, const std::string& stimulus_name, const std::string& tensor_name, void* address, const uintmax_t size) {
|
| 171 |
+
const auto file_name = utilities::get_legal_name(tensor_name);
|
| 172 |
+
auto file_path(folder);
|
| 173 |
+
file_path.append("/").append(stimulus_name).append("/").append(file_name).append(".bin");
|
| 174 |
+
|
| 175 |
+
std::vector<uint8_t> tensor_buffer(size);
|
| 176 |
+
if (!read(file_path, tensor_buffer.data(), tensor_buffer.size())) {
|
| 177 |
+
utilities::glog.print(utilities::log::type::error,
|
| 178 |
+
"Read tensor {name: %s} stimulus file {%s} failed.\n", tensor_name.c_str(), file_path.c_str());
|
| 179 |
+
return false;
|
| 180 |
+
}
|
| 181 |
+
|
| 182 |
+
if (const auto ret = axclrtMemcpy(address, tensor_buffer.data(), size, AXCL_MEMCPY_HOST_TO_DEVICE); 0 != ret) {
|
| 183 |
+
utilities::glog.print(utilities::log::type::error,
|
| 184 |
+
"Receive tensor {name: %s, addr: 0x%08X, size: %ld} from host {addr: 0x%08X} failed{0x%08X}.\n",
|
| 185 |
+
tensor_name.c_str(), address, size, tensor_buffer.data(), ret);
|
| 186 |
+
return false;
|
| 187 |
+
}
|
| 188 |
+
|
| 189 |
+
return true;
|
| 190 |
+
}
|
| 191 |
+
|
| 192 |
+
bool middleware::axcl_base::verify(const std::string& folder, const std::string& stimulus_name, const std::string& tensor_name, void* address, const uintmax_t size) {
|
| 193 |
+
const auto file_name = utilities::get_legal_name(tensor_name);
|
| 194 |
+
auto file_path(folder);
|
| 195 |
+
file_path.append("/").append(stimulus_name).append("/").append(file_name).append(".bin");
|
| 196 |
+
|
| 197 |
+
std::vector<uint8_t> file_buffer(size);
|
| 198 |
+
if (!read(file_path, file_buffer.data(), file_buffer.size())) {
|
| 199 |
+
utilities::glog.print(utilities::log::type::error,
|
| 200 |
+
"Read tensor {name: %s} file {%s} failed.\n", file_name.c_str(), file_path.c_str());
|
| 201 |
+
return false;
|
| 202 |
+
}
|
| 203 |
+
|
| 204 |
+
std::vector<uint8_t> tensor_buffer(size);
|
| 205 |
+
|
| 206 |
+
if (const auto ret = axclrtMemcpy(tensor_buffer.data(), address, size, AXCL_MEMCPY_DEVICE_TO_HOST); 0 != ret) {
|
| 207 |
+
utilities::glog.print(utilities::log::type::error,
|
| 208 |
+
"Send tensor {name: %s, addr: 0x%08X, size: %ld} to host {addr: 0x%08X} failed{0x%08X}.\n",
|
| 209 |
+
file_name.c_str(), address, size, tensor_buffer.data(), ret);
|
| 210 |
+
return false;
|
| 211 |
+
}
|
| 212 |
+
|
| 213 |
+
if (!runner::verify(file_buffer.data(), tensor_buffer.data(), file_buffer.size())) {
|
| 214 |
+
utilities::glog.print(utilities::log::type::error, "Verify tensor {name: %s} failed.\n", file_name.c_str());
|
| 215 |
+
return false;
|
| 216 |
+
}
|
| 217 |
+
|
| 218 |
+
return true;
|
| 219 |
+
}
|
| 220 |
+
|
| 221 |
+
bool middleware::axcl_base::save(const std::string& output_folder, const std::string& stimulus_name, const std::string& tensor_name, void* address, const uintmax_t size) {
|
| 222 |
+
const auto file_name = utilities::get_legal_name(tensor_name);
|
| 223 |
+
auto file_path(output_folder);
|
| 224 |
+
file_path.append("/").append(stimulus_name).append("/").append(file_name).append(".bin");
|
| 225 |
+
|
| 226 |
+
std::vector<uint8_t> tensor_buffer(size);
|
| 227 |
+
if (const auto ret = axclrtMemcpy(tensor_buffer.data(), address, size, AXCL_MEMCPY_DEVICE_TO_HOST); 0 != ret) {
|
| 228 |
+
utilities::glog.print(utilities::log::type::error,
|
| 229 |
+
"Send tensor {name: %s, addr: 0x%08X, size: %ld} to host {addr: 0x%08X} failed{0x%08X}.\n",
|
| 230 |
+
file_name.c_str(), address, size, tensor_buffer.data(), ret);
|
| 231 |
+
return false;
|
| 232 |
+
}
|
| 233 |
+
|
| 234 |
+
if (!write(file_path, tensor_buffer.data(), tensor_buffer.size())) {
|
| 235 |
+
utilities::glog.print(utilities::log::type::error, "Write tensor {name: %s} file {%s} failed.\n", file_name.c_str(), file_path.c_str());
|
| 236 |
+
return false;
|
| 237 |
+
}
|
| 238 |
+
|
| 239 |
+
return true;
|
| 240 |
+
}
|
| 241 |
+
#endif
|
cpp/src/middleware/axcl_base.hpp
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include "middleware/runner.hpp"
|
| 14 |
+
|
| 15 |
+
#if defined(ENV_AXCL_RUNTIME_API_ENABLE) || defined(ENV_AXCL_NATIVE_API_ENABLE)
|
| 16 |
+
|
| 17 |
+
#include <functional>
|
| 18 |
+
|
| 19 |
+
namespace middleware {
|
| 20 |
+
|
| 21 |
+
class axcl_base : public runner {
|
| 22 |
+
public:
|
| 23 |
+
using npu_func = std::function<bool()>;
|
| 24 |
+
|
| 25 |
+
[[nodiscard]] static bool init(const std::string& config, const uint32_t& index, const npu_func& func);
|
| 26 |
+
[[nodiscard]] static bool final(const npu_func& func);
|
| 27 |
+
|
| 28 |
+
[[nodiscard]] bool flush_input() const override;
|
| 29 |
+
[[nodiscard]] bool invalidate_output() const override;
|
| 30 |
+
|
| 31 |
+
[[nodiscard]] bool feed(const std::string& input_folder, const std::string& stimulus_name) const override;
|
| 32 |
+
[[nodiscard]] bool verify(const std::string& output_folder, const std::string& stimulus_name) const override;
|
| 33 |
+
[[nodiscard]] bool save(const std::string& output_folder, const std::string& stimulus_name) const override;
|
| 34 |
+
|
| 35 |
+
private:
|
| 36 |
+
[[nodiscard]] static bool feed(const std::string& folder, const std::string& stimulus_name, const std::string& tensor_name, void* address, uintmax_t size) ;
|
| 37 |
+
[[nodiscard]] static bool verify(const std::string &folder, const std::string &stimulus_name, const std::string &tensor_name, void *address, uintmax_t size) ;
|
| 38 |
+
[[nodiscard]] static bool save(const std::string& output_folder, const std::string& stimulus_name, const std::string& tensor_name, void* address, uintmax_t size);
|
| 39 |
+
};
|
| 40 |
+
|
| 41 |
+
}
|
| 42 |
+
#endif
|
cpp/src/middleware/axcl_native_runner.cc
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#include "middleware/axcl_native_runner_impl.hpp"
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AXCL_NATIVE_API_ENABLE)
|
| 14 |
+
middleware::native_runner::native_runner() {
|
| 15 |
+
this->impl_ = new impl();
|
| 16 |
+
}
|
| 17 |
+
|
| 18 |
+
middleware::native_runner::~native_runner() {
|
| 19 |
+
delete this->impl_;
|
| 20 |
+
}
|
| 21 |
+
|
| 22 |
+
bool middleware::native_runner::init(const std::string& config_file, const uint32_t& device_index, const uint32_t& kind) {
|
| 23 |
+
return this->impl_->init(config_file, device_index, kind);
|
| 24 |
+
}
|
| 25 |
+
|
| 26 |
+
bool middleware::native_runner::final() {
|
| 27 |
+
return this->impl_->final();
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
bool middleware::native_runner::load(const std::string& model_path) {
|
| 31 |
+
return this->impl_->load(model_path);
|
| 32 |
+
}
|
| 33 |
+
|
| 34 |
+
bool middleware::native_runner::prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 35 |
+
return this->impl_->prepare(input_cached, output_cached, group, batch);
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
bool middleware::native_runner::run(const bool& parallel) {
|
| 39 |
+
return this->impl_->run(parallel);
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
uint32_t middleware::native_runner::get_input_count() const {
|
| 43 |
+
return this->impl_->get_input_count();
|
| 44 |
+
}
|
| 45 |
+
|
| 46 |
+
uint32_t middleware::native_runner::get_output_count() const {
|
| 47 |
+
return this->impl_->get_output_count();
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
std::string middleware::native_runner::get_input_name(const uint32_t& index) const {
|
| 51 |
+
return this->impl_->get_input_name(index);
|
| 52 |
+
}
|
| 53 |
+
|
| 54 |
+
std::string middleware::native_runner::get_output_name(const uint32_t& index) const {
|
| 55 |
+
return this->impl_->get_output_name(index);
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
void *middleware::native_runner::get_input_pointer(const uint32_t& index) const {
|
| 59 |
+
return this->impl_->get_input_pointer(index);
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
void *middleware::native_runner::get_output_pointer(const uint32_t& index) const {
|
| 63 |
+
return this->impl_->get_output_pointer(index);
|
| 64 |
+
}
|
| 65 |
+
|
| 66 |
+
uintmax_t middleware::native_runner::get_input_size(const uint32_t& index) const {
|
| 67 |
+
return this->impl_->get_input_size(index);
|
| 68 |
+
}
|
| 69 |
+
|
| 70 |
+
uintmax_t middleware::native_runner::get_output_size(const uint32_t& index) const {
|
| 71 |
+
return this->impl_->get_output_size(index);
|
| 72 |
+
}
|
| 73 |
+
|
| 74 |
+
uintmax_t middleware::native_runner::get_shape_group_count() const {
|
| 75 |
+
return this->impl_->get_shape_group_count();
|
| 76 |
+
}
|
| 77 |
+
|
| 78 |
+
std::string middleware::native_runner::get_library_version() const {
|
| 79 |
+
return impl::get_library_version();
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
std::string middleware::native_runner::get_model_version() const {
|
| 83 |
+
return this->impl_->get_model_version();
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
int32_t middleware::native_runner::get_model_type() const {
|
| 87 |
+
return this->impl_->get_model_type();
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
int32_t middleware::native_runner::get_npu_type() const {
|
| 91 |
+
return this->impl_->get_npu_type();
|
| 92 |
+
}
|
| 93 |
+
|
| 94 |
+
int32_t middleware::native_runner::get_batch_size() const {
|
| 95 |
+
return this->impl_->get_batch_size();
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
intmax_t middleware::native_runner::get_sys_usage() const {
|
| 99 |
+
return this->impl_->get_sys_usage();
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
intmax_t middleware::native_runner::get_cmm_usage() const {
|
| 103 |
+
return this->impl_->get_cmm_usage();
|
| 104 |
+
}
|
| 105 |
+
#endif
|
cpp/src/middleware/axcl_native_runner.hpp
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AXCL_NATIVE_API_ENABLE)
|
| 14 |
+
|
| 15 |
+
#include "middleware/axcl_base.hpp"
|
| 16 |
+
|
| 17 |
+
namespace middleware {
|
| 18 |
+
|
| 19 |
+
class native_runner final : public axcl_base {
|
| 20 |
+
public:
|
| 21 |
+
native_runner();
|
| 22 |
+
~native_runner() override;
|
| 23 |
+
|
| 24 |
+
[[nodiscard]] bool init(const std::string& config_file, const uint32_t& device_index, const uint32_t& kind) override;
|
| 25 |
+
[[nodiscard]] bool final() override;
|
| 26 |
+
|
| 27 |
+
[[nodiscard]] bool load(const std::string& model_path) override;
|
| 28 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) override;
|
| 29 |
+
|
| 30 |
+
[[nodiscard]] bool run(const bool& parallel) override;
|
| 31 |
+
|
| 32 |
+
[[nodiscard]] uint32_t get_input_count() const override;
|
| 33 |
+
[[nodiscard]] uint32_t get_output_count() const override;
|
| 34 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) const override;
|
| 35 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) const override;
|
| 36 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) const override;
|
| 37 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) const override;
|
| 38 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) const override;
|
| 39 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) const override;
|
| 40 |
+
[[nodiscard]] uintmax_t get_shape_group_count() const override;
|
| 41 |
+
|
| 42 |
+
[[nodiscard]] std::string get_library_version() const override;
|
| 43 |
+
[[nodiscard]] std::string get_model_version() const override;
|
| 44 |
+
|
| 45 |
+
[[nodiscard]] int32_t get_model_type() const override;
|
| 46 |
+
[[nodiscard]] int32_t get_npu_type() const override;
|
| 47 |
+
[[nodiscard]] int32_t get_batch_size() const override;
|
| 48 |
+
[[nodiscard]] intmax_t get_sys_usage() const override;
|
| 49 |
+
[[nodiscard]] intmax_t get_cmm_usage() const override;
|
| 50 |
+
|
| 51 |
+
private:
|
| 52 |
+
struct impl;
|
| 53 |
+
impl *impl_;
|
| 54 |
+
};
|
| 55 |
+
|
| 56 |
+
}
|
| 57 |
+
#endif
|
cpp/src/middleware/axcl_native_runner_impl.hpp
ADDED
|
@@ -0,0 +1,504 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include "middleware/axcl_native_runner.hpp"
|
| 14 |
+
|
| 15 |
+
#include "utilities/scalar_guard.hpp"
|
| 16 |
+
#include "utilities/file.hpp"
|
| 17 |
+
#include "utilities/file_mapper.hpp"
|
| 18 |
+
#include "utilities/log.hpp"
|
| 19 |
+
|
| 20 |
+
#include <axcl.h>
|
| 21 |
+
|
| 22 |
+
#if defined(ENV_AXCL_NATIVE_API_ENABLE)
|
| 23 |
+
#define MODEL_LEADING_NAME "axcl npu "
|
| 24 |
+
|
| 25 |
+
struct middleware::native_runner::impl {
|
| 26 |
+
impl() = default;
|
| 27 |
+
|
| 28 |
+
~impl() {
|
| 29 |
+
std::ignore = this->final();
|
| 30 |
+
}
|
| 31 |
+
|
| 32 |
+
[[nodiscard]] bool init(const std::string& config_file, const uint32_t& index, const uint32_t& kind) {
|
| 33 |
+
// 0. check the NPU kind
|
| 34 |
+
if (kind >= ::AX_ENGINE_VIRTUAL_NPU_BUTT) {
|
| 35 |
+
utilities::glog.print(utilities::log::type::error,
|
| 36 |
+
"Specified NPU kind{%d} is out of range{total %d}.\n", kind, ::AX_ENGINE_VIRTUAL_NPU_BUTT);
|
| 37 |
+
return false;
|
| 38 |
+
}
|
| 39 |
+
|
| 40 |
+
// 1. build the npu init function
|
| 41 |
+
auto npu_init_func = [&kind]() {
|
| 42 |
+
::AX_ENGINE_NPU_ATTR_T attr{};
|
| 43 |
+
attr.eHardMode = static_cast<::AX_ENGINE_NPU_MODE_T>(kind);
|
| 44 |
+
if (const int ret = ::AXCL_ENGINE_Init(&attr); 0 != ret) {
|
| 45 |
+
utilities::glog.print(utilities::log::type::error,
|
| 46 |
+
"Init axcl NPU as kind{%d} failed{0x%08X}.\n", kind, ret);
|
| 47 |
+
return false;
|
| 48 |
+
}
|
| 49 |
+
utilities::glog.print(utilities::log::type::info, "axcl NPU inited.\n");
|
| 50 |
+
return true;
|
| 51 |
+
};
|
| 52 |
+
|
| 53 |
+
// 2. execute initialization
|
| 54 |
+
if (const auto ret = axcl_base::init(config_file, index, npu_init_func); !ret) {
|
| 55 |
+
return false;
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
// 3. set the flag
|
| 59 |
+
this->is_initialized_ = true;
|
| 60 |
+
return true;
|
| 61 |
+
}
|
| 62 |
+
|
| 63 |
+
[[nodiscard]] bool final() {
|
| 64 |
+
auto flag = true;
|
| 65 |
+
|
| 66 |
+
if (this->is_initialized_) {
|
| 67 |
+
if (nullptr != this->handle_) {
|
| 68 |
+
for (auto& input : inputs_) {
|
| 69 |
+
if (0 != input.phyAddr) {
|
| 70 |
+
flag &= 0 == ::axclrtFree(reinterpret_cast<void *>(input.phyAddr));
|
| 71 |
+
input.phyAddr = 0;
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
for (auto& output : outputs_) {
|
| 75 |
+
if (0 != output.phyAddr) {
|
| 76 |
+
flag &= 0 == ::axclrtFree(reinterpret_cast<void *>(output.phyAddr));
|
| 77 |
+
output.phyAddr = 0;
|
| 78 |
+
}
|
| 79 |
+
}
|
| 80 |
+
|
| 81 |
+
flag &= 0 == ::AXCL_ENGINE_DestroyHandle(this->handle_);
|
| 82 |
+
}
|
| 83 |
+
|
| 84 |
+
flag &= 0 == axcl_base::final([&]() {
|
| 85 |
+
if (const auto ret = ::AXCL_ENGINE_Deinit(); 0 != ret) {
|
| 86 |
+
utilities::glog.print(utilities::log::type::error, "Final axcl NPU failed{0x%08X}.\n", ret);
|
| 87 |
+
return false;
|
| 88 |
+
}
|
| 89 |
+
utilities::glog.print(utilities::log::type::info, "axcl NPU finalized.\n");
|
| 90 |
+
return true;
|
| 91 |
+
});
|
| 92 |
+
|
| 93 |
+
this->is_initialized_ = false;
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
return flag;
|
| 97 |
+
}
|
| 98 |
+
|
| 99 |
+
[[nodiscard]] bool load(const std::string& model_path) {
|
| 100 |
+
if (!this->is_initialized_) {
|
| 101 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 102 |
+
return false;
|
| 103 |
+
}
|
| 104 |
+
|
| 105 |
+
if (nullptr != this->handle_) {
|
| 106 |
+
utilities::glog.print(utilities::log::type::error, "Model is already loaded.\n");
|
| 107 |
+
return false;
|
| 108 |
+
}
|
| 109 |
+
|
| 110 |
+
if (!utilities::exists(model_path)
|
| 111 |
+
|| !utilities::is_regular_file(model_path)
|
| 112 |
+
|| 0 == utilities::file_size(model_path)
|
| 113 |
+
|| utilities::error_size == utilities::file_size(model_path)) {
|
| 114 |
+
utilities::glog.print(utilities::log::type::error,
|
| 115 |
+
"Model file{%s} error, please check it.\n", model_path.c_str());
|
| 116 |
+
return false;
|
| 117 |
+
}
|
| 118 |
+
|
| 119 |
+
const auto file = utilities::file_mapper(model_path);
|
| 120 |
+
if (nullptr == file.get()) {
|
| 121 |
+
utilities::glog.print(utilities::log::type::error,
|
| 122 |
+
"Model file{%s} mapping failed.\n", model_path.c_str());
|
| 123 |
+
return false;
|
| 124 |
+
}
|
| 125 |
+
|
| 126 |
+
const auto file_size = utilities::file_size(model_path);
|
| 127 |
+
auto device = utilities::scalar_guard<void*>(
|
| 128 |
+
[&file_size]() -> void* {
|
| 129 |
+
void *mem = nullptr;
|
| 130 |
+
if (const auto ret = ::axclrtMalloc(&mem, file_size, ::axclrtMemMallocPolicy{}); 0 != ret) {
|
| 131 |
+
utilities::glog.print(utilities::log::type::error,
|
| 132 |
+
"Malloc on device for model file failed{0x%08X}.\n", ret);
|
| 133 |
+
}
|
| 134 |
+
return mem;
|
| 135 |
+
},
|
| 136 |
+
[](void*& dev) { if (dev != nullptr) std::ignore = ::axclrtFree(dev); }
|
| 137 |
+
);
|
| 138 |
+
|
| 139 |
+
if (nullptr == device.get()) {
|
| 140 |
+
utilities::glog.print(utilities::log::type::error,
|
| 141 |
+
"Malloc on device for file{%s} failed.\n", model_path.c_str());
|
| 142 |
+
return false;
|
| 143 |
+
}
|
| 144 |
+
|
| 145 |
+
if (const auto ret = ::axclrtMemcpy(device.get(), file.get(), file_size, ::AXCL_MEMCPY_HOST_TO_DEVICE); 0 != ret) {
|
| 146 |
+
utilities::glog.print(utilities::log::type::error,
|
| 147 |
+
"Send model file{%s} to device failed{0x%08X}.\n", model_path.c_str(), ret);
|
| 148 |
+
return false;
|
| 149 |
+
}
|
| 150 |
+
|
| 151 |
+
const auto model_name = std::string(MODEL_LEADING_NAME) + utilities::get_file_name(model_path);
|
| 152 |
+
::AX_ENGINE_HANDLE_EXTRA_T extra_param{};
|
| 153 |
+
extra_param.pName = reinterpret_cast<::AX_S8 *>(const_cast<char *>(model_name.c_str()));
|
| 154 |
+
if (const auto ret = ::AXCL_ENGINE_CreateHandleV2(&this->handle_, device.get(), file_size, &extra_param); 0 != ret) {
|
| 155 |
+
utilities::glog.print(utilities::log::type::error,
|
| 156 |
+
"Create model{%s} handle failed.\n", model_path.c_str());
|
| 157 |
+
return false;
|
| 158 |
+
}
|
| 159 |
+
|
| 160 |
+
if (const auto ret = ::AXCL_ENGINE_CreateContextV2(this->handle_, &this->context_); 0 != ret) {
|
| 161 |
+
utilities::glog.print(utilities::log::type::error,
|
| 162 |
+
"Create model{%s} context failed.\n", model_path.c_str());
|
| 163 |
+
return false;
|
| 164 |
+
}
|
| 165 |
+
|
| 166 |
+
return true;
|
| 167 |
+
}
|
| 168 |
+
|
| 169 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 170 |
+
std::ignore = input_cached;
|
| 171 |
+
std::ignore = output_cached;
|
| 172 |
+
|
| 173 |
+
// 0. check the handle
|
| 174 |
+
if (nullptr == this->handle_) {
|
| 175 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is null.\n");
|
| 176 |
+
return false;
|
| 177 |
+
}
|
| 178 |
+
|
| 179 |
+
// 1. get the count of shape group
|
| 180 |
+
if (const auto ret = ::AXCL_ENGINE_GetGroupIOInfoCount(this->handle_, &this->group_count_); 0 != ret) {
|
| 181 |
+
utilities::glog.print(utilities::log::type::error,
|
| 182 |
+
"Get model shape group count failed{0x%08X}.\n", ret);
|
| 183 |
+
return false;
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
// 2. check the group index
|
| 187 |
+
if (group >= this->group_count_) {
|
| 188 |
+
utilities::glog.print(utilities::log::type::error,
|
| 189 |
+
"Model group{%d} is out of range{total %d}.\n", group, this->group_);
|
| 190 |
+
return false;
|
| 191 |
+
}
|
| 192 |
+
this->group_ = group;
|
| 193 |
+
|
| 194 |
+
// 3. get the IO info
|
| 195 |
+
if (1 == this->group_count_) {
|
| 196 |
+
if (const auto ret = ::AXCL_ENGINE_GetIOInfo(this->handle_, &info_); 0 != ret) {
|
| 197 |
+
utilities::glog.print(utilities::log::type::error,
|
| 198 |
+
"Get model IO info failed{0x%08X}.\n", ret);
|
| 199 |
+
return false;
|
| 200 |
+
}
|
| 201 |
+
} else {
|
| 202 |
+
if (const auto ret = ::AXCL_ENGINE_GetGroupIOInfo(this->handle_, group, &info_); 0 != ret) {
|
| 203 |
+
utilities::glog.print(utilities::log::type::error,
|
| 204 |
+
"Get model group{index: %d} IO info failed{0x%08X}.\n", group, ret);
|
| 205 |
+
return false;
|
| 206 |
+
}
|
| 207 |
+
}
|
| 208 |
+
|
| 209 |
+
// 4. check the batch size
|
| 210 |
+
this->batch_ = (0 == batch ? 1 : batch);
|
| 211 |
+
this->io_.nBatchSize = this->batch_;
|
| 212 |
+
|
| 213 |
+
// 5. prepare the input and output
|
| 214 |
+
this->inputs_.resize(this->info_->nInputSize, ::AX_ENGINE_IO_BUFFER_T{});
|
| 215 |
+
this->io_.pInputs = this->inputs_.data();
|
| 216 |
+
this->io_.nInputSize = this->info_->nInputSize;
|
| 217 |
+
|
| 218 |
+
this->outputs_.resize(this->info_->nOutputSize, ::AX_ENGINE_IO_BUFFER_T{});
|
| 219 |
+
this->io_.pOutputs = this->outputs_.data();
|
| 220 |
+
this->io_.nOutputSize = this->info_->nOutputSize;
|
| 221 |
+
|
| 222 |
+
// 6. prepare the memory, inputs
|
| 223 |
+
for (::AX_U32 i = 0; i < this->info_->nInputSize; i++) {
|
| 224 |
+
const auto meta = &(this->info_->pInputs[i]);
|
| 225 |
+
this->io_.pInputs[i].nSize = meta->nSize * this->batch_;
|
| 226 |
+
|
| 227 |
+
if (const auto ret = ::axclrtMalloc(
|
| 228 |
+
reinterpret_cast<void **>(&this->io_.pInputs[i].phyAddr),
|
| 229 |
+
this->io_.pInputs[i].nSize,
|
| 230 |
+
::axclrtMemMallocPolicy{}); 0 != ret) {
|
| 231 |
+
utilities::glog.print(utilities::log::type::error,
|
| 232 |
+
"Memory allocation for tensor {%s} failed{0x%08X}.\n", meta->pName, ret);
|
| 233 |
+
return false;
|
| 234 |
+
}
|
| 235 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta->pName);
|
| 236 |
+
|
| 237 |
+
// clean memory, some cases model may need to clean memory
|
| 238 |
+
//::axclrtMemset(reinterpret_cast<void *>(this->io_.pInputs[i].phyAddr), 0, this->io_.pInputs[i].nSize);
|
| 239 |
+
|
| 240 |
+
utilities::glog.print(utilities::log::type::info,
|
| 241 |
+
"Set tensor { name: %s, phy: %p, vir: %p, size: %u Bytes }.\n",
|
| 242 |
+
meta->pName,
|
| 243 |
+
reinterpret_cast<void*>(this->io_.pInputs[i].phyAddr),
|
| 244 |
+
this->io_.pInputs[i].pVirAddr,
|
| 245 |
+
this->io_.pInputs[i].nSize);
|
| 246 |
+
}
|
| 247 |
+
|
| 248 |
+
// 7. prepare the memory, outputs
|
| 249 |
+
for (::AX_U32 i = 0; i < this->info_->nOutputSize; i++) {
|
| 250 |
+
const auto meta = &(this->info_->pOutputs[i]);
|
| 251 |
+
this->io_.pOutputs[i].nSize = meta->nSize * this->batch_;
|
| 252 |
+
|
| 253 |
+
if (const auto ret = ::axclrtMalloc(
|
| 254 |
+
reinterpret_cast<void **>(&this->io_.pOutputs[i].phyAddr),
|
| 255 |
+
this->io_.pOutputs[i].nSize,
|
| 256 |
+
::axclrtMemMallocPolicy{}); 0 != ret) {
|
| 257 |
+
utilities::glog.print(utilities::log::type::error,
|
| 258 |
+
"Memory allocation for tensor {%s} failed{0x%08X}.\n", meta->pName, ret);
|
| 259 |
+
return false;
|
| 260 |
+
}
|
| 261 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor {%s} is allocated.\n", meta->pName);
|
| 262 |
+
|
| 263 |
+
// clean memory, some cases model may need to clean memory
|
| 264 |
+
//::axclrtMemset(reinterpret_cast<void *>(this->io_.pOutputs[i].phyAddr), 0, this->io_.pOutputs[i].nSize);
|
| 265 |
+
|
| 266 |
+
utilities::glog.print(utilities::log::type::info,
|
| 267 |
+
"Set tensor { name: %s, phy: %p, vir: %p, size: %u Bytes }.\n",
|
| 268 |
+
meta->pName,
|
| 269 |
+
reinterpret_cast<void*>(this->io_.pOutputs[i].phyAddr),
|
| 270 |
+
this->io_.pOutputs[i].pVirAddr,
|
| 271 |
+
this->io_.pOutputs[i].nSize);
|
| 272 |
+
}
|
| 273 |
+
|
| 274 |
+
return true;
|
| 275 |
+
}
|
| 276 |
+
|
| 277 |
+
[[nodiscard]] bool run(const bool& parallel) {
|
| 278 |
+
this->io_.nParallelRun = (parallel ? 1 : 0);
|
| 279 |
+
|
| 280 |
+
if (nullptr == this->handle_ || nullptr == this->context_) {
|
| 281 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 282 |
+
return false;
|
| 283 |
+
}
|
| 284 |
+
|
| 285 |
+
if (0 == this->io_.nInputSize || 0 == this->io_.nOutputSize) {
|
| 286 |
+
utilities::glog.print(utilities::log::type::error, "Model io is not set, prepare first.\n");
|
| 287 |
+
return false;
|
| 288 |
+
}
|
| 289 |
+
|
| 290 |
+
::AX_S32 ret = 0;
|
| 291 |
+
// Note: this is a workaround for the case that the model has only ONE group,
|
| 292 |
+
// currently AXCL_ENGINE_GetGroupIOInfo will return error.
|
| 293 |
+
// the bug of libax_engine.so will be fixed in the future.
|
| 294 |
+
if (1 == this->group_count_) {
|
| 295 |
+
ret = ::AXCL_ENGINE_RunSyncV2(this->handle_, this->context_, &this->io_);
|
| 296 |
+
} else {
|
| 297 |
+
ret = ::AXCL_ENGINE_RunGroupIOSync(this->handle_, this->context_, this->group_, &this->io_);
|
| 298 |
+
}
|
| 299 |
+
if (0 != ret) {
|
| 300 |
+
utilities::glog.print(utilities::log::type::error, "Run model failed{0x%08X}.\n", ret);
|
| 301 |
+
return false;
|
| 302 |
+
}
|
| 303 |
+
utilities::glog.print(utilities::log::type::info, "Running done.\n");
|
| 304 |
+
|
| 305 |
+
return true;
|
| 306 |
+
}
|
| 307 |
+
|
| 308 |
+
[[nodiscard]] uint32_t get_input_count() const {
|
| 309 |
+
if (nullptr == this->info_) {
|
| 310 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 311 |
+
return 0;
|
| 312 |
+
}
|
| 313 |
+
|
| 314 |
+
return this->info_->nInputSize;
|
| 315 |
+
}
|
| 316 |
+
|
| 317 |
+
[[nodiscard]] uint32_t get_output_count() const {
|
| 318 |
+
if (nullptr == this->info_) {
|
| 319 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 320 |
+
return 0;
|
| 321 |
+
}
|
| 322 |
+
|
| 323 |
+
return this->info_->nOutputSize;
|
| 324 |
+
}
|
| 325 |
+
|
| 326 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) const {
|
| 327 |
+
if (nullptr == this->info_) {
|
| 328 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 329 |
+
return {};
|
| 330 |
+
}
|
| 331 |
+
|
| 332 |
+
if (index >= this->info_->nInputSize) {
|
| 333 |
+
return {};
|
| 334 |
+
}
|
| 335 |
+
return std::string{this->info_->pInputs[index].pName};
|
| 336 |
+
}
|
| 337 |
+
|
| 338 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) const {
|
| 339 |
+
if (nullptr == this->info_) {
|
| 340 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 341 |
+
return {};
|
| 342 |
+
}
|
| 343 |
+
|
| 344 |
+
if (index >= this->info_->nOutputSize) {
|
| 345 |
+
return {};
|
| 346 |
+
}
|
| 347 |
+
return std::string{this->info_->pOutputs[index].pName};
|
| 348 |
+
}
|
| 349 |
+
|
| 350 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) const {
|
| 351 |
+
if (nullptr == this->info_) {
|
| 352 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 353 |
+
return nullptr;
|
| 354 |
+
}
|
| 355 |
+
|
| 356 |
+
if (index >= this->info_->nInputSize) {
|
| 357 |
+
return nullptr;
|
| 358 |
+
}
|
| 359 |
+
return reinterpret_cast<void *>(this->io_.pInputs[index].phyAddr);
|
| 360 |
+
}
|
| 361 |
+
|
| 362 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) const {
|
| 363 |
+
if (nullptr == this->info_) {
|
| 364 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 365 |
+
return nullptr;
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
if (index >= this->info_->nOutputSize) {
|
| 369 |
+
return nullptr;
|
| 370 |
+
}
|
| 371 |
+
return reinterpret_cast<void *>(this->io_.pOutputs[index].phyAddr);
|
| 372 |
+
}
|
| 373 |
+
|
| 374 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) const {
|
| 375 |
+
if (nullptr == this->info_) {
|
| 376 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 377 |
+
return 0;
|
| 378 |
+
}
|
| 379 |
+
|
| 380 |
+
if (index >= this->info_->nInputSize) {
|
| 381 |
+
return 0;
|
| 382 |
+
}
|
| 383 |
+
return this->io_.pInputs[index].nSize;
|
| 384 |
+
}
|
| 385 |
+
|
| 386 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) const {
|
| 387 |
+
if (nullptr == this->info_) {
|
| 388 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 389 |
+
return 0;
|
| 390 |
+
}
|
| 391 |
+
|
| 392 |
+
if (index >= this->info_->nOutputSize) {
|
| 393 |
+
return 0;
|
| 394 |
+
}
|
| 395 |
+
return this->io_.pOutputs[index].nSize;
|
| 396 |
+
}
|
| 397 |
+
|
| 398 |
+
[[nodiscard]] uintmax_t get_shape_group_count() const {
|
| 399 |
+
if (nullptr == this->info_) {
|
| 400 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 401 |
+
return 0;
|
| 402 |
+
}
|
| 403 |
+
|
| 404 |
+
::AX_U32 group = 0;
|
| 405 |
+
if (const auto ret = ::AXCL_ENGINE_GetGroupIOInfoCount(this->handle_, &group); 0 != ret) {
|
| 406 |
+
utilities::glog.print(utilities::log::type::error, "Get model shape group count failed{0x%08X}.\n", ret);
|
| 407 |
+
return 0;
|
| 408 |
+
}
|
| 409 |
+
return group;
|
| 410 |
+
}
|
| 411 |
+
|
| 412 |
+
[[nodiscard]] static std::string get_library_version() {
|
| 413 |
+
int32_t major = 0, minor = 0, patch = 0;
|
| 414 |
+
if (const auto ret = ::axclrtGetVersion(&major, &minor, &patch); 0 != ret) {
|
| 415 |
+
utilities::glog.print(utilities::log::type::error, "Get library version failed{0x%08X}.\n", ret);
|
| 416 |
+
return "";
|
| 417 |
+
}
|
| 418 |
+
return {std::to_string(major) + "." + std::to_string(minor) + "." + std::to_string(patch) + " " + ::AXCL_ENGINE_GetVersion()};
|
| 419 |
+
}
|
| 420 |
+
|
| 421 |
+
[[nodiscard]] std::string get_model_version() const {
|
| 422 |
+
if (nullptr == this->handle_) {
|
| 423 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 424 |
+
return "";
|
| 425 |
+
}
|
| 426 |
+
return {::AXCL_ENGINE_GetModelToolsVersion(this->handle_)};
|
| 427 |
+
}
|
| 428 |
+
|
| 429 |
+
[[nodiscard]] int32_t get_model_type() const {
|
| 430 |
+
if (nullptr == this->handle_) {
|
| 431 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 432 |
+
return -1;
|
| 433 |
+
}
|
| 434 |
+
|
| 435 |
+
::AX_ENGINE_MODEL_TYPE_T type;
|
| 436 |
+
if (const auto ret = ::AXCL_ENGINE_GetHandleModelType(this->handle_, &type); 0 != ret) {
|
| 437 |
+
utilities::glog.print(utilities::log::type::error, "Get model type failed{0x%08X}.\n", ret);
|
| 438 |
+
return -1;
|
| 439 |
+
}
|
| 440 |
+
return type;
|
| 441 |
+
}
|
| 442 |
+
|
| 443 |
+
[[nodiscard]] int32_t get_npu_type() const {
|
| 444 |
+
if (!this->is_initialized_) {
|
| 445 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 446 |
+
return false;
|
| 447 |
+
}
|
| 448 |
+
|
| 449 |
+
::AX_ENGINE_NPU_ATTR_T attr{};
|
| 450 |
+
if (const auto ret = ::AXCL_ENGINE_GetVNPUAttr(&attr); 0 != ret) {
|
| 451 |
+
utilities::glog.print(utilities::log::type::error, "Get visual NPU type failed{0x%08X}.\n", ret);
|
| 452 |
+
return -1;
|
| 453 |
+
}
|
| 454 |
+
return attr.eHardMode;
|
| 455 |
+
}
|
| 456 |
+
|
| 457 |
+
[[nodiscard]] int32_t get_batch_size() const {
|
| 458 |
+
if (nullptr == this->handle_) {
|
| 459 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 460 |
+
return -1;
|
| 461 |
+
}
|
| 462 |
+
|
| 463 |
+
return static_cast<int32_t>(this->batch_);
|
| 464 |
+
}
|
| 465 |
+
|
| 466 |
+
[[nodiscard]] intmax_t get_sys_usage() const {
|
| 467 |
+
if (nullptr == this->handle_) {
|
| 468 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 469 |
+
return -1;
|
| 470 |
+
}
|
| 471 |
+
|
| 472 |
+
return 0;
|
| 473 |
+
}
|
| 474 |
+
|
| 475 |
+
[[nodiscard]] intmax_t get_cmm_usage() const {
|
| 476 |
+
if (nullptr == this->handle_) {
|
| 477 |
+
utilities::glog.print(utilities::log::type::error, "Model handle is not set, load model first.\n");
|
| 478 |
+
return -1;
|
| 479 |
+
}
|
| 480 |
+
|
| 481 |
+
::AX_ENGINE_CMM_INFO usage{};
|
| 482 |
+
if (const auto ret = ::AXCL_ENGINE_GetCMMUsage(this->handle_, &usage); 0 != ret) {
|
| 483 |
+
utilities::glog.print(utilities::log::type::error, "Get model usage failed{0x%08X}.\n", ret);
|
| 484 |
+
return -1;
|
| 485 |
+
}
|
| 486 |
+
|
| 487 |
+
return usage.nCMMSize;
|
| 488 |
+
}
|
| 489 |
+
|
| 490 |
+
private:
|
| 491 |
+
uint32_t group_count_ = 0;
|
| 492 |
+
uint32_t group_ = 0;
|
| 493 |
+
uint32_t batch_ = 0;
|
| 494 |
+
|
| 495 |
+
::AX_ENGINE_HANDLE handle_{};
|
| 496 |
+
::AX_ENGINE_CONTEXT_T context_{};
|
| 497 |
+
::AX_ENGINE_IO_INFO_T* info_{};
|
| 498 |
+
::AX_ENGINE_IO_T io_{};
|
| 499 |
+
std::vector<::AX_ENGINE_IO_BUFFER_T> inputs_;
|
| 500 |
+
std::vector<::AX_ENGINE_IO_BUFFER_T> outputs_;
|
| 501 |
+
|
| 502 |
+
bool is_initialized_ = false;
|
| 503 |
+
};
|
| 504 |
+
#endif
|
cpp/src/middleware/axcl_runtime_runner.cc
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#include "middleware/axcl_runtime_runner_impl.hpp"
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_AXCL_RUNTIME_API_ENABLE)
|
| 14 |
+
middleware::runtime_runner::runtime_runner() {
|
| 15 |
+
this->impl_ = new impl();
|
| 16 |
+
}
|
| 17 |
+
|
| 18 |
+
middleware::runtime_runner::~runtime_runner() {
|
| 19 |
+
delete this->impl_;
|
| 20 |
+
}
|
| 21 |
+
|
| 22 |
+
bool middleware::runtime_runner::init(const std::string& config_file, const uint32_t& device_index, const uint32_t& kind) {
|
| 23 |
+
return this->impl_->init(config_file, device_index, kind);
|
| 24 |
+
}
|
| 25 |
+
|
| 26 |
+
bool middleware::runtime_runner::final() {
|
| 27 |
+
return this->impl_->final();
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
bool middleware::runtime_runner::load(const std::string& model_path) {
|
| 31 |
+
return this->impl_->load(model_path);
|
| 32 |
+
}
|
| 33 |
+
|
| 34 |
+
bool middleware::runtime_runner::prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 35 |
+
return this->impl_->prepare(input_cached, output_cached, group, batch);
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
bool middleware::runtime_runner::run(const bool& parallel) {
|
| 39 |
+
return this->impl_->run(parallel);
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
uint32_t middleware::runtime_runner::get_input_count() const {
|
| 43 |
+
return this->impl_->get_input_count();
|
| 44 |
+
}
|
| 45 |
+
|
| 46 |
+
uint32_t middleware::runtime_runner::get_output_count() const {
|
| 47 |
+
return this->impl_->get_output_count();
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
std::string middleware::runtime_runner::get_input_name(const uint32_t& index) const {
|
| 51 |
+
return this->impl_->get_input_name(index);
|
| 52 |
+
}
|
| 53 |
+
|
| 54 |
+
std::string middleware::runtime_runner::get_output_name(const uint32_t& index) const {
|
| 55 |
+
return this->impl_->get_output_name(index);
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
void *middleware::runtime_runner::get_input_pointer(const uint32_t& index) const {
|
| 59 |
+
return this->impl_->get_input_pointer(index);
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
void *middleware::runtime_runner::get_output_pointer(const uint32_t& index) const {
|
| 63 |
+
return this->impl_->get_output_pointer(index);
|
| 64 |
+
}
|
| 65 |
+
|
| 66 |
+
uintmax_t middleware::runtime_runner::get_input_size(const uint32_t& index) const {
|
| 67 |
+
return this->impl_->get_input_size(index);
|
| 68 |
+
}
|
| 69 |
+
|
| 70 |
+
uintmax_t middleware::runtime_runner::get_output_size(const uint32_t& index) const {
|
| 71 |
+
return this->impl_->get_output_size(index);
|
| 72 |
+
}
|
| 73 |
+
|
| 74 |
+
uintmax_t middleware::runtime_runner::get_shape_group_count() const {
|
| 75 |
+
return this->impl_->get_shape_group_count();
|
| 76 |
+
}
|
| 77 |
+
|
| 78 |
+
std::string middleware::runtime_runner::get_library_version() const {
|
| 79 |
+
return impl::get_library_version();
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
std::string middleware::runtime_runner::get_model_version() const {
|
| 83 |
+
return this->impl_->get_model_version();
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
int32_t middleware::runtime_runner::get_model_type() const {
|
| 87 |
+
return this->impl_->get_model_type();
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
int32_t middleware::runtime_runner::get_npu_type() const {
|
| 91 |
+
return this->impl_->get_npu_type();
|
| 92 |
+
}
|
| 93 |
+
|
| 94 |
+
int32_t middleware::runtime_runner::get_batch_size() const {
|
| 95 |
+
return this->impl_->get_batch_size();
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
intmax_t middleware::runtime_runner::get_sys_usage() const {
|
| 99 |
+
return this->impl_->get_sys_usage();
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
intmax_t middleware::runtime_runner::get_cmm_usage() const {
|
| 103 |
+
return this->impl_->get_cmm_usage();
|
| 104 |
+
}
|
| 105 |
+
#endif
|
cpp/src/middleware/axcl_runtime_runner.hpp
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include "middleware/axcl_base.hpp"
|
| 14 |
+
|
| 15 |
+
namespace middleware {
|
| 16 |
+
|
| 17 |
+
#if defined(ENV_AXCL_RUNTIME_API_ENABLE)
|
| 18 |
+
class runtime_runner final : public axcl_base {
|
| 19 |
+
public:
|
| 20 |
+
runtime_runner();
|
| 21 |
+
~runtime_runner() override;
|
| 22 |
+
|
| 23 |
+
[[nodiscard]] bool init(const std::string& config_file, const uint32_t& device_index, const uint32_t& kind) override;
|
| 24 |
+
[[nodiscard]] bool final() override;
|
| 25 |
+
|
| 26 |
+
[[nodiscard]] bool load(const std::string& model_path) override;
|
| 27 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) override;
|
| 28 |
+
|
| 29 |
+
[[nodiscard]] bool run(const bool& parallel) override;
|
| 30 |
+
|
| 31 |
+
[[nodiscard]] uint32_t get_input_count() const override;
|
| 32 |
+
[[nodiscard]] uint32_t get_output_count() const override;
|
| 33 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) const override;
|
| 34 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) const override;
|
| 35 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) const override;
|
| 36 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) const override;
|
| 37 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) const override;
|
| 38 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) const override;
|
| 39 |
+
[[nodiscard]] uintmax_t get_shape_group_count() const override;
|
| 40 |
+
|
| 41 |
+
[[nodiscard]] std::string get_library_version() const override;
|
| 42 |
+
[[nodiscard]] std::string get_model_version() const override;
|
| 43 |
+
|
| 44 |
+
[[nodiscard]] int32_t get_model_type() const override;
|
| 45 |
+
[[nodiscard]] int32_t get_npu_type() const override;
|
| 46 |
+
[[nodiscard]] int32_t get_batch_size() const override;
|
| 47 |
+
[[nodiscard]] intmax_t get_sys_usage() const override;
|
| 48 |
+
[[nodiscard]] intmax_t get_cmm_usage() const override;
|
| 49 |
+
|
| 50 |
+
private:
|
| 51 |
+
struct impl;
|
| 52 |
+
impl *impl_;
|
| 53 |
+
};
|
| 54 |
+
#endif
|
| 55 |
+
|
| 56 |
+
}
|
cpp/src/middleware/axcl_runtime_runner_impl.hpp
ADDED
|
@@ -0,0 +1,482 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include "middleware/axcl_runtime_runner.hpp"
|
| 14 |
+
|
| 15 |
+
#include "middleware/axcl_base.hpp"
|
| 16 |
+
#include "utilities/scalar_guard.hpp"
|
| 17 |
+
#include "utilities/file.hpp"
|
| 18 |
+
#include "utilities/log.hpp"
|
| 19 |
+
|
| 20 |
+
#include <axcl.h>
|
| 21 |
+
|
| 22 |
+
#if defined(ENV_AXCL_RUNTIME_API_ENABLE)
|
| 23 |
+
struct middleware::runtime_runner::impl {
|
| 24 |
+
impl() = default;
|
| 25 |
+
|
| 26 |
+
~impl() {
|
| 27 |
+
std::ignore = this->final();
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
[[nodiscard]] bool init(const std::string& config_file, const uint32_t& index, const uint32_t& kind) {
|
| 31 |
+
// 0. check the NPU kind
|
| 32 |
+
if (kind > AXCL_VNPU_LITTLE_BIG) {
|
| 33 |
+
utilities::glog.print(utilities::log::type::error,
|
| 34 |
+
"Specified NPU kind{%d} is out of range{total %d}.\n", kind, static_cast<int>(AXCL_VNPU_LITTLE_BIG) + 1);
|
| 35 |
+
return false;
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
// 1. build the npu init function
|
| 39 |
+
auto npu_init_func = [&kind]() {
|
| 40 |
+
if (const int ret = axclrtEngineInit(static_cast<axclrtEngineVNpuKind>(kind)); 0 != ret) {
|
| 41 |
+
utilities::glog.print(utilities::log::type::error, "Init AXCLRT Engine as kind{%d} failed{0x%08X}.\n", kind, ret);
|
| 42 |
+
return false;
|
| 43 |
+
}
|
| 44 |
+
utilities::glog.print(utilities::log::type::info, "AXCLRT Engine inited.\n");
|
| 45 |
+
return true;
|
| 46 |
+
};
|
| 47 |
+
|
| 48 |
+
// 2. execute initialization
|
| 49 |
+
if (const auto ret = axcl_base::init(config_file, index, npu_init_func); !ret) {
|
| 50 |
+
return false;
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
// 3. set the flag
|
| 54 |
+
this->is_initialized_ = true;
|
| 55 |
+
return true;
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
[[nodiscard]] bool final() {
|
| 59 |
+
auto flag = true;
|
| 60 |
+
|
| 61 |
+
if (this->is_initialized_) {
|
| 62 |
+
if (0 != this->model_id_) {
|
| 63 |
+
for (auto& input : inputs_) {
|
| 64 |
+
if (nullptr != input) {
|
| 65 |
+
std::ignore = axclrtFree(input);
|
| 66 |
+
input = nullptr;
|
| 67 |
+
}
|
| 68 |
+
}
|
| 69 |
+
for (auto& output : outputs_) {
|
| 70 |
+
if (nullptr != output) {
|
| 71 |
+
std::ignore = axclrtFree(output);
|
| 72 |
+
output = nullptr;
|
| 73 |
+
}
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
flag &= 0 == axclrtEngineDestroyIOInfo(this->info_);
|
| 77 |
+
flag &= 0 == axclrtEngineDestroyIO(this->io_);
|
| 78 |
+
flag &= 0 == axclrtEngineUnload(this->model_id_);
|
| 79 |
+
}
|
| 80 |
+
|
| 81 |
+
flag &= 0 == axcl_base::final([&]() {
|
| 82 |
+
if (const auto ret = axclrtEngineFinalize(); 0 != ret) {
|
| 83 |
+
utilities::glog.print(utilities::log::type::error, "Final AXCLRT Engine failed{0x%08X}.\n", ret);
|
| 84 |
+
return false;
|
| 85 |
+
}
|
| 86 |
+
utilities::glog.print(utilities::log::type::info, "AXCLRT Engine finalized.\n");
|
| 87 |
+
return true;
|
| 88 |
+
});
|
| 89 |
+
|
| 90 |
+
this->is_initialized_ = false;
|
| 91 |
+
}
|
| 92 |
+
|
| 93 |
+
return flag;
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
[[nodiscard]] bool load(const std::string& model_path) {
|
| 97 |
+
if (!this->is_initialized_) {
|
| 98 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 99 |
+
return false;
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
if (0 != this->model_id_) {
|
| 103 |
+
utilities::glog.print(utilities::log::type::error, "Model is already loaded.\n");
|
| 104 |
+
return false;
|
| 105 |
+
}
|
| 106 |
+
|
| 107 |
+
if (!utilities::exists(model_path)
|
| 108 |
+
|| !utilities::is_regular_file(model_path)
|
| 109 |
+
|| 0 == utilities::file_size(model_path)
|
| 110 |
+
|| utilities::error_size == utilities::file_size(model_path)) {
|
| 111 |
+
utilities::glog.print(utilities::log::type::error, "Model file{%s} error, please check it.\n", model_path.c_str());
|
| 112 |
+
return false;
|
| 113 |
+
}
|
| 114 |
+
|
| 115 |
+
if (const auto ret = axclrtEngineLoadFromFile(model_path.c_str(), &this->model_id_); 0 != ret) {
|
| 116 |
+
utilities::glog.print(utilities::log::type::error, "Create model{%s} handle failed.\n", model_path.c_str());
|
| 117 |
+
return false;
|
| 118 |
+
}
|
| 119 |
+
|
| 120 |
+
if (const auto ret = axclrtEngineCreateContext(this->model_id_, &this->context_id_); 0 != ret) {
|
| 121 |
+
utilities::glog.print(utilities::log::type::error, "Create model{%s} context failed.\n", model_path.c_str());
|
| 122 |
+
return false;
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
return true;
|
| 126 |
+
}
|
| 127 |
+
|
| 128 |
+
[[nodiscard]] bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) {
|
| 129 |
+
std::ignore = input_cached;
|
| 130 |
+
std::ignore = output_cached;
|
| 131 |
+
|
| 132 |
+
// 0. check the handle
|
| 133 |
+
if (0 == this->model_id_) {
|
| 134 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 135 |
+
return false;
|
| 136 |
+
}
|
| 137 |
+
|
| 138 |
+
// 1. get the IO info
|
| 139 |
+
if (const auto ret = axclrtEngineGetIOInfo(this->model_id_, &this->info_); 0 != ret) {
|
| 140 |
+
utilities::glog.print(utilities::log::type::error, "Get model io info failed{0x%08X}.\n", ret);
|
| 141 |
+
return false;
|
| 142 |
+
}
|
| 143 |
+
|
| 144 |
+
// 2. get the count of shape group
|
| 145 |
+
int32_t total_group = 0;
|
| 146 |
+
if (const auto ret = axclrtEngineGetShapeGroupsCount(this->info_, &total_group); 0 != ret) {
|
| 147 |
+
utilities::glog.print(utilities::log::type::error, "Get model shape group count failed{0x%08X}.\n", ret);
|
| 148 |
+
return false;
|
| 149 |
+
}
|
| 150 |
+
|
| 151 |
+
// 3. check the group index
|
| 152 |
+
if (group >= static_cast<decltype(group)>(total_group)) {
|
| 153 |
+
utilities::glog.print(utilities::log::type::error, "Model group{%d} is out of range{total %d}.\n", group, total_group);
|
| 154 |
+
return false;
|
| 155 |
+
}
|
| 156 |
+
this->group_ = static_cast<int32_t>(group);
|
| 157 |
+
|
| 158 |
+
// 4. check the batch size
|
| 159 |
+
this->batch_ = (0 == batch ? 1 : batch);
|
| 160 |
+
|
| 161 |
+
// 5. get the count of inputs
|
| 162 |
+
uint32_t input_count = 0;
|
| 163 |
+
if (input_count = axclrtEngineGetNumInputs(this->info_); 0 == input_count) {
|
| 164 |
+
utilities::glog.print(utilities::log::type::error, "Get model input count failed.\n");
|
| 165 |
+
return false;
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
// 6. get the count of outputs
|
| 169 |
+
uint32_t output_count = 0;
|
| 170 |
+
if (output_count = axclrtEngineGetNumOutputs(this->info_); 0 == output_count) {
|
| 171 |
+
utilities::glog.print(utilities::log::type::error, "Get model output count failed.\n");
|
| 172 |
+
return false;
|
| 173 |
+
}
|
| 174 |
+
|
| 175 |
+
// 7. prepare the input and output
|
| 176 |
+
this->inputs_.resize(input_count, nullptr);
|
| 177 |
+
this->inputs_size_.resize(input_count, 0);
|
| 178 |
+
this->outputs_.resize(output_count, nullptr);
|
| 179 |
+
this->outputs_size_.resize(output_count, 0);
|
| 180 |
+
|
| 181 |
+
// 8. prepare the memory, inputs
|
| 182 |
+
for (uint32_t i = 0; i < input_count; i++) {
|
| 183 |
+
uint32_t original_size = 0;
|
| 184 |
+
if (original_size = axclrtEngineGetInputSizeByIndex(this->info_, group, i); 0 == original_size) {
|
| 185 |
+
utilities::glog.print(utilities::log::type::error, "Get model input{index: %d} size failed.\n", i);
|
| 186 |
+
return false;
|
| 187 |
+
}
|
| 188 |
+
|
| 189 |
+
this->inputs_size_[i] = original_size * this->batch_;
|
| 190 |
+
|
| 191 |
+
if (const auto ret = axclrtMalloc(&this->inputs_[i], this->inputs_size_[i], axclrtMemMallocPolicy{}); 0 != ret) {
|
| 192 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor{index: %d} failed{0x%08X}.\n", i, ret);
|
| 193 |
+
return false;
|
| 194 |
+
}
|
| 195 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor{index: %d} is allocated.\n", i);
|
| 196 |
+
|
| 197 |
+
// clean memory, some cases model may need to clean memory
|
| 198 |
+
// axclrtMemset(this->inputs_[i], 0, size);
|
| 199 |
+
|
| 200 |
+
utilities::glog.print(utilities::log::type::info,
|
| 201 |
+
"Set tensor { index: %d, address: %p, size: %u Bytes }.\n", i, this->inputs_[i], this->inputs_size_[i]);
|
| 202 |
+
}
|
| 203 |
+
|
| 204 |
+
// 9. prepare the memory, outputs
|
| 205 |
+
for (uint32_t i = 0; i < output_count; i++) {
|
| 206 |
+
uint32_t original_size = 0;
|
| 207 |
+
if (original_size = axclrtEngineGetOutputSizeByIndex(this->info_, group, i); 0 == original_size) {
|
| 208 |
+
utilities::glog.print(utilities::log::type::error, "Get model output{index: %d} size failed.\n", i);
|
| 209 |
+
return false;
|
| 210 |
+
}
|
| 211 |
+
|
| 212 |
+
this->outputs_size_[i] = original_size * this->batch_;
|
| 213 |
+
|
| 214 |
+
if (const auto ret = axclrtMalloc(&this->outputs_[i], this->outputs_size_[i], axclrtMemMallocPolicy{}); 0 != ret) {
|
| 215 |
+
utilities::glog.print(utilities::log::type::error, "Memory allocation for tensor{index: %d} failed{0x%08X}.\n", i, ret);
|
| 216 |
+
return false;
|
| 217 |
+
}
|
| 218 |
+
utilities::glog.print(utilities::log::type::info, "Memory for tensor{index: %d} is allocated.\n", i);
|
| 219 |
+
|
| 220 |
+
// clean memory, some cases model may need to clean memory
|
| 221 |
+
// axclrtMemset(this->outputs_[i], 0, size);
|
| 222 |
+
|
| 223 |
+
utilities::glog.print(utilities::log::type::info,
|
| 224 |
+
"Set tensor { index: %d, address: %p, size: %u Bytes }.\n", i, this->outputs_[i], this->outputs_size_[i]);
|
| 225 |
+
}
|
| 226 |
+
|
| 227 |
+
// 10. create the IO
|
| 228 |
+
if (const auto ret = axclrtEngineCreateIO(this->info_, &this->io_); 0 != ret) {
|
| 229 |
+
utilities::glog.print(utilities::log::type::error, "Create model io failed{0x%08X}.\n", ret);
|
| 230 |
+
return false;
|
| 231 |
+
}
|
| 232 |
+
utilities::glog.print(utilities::log::type::info, "AXCLRT Engine inited.\n");
|
| 233 |
+
|
| 234 |
+
// 11. set the input and output buffer
|
| 235 |
+
for (uint32_t i = 0; i < input_count; i++) {
|
| 236 |
+
if (const auto ret = axclrtEngineSetInputBufferByIndex(this->io_, i, this->inputs_[i], this->inputs_size_[i]); 0 != ret) {
|
| 237 |
+
utilities::glog.print(utilities::log::type::error, "Set input buffer{index: %d} failed{0x%08X}.\n", i, ret);
|
| 238 |
+
return false;
|
| 239 |
+
}
|
| 240 |
+
}
|
| 241 |
+
for (uint32_t i = 0; i < output_count; i++) {
|
| 242 |
+
if (const auto ret = axclrtEngineSetOutputBufferByIndex(this->io_, i, this->outputs_[i], this->outputs_size_[i]); 0 != ret) {
|
| 243 |
+
utilities::glog.print(utilities::log::type::error, "Set output buffer{index: %d} failed{0x%08X}.\n", i, ret);
|
| 244 |
+
return false;
|
| 245 |
+
}
|
| 246 |
+
}
|
| 247 |
+
|
| 248 |
+
// 12. set the batch size
|
| 249 |
+
if (const auto ret = axclrtEngineSetDynamicBatchSize(this->io_, this->batch_); 0 != ret) {
|
| 250 |
+
utilities::glog.print(utilities::log::type::error, "Set batch size{%d} failed{0x%08X}.\n", this->batch_, ret);
|
| 251 |
+
return false;
|
| 252 |
+
}
|
| 253 |
+
|
| 254 |
+
return true;
|
| 255 |
+
}
|
| 256 |
+
|
| 257 |
+
[[nodiscard]] bool run(const bool& parallel) const {
|
| 258 |
+
std::ignore = parallel;
|
| 259 |
+
|
| 260 |
+
if (0 == this->model_id_ || 0 == this->context_id_) {
|
| 261 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 262 |
+
return false;
|
| 263 |
+
}
|
| 264 |
+
|
| 265 |
+
if (nullptr == this->io_) {
|
| 266 |
+
utilities::glog.print(utilities::log::type::error, "Model io is not set, prepare first.\n");
|
| 267 |
+
return false;
|
| 268 |
+
}
|
| 269 |
+
|
| 270 |
+
if (const auto ret = axclrtEngineExecute(this->model_id_, this->context_id_, this->group_, this->io_); 0 != ret) {
|
| 271 |
+
utilities::glog.print(utilities::log::type::error, "Run model failed{0x%08X}.\n", ret);
|
| 272 |
+
return false;
|
| 273 |
+
}
|
| 274 |
+
utilities::glog.print(utilities::log::type::info, "Running done.\n");
|
| 275 |
+
|
| 276 |
+
return true;
|
| 277 |
+
}
|
| 278 |
+
|
| 279 |
+
[[nodiscard]] uint32_t get_input_count() const {
|
| 280 |
+
if (nullptr == this->info_) {
|
| 281 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 282 |
+
return 0;
|
| 283 |
+
}
|
| 284 |
+
|
| 285 |
+
return axclrtEngineGetNumInputs(this->info_);
|
| 286 |
+
}
|
| 287 |
+
|
| 288 |
+
[[nodiscard]] uint32_t get_output_count() const {
|
| 289 |
+
if (nullptr == this->info_) {
|
| 290 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 291 |
+
return 0;
|
| 292 |
+
}
|
| 293 |
+
|
| 294 |
+
return axclrtEngineGetNumOutputs(this->info_);
|
| 295 |
+
}
|
| 296 |
+
|
| 297 |
+
[[nodiscard]] std::string get_input_name(const uint32_t& index) const {
|
| 298 |
+
if (nullptr == this->info_) {
|
| 299 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 300 |
+
return {};
|
| 301 |
+
}
|
| 302 |
+
|
| 303 |
+
if (index >= get_input_count()) {
|
| 304 |
+
return {};
|
| 305 |
+
}
|
| 306 |
+
return std::string{axclrtEngineGetInputNameByIndex(this->info_, index)};
|
| 307 |
+
}
|
| 308 |
+
|
| 309 |
+
[[nodiscard]] std::string get_output_name(const uint32_t& index) const {
|
| 310 |
+
if (nullptr == this->info_) {
|
| 311 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 312 |
+
return {};
|
| 313 |
+
}
|
| 314 |
+
|
| 315 |
+
if (index >= get_output_count()) {
|
| 316 |
+
return {};
|
| 317 |
+
}
|
| 318 |
+
return std::string{axclrtEngineGetOutputNameByIndex(this->info_, index)};
|
| 319 |
+
}
|
| 320 |
+
|
| 321 |
+
[[nodiscard]] void *get_input_pointer(const uint32_t& index) const {
|
| 322 |
+
if (nullptr == this->info_) {
|
| 323 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 324 |
+
return nullptr;
|
| 325 |
+
}
|
| 326 |
+
|
| 327 |
+
if (index >= get_input_count()) {
|
| 328 |
+
return nullptr;
|
| 329 |
+
}
|
| 330 |
+
return this->inputs_[index];
|
| 331 |
+
}
|
| 332 |
+
|
| 333 |
+
[[nodiscard]] void *get_output_pointer(const uint32_t& index) const {
|
| 334 |
+
if (nullptr == this->info_) {
|
| 335 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 336 |
+
return nullptr;
|
| 337 |
+
}
|
| 338 |
+
|
| 339 |
+
if (index >= get_output_count()) {
|
| 340 |
+
return nullptr;
|
| 341 |
+
}
|
| 342 |
+
return this->outputs_[index];
|
| 343 |
+
}
|
| 344 |
+
|
| 345 |
+
[[nodiscard]] uintmax_t get_input_size(const uint32_t& index) const {
|
| 346 |
+
if (nullptr == this->info_) {
|
| 347 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 348 |
+
return 0;
|
| 349 |
+
}
|
| 350 |
+
|
| 351 |
+
if (index >= get_input_count()) {
|
| 352 |
+
return 0;
|
| 353 |
+
}
|
| 354 |
+
return axclrtEngineGetInputSizeByIndex(this->info_, this->group_, index);
|
| 355 |
+
}
|
| 356 |
+
|
| 357 |
+
[[nodiscard]] uintmax_t get_output_size(const uint32_t& index) const {
|
| 358 |
+
if (nullptr == this->info_) {
|
| 359 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 360 |
+
return 0;
|
| 361 |
+
}
|
| 362 |
+
|
| 363 |
+
if (index >= get_output_count()) {
|
| 364 |
+
return 0;
|
| 365 |
+
}
|
| 366 |
+
return axclrtEngineGetOutputSizeByIndex(this->info_, this->group_, index);
|
| 367 |
+
}
|
| 368 |
+
|
| 369 |
+
[[nodiscard]] uintmax_t get_shape_group_count() const {
|
| 370 |
+
if (nullptr == this->info_) {
|
| 371 |
+
utilities::glog.print(utilities::log::type::error, "Model io info is not set, prepare first.\n");
|
| 372 |
+
return 0;
|
| 373 |
+
}
|
| 374 |
+
|
| 375 |
+
int group = 0;
|
| 376 |
+
if (const auto ret = axclrtEngineGetShapeGroupsCount(this->info_, &group); 0 != ret) {
|
| 377 |
+
utilities::glog.print(utilities::log::type::error, "Get model shape group count failed{0x%08X}.\n", ret);
|
| 378 |
+
return 0;
|
| 379 |
+
}
|
| 380 |
+
return group;
|
| 381 |
+
}
|
| 382 |
+
|
| 383 |
+
[[nodiscard]] static std::string get_library_version() {
|
| 384 |
+
int32_t major = 0, minor = 0, patch = 0;
|
| 385 |
+
if (const auto ret = axclrtGetVersion(&major, &minor, &patch); 0 != ret) {
|
| 386 |
+
utilities::glog.print(utilities::log::type::error, "Get library version failed{0x%08X}.\n", ret);
|
| 387 |
+
return "";
|
| 388 |
+
}
|
| 389 |
+
return {std::to_string(major) + "." + std::to_string(minor) + "." + std::to_string(patch)};
|
| 390 |
+
}
|
| 391 |
+
|
| 392 |
+
[[nodiscard]] std::string get_model_version() const {
|
| 393 |
+
if (0 == this->model_id_) {
|
| 394 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 395 |
+
return "";
|
| 396 |
+
}
|
| 397 |
+
return {axclrtEngineGetModelCompilerVersion(this->model_id_)};
|
| 398 |
+
}
|
| 399 |
+
|
| 400 |
+
[[nodiscard]] int32_t get_model_type() const {
|
| 401 |
+
if (0 == this->model_id_) {
|
| 402 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 403 |
+
return -1;
|
| 404 |
+
}
|
| 405 |
+
|
| 406 |
+
axclrtEngineModelKind type;
|
| 407 |
+
if (const auto ret = axclrtEngineGetModelTypeFromModelId(this->model_id_, &type); 0 != ret) {
|
| 408 |
+
utilities::glog.print(utilities::log::type::error, "Get model type failed{0x%08X}.\n", ret);
|
| 409 |
+
return -1;
|
| 410 |
+
}
|
| 411 |
+
return type;
|
| 412 |
+
}
|
| 413 |
+
|
| 414 |
+
[[nodiscard]] int32_t get_npu_type() const {
|
| 415 |
+
if (!this->is_initialized_) {
|
| 416 |
+
utilities::glog.print(utilities::log::type::error, "axcl is not initialized.\n");
|
| 417 |
+
return false;
|
| 418 |
+
}
|
| 419 |
+
|
| 420 |
+
axclrtEngineVNpuKind kind;
|
| 421 |
+
if (const auto ret = axclrtEngineGetVNpuKind(&kind); 0 != ret) {
|
| 422 |
+
utilities::glog.print(utilities::log::type::error, "Get visual NPU type failed{0x%08X}.\n", ret);
|
| 423 |
+
return -1;
|
| 424 |
+
}
|
| 425 |
+
return kind;
|
| 426 |
+
}
|
| 427 |
+
|
| 428 |
+
[[nodiscard]] int32_t get_batch_size() const {
|
| 429 |
+
if (0 == this->model_id_) {
|
| 430 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 431 |
+
return -1;
|
| 432 |
+
}
|
| 433 |
+
|
| 434 |
+
return static_cast<int32_t>(this->batch_);
|
| 435 |
+
}
|
| 436 |
+
|
| 437 |
+
[[nodiscard]] intmax_t get_sys_usage() const {
|
| 438 |
+
if (0 == this->model_id_) {
|
| 439 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 440 |
+
return -1;
|
| 441 |
+
}
|
| 442 |
+
|
| 443 |
+
int64_t sys_size = 0, cmm_size = 0;
|
| 444 |
+
if (const auto ret = axclrtEngineGetUsageFromModelId(this->model_id_, &sys_size, &cmm_size); 0 != ret) {
|
| 445 |
+
utilities::glog.print(utilities::log::type::error, "Get model usage failed{0x%08X}.\n", ret);
|
| 446 |
+
return -1;
|
| 447 |
+
}
|
| 448 |
+
|
| 449 |
+
return sys_size;
|
| 450 |
+
}
|
| 451 |
+
|
| 452 |
+
[[nodiscard]] intmax_t get_cmm_usage() const {
|
| 453 |
+
if (0 == this->model_id_) {
|
| 454 |
+
utilities::glog.print(utilities::log::type::error, "Model id is not set, load model first.\n");
|
| 455 |
+
return -1;
|
| 456 |
+
}
|
| 457 |
+
|
| 458 |
+
int64_t sys_size = 0, cmm_size = 0;
|
| 459 |
+
if (const auto ret = axclrtEngineGetUsageFromModelId(this->model_id_, &sys_size, &cmm_size); 0 != ret) {
|
| 460 |
+
utilities::glog.print(utilities::log::type::error, "Get model usage failed{0x%08X}.\n", ret);
|
| 461 |
+
return -1;
|
| 462 |
+
}
|
| 463 |
+
|
| 464 |
+
return cmm_size;
|
| 465 |
+
}
|
| 466 |
+
|
| 467 |
+
private:
|
| 468 |
+
int32_t group_ = 0;
|
| 469 |
+
uint32_t batch_ = 0;
|
| 470 |
+
|
| 471 |
+
uint64_t model_id_{};
|
| 472 |
+
uint64_t context_id_{};
|
| 473 |
+
axclrtEngineIOInfo info_{};
|
| 474 |
+
axclrtEngineIO io_{};
|
| 475 |
+
std::vector<void*> inputs_;
|
| 476 |
+
std::vector<void*> outputs_;
|
| 477 |
+
std::vector<uintmax_t> inputs_size_;
|
| 478 |
+
std::vector<uintmax_t> outputs_size_;
|
| 479 |
+
|
| 480 |
+
bool is_initialized_ = false;
|
| 481 |
+
};
|
| 482 |
+
#endif
|
cpp/src/middleware/runner.cc
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#include "middleware/runner.hpp"
|
| 12 |
+
|
| 13 |
+
#include "utilities/file.hpp"
|
| 14 |
+
#include "utilities/log.hpp"
|
| 15 |
+
|
| 16 |
+
#include <chrono>
|
| 17 |
+
#include <thread>
|
| 18 |
+
|
| 19 |
+
void middleware::runner::sleep_for(const uint32_t sleep_duration) const {
|
| 20 |
+
std::this_thread::sleep_for(std::chrono::milliseconds(sleep_duration));
|
| 21 |
+
}
|
| 22 |
+
|
| 23 |
+
std::vector<std::string> middleware::runner::read(const std::string& list_file) {
|
| 24 |
+
if (!utilities::exists(list_file) || !utilities::is_regular_file(list_file)) {
|
| 25 |
+
utilities::glog.print(utilities::log::type::error,
|
| 26 |
+
"Stimulus list file{%s} is not exist.\n", list_file.c_str());
|
| 27 |
+
return {};
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
std::vector<std::string> list;
|
| 31 |
+
|
| 32 |
+
std::ifstream list_stream;
|
| 33 |
+
list_stream.open(list_file, std::ios::in);
|
| 34 |
+
if (!list_stream.is_open()) {
|
| 35 |
+
utilities::glog.print(utilities::log::type::error,
|
| 36 |
+
"Read stimulus list file{%s} failed.\n", list_file.c_str());
|
| 37 |
+
return {};
|
| 38 |
+
}
|
| 39 |
+
|
| 40 |
+
std::string file_path;
|
| 41 |
+
while (std::getline(list_stream, file_path)) {
|
| 42 |
+
list.push_back(file_path);
|
| 43 |
+
}
|
| 44 |
+
list_stream.close();
|
| 45 |
+
|
| 46 |
+
return list;
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
bool middleware::runner::read(const std::string& binary_file, void* address, const uintmax_t& size) {
|
| 50 |
+
if (!utilities::exists(binary_file) || !utilities::is_regular_file(binary_file)) {
|
| 51 |
+
utilities::glog.print(utilities::log::type::error,
|
| 52 |
+
"Stimulus file{%s} is not exist.\n", binary_file.c_str());
|
| 53 |
+
return false;
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
const auto file_size = utilities::file_size(binary_file);
|
| 57 |
+
if (utilities::error_size == file_size) {
|
| 58 |
+
utilities::glog.print(utilities::log::type::error, "Get stimulus file size failed.\n");
|
| 59 |
+
return false;
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
if (file_size != size) {
|
| 63 |
+
utilities::glog.print(utilities::log::type::error,
|
| 64 |
+
"Size of stimulus file{%s} is not equal the specified{file: %d vs tensor: %d}.\n", binary_file.c_str(), file_size, size);
|
| 65 |
+
return false;
|
| 66 |
+
}
|
| 67 |
+
|
| 68 |
+
return utilities::read(binary_file, address, size);
|
| 69 |
+
}
|
| 70 |
+
|
| 71 |
+
bool middleware::runner::write(const std::string& binary_file, const void* address, const uintmax_t& size) {
|
| 72 |
+
return utilities::write(binary_file, address, size);
|
| 73 |
+
}
|
| 74 |
+
|
| 75 |
+
bool middleware::runner::verify(const void* dst, const void* src, const uintmax_t& size) {
|
| 76 |
+
const auto* src_ptr = static_cast<const uint8_t*>(src);
|
| 77 |
+
const auto* dst_ptr = static_cast<const uint8_t*>(dst);
|
| 78 |
+
|
| 79 |
+
for (uintmax_t i = 0; i < size; i++) {
|
| 80 |
+
if (src_ptr[i] != dst_ptr[i]) {
|
| 81 |
+
utilities::glog.print(utilities::log::type::error,
|
| 82 |
+
"Verify stimulus failed{offset: 0x%x; 0x%02X vs. 0x%02X}.\n", i, src_ptr[i], dst_ptr[i]);
|
| 83 |
+
return false;
|
| 84 |
+
}
|
| 85 |
+
}
|
| 86 |
+
|
| 87 |
+
return true;
|
| 88 |
+
}
|
cpp/src/middleware/runner.hpp
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <string>
|
| 14 |
+
#include <vector>
|
| 15 |
+
#include <cstdint>
|
| 16 |
+
|
| 17 |
+
namespace middleware {
|
| 18 |
+
|
| 19 |
+
class runner {
|
| 20 |
+
public:
|
| 21 |
+
runner() = default;
|
| 22 |
+
virtual ~runner() = default;
|
| 23 |
+
|
| 24 |
+
[[nodiscard]] virtual bool init(const std::string& config_file, const uint32_t& device_index, const uint32_t& kind) = 0;
|
| 25 |
+
[[nodiscard]] virtual bool final() = 0;
|
| 26 |
+
|
| 27 |
+
[[nodiscard]] virtual bool load(const std::string& model_path) = 0;
|
| 28 |
+
[[nodiscard]] virtual bool prepare(const bool& input_cached, const bool& output_cached, const uint32_t& group, const uint32_t& batch) = 0;
|
| 29 |
+
|
| 30 |
+
[[nodiscard]] virtual bool run(const bool& parallel) = 0;
|
| 31 |
+
|
| 32 |
+
[[nodiscard]] virtual uint32_t get_input_count() const = 0;
|
| 33 |
+
[[nodiscard]] virtual uint32_t get_output_count() const = 0;
|
| 34 |
+
[[nodiscard]] virtual std::string get_input_name(const uint32_t& index) const = 0;
|
| 35 |
+
[[nodiscard]] virtual std::string get_output_name(const uint32_t& index) const = 0;
|
| 36 |
+
[[nodiscard]] virtual void *get_input_pointer(const uint32_t& index) const = 0;
|
| 37 |
+
[[nodiscard]] virtual void *get_output_pointer(const uint32_t& index) const = 0;
|
| 38 |
+
[[nodiscard]] virtual uintmax_t get_input_size(const uint32_t& index) const = 0;
|
| 39 |
+
[[nodiscard]] virtual uintmax_t get_output_size(const uint32_t& index) const = 0;
|
| 40 |
+
|
| 41 |
+
[[nodiscard]] virtual uintmax_t get_shape_group_count() const = 0;
|
| 42 |
+
|
| 43 |
+
[[nodiscard]] virtual bool flush_input() const = 0;
|
| 44 |
+
[[nodiscard]] virtual bool invalidate_output() const = 0;
|
| 45 |
+
|
| 46 |
+
[[nodiscard]] virtual bool feed(const std::string& input_folder, const std::string& stimulus_name) const = 0;
|
| 47 |
+
[[nodiscard]] virtual bool verify(const std::string& output_folder, const std::string& stimulus_name) const = 0;
|
| 48 |
+
[[nodiscard]] virtual bool save(const std::string& output_folder, const std::string& stimulus_name) const = 0;
|
| 49 |
+
|
| 50 |
+
[[nodiscard]] virtual std::string get_library_version() const = 0;
|
| 51 |
+
[[nodiscard]] virtual std::string get_model_version() const = 0;
|
| 52 |
+
|
| 53 |
+
[[nodiscard]] virtual int32_t get_model_type() const = 0;
|
| 54 |
+
[[nodiscard]] virtual int32_t get_npu_type() const = 0;
|
| 55 |
+
[[nodiscard]] virtual int32_t get_batch_size() const = 0;
|
| 56 |
+
[[nodiscard]] virtual intmax_t get_sys_usage() const = 0;
|
| 57 |
+
[[nodiscard]] virtual intmax_t get_cmm_usage() const = 0;
|
| 58 |
+
|
| 59 |
+
virtual void sleep_for(uint32_t sleep_duration) const;
|
| 60 |
+
|
| 61 |
+
[[nodiscard]] static std::vector<std::string> read(const std::string& list_file);
|
| 62 |
+
[[nodiscard]] static bool read(const std::string& binary_file, void* address, const uintmax_t& size);
|
| 63 |
+
[[nodiscard]] static bool write(const std::string& binary_file, const void* address, const uintmax_t& size);
|
| 64 |
+
[[nodiscard]] static bool verify(const void* dst, const void* src, const uintmax_t& size);
|
| 65 |
+
};
|
| 66 |
+
|
| 67 |
+
}
|
cpp/src/utilities/file.hpp
ADDED
|
@@ -0,0 +1,251 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <fstream>
|
| 14 |
+
#include <cctype>
|
| 15 |
+
|
| 16 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 17 |
+
#include <filesystem>
|
| 18 |
+
#endif // ENV_HAS_STD_FILESYSTEM
|
| 19 |
+
#if defined(ENV_HAS_POSIX_FILE_STAT)
|
| 20 |
+
#include <sys/types.h>
|
| 21 |
+
#include <sys/stat.h>
|
| 22 |
+
#include <dirent.h>
|
| 23 |
+
#include <cstring>
|
| 24 |
+
#include "utilities/scalar_guard.hpp"
|
| 25 |
+
#endif // ENV_HAS_POSIX_FILE_STAT
|
| 26 |
+
#if defined(ENV_HAS_WIN_API)
|
| 27 |
+
#include <Windows.h>
|
| 28 |
+
#endif // ENV_HAS_WIN_API
|
| 29 |
+
|
| 30 |
+
#include <string>
|
| 31 |
+
|
| 32 |
+
namespace utilities {
|
| 33 |
+
constexpr auto error_size = static_cast<uintmax_t>(-1);
|
| 34 |
+
|
| 35 |
+
enum class file_type {
|
| 36 |
+
none,
|
| 37 |
+
not_found,
|
| 38 |
+
regular,
|
| 39 |
+
directory,
|
| 40 |
+
symlink,
|
| 41 |
+
block,
|
| 42 |
+
character,
|
| 43 |
+
fifo,
|
| 44 |
+
socket,
|
| 45 |
+
unknown,
|
| 46 |
+
};
|
| 47 |
+
|
| 48 |
+
inline bool exists(const std::string& path) {
|
| 49 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 50 |
+
return std::filesystem::exists(path);
|
| 51 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 52 |
+
struct ::stat path_stat{};
|
| 53 |
+
return 0 == ::stat(path.c_str(), &path_stat);
|
| 54 |
+
#else
|
| 55 |
+
#error "Unsupported platform, native file system API is required."
|
| 56 |
+
#endif
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
inline uintmax_t file_size(const std::string& path) {
|
| 60 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 61 |
+
return std::filesystem::file_size(path);
|
| 62 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 63 |
+
struct ::stat path_stat{};
|
| 64 |
+
if (0 == ::stat(path.c_str(), &path_stat)) {
|
| 65 |
+
return path_stat.st_size;
|
| 66 |
+
}
|
| 67 |
+
return error_size;
|
| 68 |
+
#else
|
| 69 |
+
#error "Unsupported platform, native file system API is required."
|
| 70 |
+
#endif
|
| 71 |
+
}
|
| 72 |
+
|
| 73 |
+
inline file_type status(const std::string& path) {
|
| 74 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 75 |
+
if (!exists(path)) {
|
| 76 |
+
return file_type::not_found;
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
+
const auto status = std::filesystem::status(path);
|
| 80 |
+
if (std::filesystem::is_regular_file(status)) {
|
| 81 |
+
return file_type::regular;
|
| 82 |
+
}
|
| 83 |
+
if (std::filesystem::is_directory(status)) {
|
| 84 |
+
return file_type::directory;
|
| 85 |
+
}
|
| 86 |
+
if (std::filesystem::is_symlink(status)) {
|
| 87 |
+
return file_type::symlink;
|
| 88 |
+
}
|
| 89 |
+
if (std::filesystem::is_block_file(status)) {
|
| 90 |
+
return file_type::block;
|
| 91 |
+
}
|
| 92 |
+
if (std::filesystem::is_character_file(status)) {
|
| 93 |
+
return file_type::character;
|
| 94 |
+
}
|
| 95 |
+
if (std::filesystem::is_fifo(status)) {
|
| 96 |
+
return file_type::fifo;
|
| 97 |
+
}
|
| 98 |
+
if (std::filesystem::is_socket(status)) {
|
| 99 |
+
return file_type::socket;
|
| 100 |
+
}
|
| 101 |
+
return file_type::unknown;
|
| 102 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 103 |
+
struct ::stat path_stat{};
|
| 104 |
+
if (0 != ::stat(path.c_str(), &path_stat)) {
|
| 105 |
+
return file_type::not_found;
|
| 106 |
+
}
|
| 107 |
+
|
| 108 |
+
if (S_ISREG(path_stat.st_mode)) {
|
| 109 |
+
return file_type::regular;
|
| 110 |
+
}
|
| 111 |
+
if (S_ISDIR(path_stat.st_mode)) {
|
| 112 |
+
return file_type::directory;
|
| 113 |
+
}
|
| 114 |
+
if (S_ISLNK(path_stat.st_mode)) {
|
| 115 |
+
return file_type::symlink;
|
| 116 |
+
}
|
| 117 |
+
if (S_ISBLK(path_stat.st_mode)) {
|
| 118 |
+
return file_type::block;
|
| 119 |
+
}
|
| 120 |
+
if (S_ISCHR(path_stat.st_mode)) {
|
| 121 |
+
return file_type::character;
|
| 122 |
+
}
|
| 123 |
+
if (S_ISFIFO(path_stat.st_mode)) {
|
| 124 |
+
return file_type::fifo;
|
| 125 |
+
}
|
| 126 |
+
if (S_ISSOCK(path_stat.st_mode)) {
|
| 127 |
+
return file_type::socket;
|
| 128 |
+
}
|
| 129 |
+
return file_type::unknown;
|
| 130 |
+
#else
|
| 131 |
+
#error "Unsupported platform, native file system API is required."
|
| 132 |
+
#endif
|
| 133 |
+
}
|
| 134 |
+
|
| 135 |
+
inline bool is_regular_file(const std::string& path) {
|
| 136 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 137 |
+
return std::filesystem::is_regular_file(path);
|
| 138 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 139 |
+
struct ::stat path_stat{};
|
| 140 |
+
return 0 == ::stat(path.c_str(), &path_stat) && S_ISREG(path_stat.st_mode);
|
| 141 |
+
#else
|
| 142 |
+
#error "Unsupported platform, native file system API is required."
|
| 143 |
+
#endif
|
| 144 |
+
}
|
| 145 |
+
|
| 146 |
+
inline bool is_directory(const std::string& path) {
|
| 147 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 148 |
+
return std::filesystem::is_directory(path);
|
| 149 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 150 |
+
struct ::stat path_stat{};
|
| 151 |
+
return 0 == ::stat(path.c_str(), &path_stat) && S_ISDIR(path_stat.st_mode);
|
| 152 |
+
#else
|
| 153 |
+
#error "Unsupported platform, native file system API is required."
|
| 154 |
+
#endif
|
| 155 |
+
}
|
| 156 |
+
|
| 157 |
+
inline bool is_empty(const std::string& path) {
|
| 158 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 159 |
+
return std::filesystem::is_empty(path);
|
| 160 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 161 |
+
if (!exists(path))
|
| 162 |
+
return false;
|
| 163 |
+
|
| 164 |
+
if (is_regular_file(path))
|
| 165 |
+
return 0 == file_size(path);
|
| 166 |
+
if (is_directory(path)) {
|
| 167 |
+
auto dir = scalar_guard<DIR*>(::opendir(path.c_str()), ::closedir);
|
| 168 |
+
if (nullptr == dir.get()) {
|
| 169 |
+
return false;
|
| 170 |
+
}
|
| 171 |
+
|
| 172 |
+
struct ::dirent* entry;
|
| 173 |
+
while ((entry = ::readdir(dir.get())) != nullptr) {
|
| 174 |
+
if (0 != ::strcmp(entry->d_name, ".") && 0 != ::strcmp(entry->d_name, "..")) {
|
| 175 |
+
return false;
|
| 176 |
+
}
|
| 177 |
+
}
|
| 178 |
+
return true;
|
| 179 |
+
}
|
| 180 |
+
return false;
|
| 181 |
+
#else
|
| 182 |
+
#error "Unsupported platform, native file system API is required."
|
| 183 |
+
#endif
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
inline bool read(const std::string& file, void* data, const uintmax_t& size) {
|
| 187 |
+
std::ifstream stream;
|
| 188 |
+
stream.open(file, std::ios_base::binary | std::ios_base::in);
|
| 189 |
+
if (stream.is_open()) {
|
| 190 |
+
stream.read(static_cast<char*>(data), static_cast<std::streamsize>(size));
|
| 191 |
+
stream.close();
|
| 192 |
+
return true;
|
| 193 |
+
}
|
| 194 |
+
return false;
|
| 195 |
+
}
|
| 196 |
+
|
| 197 |
+
inline bool write(const std::string& file, const void* data, const uintmax_t& size) {
|
| 198 |
+
std::ofstream stream;
|
| 199 |
+
stream.open(file, std::ios_base::binary | std::ios_base::out);
|
| 200 |
+
if (stream.is_open()) {
|
| 201 |
+
stream.write(static_cast<const char*>(data), static_cast<std::streamsize>(size));
|
| 202 |
+
stream.close();
|
| 203 |
+
return true;
|
| 204 |
+
}
|
| 205 |
+
return false;
|
| 206 |
+
}
|
| 207 |
+
|
| 208 |
+
inline bool create_directory(const std::string& path) {
|
| 209 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 210 |
+
return std::filesystem::create_directories(path);
|
| 211 |
+
#elif defined(ENV_HAS_POSIX_FILE_STAT)
|
| 212 |
+
return 0 == ::mkdir(path.c_str(), 0755);
|
| 213 |
+
#else
|
| 214 |
+
#error "Unsupported platform, native file system API is required."
|
| 215 |
+
#endif
|
| 216 |
+
}
|
| 217 |
+
|
| 218 |
+
inline std::string get_file_name(const std::string& file) {
|
| 219 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 220 |
+
return std::filesystem::path(file).filename().string();
|
| 221 |
+
#else
|
| 222 |
+
if (const auto pos = file.find_last_of("/\\"); std::string::npos != pos) {
|
| 223 |
+
return file.substr(pos + 1);
|
| 224 |
+
}
|
| 225 |
+
return file;
|
| 226 |
+
#endif
|
| 227 |
+
}
|
| 228 |
+
|
| 229 |
+
inline std::string get_file_extension(const std::string& file) {
|
| 230 |
+
#if defined(ENV_HAS_STD_FILESYSTEM)
|
| 231 |
+
return std::filesystem::path(file).extension().string();
|
| 232 |
+
#else
|
| 233 |
+
if (const auto pos = file.find_last_of('.'); std::string::npos != pos) {
|
| 234 |
+
return file.substr(pos + 1);
|
| 235 |
+
}
|
| 236 |
+
return "";
|
| 237 |
+
#endif
|
| 238 |
+
}
|
| 239 |
+
|
| 240 |
+
inline std::string get_legal_name(const std::string& name) {
|
| 241 |
+
std::string temp = name;
|
| 242 |
+
|
| 243 |
+
size_t pos = temp.find_last_of("/\\");
|
| 244 |
+
if (pos != std::string::npos) pos = 0;
|
| 245 |
+
|
| 246 |
+
for (size_t i = pos; i < temp.size(); ++i)
|
| 247 |
+
if (!(::isalnum(temp[i]) || '-' == temp[i] || '_' == temp[i])) temp[i] = '_';
|
| 248 |
+
return temp;
|
| 249 |
+
}
|
| 250 |
+
|
| 251 |
+
}
|
cpp/src/utilities/file_mapper.hpp
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#if defined(ENV_HAS_POSIX_FILE_STAT)
|
| 14 |
+
#include "utilities/scalar_guard.hpp"
|
| 15 |
+
#include <sys/mman.h>
|
| 16 |
+
#include <fcntl.h>
|
| 17 |
+
#include <unistd.h>
|
| 18 |
+
#endif // ENV_HAS_POSIX_FILE_STAT
|
| 19 |
+
|
| 20 |
+
#if defined(ENV_HAS_WIN_API)
|
| 21 |
+
#include <Windows.h>
|
| 22 |
+
#endif // ENV_HAS_WIN_API
|
| 23 |
+
|
| 24 |
+
#include <utilities/file.hpp>
|
| 25 |
+
|
| 26 |
+
namespace utilities {
|
| 27 |
+
|
| 28 |
+
struct file_mapper {
|
| 29 |
+
file_mapper() = delete;
|
| 30 |
+
|
| 31 |
+
explicit file_mapper(const std::string& path) {
|
| 32 |
+
#if defined(ENV_HAS_POSIX_FILE_STAT)
|
| 33 |
+
|
| 34 |
+
auto fd_guard = scalar_guard<int>(
|
| 35 |
+
::open(path.c_str(), O_RDONLY),
|
| 36 |
+
[](const int& fd) { if (fd != -1) ::close(fd); }
|
| 37 |
+
);
|
| 38 |
+
|
| 39 |
+
if (-1 == fd_guard.get()) {
|
| 40 |
+
return;
|
| 41 |
+
}
|
| 42 |
+
|
| 43 |
+
auto size = file_size(path);
|
| 44 |
+
auto map_guard = scalar_guard<void*>(
|
| 45 |
+
::mmap(nullptr, size, PROT_READ, MAP_SHARED, fd_guard.get(), 0),
|
| 46 |
+
[&size](void*& addr) { if (MAP_FAILED != addr && nullptr != addr) ::munmap(addr, size); }
|
| 47 |
+
);
|
| 48 |
+
|
| 49 |
+
if (MAP_FAILED == map_guard.get()) {
|
| 50 |
+
return;
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
this->size_ = size;
|
| 54 |
+
std::swap(this->fd_, fd_guard.get());
|
| 55 |
+
std::swap(this->buffer_, map_guard.get());
|
| 56 |
+
#endif // ENV_HAS_POSIX_FILE_STAT
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
~file_mapper() {
|
| 60 |
+
#if defined(ENV_HAS_POSIX_FILE_STAT)
|
| 61 |
+
if (nullptr != this->buffer_) {
|
| 62 |
+
::munmap(this->buffer_, this->size_);
|
| 63 |
+
this->buffer_ = nullptr;
|
| 64 |
+
}
|
| 65 |
+
if (-1 != this->fd_) {
|
| 66 |
+
::close(this->fd_);
|
| 67 |
+
this->fd_ = -1;
|
| 68 |
+
}
|
| 69 |
+
#endif // ENV_HAS_POSIX_FILE_STAT
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
[[nodiscard]] void* get() const {
|
| 73 |
+
return this->buffer_;
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
[[nodiscard]] uint64_t size() const {
|
| 77 |
+
return this->size_;
|
| 78 |
+
}
|
| 79 |
+
|
| 80 |
+
private:
|
| 81 |
+
int fd_ = -1;
|
| 82 |
+
void* buffer_ = nullptr;
|
| 83 |
+
uint64_t size_ = 0;
|
| 84 |
+
};
|
| 85 |
+
|
| 86 |
+
}
|
cpp/src/utilities/log.hpp
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/*
|
| 2 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 3 |
+
*
|
| 4 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 5 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 6 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 7 |
+
*
|
| 8 |
+
* Author: wanglusheng@axera-tech.com
|
| 9 |
+
*/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <cstdarg>
|
| 14 |
+
#include <cstdio>
|
| 15 |
+
|
| 16 |
+
namespace utilities {
|
| 17 |
+
class log
|
| 18 |
+
{
|
| 19 |
+
public:
|
| 20 |
+
enum class type {
|
| 21 |
+
debug = 0,
|
| 22 |
+
info = 1,
|
| 23 |
+
warn = 2,
|
| 24 |
+
error = 3,
|
| 25 |
+
};
|
| 26 |
+
|
| 27 |
+
log() = default;
|
| 28 |
+
explicit log(const type& mode) : type_(mode) {}
|
| 29 |
+
~log() = default;
|
| 30 |
+
|
| 31 |
+
void print(const type& mode, const char *fmt, ...) const {
|
| 32 |
+
if (mode >= this->type_) {
|
| 33 |
+
switch (mode) {
|
| 34 |
+
case type::debug:
|
| 35 |
+
::fprintf(stdout, "[DEBUG] ");
|
| 36 |
+
break;
|
| 37 |
+
case type::info:
|
| 38 |
+
::fprintf(stdout, "[INFO] ");
|
| 39 |
+
break;
|
| 40 |
+
case type::warn:
|
| 41 |
+
::fprintf(stdout, "[WARNING] ");
|
| 42 |
+
break;
|
| 43 |
+
case type::error:
|
| 44 |
+
::fprintf(stdout, "[ERROR] ");
|
| 45 |
+
break;
|
| 46 |
+
}
|
| 47 |
+
|
| 48 |
+
::va_list args;
|
| 49 |
+
va_start(args, fmt);
|
| 50 |
+
::vprintf( fmt, args);
|
| 51 |
+
va_end(args);
|
| 52 |
+
}
|
| 53 |
+
::fflush(stdout);
|
| 54 |
+
}
|
| 55 |
+
void set_level(const type& mode) {
|
| 56 |
+
this->type_ = mode;
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
private:
|
| 60 |
+
type type_ = type::warn;
|
| 61 |
+
};
|
| 62 |
+
|
| 63 |
+
static log glog;
|
| 64 |
+
}
|
cpp/src/utilities/scalar_guard.hpp
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <functional>
|
| 14 |
+
#include <vector>
|
| 15 |
+
#include <utility>
|
| 16 |
+
#include <stdexcept>
|
| 17 |
+
|
| 18 |
+
namespace utilities {
|
| 19 |
+
|
| 20 |
+
template<typename T>
|
| 21 |
+
class scalar_guard {
|
| 22 |
+
public:
|
| 23 |
+
scalar_guard() = delete;
|
| 24 |
+
|
| 25 |
+
scalar_guard(T resource, std::function<void(T&)> destructor)
|
| 26 |
+
: instance_(resource), destructor_(std::move(destructor)) {}
|
| 27 |
+
|
| 28 |
+
scalar_guard(std::function<T()> resource_creator, std::function<void(T&)> destructor)
|
| 29 |
+
: instance_(resource_creator()), destructor_(std::move(destructor)) {}
|
| 30 |
+
|
| 31 |
+
scalar_guard(scalar_guard&& other) noexcept
|
| 32 |
+
: instance_(std::exchange(other.instance_, T{})), destructor_(std::move(other.destructor_)) {}
|
| 33 |
+
|
| 34 |
+
scalar_guard& operator=(scalar_guard&& other) noexcept {
|
| 35 |
+
if (this != &other) {
|
| 36 |
+
instance_ = std::exchange(other.instance_, T{});
|
| 37 |
+
destructor_ = std::move(other.destructor_);
|
| 38 |
+
}
|
| 39 |
+
return *this;
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
scalar_guard(const scalar_guard&) = delete;
|
| 43 |
+
scalar_guard& operator=(const scalar_guard&) = delete;
|
| 44 |
+
|
| 45 |
+
~scalar_guard() noexcept {
|
| 46 |
+
if (destructor_) {
|
| 47 |
+
destructor_(instance_);
|
| 48 |
+
}
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
T& get() {
|
| 52 |
+
return instance_;
|
| 53 |
+
}
|
| 54 |
+
|
| 55 |
+
private:
|
| 56 |
+
T instance_;
|
| 57 |
+
std::function<void(T&)> destructor_;
|
| 58 |
+
};
|
| 59 |
+
|
| 60 |
+
}
|
cpp/src/utilities/timer.hpp
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <chrono>
|
| 14 |
+
|
| 15 |
+
namespace utilities {
|
| 16 |
+
|
| 17 |
+
class timer {
|
| 18 |
+
public:
|
| 19 |
+
using nanoseconds = std::chrono::nanoseconds;
|
| 20 |
+
using microseconds = std::chrono::microseconds;
|
| 21 |
+
using milliseconds = std::chrono::milliseconds;
|
| 22 |
+
using seconds = std::chrono::seconds;
|
| 23 |
+
using minutes = std::chrono::minutes;
|
| 24 |
+
using hours = std::chrono::hours;
|
| 25 |
+
|
| 26 |
+
timer() {
|
| 27 |
+
start();
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
void start() {
|
| 31 |
+
stop();
|
| 32 |
+
this->start_ = this->end_;
|
| 33 |
+
}
|
| 34 |
+
|
| 35 |
+
void stop() {
|
| 36 |
+
#ifdef _MSC_VER
|
| 37 |
+
this->end_ = std::chrono::system_clock::now();
|
| 38 |
+
#else
|
| 39 |
+
this->end_ = std::chrono::high_resolution_clock::now();
|
| 40 |
+
#endif
|
| 41 |
+
}
|
| 42 |
+
|
| 43 |
+
template<typename T>
|
| 44 |
+
float elapsed() {
|
| 45 |
+
if (this->end_ <= this->start_) {
|
| 46 |
+
this->stop();
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
if (std::is_same_v<T, std::chrono::nanoseconds>) {
|
| 50 |
+
const auto t = std::chrono::duration_cast<std::chrono::nanoseconds>(this->end_ - this->start_).count();
|
| 51 |
+
return static_cast<float>(t);
|
| 52 |
+
}
|
| 53 |
+
if (std::is_same_v<T, std::chrono::microseconds>) {
|
| 54 |
+
const auto t = std::chrono::duration_cast<std::chrono::nanoseconds>(this->end_ - this->start_).count();
|
| 55 |
+
return static_cast<float>(t) / 1000.f;
|
| 56 |
+
}
|
| 57 |
+
if (std::is_same_v<T, std::chrono::milliseconds>) {
|
| 58 |
+
const auto t = std::chrono::duration_cast<std::chrono::microseconds>(this->end_ - this->start_).count();
|
| 59 |
+
return static_cast<float>(t) / 1000.f;
|
| 60 |
+
}
|
| 61 |
+
if (std::is_same_v<T, std::chrono::seconds>) {
|
| 62 |
+
const auto t = std::chrono::duration_cast<std::chrono::milliseconds>(this->end_ - this->start_).count();
|
| 63 |
+
return static_cast<float>(t) / 1000.f;
|
| 64 |
+
}
|
| 65 |
+
if (std::is_same_v<T, std::chrono::minutes>) {
|
| 66 |
+
const auto t = std::chrono::duration_cast<std::chrono::milliseconds>(this->end_ - this->start_).count();
|
| 67 |
+
return static_cast<float>(t) / (60.f * 1000.f);
|
| 68 |
+
}
|
| 69 |
+
if (std::is_same_v<T, std::chrono::hours>) {
|
| 70 |
+
const auto t = std::chrono::duration_cast<std::chrono::seconds>(this->end_ - this->start_).count();
|
| 71 |
+
return static_cast<float>(t) / (60.f * 60.f);
|
| 72 |
+
}
|
| 73 |
+
return 0.f;
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
float elapsed() {
|
| 77 |
+
return elapsed<std::chrono::milliseconds>();
|
| 78 |
+
}
|
| 79 |
+
|
| 80 |
+
private:
|
| 81 |
+
std::chrono::system_clock::time_point start_, end_;
|
| 82 |
+
};
|
| 83 |
+
|
| 84 |
+
}
|
cpp/src/utilities/vector_guard.hpp
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**************************************************************************************************
|
| 2 |
+
*
|
| 3 |
+
* Copyright (c) 2019-2024 Axera Semiconductor Co., Ltd. All Rights Reserved.
|
| 4 |
+
*
|
| 5 |
+
* This source file is the property of Axera Semiconductor Co., Ltd. and
|
| 6 |
+
* may not be copied or distributed in any isomorphic form without the prior
|
| 7 |
+
* written consent of Axera Semiconductor Co., Ltd.
|
| 8 |
+
*
|
| 9 |
+
**************************************************************************************************/
|
| 10 |
+
|
| 11 |
+
#pragma once
|
| 12 |
+
|
| 13 |
+
#include <functional>
|
| 14 |
+
#include <vector>
|
| 15 |
+
|
| 16 |
+
namespace utilities {
|
| 17 |
+
|
| 18 |
+
template<typename T>
|
| 19 |
+
class vector_guard {
|
| 20 |
+
public:
|
| 21 |
+
vector_guard() = delete;
|
| 22 |
+
|
| 23 |
+
vector_guard(std::vector<T> resources, std::function<void(T&)> destructor)
|
| 24 |
+
: instance_(std::move(resources)), destructor_(std::move(destructor)) {}
|
| 25 |
+
|
| 26 |
+
vector_guard(std::function<std::vector<T>()> resource_creator, std::function<void(T&)> destructor)
|
| 27 |
+
: instance_(resource_creator()), destructor_(std::move(destructor)) {}
|
| 28 |
+
|
| 29 |
+
vector_guard(vector_guard&& other) noexcept
|
| 30 |
+
: instance_(std::exchange(other.instance_, std::vector<T>{})), destructor_(std::move(other.destructor_)) {}
|
| 31 |
+
|
| 32 |
+
vector_guard& operator=(vector_guard&& other) noexcept {
|
| 33 |
+
if (this != &other) {
|
| 34 |
+
instance_ = std::exchange(other.instance_, std::vector<T>{});
|
| 35 |
+
destructor_ = std::move(other.destructor_);
|
| 36 |
+
}
|
| 37 |
+
return *this;
|
| 38 |
+
}
|
| 39 |
+
|
| 40 |
+
vector_guard(const vector_guard&) = delete;
|
| 41 |
+
vector_guard& operator=(const vector_guard&) = delete;
|
| 42 |
+
|
| 43 |
+
~vector_guard() noexcept {
|
| 44 |
+
if (destructor_) {
|
| 45 |
+
for (auto& resource : instance_) {
|
| 46 |
+
destructor_(resource);
|
| 47 |
+
}
|
| 48 |
+
}
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
std::vector<T>& get() {
|
| 52 |
+
return instance_;
|
| 53 |
+
}
|
| 54 |
+
|
| 55 |
+
T* data() {
|
| 56 |
+
return instance_.data();
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
private:
|
| 60 |
+
std::vector<T> instance_;
|
| 61 |
+
std::function<void(T&)> destructor_;
|
| 62 |
+
};
|
| 63 |
+
|
| 64 |
+
}
|
cpp/src/vocoder.cpp
CHANGED
|
@@ -38,7 +38,7 @@ int Vocoder::Init(const Config& cfg) {
|
|
| 38 |
|
| 39 |
// Load vocos_full.axmodel
|
| 40 |
m_session = std::make_unique<EngineWrapper>();
|
| 41 |
-
if (m_session->Init(cfg.model_path.c_str()) != 0) {
|
| 42 |
printf("Failed to load vocoder model: %s\n", cfg.model_path.c_str());
|
| 43 |
return -1;
|
| 44 |
}
|
|
|
|
| 38 |
|
| 39 |
// Load vocos_full.axmodel
|
| 40 |
m_session = std::make_unique<EngineWrapper>();
|
| 41 |
+
if (m_session->Init(cfg.model_path.c_str(), 0, cfg.axclConfig) != 0) {
|
| 42 |
printf("Failed to load vocoder model: %s\n", cfg.model_path.c_str());
|
| 43 |
return -1;
|
| 44 |
}
|
cpp/src/vocoder.hpp
CHANGED
|
@@ -17,6 +17,7 @@ class Vocoder {
|
|
| 17 |
public:
|
| 18 |
struct Config {
|
| 19 |
std::string model_path;
|
|
|
|
| 20 |
int n_fft = 1024;
|
| 21 |
int hop_length = 256;
|
| 22 |
int win_length = 1024;
|
|
|
|
| 17 |
public:
|
| 18 |
struct Config {
|
| 19 |
std::string model_path;
|
| 20 |
+
const char* axclConfig = nullptr; // AXCL only: path to axcl.json. Ignored on AXERA.
|
| 21 |
int n_fft = 1024;
|
| 22 |
int hop_length = 256;
|
| 23 |
int win_length = 1024;
|
cpp/src/zipvoice_engine.cpp
CHANGED
|
@@ -136,10 +136,10 @@ double ZipVoiceEngine::GetCurrentTimeMs() {
|
|
| 136 |
return tv.tv_sec * 1000.0 + tv.tv_usec / 1000.0;
|
| 137 |
}
|
| 138 |
|
| 139 |
-
int ZipVoiceEngine::Init(const std::string& model_dir) {
|
| 140 |
if (LoadConfig(model_dir) != 0) return -1;
|
| 141 |
if (LoadManifest(model_dir) != 0) return -1;
|
| 142 |
-
if (LoadModels(model_dir) != 0) return -1;
|
| 143 |
if (LoadDecoderMetadata() != 0) return -1;
|
| 144 |
|
| 145 |
m_has_init = true;
|
|
@@ -227,11 +227,11 @@ int ZipVoiceEngine::LoadManifest(const std::string& model_dir) {
|
|
| 227 |
return 0;
|
| 228 |
}
|
| 229 |
|
| 230 |
-
int ZipVoiceEngine::LoadModels(const std::string& model_dir) {
|
| 231 |
// Load encoder
|
| 232 |
std::string enc_path = model_dir + "/" + m_encoder_info.file;
|
| 233 |
auto enc = std::make_unique<EngineWrapper>();
|
| 234 |
-
if (enc->Init(enc_path.c_str()) != 0) {
|
| 235 |
printf("Failed to load encoder: %s\n", enc_path.c_str());
|
| 236 |
return -1;
|
| 237 |
}
|
|
@@ -241,7 +241,7 @@ int ZipVoiceEngine::LoadModels(const std::string& model_dir) {
|
|
| 241 |
for (auto& part : m_decoder_parts) {
|
| 242 |
std::string path = model_dir + "/" + part.file;
|
| 243 |
auto sess = std::make_unique<EngineWrapper>();
|
| 244 |
-
if (sess->Init(path.c_str()) != 0) {
|
| 245 |
printf("Failed to load decoder part: %s\n", path.c_str());
|
| 246 |
return -1;
|
| 247 |
}
|
|
|
|
| 136 |
return tv.tv_sec * 1000.0 + tv.tv_usec / 1000.0;
|
| 137 |
}
|
| 138 |
|
| 139 |
+
int ZipVoiceEngine::Init(const std::string& model_dir, const char* axclConfig) {
|
| 140 |
if (LoadConfig(model_dir) != 0) return -1;
|
| 141 |
if (LoadManifest(model_dir) != 0) return -1;
|
| 142 |
+
if (LoadModels(model_dir, axclConfig) != 0) return -1;
|
| 143 |
if (LoadDecoderMetadata() != 0) return -1;
|
| 144 |
|
| 145 |
m_has_init = true;
|
|
|
|
| 227 |
return 0;
|
| 228 |
}
|
| 229 |
|
| 230 |
+
int ZipVoiceEngine::LoadModels(const std::string& model_dir, const char* axclConfig) {
|
| 231 |
// Load encoder
|
| 232 |
std::string enc_path = model_dir + "/" + m_encoder_info.file;
|
| 233 |
auto enc = std::make_unique<EngineWrapper>();
|
| 234 |
+
if (enc->Init(enc_path.c_str(), 0, axclConfig) != 0) {
|
| 235 |
printf("Failed to load encoder: %s\n", enc_path.c_str());
|
| 236 |
return -1;
|
| 237 |
}
|
|
|
|
| 241 |
for (auto& part : m_decoder_parts) {
|
| 242 |
std::string path = model_dir + "/" + part.file;
|
| 243 |
auto sess = std::make_unique<EngineWrapper>();
|
| 244 |
+
if (sess->Init(path.c_str(), 0, axclConfig) != 0) {
|
| 245 |
printf("Failed to load decoder part: %s\n", path.c_str());
|
| 246 |
return -1;
|
| 247 |
}
|
cpp/src/zipvoice_engine.hpp
CHANGED
|
@@ -61,13 +61,11 @@ public:
|
|
| 61 |
~ZipVoiceEngine();
|
| 62 |
|
| 63 |
/**
|
| 64 |
-
* Initialize from a model directory
|
| 65 |
-
*
|
| 66 |
-
*
|
| 67 |
-
* - encoder.axmodel
|
| 68 |
-
* - decoder_part0.axmodel .. decoder_part3.axmodel
|
| 69 |
*/
|
| 70 |
-
int Init(const std::string& model_dir);
|
| 71 |
|
| 72 |
/**
|
| 73 |
* Run encoder: cat_tokens[1, max_tokens] → encoded[1, max_tokens, feat_dim].
|
|
@@ -130,7 +128,7 @@ private:
|
|
| 130 |
|
| 131 |
int LoadConfig(const std::string& model_dir);
|
| 132 |
int LoadManifest(const std::string& model_dir);
|
| 133 |
-
int LoadModels(const std::string& model_dir);
|
| 134 |
int LoadDecoderMetadata();
|
| 135 |
|
| 136 |
std::vector<float> GetTimesteps(int num_step, float t_shift) const;
|
|
|
|
| 61 |
~ZipVoiceEngine();
|
| 62 |
|
| 63 |
/**
|
| 64 |
+
* Initialize from a model directory.
|
| 65 |
+
* @param model_dir Path containing axmodel files and JSON configs
|
| 66 |
+
* @param axclConfig AXCL only: path to axcl.json (nullptr → default). Ignored on AXERA.
|
|
|
|
|
|
|
| 67 |
*/
|
| 68 |
+
int Init(const std::string& model_dir, const char* axclConfig = nullptr);
|
| 69 |
|
| 70 |
/**
|
| 71 |
* Run encoder: cat_tokens[1, max_tokens] → encoded[1, max_tokens, feat_dim].
|
|
|
|
| 128 |
|
| 129 |
int LoadConfig(const std::string& model_dir);
|
| 130 |
int LoadManifest(const std::string& model_dir);
|
| 131 |
+
int LoadModels(const std::string& model_dir, const char* axclConfig = nullptr);
|
| 132 |
int LoadDecoderMetadata();
|
| 133 |
|
| 134 |
std::vector<float> GetTimesteps(int num_step, float t_shift) const;
|
cpp/toolchains/aarch64-none-linux-gnu.toolchain.cmake
CHANGED
|
@@ -1,21 +1,10 @@
|
|
| 1 |
-
#
|
| 2 |
-
#
|
| 3 |
-
# Download:
|
| 4 |
-
# wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 5 |
-
# tar -xf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
|
| 6 |
-
#
|
| 7 |
-
# Set TOOLCHAIN_DIR: export TOOLCHAIN_DIR=/path/to/gcc-arm-...
|
| 8 |
-
# or pass: cmake -DTOOLCHAIN_DIR=/path/to/...
|
| 9 |
-
|
| 10 |
-
if(NOT DEFINED TOOLCHAIN_DIR)
|
| 11 |
-
set(TOOLCHAIN_DIR "$ENV{HOME}/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu")
|
| 12 |
-
endif()
|
| 13 |
-
|
| 14 |
set(CMAKE_SYSTEM_NAME Linux)
|
| 15 |
set(CMAKE_SYSTEM_PROCESSOR aarch64)
|
| 16 |
|
| 17 |
-
set(
|
| 18 |
-
set(
|
|
|
|
| 19 |
|
| 20 |
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
| 21 |
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
|
|
|
| 1 |
+
# Cross-compile toolchain for AXCL aarch64
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
set(CMAKE_SYSTEM_NAME Linux)
|
| 3 |
set(CMAKE_SYSTEM_PROCESSOR aarch64)
|
| 4 |
|
| 5 |
+
set(TOOLCHAIN_DIR /data/shared/huyuan/toolchains/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin)
|
| 6 |
+
set(CMAKE_C_COMPILER "${TOOLCHAIN_DIR}/aarch64-none-linux-gnu-gcc")
|
| 7 |
+
set(CMAKE_CXX_COMPILER "${TOOLCHAIN_DIR}/aarch64-none-linux-gnu-g++")
|
| 8 |
|
| 9 |
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
| 10 |
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
cpp/zipvoice.cpp
CHANGED
|
@@ -38,7 +38,10 @@
|
|
| 38 |
#include <unistd.h>
|
| 39 |
#include <sys/wait.h>
|
| 40 |
|
|
|
|
| 41 |
#include "ax_sys_api.h"
|
|
|
|
|
|
|
| 42 |
#include "src/cmdline.hpp"
|
| 43 |
#include "src/EngineWrapper.hpp"
|
| 44 |
#include "src/tokenizer.hpp"
|
|
@@ -620,6 +623,9 @@ int main(int argc, char** argv) {
|
|
| 620 |
false, "");
|
| 621 |
cmd.add<std::string>("vocoder-model", 0, "Path to vocos_full.axmodel for C++ vocoder. When set, all-C++ pipeline, no Python.",
|
| 622 |
false, "");
|
|
|
|
|
|
|
|
|
|
| 623 |
cmd.add<std::string>("cat-tokens-file", 0, "Pre-computed cat_tokens int32 binary (from export_tokens.py). "
|
| 624 |
"When set, skips built-in tokenizer.",
|
| 625 |
false, "");
|
|
@@ -652,6 +658,8 @@ int main(int argc, char** argv) {
|
|
| 652 |
auto output_feat = cmd.get<std::string>("output-feat");
|
| 653 |
auto repo_dir = cmd.get<std::string>("repo-dir");
|
| 654 |
auto vocoder_model = cmd.get<std::string>("vocoder-model");
|
|
|
|
|
|
|
| 655 |
auto cat_tokens_file = cmd.get<std::string>("cat-tokens-file");
|
| 656 |
int prompt_tokens_len_cmd = cmd.get<int>("prompt-tokens-len");
|
| 657 |
int text_tokens_len_cmd = cmd.get<int>("text-tokens-len");
|
|
@@ -787,7 +795,8 @@ int main(int argc, char** argv) {
|
|
| 787 |
printf("seed: %d\n", seed);
|
| 788 |
printf("========================================\n");
|
| 789 |
|
| 790 |
-
// --- Init AX system ---
|
|
|
|
| 791 |
double t_total_start = get_current_time_ms();
|
| 792 |
int ret = AX_SYS_Init();
|
| 793 |
if (0 != ret) {
|
|
@@ -803,6 +812,7 @@ int main(int argc, char** argv) {
|
|
| 803 |
fprintf(stderr, "AX_ENGINE_Init failed{0x%8x}.\n", ret);
|
| 804 |
return -1;
|
| 805 |
}
|
|
|
|
| 806 |
|
| 807 |
// tokenizer already loaded above for Python-aligned segmentation
|
| 808 |
|
|
@@ -849,7 +859,7 @@ int main(int argc, char** argv) {
|
|
| 849 |
double t_load_start = get_current_time_ms();
|
| 850 |
ZipVoiceEngine engine;
|
| 851 |
|
| 852 |
-
if (engine.Init(model_dir) != 0) {
|
| 853 |
printf("Failed to initialize ZipVoice engine\n");
|
| 854 |
return -1;
|
| 855 |
}
|
|
@@ -861,6 +871,7 @@ int main(int argc, char** argv) {
|
|
| 861 |
double t_vocoder_load = get_current_time_ms();
|
| 862 |
Vocoder::Config vcfg;
|
| 863 |
vcfg.model_path = vocoder_model;
|
|
|
|
| 864 |
if (vocoder.Init(vcfg) != 0) {
|
| 865 |
printf("ERROR: Failed to load vocoder model\n");
|
| 866 |
return -1;
|
|
|
|
| 38 |
#include <unistd.h>
|
| 39 |
#include <sys/wait.h>
|
| 40 |
|
| 41 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 42 |
#include "ax_sys_api.h"
|
| 43 |
+
#endif
|
| 44 |
+
|
| 45 |
#include "src/cmdline.hpp"
|
| 46 |
#include "src/EngineWrapper.hpp"
|
| 47 |
#include "src/tokenizer.hpp"
|
|
|
|
| 623 |
false, "");
|
| 624 |
cmd.add<std::string>("vocoder-model", 0, "Path to vocos_full.axmodel for C++ vocoder. When set, all-C++ pipeline, no Python.",
|
| 625 |
false, "");
|
| 626 |
+
cmd.add<std::string>("axcl-config", 0, "AXCL only: path to axcl.json config file",
|
| 627 |
+
false, "/usr/local/axcl/axcl.json");
|
| 628 |
+
cmd.add<int>("device-index", 0, "AXCL only: device index (0=first card)", false, 0);
|
| 629 |
cmd.add<std::string>("cat-tokens-file", 0, "Pre-computed cat_tokens int32 binary (from export_tokens.py). "
|
| 630 |
"When set, skips built-in tokenizer.",
|
| 631 |
false, "");
|
|
|
|
| 658 |
auto output_feat = cmd.get<std::string>("output-feat");
|
| 659 |
auto repo_dir = cmd.get<std::string>("repo-dir");
|
| 660 |
auto vocoder_model = cmd.get<std::string>("vocoder-model");
|
| 661 |
+
auto axcl_config = cmd.get<std::string>("axcl-config");
|
| 662 |
+
int device_index = cmd.get<int>("device-index");
|
| 663 |
auto cat_tokens_file = cmd.get<std::string>("cat-tokens-file");
|
| 664 |
int prompt_tokens_len_cmd = cmd.get<int>("prompt-tokens-len");
|
| 665 |
int text_tokens_len_cmd = cmd.get<int>("text-tokens-len");
|
|
|
|
| 795 |
printf("seed: %d\n", seed);
|
| 796 |
printf("========================================\n");
|
| 797 |
|
| 798 |
+
// --- Init AX system (AXERA only; AXCL handles init internally) ---
|
| 799 |
+
#if defined(AX650) || defined(AX630C) || defined(AX620Q)
|
| 800 |
double t_total_start = get_current_time_ms();
|
| 801 |
int ret = AX_SYS_Init();
|
| 802 |
if (0 != ret) {
|
|
|
|
| 812 |
fprintf(stderr, "AX_ENGINE_Init failed{0x%8x}.\n", ret);
|
| 813 |
return -1;
|
| 814 |
}
|
| 815 |
+
#endif
|
| 816 |
|
| 817 |
// tokenizer already loaded above for Python-aligned segmentation
|
| 818 |
|
|
|
|
| 859 |
double t_load_start = get_current_time_ms();
|
| 860 |
ZipVoiceEngine engine;
|
| 861 |
|
| 862 |
+
if (engine.Init(model_dir, axcl_config.c_str()) != 0) {
|
| 863 |
printf("Failed to initialize ZipVoice engine\n");
|
| 864 |
return -1;
|
| 865 |
}
|
|
|
|
| 871 |
double t_vocoder_load = get_current_time_ms();
|
| 872 |
Vocoder::Config vcfg;
|
| 873 |
vcfg.model_path = vocoder_model;
|
| 874 |
+
vcfg.axclConfig = axcl_config.c_str();
|
| 875 |
if (vocoder.Init(vcfg) != 0) {
|
| 876 |
printf("ERROR: Failed to load vocoder model\n");
|
| 877 |
return -1;
|
run_ax650.sh
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# ZipVoice AXERA 板端一键运行
|
| 3 |
+
# 用法: bash run_ax650.sh [standard|distill] [zh|en] [sentence|paragraph]
|
| 4 |
+
|
| 5 |
+
MODEL="${1:-distill}"
|
| 6 |
+
LANG="${2:-zh}"
|
| 7 |
+
MODE="${3:-sentence}"
|
| 8 |
+
|
| 9 |
+
BIN="cpp/install/ax650/zipvoice"
|
| 10 |
+
TOKEN="./resources/zipvoice_hf/zipvoice/tokens.txt"
|
| 11 |
+
VOCODER="./cpp/vocoder/vocos_full.axmodel"
|
| 12 |
+
|
| 13 |
+
cd "$(dirname "$0")"
|
| 14 |
+
|
| 15 |
+
if [ "$MODEL" = "distill" ]; then
|
| 16 |
+
MODEL_DIR="./models/zipvoice_distill_ax650"
|
| 17 |
+
else
|
| 18 |
+
MODEL_DIR="./models/zipvoice_ax650"
|
| 19 |
+
fi
|
| 20 |
+
|
| 21 |
+
if [ "$LANG" = "en" ]; then
|
| 22 |
+
PWAV="./assets/moss_prompts/en_4_4p5s.wav"
|
| 23 |
+
PTEXT="This is almost twice the current industry production level per train."
|
| 24 |
+
DTEXT="This morning, a small train left the station, carrying sleepy passengers toward a bright coastal town."
|
| 25 |
+
TFILE="./assets/paragraphs/en_scavenger.txt"
|
| 26 |
+
REPO="--repo-dir ."
|
| 27 |
+
else
|
| 28 |
+
PWAV="./assets/moss_prompts/zh_1_4p5s.wav"
|
| 29 |
+
PTEXT="不管怎么样我和汤姆还是要感谢贝尔卡金的援手"
|
| 30 |
+
DTEXT="今天午后天气很好，我打开窗户，听见远处有人聊天，水杯也轻轻晃了一下。"
|
| 31 |
+
TFILE="./assets/paragraphs/zh_ginkgo.txt"
|
| 32 |
+
REPO=""
|
| 33 |
+
fi
|
| 34 |
+
|
| 35 |
+
OUT="output_${LANG}_${MODEL}_${MODE}.wav"
|
| 36 |
+
|
| 37 |
+
echo "=========================================="
|
| 38 |
+
echo " ZipVoice AX650"
|
| 39 |
+
echo "=========================================="
|
| 40 |
+
echo " MODEL: $MODEL_DIR"
|
| 41 |
+
echo " LANG: $LANG MODE: $MODE"
|
| 42 |
+
echo " OUTPUT: $OUT"
|
| 43 |
+
echo "=========================================="
|
| 44 |
+
|
| 45 |
+
if [ "$MODE" = "paragraph" ]; then
|
| 46 |
+
exec $BIN \
|
| 47 |
+
--model-dir "$MODEL_DIR" --token-file "$TOKEN" \
|
| 48 |
+
--prompt-wav "$PWAV" --prompt-text "$PTEXT" \
|
| 49 |
+
--text-file "$TFILE" \
|
| 50 |
+
--vocoder-model "$VOCODER" --output-wav "$OUT" --seed 42 $REPO
|
| 51 |
+
else
|
| 52 |
+
exec $BIN \
|
| 53 |
+
--model-dir "$MODEL_DIR" --token-file "$TOKEN" \
|
| 54 |
+
--prompt-wav "$PWAV" --prompt-text "$PTEXT" \
|
| 55 |
+
--text "$DTEXT" \
|
| 56 |
+
--vocoder-model "$VOCODER" --output-wav "$OUT" --seed 42 $REPO
|
| 57 |
+
fi
|
run_axcl.sh
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# ZipVoice AXCL 算力卡一键运行
|
| 3 |
+
# 用法: bash run_axcl.sh [standard|distill] [zh|en] [sentence|paragraph]
|
| 4 |
+
|
| 5 |
+
MODEL="${1:-distill}"
|
| 6 |
+
LANG="${2:-zh}"
|
| 7 |
+
MODE="${3:-sentence}"
|
| 8 |
+
|
| 9 |
+
BIN="cpp/install/axcl/zipvoice"
|
| 10 |
+
TOKEN="./resources/zipvoice_hf/zipvoice/tokens.txt"
|
| 11 |
+
VOCODER="./cpp/vocoder/vocos_full.axmodel"
|
| 12 |
+
|
| 13 |
+
cd "$(dirname "$0")"
|
| 14 |
+
|
| 15 |
+
if [ "$MODEL" = "distill" ]; then
|
| 16 |
+
MODEL_DIR="./models/zipvoice_distill_ax650"
|
| 17 |
+
else
|
| 18 |
+
MODEL_DIR="./models/zipvoice_ax650"
|
| 19 |
+
fi
|
| 20 |
+
|
| 21 |
+
if [ "$LANG" = "en" ]; then
|
| 22 |
+
PWAV="./assets/moss_prompts/en_4_4p5s.wav"
|
| 23 |
+
PTEXT="This is almost twice the current industry production level per train."
|
| 24 |
+
DTEXT="This morning, a small train left the station, carrying sleepy passengers toward a bright coastal town."
|
| 25 |
+
TFILE="./assets/paragraphs/en_scavenger.txt"
|
| 26 |
+
REPO="--repo-dir ."
|
| 27 |
+
else
|
| 28 |
+
PWAV="./assets/moss_prompts/zh_1_4p5s.wav"
|
| 29 |
+
PTEXT="不管怎么样我和汤姆还是要感谢贝尔卡金的援手"
|
| 30 |
+
DTEXT="今天午后天气很好，我打开窗户，听见远处有人聊天，水杯也轻轻晃了一下。"
|
| 31 |
+
TFILE="./assets/paragraphs/zh_ginkgo.txt"
|
| 32 |
+
REPO=""
|
| 33 |
+
fi
|
| 34 |
+
|
| 35 |
+
OUT="output_${LANG}_${MODEL}_${MODE}.wav"
|
| 36 |
+
|
| 37 |
+
echo "=========================================="
|
| 38 |
+
echo " ZipVoice AXCL"
|
| 39 |
+
echo "=========================================="
|
| 40 |
+
echo " MODEL: $MODEL_DIR"
|
| 41 |
+
echo " LANG: $LANG MODE: $MODE"
|
| 42 |
+
echo " OUTPUT: $OUT"
|
| 43 |
+
echo "=========================================="
|
| 44 |
+
|
| 45 |
+
if [ "$MODE" = "paragraph" ]; then
|
| 46 |
+
exec $BIN \
|
| 47 |
+
--model-dir "$MODEL_DIR" --token-file "$TOKEN" \
|
| 48 |
+
--prompt-wav "$PWAV" --prompt-text "$PTEXT" \
|
| 49 |
+
--text-file "$TFILE" \
|
| 50 |
+
--vocoder-model "$VOCODER" --output-wav "$OUT" --seed 42 $REPO
|
| 51 |
+
else
|
| 52 |
+
exec $BIN \
|
| 53 |
+
--model-dir "$MODEL_DIR" --token-file "$TOKEN" \
|
| 54 |
+
--prompt-wav "$PWAV" --prompt-text "$PTEXT" \
|
| 55 |
+
--text "$DTEXT" \
|
| 56 |
+
--vocoder-model "$VOCODER" --output-wav "$OUT" --seed 42 $REPO
|
| 57 |
+
fi
|
scripts/__pycache__/__init__.cpython-310.pyc
ADDED
|
Binary file (155 Bytes). View file
|
|
|
scripts/__pycache__/local_tokenizer.cpython-310.pyc
ADDED
|
Binary file (7.28 kB). View file
|
|
|
scripts/__pycache__/text_processing.cpython-310.pyc
ADDED
|
Binary file (8.46 kB). View file
|
|
|