it's sad that nsys and ncu can't be used on these cards
==ERROR== ERR_NVCMPGPU - Profiling is not supported on the NVIDIA Crypto Mining Processors (CMP) of the target device 0. For more information, please visit https://developer.nvidia.com/ERR_NVCMPGPU

but to add more color to the above results:
GPU utilization during inference — nvidia-smi dmon -s u while running the above benchmarks

Regime


MoltenVK, running in FP16 mode on FP32 only chip


GPU supports coopmat but pp512 is faster with it turned off