This article provides an overview of how to use the NPU. After a user has trained an NN model, it needs to be converted into a model that the NPU can recognize before it can be used in the C3V Linux environment.
Software Introduction
We will use the following Vivante software:
Vivante ACUITY Toolkit:
The Vivante ACUITY™ Toolkit serves to translate model formats, optimize networks, and perform model training, quantization, and inference until final integrations with devices (Vivante NPUs). The ACUITY Toolkit provides both binary and Python versions.
Document please refer to: Vivante.Programming.ACUITY.Toolkit.User.Guide.pdf
Vivante IDE:
VivanteIDE provides many powerful features for every stage of software development and enables programmers to efficiently accomplish tasks of editing, building, and debugging software on a Vivante simulator.
The Vivante IDE software provides both X86 Windows 64-bit and X86 Linux 64-bit versions. The Vivante IDE can debug and profile the above OpenVX application via the Windows interface.
Document please refer to: Vivante_IDE_User_Guide.pdf
Vivante OVXLIB:
The Vivante OVXLIB C library is in the NPU accelerator platform to run NBG OpenVX applications exported from an ACUITY model. The OVXLIB C library files provided in the C3V Linux platform would install OVXLIB and OpenVX driver by default.
Document please refer to: https://github.com/VeriSilicon/TIM-VX/blob/main/docs/Programming_Guide.md
Environment Preparation
Compile the C3V Linux environment that supports the NPU.
Please first refer to the document https://sunplus.atlassian.net/wiki/x/CYCMdw to compile the C3V runtime environment that supports the NPU.
Set up the working environment for model conversion.
In the NPU series of documents, Refer to the documents Acuity Toolkit Environment and VivanteIDE Install to set up the working environment for model conversion.
Development Steps
The entire process can be divided into the following steps:
Convert the trained model into a model that can run on the NPU.
The output file types of the OpenVX application are .c, .h, and .export.data. Or, .c, .h, and .nbg, if packing binary graph is enabled. (pack_nbg_unify=True).
In the development step, we need to enable binary graph to generate a .nbg file which is working on the C3V Linux. Please refer to the document NN model conversion to complete the model conversion.
Use Vivante IDE to analyze the performance and accuracy of the Acuity model via the above OpenVX application (Optional).
Compile an application that can run in the C3V Linux environment.
Please follow the document Run NN Convert Model on C3V Linux to use Vivante OVXLIB to run the Acuity model via the application in the C3V Linux platform.