This article provides an overview of how to use the NPU. After a user has trained an NN model, it needs to be converted into a model that the NPU can recognize before it can be used in the C3V Linux environment.
Software Introduction
We will use the following NPU reference software:
NPU Toolkit:
The NPU Toolkit translates model formats, optimizes networks, and performs model training, quantization, and inference until final integrations with devices (NPUs). The NPU Toolkit provides both binary and Python versions.
Document please refer to: Vivante.Programming.ACUITY.Toolkit.User.Guide.pdf
NPU IDE:
NPU IDE provides many powerful features for every stage of software development and enables programmers to efficiently accomplish tasks of editing, building, and debugging software on an NPU simulator.
The NPU IDE software provides both X86 Windows 64-bit and X86 Linux 64-bit versions. The NPU IDE can debug and profile the above OpenVX application via the Windows interface.
Document please refer to: Vivante_IDE_User_Guide.pdf
NN SDK LIB:
The NN SDK C library is in the NPU accelerator platform to run NBG NN applications exported from an NPU Toolkit. The NN SDK C library files provided in the C3V Linux platform would install the NN SDK library and NPU driver by default.
Document please refer to: https://github.com/VeriSilicon/TIM-VX/blob/main/docs/Programming_Guide.md
Environment Preparation
Compile the C3V Linux environment that supports the NPU.
Please first refer to the document https://sunplus.atlassian.net/wiki/x/CYCMdw to compile the C3V runtime environment that supports the NPU.
Set up the working environment for model conversion.
In the NPU series of documents, Refer to the documents Acuity Toolkit Environment and VivanteIDE Install to set up the working environment for model conversion.
Development Steps
The entire process can be divided into the following steps:
Convert the trained model into a model that can run on the NPU.
The output file types of the NN application are .c, .h, and .export.data. Or, .c, .h, and .nbg, if packing binary graph is enabled. (pack_nbg_unify=True).
In the development step, we need to enable binary graph to generate a .nbg file which is working on the C3V Linux. Please refer to the document NN model conversion to complete the model conversion.
Use NPU IDE to analyze the performance and accuracy of the Acuity model via the above NN application (Optional).
Compile an application that can run in the C3V Linux environment.
Please follow the document Run NN Convert Model on C3V Linux to use NN SDK LIB to run the NN model via the application in the C3V Linux platform.