This document will provide a detailed description of:
How to convert the YOLOV8 ONNX model into a model for use on the C3V platform
Write sample code for object detection based on YOLOV8
Execute object detection program and obtain recognition results in the C3V Linux environment
The tool versions involved in the current document are as follows:
NPU Kernel Driver | v6.4.15.9 |
Acuity Toolkit | 6.21.1 |
ViviantelIDE | 5.8.2 |
1. Model Conversation
Before the conversion, it is necessary to first set up the environment for model conversion. Please refer to the following document to prepare the environment：NN Model Conversion
1.1. Project Preparation
Create Model folder
Create a folder yolov8s in path ~/c3v/Models. Please ensure the folder name is the same as the ONNX file name.
~/c3v/Models$ mkdir yolov8s && cd yolov8s
Copy the ONNX file and input.jpg which resolution is 640x640 to the folder yolov8s. These two files will be used as input files during model conversion.
~/c3v/Models$ cp yolov8s.onnx yolov8s/ ~/c3v/Models$ cp input.jpg yolov8s/
Create a dataset.txt file, the content of dataset.txt is the input.jpg file name.
./input.jpg
Create inputs_outputs.txt file and get the information from yolov8s.onnx via netron tool/webpage. Here is the onnx file:
.
Select the three operators within the red box as the output. write --input-size-list and --outputs informations to inputs_outputs.txt:
--outputs '/model.22/Sigmoid_output_0 /model.22/Mul_2_output_0'
After completing the above steps, there will be the following files under the yolov8s path：
1.2. Implementing
Using shell script tools to convert the model from ONNX to the NB file. There are 4 steps: import quantize inference and export. Tools are in ~/c3v/Models:
pegasus_import.sh
pegasus_quantize.sh
pegasus_inference.sh
pegasus_export_ovx.sh
Import
Execute the command in the console or terminal, and wait for it to complete. It will import and translate an NN model to NN formats.
./pegasus_import.sh yolov8s
Wait until the tool execution is complete and check there are no errors like this:
Then we will see the following four files added under the folder ~/c3v/Models/yolov8s.
Quantize
Modify the scale value(1/255=0.003921569) of the yolov8s_inputmeta.yml file, which is in ~/c3v/Models/yolov8s.
Select one quantized type for your need, such as uint8 / int16 / bf16 / pcq. In this sample we use uint8.
./pegasus_quantize.sh yolov8s uint8
Wait until the tool execution is complete and check there are no errors like this:
Then we will see the following four files added under the folder ~/c3v/Models/yolov8s.
Inference
Inference the NN model with the quantization data type.
./pegasus_inference.sh yolov8s uint8
Wait until the tool execution is complete and check there are no errors like this:
Export
Export the quantized application for device deployment. Please modify the pegasus_export_ovx.sh for the nb file generating, and add both 3 lines marked in the red box.
./pegasus_export_ovx.sh yolov8s uint8
Wait until the tool execution is complete and check there are no errors like this:
In the path ~/c3v/Models/yolov8s/wksp, you will find a folder named yolov8s_uint8_nbg_unify.
We can get the nb file and a c file for NN graph setup information.
2. Object Detection Program
2.1. Post Processing
The post-processing of the example code automatically transferred out by the tool will print the top 5. We need to increase the parsing of the results to obtain complete results of target recognition. The relevant post-processing functions are located in the file vnn_post_process.c.
We provide an example function for post-processing, which can complete the parsing of NN processing results：
post_proc_init
post_proc_process
post_proc_deinit
The function needs to be modified：vnn_PostProcessYolov8Uint8
vsi_status vnn_PostProcessYolov8sUint8(vsi_nn_graph_t *graph)
{
vsi_status status = VSI_FAILURE;
#if DETECT_RESULT_IMPL
/*detect result sample implement*/
post_proc_init(graph);
post_proc_process(graph);
post_proc_deinit();
#else
/* Show the top5 result */
status = show_top5(graph, vsi_nn_GetTensor(graph, graph->output.tensors[0]));
TEST_CHECK_STATUS(status, final);
/* Save all output tensor data to txt file */
save_output_data(graph);
final:
#endif
return VSI_SUCCESS;
}
For detailed function implementation, please refer to the following file:
2.2. Program Compile
When compiling NN-related applications, SDK's headers and libraries must be included.
Example of SDK Includes Path：
INCLUDES+=-I$(NN_SDK_DIR)/include/ \ -I$(NN_SDK_DIR)/include/CL \ -I$(NN_SDK_DIR)/include/VX \ -I$(NN_SDK_DIR)/include/ovxlib \ -I$(NN_SDK_DIR)/include/jpeg
Example of SDK Link Libraries：
LIBS+=-lOpenVX -lOpenVXU -lCLC -lVSC -lGAL -ljpeg -lovxlib
3. Example flow of the program build and run
Unzipped and
then placed them in ~/c3v/Models/yolov8s-detection/wksp/yolov8s-detection_uint8_nbg_unify Folder. The brief folder of the project is like this:
3.1. build in c3v
If you want to build the project in c3v directly, please modify these contents of Makefile:
BIN=yolov8s-detection-uint8 # 2.build in c3v NN_SDK_DIR=/usr CC=gcc CXX=g++
then copy the whole folder yolov8s-detection_uint8_nbg_unify to the c3v Linux system. Then using make to compile the project.
cd /sample/yolov8s-detection_uint8_nbg_unify make -j
After compilation, you can see the corresponding application program：yolov8s-detection-uint8.
You can run the application directly on c3v:
The param1 is the network_binary.nb file that converts from the acuity toolkit.
The param2 is the image that is for detection. Please prepare the image file which format is jpg and the pixel size is 640 * 640.
./yolov8s-detection-uint8 ./network_binary.nb ./input.jpg
The result is like this:
/mnt/yolov8s-detection_uint8_nbg_unify # ./yolov8s-detection-uint8 ./network_binary.nb ./input.jpg Create Neural Network: 28ms or 28375us Verify... Verify Graph: 21ms or 21116us Start run graph [1] times... Run the 1 time: 57.55ms or 57548.24us vxProcessGraph execution time: Total 58.05ms or 58053.36us Average 58.05ms or 58053.36us obj: L: 0 P:0.93, [(0, 42) - (200, 599)] obj: L: 0 P:0.91, [(309, 279) - (180, 361)] obj: L: 0 P:0.58, [(344, 171) - (170, 301)]
3.2. cross-compile in Linux
If you want to build the project in host Linux, please modify these contents of Makefile:
BIN=yolov8s-detection-uint8 # 1.cross compile NN_SDK_DIR=Path to NN SDK directory TOOLCHAIN=Path to toolchain directory CROSS_COMPILE=$(TOOLCHAIN)/aarch64-none-linux-gnu- CC=$(CROSS_COMPILE)gcc CXX=$(CROSS_COMPILE)g++
you need to set the right path of NN_SDK_DIR and TOOLCHAIN
NN_SDK_DIR: The path to NPU SDK
TOOLCHAIN: The cross-compile toolchain path. which format may be like this:
TOOLCHAIN=/pub/toolchain/crossgcc/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin
then using make to compile the project.
make
Copy the application, network_binary.nb file and related libraries into C3V Linux and run:
The param1 is the nb file that converts from the acuity toolkit.
The param2 is the image that is for detection. Please prepare the image file which format is jpg and the pixel size is 640 * 640.
./yolov8s-detection-uint8 ./network_binary.nb ./input.jpg
The result is like this:
/mnt/yolov8s-detection_uint8_nbg_unify # ./yolov8s-detection-uint8 ./network_binary.nb ./input.jpg Create Neural Network: 28ms or 28375us Verify... Verify Graph: 21ms or 21116us Start run graph [1] times... Run the 1 time: 57.55ms or 57548.24us vxProcessGraph execution time: Total 58.05ms or 58053.36us Average 58.05ms or 58053.36us obj: L: 0 P:0.93, [(0, 42) - (200, 599)] obj: L: 0 P:0.91, [(309, 279) - (180, 361)] obj: L: 0 P:0.58, [(344, 171) - (170, 301)]