This document will provide a detailed description of:
How to convert the YOLOV5 ONNX model into a model for use on the C3V platform
Write sample code for object detection based on YOLOV5
Execute object detection program and obtain recognition results in the C3V Linux environment
The tool versions involved in the current document are as follows:
VIP9000 NPU Kernel Driver | v6.4.15.9 |
Acuity Toolkit | 6.21.1 |
ViviantelIDE | 5.8.2 |
1. Model Conversation
Before the conversion, it is necessary to first set up the environment for model conversion. Please refer to the following document to prepare the environment：NN model conversion
1.1. Project Preparation
Create Model folder
Create a folder yolov5s in path ~/c3v/Models. Please make sure the folder name is as same as the ONNX file name.
~/c3v/Models$ mkdir yolov5s && cd yolov5s
Copy the ONNX file and input.jpg which resolution is 640x640 to the folder yolov5s. These two files will be used as input files during model conversion.
~/c3v/Models$ cp yolov5s.onnx yolov5s/ ~/c3v/Models$ cp input.jpg yolov5s/
Create a dataset.txt file, the content of dataset.txt is the input.jpg file name.
./input.jpg
Create inputs_outputs.txt file and get the information from yolov5s.onnx via netron tool/webpage. Here is the onnx file:
.
write --input-size-list and --outputs informations to inputs_outputs.txt:
--outputs 'onnx::Sigmoid_375 onnx::Sigmoid_419 onnx::Sigmoid_462'
After completing the above steps, there will be the following files under the yolov5s path：
1.2. Implementing
Using shell script tools to convert the model from ONNX to the NB file. There are 4 steps: import quantize inference and export. Tools are in ~/c3v/Models:
pegasus_import.sh
pegasus_quantize.sh
pegasus_inference.sh
pegasus_export_ovx.sh
Import
Execute the command in the console or terminal, and wait for it to complete. It will import and translate an NN model to ACUITY formats.
./pegasus_import.sh yolov5s
Wait until the tool execution is complete and check there are no errors like this:
Then we will see the following four files added under the folder ~/c3v/Models/yolov5s.
Quantize
Modify the scale value(1/255=0.003921569) of the yolov5s_inputmeta.yml file, which is in ~/c3v/Models/yolov5s.
Select one quantized type for your need, such as uint8 / int16 / bf16 / pcq. In this sample we use uint8.
./pegasus_quantize.sh yolov5s uint8
Wait until the tool execution is complete and check there are no errors like this:
Then we will see the following four files added under the folder ~/c3v/Models/yolov5s.
Inference
Inference the ACUITY model with the quantization data type.
./pegasus_inference.sh yolov5s uint8
Wait until the tool execution is complete and check there are no errors like this:
Export
Export the quantized application for device deployment. Please modify the pegasus_export_ovx.sh for the nb file generating, and add both 3 lines marked in the red box.
./pegasus_export_ovx.sh yolov5s uint8
Wait until the tool execution is complete and check there are no errors like this:
In the path ~/c3v/Models/yolov5s/wksp, you will find a folder named yolov5s_uint8_nbg_unify.
We can get the nb file and a c file for NN graph setup information.
2. Object Detection Program
2.1. Post Processing
The post-processing of the example code automatically transferred out by the tool will print the top 5. We need to increase the parsing of the results to obtain complete results of target recognition. The relevant post-processing functions are located in the file vnn_post_process.c.
We provide an example function for post-processing, which can complete the parsing of NN processing results：
post_proc_init
post_proc_process
post_proc_deinit
The function needs to be modified：vnn_PostProcessYolov5Uint8
vsi_status vnn_PostProcessYolov5sUint8(vsi_nn_graph_t *graph)
{
vsi_status status = VSI_FAILURE;
#if DETECT_RESULT_IMPL
/*detect result sample implement*/
post_proc_init(graph);
post_proc_process(graph);
post_proc_deinit();
#else
/* Show the top5 result */
status = show_top5(graph, vsi_nn_GetTensor(graph, graph->output.tensors[0]));
TEST_CHECK_STATUS(status, final);
/* Save all output tensor data to txt file */
save_output_data(graph);
final:
#endif
return VSI_SUCCESS;
}
For detailed function implementation, please refer to the following file:
2.2. Linux Terminal Cross Compile
When compiling NN-related applications, it is necessary to include SDK's headers and libraries.
Example of SDK Includes Path：
INCLUDES+=-I$(VIVANTE_SDK_DIR)/include/ \ -I$(VIVANTE_SDK_DIR)/include/CL \ -I$(VIVANTE_SDK_DIR)/include/VX \ -I$(VIVANTE_SDK_DIR)/include/ovxlib \ -I$(VIVANTE_SDK_DIR)/include/jpeg
Example of SDK Link Libraries：
LIBS+=-lOpenVX -lOpenVXU -lCLC -lVSC -lGAL -ljpeg -lovxlib
This is an example Makefile that just needs to be placed in ~/c3v/Models/yolov5s/wksp/yolov5s_uint8_nbg_unify Folder. And set the relevant VIVIANTE_ SDK_ DIR and TOOLCHAIN can complete the compilation of the app:
BIN=sampleApp
VIVANTE_SDK_DIR=Path to VIVANTE SDK directory
TOOLCHAIN=Path to toolchain directory
VIVANTE_SDK_INC=$(VIVANTE_SDK_DIR)/include
VIVANTE_SDK_LIB=$(VIVANTE_SDK_DIR)/lib
CROSS_COMPILE=$(TOOLCHAIN)/aarch64-none-linux-gnu-
CC=$(CROSS_COMPILE)gcc
CXX=$(CROSS_COMPILE)g++
CFLAGS=-Wall -O3
INCLUDE += -I$(VIVANTE_SDK_INC) -I$(VIVANTE_SDK_INC)/HAL -I$(VIVANTE_SDK_INC)/ovxlib
LIBS += -L$(VIVANTE_SDK_LIB) -L./ -L$(STD_LOG_INC)
LIBS += -lOpenVX -lOpenVXU -lOpenVX -lCLC -lVSC -lGAL -ljpeg -lovxlib -lm
LIBS += -lNNArchPerf -lArchModelSw
LIBS += -lstdc++ -ldl -lpthread -lgcc_s
CFLAGS += $(INCLUDE) -fPIC
SRCS=${wildcard *.c}
SRCS+=${wildcard *.cpp}
OBJS=$(addsuffix .o, $(basename $(SRCS)))
.SUFFIXES: .hpp .cpp .c
.cpp.o:
$(CXX) $(CFLAGS) -std=c++11 -c $<
.c.o:
$(CC) $(CFLAGS) -c $<
all: $(BIN)
$(BIN): $(OBJS)
$(CC) $(CFLAGS) $(LFLAGS) $(OBJS) -o $@ $(LIBS)
rm -rf *.o
clean:
rm -rf *.o
rm -rf $(BIN) $(LIB)
rm -rf *~
3. Running on the C3V Linux
Insmod to kernel
insmod ./galcore.ko [14358.019373] galcore f8140000.galcore: NPU get power success [14358.019458] galcore f8140000.galcore: galcore irq number is 44 [14358.020542] galcore f8140000.galcore: NPU clock: 900000000 [14358.026015] Galcore version 6.4.15.9.700103
Copy the application and related libraries into C3V Linux and run:
./yolov5suint8 ./network_binary.nb ./input.jpg
The result is like this:
/mnt/yolov5s_uint8_nbg_unify# ./yolov5suint8 ./network_binary.nb input.jpg Create Neural Network: 30ms or 30801us Verify... Verify Graph: 23ms or 23647us Start run graph [1] times... Run the 1 time: 43.14ms or 43144.08us vxProcessGraph execution time: Total 43.28ms or 43277.80us Average 43.28ms or 43277.80us obj: L: 0 P:0.88, [(302, 266) - (200, 378)] obj: L: 0 P:0.83, [(-3, 55) - (221, 582)] obj: L:15 P:0.39, [(1, 520) - (55, 117)] obj: L: 0 P:0.39, [(549, 259) - (92, 372)] obj: L: 0 P:0.31, [(348, 166) - (181, 277)]