/
V1.1.0 of SNNF

V1.1.0 of SNNF

Owned by 杨彬
Last updated: Jan 15, 2025
7 min read

This is the formal release V1.1.0 of SNNF(Sunplus Neural Network Framework).

Target of the V1.1.0

On the basis of v1.0.0, v1.1.0 mainly has the following updates：

  1. Add some NN module.

    • genderage: gender and age

      • Det_10g + genderage

      • LightFace + genderage

  2. Based on the current main framework of SNNF, provide a porting tool so that users can easily add a new model to the Customized zone using the official generated nbg code as the source. The initial version of this tool is currently available, and there will be further updates in the next version.

  3. Fix some known issues, such as:

    • OCR have a probability of crashing.

    • HumanAttr have a probability of crashing.

    • When multiple models are connected in series, if the last model comes out, the result may have an unfinished flag fin=false.

  4. Standardize post-process code.

  5. Organize samples and unittest codes:

    • samples is a sample code for snnf command sample which is just for snnf_run.sh.

    • unittest/modelTest is just for a buildin model's test sample.

    • unittest/pluginTest is just for the plugin model's test sample.

    • unittest/sequntialTest is the sample for the pipeline in which several models are sequentially linked together in series.

  6. Add NN filter, which can facilitate users to do their own private data processing at the application layer. Users only need to insert the filter directly into the corresponding position of the pipeline when creating the model flow, and then perform private and correct processing at the most appropriate location. The advantage of this is that there is no need to write a dedicated module within SNNF for this private processing, only the official interface of SNNF needs to be called. Of course, SNNF provides some commonly used private processing filters internally, such as label filtering. For example, when Object Detect+HumanAttr, the label results of Object Detect can be filtered, and only the person can be passed to HumanAttr. If the requirements of the application layer happen to match this usage, there is no need to implement it yourself at the application layer. Simply call the methods provided by SNNF.

  7. Provide users with a mAP test sample to compare the precision of the C3V platform's model running results with the precision of the model's official PC running results. The default test model is yolov8s object detection. Users can refer to this mAP sample to rewrite the model they need to test mAP values for. Please refer the SOP.

  8. Improve the toString() function for each model's result, and support the detection result toString JSON format.

  9. Update some make config.

    • In addition to cross compilation on the server, we also provide compilation methods on the C3V platform.

      • If it is cross compilation on the server, please confirm the config CROSS_COMPILE := ON in makefile_config.mk .

      • If compilation is required on the C3V platform, please confirm the config CROSS_COMPILE := OFF in makefile_config.mk .

    • Users can choose the model plugin in makefile_config.mk that needs to be compiled according to their own needs, which can effectively reduce the code size of specific projects and shorten compilation time.

# ------------------------------------------------------------------------ # define which PLUGIN to build NN_PLUGIN_ENABLE_YOLOV5S_V2 := yes NN_PLUGIN_ENABLE_RTMDETS := no NN_PLUGIN_ENABLE_GEDERAGE := no NN_PLUGIN_ENABLE_YOLOV8S_DETECTION := no NN_PLUGIN_ENABLE_YOLOV8S_OBB := no NN_PLUGIN_ENABLE_YOLOV8S_POSE := no NN_PLUGIN_ENABLE_YOLOV8S_SEGMENT := no NN_PLUGIN_ENABLE_YOLOV10S_DETECTION := no NN_PLUGIN_ENABLE_YOLOV8N_CCPD := no NN_PLUGIN_ENABLE_YOLOV8N_OBB_OPTI := no NN_PLUGIN_ENABLE_YOLOV8N_POSE_OPTI := no NN_PLUGIN_ENABLE_YOLOV8N_SEGMENT_OPTI := no NN_PLUGIN_ENABLE_YOLOV8N_DETECTION_OPTI := no NN_PLUGIN_ENABLE_YOLOV8S_DETECTION_OPTI := no NN_PLUGIN_ENABLE_BOTSORT_TRACK := no

  1. Adjust the access of certain model header files and provide more necessary header files for the application layer to use.

  2. Change the bbox value type from uint to float. Although this change does not have a substantial impact on the rendering of the results, it can make the precision value more accurate when calculating mAP.

  3. In previous versions, assist modules such as imageWriter/videoWriter/JsonWriter only supported the default filename. Starting from this version, we will support filename config.

  4. Also provide release folder so that new users can easily preview or evaluate our SNNF demo without spending too much time on environment setup and code compilation.

Resource

Please get the V1.1.0 release resource here.

Before starting any work, please carefully read the instruction files such as readme.md in the document directory.

Usage of V1.1.0

How to verify Official Demos

You can use the script we provide to start as follows：

/SNNF/release # ./snnf_run.sh Usage: ./bin/snnf_nnsample [-m|-s|-a|-h] [-i|-v|-o|option] Version: 1.1.0_ Time: [-m,--model <model>] run a single model <model>:Age Det10g HumanAttr HumanFilter LightFace OcrCls OcrDet OcrRec VehicleAttr VehicleFilter W600kR50 YoloV8sOdMap Yolov5sDetection Yolov5sV1 Yolov8nClassify Yolov8sClassify stgcn Yolov5sV2 BotSortTrackStgcn GenderAge Rtmdets YoloV10sDetection YoloV8nCcpdOpti YoloV8nDetectionBaseOpti YoloV8nDetectionOpti YoloV8nObbOpti YoloV8nPoseOpti YoloV8nSegmentOpti YoloV8sDetection YoloV8sDetectionBaseOpti YoloV8sDetectionOpti YoloV8sObb YoloV8sPose YoloV8sSegment example:./bin/snnf_nnsample -m Yolov5sDetection ./bin/snnf_nnsample --model HumanAttr [-s,--sequential <model1,model2,...>] run sequential models <models>:Yolov5sDetection,HumanFilter,HumanAttr LightFace,Age OcrDet,OcrRec OcrDet,OcrCls,OcrRec YoloV8nCcpdOpti,OcrRec Det10g,W600kR50 YoloV8sDetection,VehicleFilter,VehicleAttr YoloV8nDetectionOpti,BotSortTrack YoloV8nPoseOpti,BotSortTrackStgcn YoloV8nCcpdOpti,OcrRec YoloV8nCcpdOpti,OcrCls,OcrRec example:./bin/snnf_nnsample -s Yolov5sDetection,HumanFilter,HumanAttr ./bin/snnf_nnsample --sequential ocrDet,ocrCls,ocrRec ./bin/snnf_nnsample -s YoloV8nCcpdOpti,OcrRec,imageWriter ./bin/snnf_nnsample -s YoloV8sDetectionOpti,BotSortTrack,videoWriter -v resource/video/pk-12.mp4 ./bin/snnf_nnsample -s YoloV8nPoseOpti,BotSortTrackStgcn,videoWriter -v resource/video/person-falling.mp4 [-i,--image file] set image file to nn detection. <file>: file name [-c | option]: test count, this parameter is only match with -i example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr -i filename -c testCount ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr --image filename -c testCount [-v,--video file] set video file to nn detection. <file>: file name example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr -v filename ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr --video filename [-o,--output file] specify the output file name for saving results. <file>: file name with extension (e.g., output.jpg, output.json, output.mp4) This parameter must be used in conjunction with imageWriter, jsonWriter, or videoWriter. example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,imageWriter -i filename -o output.jpg ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,jsonWriter -i filename -o output.json ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,videoWriter -v filename -o output.mp4 [-a,--all] run all model testing assist tools: imageWriter videoWriter jsonWriter BotSortTrack HumanFilter VehicleFilter

Release folder structure

image-20250107-081448.png

  • bin: snnf_nnsample. Prebuild sample programs that can run on the c3v Linux platform.

  • include：header file of sunplus NN framework SDK.

  • lib：libraries of sunplus NN framework SDK.

  • resource

    • config: some config files for features.

    • font: ttf file for plotting sample.

    • image: image files used for test.

    • model: models to be used in the sample program.

    • video: video files used for test.

  • samples：example code for using sunplus NN framework.

  • snnf_run.sh：executable script for running sample code.

  • snnf_build_samples.sh: executable script for building sample code.

  • snnf_env.sh: executable script for compiling environment.

  • thirdparty: just as its name implies.

How to run SNNF sample

  1. Copy the release foler to C3V Linux.

/SNNF/release # ls -alh total 44K drwxr-xr-x 8 101065 11400 4.0K Jan 3 2025 . drwxr-xr-x 5 101065 11400 4.0K Jan 3 2025 .. drwxr-xr-x 2 101065 11400 4.0K Jan 3 2025 bin drwxr-xr-x 6 101065 11400 4.0K Jan 3 2025 include drwxr-xr-x 3 101065 11400 4.0K Jan 3 2025 lib drwxr-xr-x 7 101065 11400 4.0K Jan 3 2025 resource drwxr-xr-x 4 101065 11400 4.0K Jan 3 2025 samples -rwxr-xr-x 1 101065 11400 223 Jan 3 2025 snnf_build_samples.sh -rwxr-xr-x 1 101065 11400 1.6K Jan 3 2025 snnf_env.sh -rwxr-xr-x 1 101065 11400 430 Jan 3 2025 snnf_run.sh drwxr-xr-x 7 101065 11400 4.0K Jan 3 2025 thirdparty

  1. Run nnf_run.sh to run the SNNF sample.

a. One-time input

./snnf_run.sh -m YoloV8sDetection

b. Read input from the image file

./snnf_run.sh -m YoloV8nDetectionOpti -i resource/image/person640x640.jpg

c. Read inputs from the video file.

./snnf_run.sh -m YoloV8nDetectionOpti -v resource/video/humanCount.mp4

d. Sequential models

./snnf_run.sh -s Yolov5sDetection,HumanAttr

e. Model inference results save to image.

./snnf_run.sh -s YoloV8sPose,imageWriter

Results will save to the image detected_1883_0931_1735827676535.jpg.

f. Model inference results save to json file.

./snnf_run.sh -s YoloV8sPose,jsonWriter -o yolov8PoseResults.json

Results will save to yolov8PoseResults.json. If the - o option is unused, it will be saved as a file by default: default_result.json.

How to build SNNF

  1. Cross-compile for C3V environment.

a. Please use snnf_build.sh for SNNF compiling.

b. All the resource will be installed to the release folder.

  1. Copy the release folder to the C3V platform.

  2. Setup environment variable.

a. Setting environment variables independently.

b. Run snnf_run.sh will auto set environment variables.

  1. Then, you can run snnf_run.sh for SNNF sample.

Models reference

Model Name

Version or Path

genderage

GitHub - deepinsight/insightface: State-of-the-art 2D and 3D Face Analysis Project

User API

Please refer to API DOC v2.0 .