Versions Compared

Version Old Version 1 New Version 2
Changes made by

杨彬

杨彬

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Target of the V1.1.0

This version is the first time, we formal release the V1.1.0 of SNNF(Sunplus Neural Network Framework), the original version is V1.

On the basis of v1.0.0.

There are some release notes for the version which are as below：

  1. There are some NN modules and Samples.

    1. lightFace;

    2. age;

    3. lightFace + age;

    4. det_10g;

    5. det_10g + w600k_r50;

    6. humanAttr;

    7. yolov5s + humanAttr;

    8. vehicleAttr;

    9. yolov8s_detection + vehicleAttr;

    10. yolov8s_detection;

    11. yolov8s_obb;

    12. yolov8s_pose;

    13. yolov8s_segment;

    14. yolov8s_classify;

    15. yolov10s_detection;

    16. yolov8n_obb_opti;

    17. yolov8n_pose_opti;

    18. yolov8n_segment_opti;

    19. yolov8n_detection_opti;

    20. yolov8n_classify;

    21. RTMDet-s;

    22. Human tracking;

    23. Human falling detection;

    24. yolov5s_detection;

    25. OCRDet；

    26. OCRCls；

    27. OCRRec；

    28. OCRDet + OCRRec;

    29. OCRDet + OCRCls + OCRRec;

    30. yolov8n_CCPD;

    31. yolov8n_CCPD + OCRRec;

    32. yolov8n_CCPD + OCRCls + OCRRec.

  2. A command mechanism for model list, you can get the all the available models，combination of models and applications and so on, which showed above.

  3. Provide a sample demo for users to refer to, and they can directly run the sample app to verify the NN environment of C3V.

  4. Parallel operations in multi-model applications can maximize the utilization of NPU and CPU, ultimately maximizing the processing frame rate of the application.

  5. Implement official and customized partitions, you can copy an offical model to customized zone for further customizing, or just put your own model in the customized zone as a new model.

  6. Define some specified paths for some resource, for example, config, font, test image, model file, test video and so on.

  7. Define some specified folders such as botSortTrack, imageWriter, videoWriter for assist.

  8. Provide Libs and header files for users to integrate specific NN modules into their applications using the standard interfaces we provide.

  9. Provide COMPILE SH to enable users to easily compile SNNF and locate generated resources in the specified release directory.

  10. Open source the SNNF to facilitate users to better understand the internal operations of SNNF.

  11. Optimize the system to make it more robust, and after long-term testing, the system is stable.

Resource

Please get the V1.0.0 release resource here.

Before starting any work, please carefully read the instruction files such as readme.md in the document directory.

Usage of V1.0.0

How to verify Official Demos

You can use the script we provide to start as follows：

...

, v1.1.0 mainly has the following updates：

  1. Add some NN module.

    • genderage: gender and age

      • Det_10g + genderage

      • LightFace + genderage

  2. Based on the current main framework of SNNF, provide a porting tool so that users can easily add a new model to the Customized zone using the official generated nbg code as the source. The initial version of this tool is currently available, and there will be further updates in the next version.

  3. Fix some known issues, such as:

  4. OCR have a probability of crashing.

  5. HumanAttr have a probability of crashing.

  6. When multiple models are connected in series, if the last model comes out, the result may have an unfinished flag fin=false.

  7. Standardize post-process code.

  8. Organize samples and unittest codes:

  9. samples is a sample code for snnf command sample which is just for snnf_run.sh.

  10. unittest/modelTest is just for a buildin model's test sample.

  11. unittest/pluginTest is just for the plugin model's test sample.

  12. unittest/sequntialTest is the sample for the pipeline in which several models are sequentially linked together in series.

  13. Add NN filter, which can facilitate users to do their own private data processing at the application layer. Users only need to insert the filter directly into the corresponding position of the pipeline when creating the model flow, and then perform private and correct processing at the most appropriate location. The advantage of this is that there is no need to write a dedicated module within SNNF for this private processing, only the official interface of SNNF needs to be called. Of course, SNNF provides some commonly used private processing filters internally, such as label filtering. For example, when Object Detect+HumanAttr, the label results of Object Detect can be filtered, and only the person can be passed to HumanAttr. If the requirements of the application layer happen to match this usage, there is no need to implement it yourself at the application layer. Simply call the methods provided by SNNF.

  14. Provide users with a mAP test sample to compare the precision of the C3V platform's model running results with the precision of the model's official PC running results. The default test model is yolov8s object detection. Users can refer to this mAP sample to rewrite the model they need to test mAP values for.

  15. Improve the toString() function for each model's result, and support the detection result toString JSON format.

  16. Update some make config.

  17. In addition to cross compilation on the server, we also provide compilation methods on the C3V platform.

  18. If it is cross compilation on the server, please confirm the config CROSS_COMPILE := ON in makefile_config.mk .

  19. If compilation is required on the C3V platform, please confirm the config CROSS_COMPILE := OFF in makefile_config.mk .

  20. Users can choose the model plugin in makefile_config.mk that needs to be compiled according to their own needs, which can effectively reduce the code size of specific projects and shorten compilation time.

Code Block
languageplain
# ------------------------------------------------------------------------
# define which PLUGIN to build
  NN_PLUGIN_ENABLE_YOLOV5S_V2                 := yes

  NN_PLUGIN_ENABLE_RTMDETS                    := no
  NN_PLUGIN_ENABLE_GEDERAGE                   := no
  NN_PLUGIN_ENABLE_YOLOV8S_DETECTION          := no
  NN_PLUGIN_ENABLE_YOLOV8S_OBB                := no
  NN_PLUGIN_ENABLE_YOLOV8S_POSE               := no
  NN_PLUGIN_ENABLE_YOLOV8S_SEGMENT            := no
  NN_PLUGIN_ENABLE_YOLOV10S_DETECTION         := no

  NN_PLUGIN_ENABLE_YOLOV8N_CCPD               := no
  NN_PLUGIN_ENABLE_YOLOV8N_OBB_OPTI           := no
  NN_PLUGIN_ENABLE_YOLOV8N_POSE_OPTI          := no
  NN_PLUGIN_ENABLE_YOLOV8N_SEGMENT_OPTI       := no
  NN_PLUGIN_ENABLE_YOLOV8N_DETECTION_OPTI     := no
  NN_PLUGIN_ENABLE_YOLOV8S_DETECTION_OPTI     := no

  NN_PLUGIN_ENABLE_BOTSORT_TRACK              := no

  1. Adjust the access of certain model header files and provide more necessary header files for the application layer to use.

  2. Change the bbox value type from uint to float. Although this change does not have a substantial impact on the rendering of the results, it can make the precision value more accurate when calculating mAP.

  3. In previous versions, assist modules such as imageWriter/videoWriter/JsonWriter only supported the default filename. Starting from this version, we will support filename config.

  4. Also provide release folder so that new users can easily preview or evaluate our SNNF demo without spending too much time on environment setup and code compilation.

Resource

Please get the V1.1.0 release resource here.

Before starting any work, please carefully read the instruction files such as readme.md in the document directory.

Usage of V1.1.0

How to verify Official Demos

You can use the script we provide to start as follows：

Code Block
/SNNF/release # ./snnf_run.sh
Usage: ./bin/snnf_nnsample [-m|-s|-a|-h] [-i|-v|-o|option]
        Version: 1.1.0_
        Time:
        [-m,--model <model>] run a single model
                <model>:Age                        Det10g                     HumanAttr
                        HumanFilter                LightFace                  OcrCls
                        OcrDet                     OcrRec                     VehicleAttr
                        VehicleFilter              W600kR50                   YoloV8sOdMap
          Version: 1.0.0_         Time:    Yolov5sDetection     [-m,--model <model>] run a single model Yolov5sV1                <model>:Age  Yolov8nClassify
                     Det10g   Yolov8sClassify            stgcn      HumanAttr                Yolov5sV2
        LightFace                BotSortTrackStgcn  OcrCls        GenderAge             OcrDet     Rtmdets
                   OcrRec     YoloV10sDetection          YoloV8nCcpdOpti      VehicleAttr      YoloV8nDetectionBaseOpti
         W600kR50               YoloV8nDetectionOpti       YoloV8nObbOpti   Yolov5sDetection          YoloV8nPoseOpti
Yolov5sV1                  Yolov5sV2      YoloV8nSegmentOpti         YoloV8sDetection          Yolov8nClassify YoloV8sDetectionBaseOpti
          Yolov8sClassify            Yolov5sV3  YoloV8sDetectionOpti       YoloV8sObb                BotSortTrackStgcn YoloV8sPose
        Rtmdets                YoloV8sSegment
   YoloV10sDetection             example:./bin/snnf_nnsample -m Yolov5sDetection
         YoloV8nCcpd                YoloV8nDetectionBaseOpti ./bin/snnf_nnsample --model HumanAttr

YoloV8nDetectionOpti        [-s,--sequential <model1,model2,...>] run sequential models
            YoloV8nObbOpti    <models>:Yolov5sDetection,HumanFilter,HumanAttr
        YoloV8nPoseOpti            YoloV8nSegmentOpti    LightFace,Age
                    YoloV8sDetection    OcrDet,OcrRec
      YoloV8sDetectionBaseOpti   YoloV8sDetectionOpti               OcrDet,OcrCls,OcrRec
         YoloV8sObb               YoloV8nCcpdOpti,OcrRec
 YoloV8sPose                YoloV8sSegment       Det10g,W600kR50
          example:./bin/snnf_nnsample -m Yolov5sDetection               YoloV8sDetection,VehicleFilter,VehicleAttr
            ./bin/snnf_nnsample --model HumanAttr          [-s,--sequential <model1,model2,...>] run sequential modelsYoloV8nDetectionOpti,BotSortTrack
                        <models>:Yolov5sDetectionYoloV8nPoseOpti,HumanAttrBotSortTrackStgcn
                        LightFaceYoloV8nCcpdOpti,AgeOcrRec
                        OcrDetYoloV8nCcpdOpti,OcrCls,OcrRec
                example:./bin/snnf_nnsample        OcrDet,OcrCls,OcrRec-s Yolov5sDetection,HumanFilter,HumanAttr
                        ./bin/snnf_nnsample  YoloV8nCcpd,OcrRec--sequential ocrDet,ocrCls,ocrRec
                        Det10g,W600kR50
 ./bin/snnf_nnsample -s YoloV8nCcpdOpti,OcrRec,imageWriter
                      YoloV8sDetection,VehicleAttr  ./bin/snnf_nnsample -s YoloV8sDetectionOpti,BotSortTrack,videoWriter -v resource/video/humanCount.mp4
                  YoloV8nDetectionOpti,BotSortTrack           ./bin/snnf_nnsample -s YoloV8nPoseOpti,BotSortTrackStgcn,videoWriter -v resource/video/person-falling.mp4

        [-i,--image file] set image YoloV8nPoseOpti,BotSortTrackStgcnfile to nn detection.
                <file>: file name
  YoloV8nCcpd,OcrRec              [-c | option]: test count, this parameter is only match  YoloV8nCcpd,OcrCls,OcrRecwith -i
                 example:./bin/snnf_nnsample_nnsample -s Yolov5sDetection,HumanAttr -i filename -sc Yolov5s,HumanAttrtestCount
                        ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr --sequential ocrDet,ocrCls,ocrRecimage filename -c testCount

        [-v,--video file] set video file to nn detection.
        ./bin/snnf_nnsample -s YoloV8nCcpd,OcrRec,imageWriter      <file>: file name
                example:./bin/snnf_nnsample -s YoloV8nDetectionOptiYolov5sDetection,BotSortTrack,videoWriterHumanAttr -v resource/video/humanCount.mp4 filename
                        ./bin/snnf_nnsample -s YoloV8nPoseOptiYolov5sDetection,BotSortTrackStgcn,videoWriterHumanAttr -v resource/video/person-falling.mp4-video filename

        [-io,--imageoutput file] setspecify image file to nn detection.
                <file>: the output file name for saving results.
             [-c | option]: test count, this parameter is<file>: onlyfile matchname with -i

  extension (e.g., output.jpg, output.json, output.mp4)
             example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr -iThis filenameparameter -cmust testCountbe used in conjunction with imageWriter, jsonWriter, or videoWriter.
                example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,imageWriter --imagei filename -c testCounto output.jpg
           [-v,--video file] set video file to nn detection.      ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,jsonWriter -i filename -o output.json
    <file>: file name                  example:./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr,videoWriter -v filename -o output.mp4
                      ./bin/snnf_nnsample -s Yolov5sDetection,HumanAttr
--video filename          [-a,--all] run all model testing

        assist tools: imageWriter videoWriter jsonWriter BotSortTrack HumanFilter VehicleFilter

Release folder structure

...

  • bin: nnf_nnsample. Prebuild sample programs that can run on the c3v Linux platform.

  • include：header file of NN framework SDK.

  • lib：libraries of NN framework SDK.

  • resource

    • config: some config files for features.

    • font: ttf file for plotting sample.

    • image: image files used for test.

    • model: models to be used in the sample program.

    • video: video files used for test.

  • samples：example code for using NN framework.

  • snnf_run.sh：executable script for running sample code.

  • thirdparty: just as its name implies.

...