SNNF demo includes the following model samples

• yolov8n_pose_opti;

• yolov8n_segment_opti;

• yolov8n_detection_opti;

• det_10g + w600k_r50; (face match)

• lightFace;

• age;

• yolov8n_detection_opti + humanAttr;

• Human tracking;

• Human falling detection;

• OCRDet + OCRCls + OCRRec;

• CCPD + OCRRec

• yolov8n_detection_opti + vehicleAttr

• yolov10s

• Rtmdets

• lightFace + genderAge

If you want to see the demo's effects more quickly, please refer to the SNNF Demo Introduction.
Please read this document carefully if you want to compile the SNNF demo.

HW Environment

Open

C3V_IO_BOARD_V1.1

NPU Driver

The NPU driver version must be set to v6.4.15.9, otherwise, unnecessary errors may occur.

Set the NPU driver to v6.4.15.9 before building the C3V image:

make kconfig, and enter the kernel configuration.

Open

Camera

The SNNF demo supports obtaining input data through a camera or a video file. The camera can be connected through MIPI or USB.

MIPI

Currently, the sensor we support for the MIPI interface is OV5640. The sensor needs to be connected to MIPI-RX4_2D on the platform, the MIPI Sensor interface as shown in the diagram above.

If you plan to use it, you must make the following modifications.

1. Modify the linux/kernel/arch/arm64/boot/dts/sunplus/sp7350-dm.dts before building the C3V image, enable OV5640_IN4.

2. Enable OV5640 sensor support before building the C3V image.

   make kconfig, and enter the kernel configuration.

Open

3. To improve the speed of reading data from the sensor, execute the following command before running the SNNF demo.

USB

The USB Camera supports USB 2.0 and USB 3.0. If you want to use USB 3.0, Set the jump cap in the figure below to the OFF state.

Open

Display

The display screen supports MIPI or HDMI connection, and the currently supported MIPI display screen is the official 7-inch display screen of Raspberry Pi.

MIPI and HDMI displays cannot be used simultaneously. The specific switching method is as follows:

Open

Switch to MIPI

Set the jump cap in the above figure to the ON state and unplug the HDMI cable.

Switch to HDMI

Set the jump cap in the above figure to OFF, power off the Panel connected to MIPI, and plug in the HDMI cable.

Source code

Please get the SNNF Demo V1.1.0 release source code here.

Folder structure

Open

Open

Open

• dependency_libraries: the library that snnf_demo depends on.

  • demo_assist

  • face_match

  • ffmpeg-4.4.2-prebuilt

  • freetype-2.12.1

  • libdrm-2.4.100-prebuilt

  • libpng-1.6.37

  • lvgl-9.1.0-prebuilt

  • opencv-4.6.0-prebuilt

  • opencv-4.2.0-prebuilt

  • snnf_release

  • vsi_lib

  • yuvconvert

• snnf_demo_src: demo source code.

  • src

  • menu: code related to UI.

  • nn: sample code for SNNF and code for processing the NN results.

  • stream: reading video stream and displaying NN results.

  • makefile

  • out: the generation path of the snnf_demo program.

• bin: snnf_demo, Pre-compiled sample programs that can run on the c3v Linux platform.

• snnf_demo_env.sh: setup environment variable.

How to build SNNF Demo

Compile on the C3V platform

1. Copy the release folder to C3V Linux；

2. cd snnf_demo_src

3. Modify makefile ;

   PLATFORM_ROOT = ../../../../../../ PROJECT_ROOT = ../ OUT_DIR = out CROSS_COMPILE := NO GCC_VERSION_LESS_THAN_12 := NO USING_PREBUILD_OPENCV := YES USING_PREBUILD_FFMPEG := YES USING_VSI_LIB_VERSION_6_4_15 := YES BUILD_TYPE := Release ifeq ($(CROSS_COMPILE), YES) ###config toolchain ifeq ($(GCC_VERSION_LESS_THAN_12), YES) TOOLCHAIN_PREFIX = $(PLATFORM_ROOT)/crossgcc/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- else TOOLCHAIN_PREFIX = $(PLATFORM_ROOT)/crossgcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- endif CC = ${TOOLCHAIN_PREFIX}gcc CXX = ${TOOLCHAIN_PREFIX}g++ else CC = gcc CXX = g++ endif

   1. Ensure that "CROSS_COMPILE" in the makefile is set to "NO";

   2. Modify the gcc configuration in the makefile according to the current gcc version on the C3V platform;

      • gcc version < 12, set “GCC_VERSION_LESS_THAN_12” to "YES".

      • gcc version >= 12, set “GCC_VERSION_LESS_THAN_12” to "NO";

      • Check the version of gcc: gcc --version

        For example：

        • Ubuntu 24.04

          root@ubuntu:/snnf_demo_release/snnf_demo_src# gcc --version gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

          set GCC_VERSION_LESS_THAN_12 to "NO";

        • Ubuntu 20.04

          root@ubuntu:/snnf_demo_release/snnf_demo_src# gcc --version gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

          set GCC_VERSION_LESS_THAN_12 to "YES"

   3. Modify the OpenCV configuration in the makefile;

      • Use the pre-compiled OpenCV in the dependencies_libraries folder, set “USING_PREBUILD_OPENCV” to "YES".

      • Use your own installed OpenCV, set “USING_PREBUILD_OPENCV” to "NO";

   4. Modify the FFmpeg configuration in the makefile;

      • Use the pre-compiled FFmpeg in the dependencies_libraries folder, set “ USING_PREBUILD_FFMPEG” to "YES".

      • Use your own installed FFmpeg, set “ USING_PREBUILD_FFMPEG” to "NO";

4. make

5. Run snnf_demo, please refer to the How to run SNNF Demo.

Cross-compile SNNF Demo

1. Copy the release folder to Cross-compile Linux；

2. cd snnf_demo_src

3. Modify makefile ;

   PLATFORM_ROOT = ../../../../../../ PROJECT_ROOT = ../ OUT_DIR = out CROSS_COMPILE := YES GCC_VERSION_LESS_THAN_12 := NO USING_PREBUILD_OPENCV := YES USING_PREBUILD_FFMPEG := YES USING_VSI_LIB_VERSION_6_4_15 := YES BUILD_TYPE := Release ifeq ($(CROSS_COMPILE), YES) ###config toolchain ifeq ($(GCC_VERSION_LESS_THAN_12), YES) TOOLCHAIN_PREFIX = $(PLATFORM_ROOT)/crossgcc/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- else TOOLCHAIN_PREFIX = $(PLATFORM_ROOT)/crossgcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- endif CC = ${TOOLCHAIN_PREFIX}gcc CXX = ${TOOLCHAIN_PREFIX}g++ else CC = gcc CXX = g++ endif

   1. Ensure that "CROSS_COMPILE", " USING_PREBUILD_OPENCV" and " USING_PREBUILD_FFMPEG" in the makefile is set to "YES";

   2. Modify the toolchain path("TOOLCHAIN_PREFIX") in the makefile;

   3. Then, according to the gcc version of the toolchain you have set, modify the value of GCC_VERSION_LES_THAN_12;

      • gcc version < 12, set “GCC_VERSION_LESS_THAN_12” to "YES".

      • gcc version >= 12, set “GCC_VERSION_LESS_THAN_12” to "NO";

4. make

5. Copy the newly compiled snnf_demo program and the folder named "dependency_libraries" from the snnf_demo_delease folder to C3V Linux.

6. Run snnf_demo, please refer to the How to run SNNF Demo.

Setup the running environment

1. Before running, please ensure that the account you are logged into has root privileges.

   • How to create a root account

     sunplus@ubuntu:~$ sudo passwd root New password: Retype new password: passwd: password updated successfully sunplus@ubuntu:~$ sunplus@ubuntu:~$ exit logout Ubuntu 24.04.1 LTS ubuntu ttyS0 ubuntu login: root Password: Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 6.6.47-SUNPLUS-v8 aarch64) root@ubuntu:~#

   • If you are logging in with a non-root account, please execute the following command before running the demonstration program.

     sunplus@ubuntu:~$ sudo -s [sudo] password for sunplus: root@ubuntu:/home/sunplus# root@ubuntu:/home/sunplus#

2. Turn off Display Manager Service (This step is only required for Ubuntu XFCE)

The status of the lightdm service enabled is as follows.

Open

• Temporary stop

Open

• Permanently Closed

Open

Open

3. Copy the newly compiled snnf_demo program, pre-compiled snnf_demo program, and the folder named "dependency_libraries" from the snnf_demo_release folder to C3V Linux.

   /snnf_demo_release# cp bin ==> <C3V Platform>/snnf_demo_release /snnf_demo_release# cp snnf_demo_src/out/snnf_demo ==> <C3V Platform>/snnf_demo_release /snnf_demo_release# cp snnf_demo_env.sh ==> <C3V Platform>/snnf_demo_release /snnf_demo_release# cp dependency_libraries ==> <C3V Platform>/snnf_demo_release

   Copy according to the following directory structure.

   root@ubuntu:/snnf_demo_release# ls -al total 864 drwxr-xr-x 5 10860 11400 4096 Jan 22 2025 . drwxr-xr-x 9 10860 11400 69632 Jan 22 2025 .. drwxr-xr-x 2 10860 11400 4096 Jan 21 19:02 bin drwxr-xr-x 18 10860 11400 4096 Jan 22 2025 dependency_libraries -rwxr-xr-x 1 root root 792832 Jan 22 2025 snnf_demo -rw-r--r-- 1 10860 11400 3170 Jan 22 11:01 snnf_demo_env.sh drwxr-xr-x 4 10860 11400 4096 Jan 22 2025 snnf_demo_src root@ubuntu:/snnf_demo_release# ls -al dependency_libraries/ total 72 drwxr-xr-x 18 10860 11400 4096 Jan 22 2025 . drwxr-xr-x 3 10860 11400 4096 Jan 22 2025 .. lrwxrwxrwx 1 root root 29 Jan 22 2025 demo_assist -> demo_assist_gcc-arm-12.3.rel1 drwxr-xr-x 4 10860 11400 4096 Jan 21 20:08 demo_assist_gcc-arm-12.3.rel1 drwxr-xr-x 4 10860 11400 4096 Jan 21 20:08 demo_assist_gcc-arm-9.2-2019.12 lrwxrwxrwx 1 root root 36 Jan 22 2025 face_match -> face_match_release_gcc-arm-12.3.rel1 drwxr-xr-x 4 10860 11400 4096 Dec 27 18:09 face_match_release_gcc-arm-12.3.rel1 drwxr-xr-x 4 10860 11400 4096 Dec 27 18:09 face_match_release_gcc-arm-9.2-2019.12 drwxr-xr-x 3 10860 11400 4096 Nov 7 10:41 ffmpeg-4.4.2-prebuilt drwxr-xr-x 3 10860 11400 4096 Nov 7 10:41 freetype-2.12.1 drwxr-xr-x 4 10860 11400 4096 Nov 7 10:41 libdrm-2.4.100-prebuilt drwxr-xr-x 3 10860 11400 4096 Nov 7 10:41 libpng-1.6.37 drwxr-xr-x 4 10860 11400 4096 Jan 16 16:33 lvgl-9.1.0-prebuilt drwxr-xr-x 3 10860 11400 4096 Dec 27 18:10 opencv-4.2.0-prebuilt drwxr-xr-x 3 root root 4096 Jan 3 10:17 opencv-4.6.0-prebuilt lrwxrwxrwx 1 root root 30 Jan 22 2025 snnf_release -> snnf_release_gcc-arm-12.3.rel1 drwxr-xr-x 8 10860 11400 4096 Jan 21 18:38 snnf_release_gcc-arm-12.3.rel1 drwxr-xr-x 8 10860 11400 4096 Jan 22 09:40 snnf_release_gcc-arm-9.2-2019.12 lrwxrwxrwx 1 root root 14 Jan 22 2025 vsi_lib -> vsi_lib_6.4.15 drwxr-xr-x 2 10860 11400 4096 Dec 5 16:10 vsi_lib_6.4.15 drwxr-xr-x 2 10860 11400 4096 Dec 5 16:10 vsi_lib_6.4.18 drwxr-xr-x 2 10860 11400 4096 Nov 7 10:41 yuvconvert

4. Copy the resource files to C3V Linux.

   The path of resource files：dependency_libraries/snnf_release/resource

   #cp dependency_libraries/snnf_release/resource/* ==> <C3V Platform>/etc/snnf/
   root@ubuntu:/snnf_demo_release# mkdir -p /etc/snnf root@ubuntu:/snnf_demo_release# cp dependency_libraries/snnf_release/resource/* /etc/snnf -rf root@ubuntu:/snnf_demo_release# ls -l /etc/snnf/ total 20 drwxr-xr-x 2 root root 4096 Jan 21 19:26 config drwxr-xr-x 2 root root 4096 Jan 21 19:26 font drwxr-xr-x 2 root root 4096 Jan 21 19:26 image drwxr-xr-x 2 root root 4096 Jan 21 19:26 model drwxr-xr-x 2 root root 4096 Jan 21 19:26 video

5. Set environment variables, which need to be configured every time the platform is launched.

   1. Switch to the directory where the dependency_libraries are located, then set the environment variable;

      root@ubuntu:/snnf_demo_release/snnf_demo_src#cd ../ root@ubuntu:/snnf_demo_release/# ls -al total 864 drwxr-xr-x 5 10860 11400 4096 Jan 22 2025 . drwxr-xr-x 9 10860 11400 69632 Jan 22 2025 .. drwxr-xr-x 2 10860 11400 4096 Jan 21 19:02 bin drwxr-xr-x 18 10860 11400 4096 Jan 22 2025 dependency_libraries -rwxr-xr-x 1 root root 792832 Jan 22 2025 snnf_demo -rw-r--r-- 1 10860 11400 3170 Jan 22 11:01 snnf_demo_env.sh drwxr-xr-x 4 10860 11400 4096 Jan 22 2025 snnf_demo_src

   2. Set the environment variable: source snnf_demo_env.sh rootfs_name. The supported parameters are：'ubuntu_20', 'ubuntu_22', 'ubuntu_24', 'buildroot'.

      For example：

      • Ubuntu 24.04

        root@ubuntu:/snnf_demo_release# source snnf_demo_env.sh ubuntu_24

      • Ubuntu 22.04

        root@ubuntu:/snnf_demo_release# source snnf_demo_env.sh ubuntu_22

      • Ubuntu 20.04

        root@ubuntu:/snnf_demo_release# source snnf_demo_env.sh ubuntu_20

      • Buildroot

        # source snnf_demo_env.sh buildroot

How to run SNNF Demo

1. Before running snnf_demo, please refer to Setup the running environment.

2. Run snnf_demo.

   root@ubuntu:/snnf_demo_release# ./snnf_demo -h Version: 1.1.0 Usage: ./snnf_demo [options] Options: -h, --help Print this help message and exit -m, --model ARG Specify the model (default: pose, support model list: pose, segment, object, humanattr, ocr, lightface, age, face match, tracking, falling, CCPD, VehicleAttr, RTMDet-s, yolov10s, genderAge) -v, --video ARG Specify the video file or device (default: /dev/video0, e.g., /dev/video40, /mnt/test.mp4) -f, --format ARG Specify the video capture format (default: YUYV, only YUYV, UYVY, YUY2, H264, MJPG are supported) -s, --size ARG Specify the video capture size (default: 640x480, e.g., 1920x1080, 1280x720) -r, --framerate ARG Specify the video capture frame rate (default: 30)

   1. Obtaining input data through the camera

      • Use default param

        root@ubuntu:/snnf_demo_release# ./snnf_demo

      • Set the video size/fps/YUV format by cmd.

        root@ubuntu:/snnf_demo_release# ./snnf_demo -v /dev/video40 -s 1280x720 -f UYVY -r 30

      • How to get the format supported by the camera?

        root@ubuntu:/snnf_demo_release# apt install v4l-utils

        • v4l2-ctl --list-formats -d /dev/video40

          Open

          

        • v4l2-ctl --list-formats-ext -d /dev/video40

          Open

          

   2. Obtaining input data through the video file

      root@ubuntu:/snnf_demo_release# ./snnf_demo -v /etc/snnf/video/humanCount.mp4

   3. Obtaining input data through the image file

      root@ubuntu:/snnf_demo_release# ./snnf_demo -v /etc/snnf/image/pose_input.jpg

3. See the demo's effects please refer to the SNNF Demo Introduction.