The SP7350 boasts a powerful neural processor unit (NPU), designed to efficiently handle complex neural network operations. However, applications requiring neural network inference cannot directly interact with the NPU. Instead, they must utilize the NPU driver along with prebuilt shared libraries.
Table of Contents
| Table of Contents | ||
|---|---|---|
|
1.
...
NPU Driver
...
The C3V Linux working environment compiles NN drivers by default, with the default driver version being V6SP7350 Linux includes the NPU driver by default, supporting three versions: 6.4.13.8, 6.4.15.9. If you want to configure the driver version, please refer to 1.1, otherwise please skip it, and 6.4.18.5. By default, version 6.4.15.9 is selected.
1.1.
...
Changing the Version
To switch to a different version, run the ‘make kconfig’ command in the project's top directory. When the Linux Kernel Configuration menu appears, move cursor to select the desired driver version. The path for driver configuration is as follows:
| Code Block | ||
|---|---|---|
| ||
Device Drivers -> Multimedia support -> Media drivers -> ML Inference Processor VIP9000_NPU |
...
NPU |
Refer to the snapshot of the NPU Version choice menu below:
...
After selecting the desired driver version, complete the Linux compilation process .
...
by running the ‘make’ command.
1.2.
...
Power Management
If you need to enable the power management function of the NN driver, you need to set the parameter “powerManagement=1” when probe the galcore.ko module.The NPU driver supports power management. When loading the NPU driver with the ‘powerManagement=0’ parameter, the NPU power is turned on when the driver is probed and turned off when the driver is removed. To further save power, load the NPU driver with the ‘powerManagement=1’ parameter. This ensures that the NPU power is only turned on when the NPU is actively running.
1.3. Loading the NPU driver
To load the NPU driver (module), use the ‘modprobe’ command as shown below:
| Code Block | ||
|---|---|---|
| ||
modprobe galcore powerManagement=1 |
2. VIP9000 SDK Usage
2.1. App Complication
The VIP9000 SDK has been placed in the C3V Linux system directory. The path of VIP9000 SDK is: workspace/linux/rootfs/initramfs/prebuilt/vip9000sdk.
When compiling NN related applicaiton, it is necessary to include SDK headers and libraries.
...
or:
| Code Block |
|---|
modprobe galcore powerManagement=0 |
Here, 'galcore' is the module name of the NPU driver. When the NPU driver is being probed, the Linux kernel message will appear as follows:
| Code Block |
|---|
[ 29.498289] galcore f8140000.galcore: NPU get power success
[ 29.498971] galcore f8140000.galcore: galcore irq number is 44
[ 29.499073] galcore f8140000.galcore: NPU clock: 900000000
[ 29.503967] Galcore version 6.4.15.9.700103
[ 29.509197] galcore f8140000.galcore: _SetPower 0 0 enable
[ 29.518719] galcore f8140000.galcore: regulator enable success
[ 29.519415] galcore f8140000.galcore: NPU ISO disable
[ 29.524477] galcore f8140000.galcore: NPU HW reset SMS
[ 29.529566] galcore f8140000.galcore: NPU HW reset deassert
[ 29.535127] galcore f8140000.galcore: NPU HW clock enable
[ 29.540803] galcore f8140000.galcore: _SetPower enable ret=0
[ 29.546162] galcore f8140000.galcore: NPU clock enable success
[ 29.559172] galcore f8140000.galcore: NPU clock disable success
[ 29.559227] galcore f8140000.galcore: _SetPower 0 0 disable
[ 29.563427] galcore f8140000.galcore: NPU HW clock disabled
[ 29.568980] galcore f8140000.galcore: NPU HW reset assert
[ 29.574355] galcore f8140000.galcore: NPU ISO enable
[ 29.579308] galcore f8140000.galcore: regulator disable success
[ 29.585206] galcore f8140000.galcore: _SetPower disable ret=0 |
Upon successful NPU driver probing, the NPU device ('galcore') will appear in the Linux device directory '/dev'.
| Code Block |
|---|
~ # ls -l /dev/galcore
crw-rw---- 1 root root 199, 0 Jan 1 00:00 /dev/galcore |
2. NN Shared Libraries
Neural network (NN) applications cannot directly interact with the NPU but instead rely on the NPU driver and prebuilt NN shared libraries. These libraries, along with relevant header files, are stored in the project directory of the SP7350 for different versions.
| Code Block |
|---|
wells@wells-linux:~/Q654$ ll linux/rootfs/initramfs/prebuilt/vip9000sdk
total 20
drwxrwxr-x 7 wells wells 4096 Jun 12 21:31 ./
drwxrwxr-x 9 wells wells 4096 Jun 12 21:31 ../
drwxrwxr-x 4 wells wells 4096 Jun 12 21:31 6.4.13.8/
drwxrwxr-x 4 wells wells 4096 Jun 12 21:31 6.4.15.9/
drwxrwxr-x 4 wells wells 4096 Jun 12 21:31 6.4.18.5/ |
The ‘include’ subdirectory includes the relevant header files:
| Code Block |
|---|
wells@wells-linux:~/Q654$ ll linux/rootfs/initramfs/prebuilt/vip9000sdk/6.4.15.9/include/
total 28
drwxrwxr-x 7 wells wells 4096 Jun 12 21:31 ./
drwxrwxr-x 4 wells wells 4096 Jun 12 21:31 ../
drwxrwxr-x 2 wells wells 4096 Jun 12 21:31 CL/
drwxrwxr-x 3 wells wells 4096 Jun 12 21:31 HAL/
drwxrwxr-x 2 wells wells 4096 Jun 12 21:31 jpeg/
drwxrwxr-x 12 wells wells 4096 Jun 12 21:31 ovxlib/
drwxrwxr-x 2 wells wells 4096 Jun 12 21:31 VX/ |
The ‘drivers’ subdirectory includes the NN shared libraries:
| Code Block |
|---|
wells@wells-linux:~/Q654$ ll linux/rootfs/initramfs/prebuilt/vip9000sdk/6.4.15.9/drivers/
total 153900
drwxrwxr-x 2 wells wells 4096 Jun 12 21:31 ./
drwxrwxr-x 4 wells wells 4096 Jun 12 21:31 ../
-rw-rw-r-- 1 wells wells 419144 Jun 12 21:31 libArchModelSw.so
-rw-rw-r-- 1 wells wells 2663528 Jun 12 21:31 libCLC.so
-rwxrwxr-x 1 wells wells 2083472 Jun 12 21:31 libGAL.so*
-rw-rw-r-- 1 wells wells 1338864 Jun 12 21:31 libGLSLC.so
-rw-rw-r-- 1 wells wells 324236 Jun 12 21:31 libjpeg.a
-rw-rw-r-- 1 wells wells 418512 Jun 12 21:31 libNNArchPerf.so
-rwxrwxr-x 1 wells wells 21718760 Jun 12 21:31 libNNGPUBinary.so*
-rwxrwxr-x 1 wells wells 65467968 Jun 12 21:31 libNNVXCBinary.so*
lrwxrwxrwx 1 wells wells 18 Jun 12 21:31 libOpenCL.so -> libOpenCL.so.3.0.0*
lrwxrwxrwx 1 wells wells 18 Jun 12 21:31 libOpenCL.so.1 -> libOpenCL.so.3.0.0*
lrwxrwxrwx 1 wells wells 18 Jun 12 21:31 libOpenCL.so.3 -> libOpenCL.so.3.0.0*
-rwxrwxr-x 1 wells wells 924856 Jun 12 21:31 libOpenCL.so.3.0.0*
lrwxrwxrwx 1 wells wells 18 Jun 12 21:31 libOpenVX.so -> libOpenVX.so.1.3.0*
lrwxrwxrwx 1 wells wells 18 Jun 12 21:31 libOpenVX.so.1 -> libOpenVX.so.1.3.0*
-rwxrwxr-x 1 wells wells 4997304 Jun 12 21:31 libOpenVX.so.1.3.0*
-rwxrwxr-x 1 wells wells 40712 Jun 12 21:31 libOpenVXU.so*
-rwxrwxr-x 1 wells wells 11961312 Jun 12 21:31 libOvx12VXCBinary.so*
-rwxrwxr-x 1 wells wells 5955904 Jun 12 21:31 libOvxGPUVXCBinary.so*
-rwxrwxr-x 1 wells wells 5571776 Jun 12 21:31 libovxlib.so*
-rw-rw-r-- 1 wells wells 3000360 Jun 12 21:31 libSPIRV_viv.so
-rw-rw-r-- 1 wells wells 15502848 Jun 12 21:31 libtim-vx.so
-rw-rw-r-- 1 wells wells 17830752 Jun 12 21:31 libVSC.so
-rw-rw-r-- 1 wells wells 1181832 Jun 12 21:31 libvx_delegate.so |
Upon selecting a version of the NPU driver, the corresponding NN shared libraries are automatically copied to the '/lib' or '/lib64' directory of the root file system on the target SP7350 system. This ensures that the required libraries are readily accessible for NN applications.
3. NN Applications
Neural network (NN) applications leverage the NPU to execute complex neural network operations by invoking functions within NN shared libraries.
3.1. Compiling NN Applications
When compiling NN applications, users must include the appropriate header files and shared libraries, as demonstrated below:
| Code Block | ||
|---|---|---|
| ||
INCLUDES+=-I$(VIVANTENN_SDK_DIR)/include/HAL \ -I$(VIVANTENN_SDK_DIR)/include/CL \ -I$(VIVANTENN_SDK_DIR)/include/VX \ -I$(VIVANTENN_SDK_DIR)/include/ovxlib \ -I$(VIVANTENN_SDK_DIR)/include/jpeg |
Example of SDK Link Libraries：
| Code Block | ||
|---|---|---|
| ||
LIBS+=-lOpenVX -lOpenVXU -lCLC -lVSC -lGAL -ljpeg -lovxlib |
...
3.2.
...
Running
...
NN Application
During runtime, ensure that the NN shared libraries are accessible by setting the library path accordingly:
| Code Block | ||
|---|---|---|
| ||
export LD_LIBRARY_PATH=/lib64:$LD_LIBRARY_PATH |
...