NPU Driver and Shared Libraries in SP7350 Linux

The SP7350 boasts a powerful neural processor unit (NPU), designed to efficiently handle complex neural network operations. However, applications requiring neural network inference cannot directly interact with the NPU. Instead, they must utilize the NPU driver along with prebuilt shared libraries.

Table of Contents

1 1. NPU Driver
- 1.1 1.1. Changing the Version
- 1.2 1.2. Power Management
- 1.3 1.3. Loading the NPU driver
2 2. NN Shared Libraries
3 3. NN Applications
- 3.1 3.1. Compiling NN Applications
- 3.2 3.2. Running NN Application

1. NPU Driver

The SP7350 Linux includes the NPU driver by default, supporting three versions: 6.4.13.8, 6.4.15.9, and 6.4.18.5. By default, version 6.4.15.9 is selected.

1.1. Changing the Version

To switch to a different version, run the ‘make kconfig’ command in the project's top directory. When the Linux Kernel Configuration menu appears, move cursor to select the desired driver version. The path is:

Device Drivers ->
	Multimedia support ->
  	Media drivers ->
    	ML Inference Processor NPU

Refer to the snapshot of the NPU Version choice menu below:

After selecting the desired driver version, complete the Linux compilation process by running the ‘make’ command.

1.2. Power Management

The NPU driver supports power management. When loading the NPU driver with the ‘powerManagement=0’ parameter, the NPU power is turned on when the driver is probed and turned off when the driver is removed. To further save power, load the NPU driver with the ‘powerManagement=1’ parameter. This ensures that the NPU power is only turned on when the NPU is actively running.

1.3. Loading the NPU driver

To load the NPU driver (module), use the ‘modprobe’ command as shown below:

modprobe galcore powerManagement=1

or:

modprobe galcore powerManagement=0

Here, 'galcore' is the module name of the NPU driver. When the NPU driver is being probed, the Linux kernel message will appear as follows:

Upon successful NPU driver probing, the NPU device ('galcore') will appear in the Linux device directory '/dev'.

2. NN Shared Libraries

Neural network (NN) applications cannot directly interact with the NPU but instead rely on the NPU driver and prebuilt NN shared libraries. These libraries, along with relevant header files, are stored in the project directory of the SP7350 for different versions.

The ‘include’ subdirectory includes the relevant header files:

The ‘drivers’ subdirectory includes the NN shared libraries:

Upon selecting a version of the NPU driver, the corresponding NN shared libraries are automatically copied to the '/lib' or '/lib64' directory of the root file system on the target SP7350 system. This ensures that the required libraries are readily accessible for NN applications.

3. NN Applications

Neural network (NN) applications leverage the NPU to execute complex neural network operations by invoking functions within NN shared libraries.

3.1. Compiling NN Applications

When compiling NN applications, users must include the appropriate header files and shared libraries, as demonstrated below:

3.2. Running NN Application

During runtime, ensure that the NN shared libraries are accessible by setting the library path accordingly: