In this guide, we will step-by-step guide you to use video camera D435i on SP7350 platforms.

Table of Contents

1. D435i Specification

2. Setup

2.1 Enable D435i Gyroscope and Accelerometer in Linux Kernel Configuration

Enter the kernel configuration by running the following command in the top directory of project

make kconfig

Selects sub-menus:

Device Drivers>HID support>User-space I/O support for HID subsystem

Device Drivers>HID support>Special HID drivers>HID Sensors framework support

> HID Sensors hub custom sensor support

Device Drivers>HID support>USB HID drivers>PID device support

Device Drivers>Industrial I/O support>HID Sensor IIO Common>Common modules for all HID Sensor IIO drivers

Device Drivers>Industrial I/O support>HID Sensor IIO Common>Common module (trigger) for all HID Sensor IIO drivers

Device Drivers>Industrial I/O support>Digital gyroscope sensors>HID Gyroscope 3D

Device Drivers>Industrial I/O support>Digital Accelerometers>HID Accelerometers 3D

2.2 Apply uvc, metadata, hid and powerlinefrequency-fix patch for kernel

Patch file :

Patch process is below:

cp uvc_realsense.patch Q654_code_path/linux/kernel
cd Q654_code_path/linux/kernel
patch -p1 < uvc_realsense.patch

2.3 Compile and Burn

https://sunplus.atlassian.net/wiki/spaces/C3/pages/1988034774/A+guide+to+downloading+and+compiling+SP7350+code#7.-Build-code

Because it needs to build realsense SDK tool to use, please choose Ubuntu XFCE or Ubuntu MATE as rootfs when doing “make config”.

2.4 Connect Sensor Module

Using C3V-W board USB3 TYPE C to connect Intel Realsense D435i.

2.5 Check

After the burning is completed, start the board. The system should print the following information through the serial port.

[  138.864182] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435i (8086:0b3a)
[  138.866850] input: Intel(R) RealSense(TM) Depth Ca as /devices/platform/usb3drd0/f80a1000.dwc3/xhci-hcd.1.auto/usb4/4-1/4-1:1.0/input/input6
[  138.868558] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435i (8086:0b3a)
[  139.222833] hid-sensor-hub 0003:8086:0B3A.0004: No report with id 0xffffffff found
[  143.179993] hid-sensor-hub 0003:8086:0B3A.0004: No report with id 0xffffffff found

After the system startup is completed, the video device can be viewed.

Device video0~5 are for Realsense D435i

sunplus@ubuntu:~$ ll /dev/video*
crw-rw-rw- 1 root plugdev 81,  5 Jun 11 16:00 /dev/video0
crw-rw-rw- 1 root plugdev 81,  6 Jun 11 16:00 /dev/video1
crw-rw---- 1 root video   81,  0 Jun 11 15:58 /dev/video10
crw-rw---- 1 root video   81,  1 Jun 11 15:58 /dev/video11
crw-rw---- 1 root video   81,  2 Jun 11 15:58 /dev/video12
crw-rw---- 1 root video   81,  3 Jun 11 15:58 /dev/video13
crw-rw---- 1 root video   81,  4 Jun 11 15:58 /dev/video14
crw-rw-rw- 1 root plugdev 81,  7 Jun 11 16:00 /dev/video2
crw-rw-rw- 1 root plugdev 81,  8 Jun 11 16:00 /dev/video3
crw-rw-rw- 1 root plugdev 81,  9 Jun 11 16:00 /dev/video4
crw-rw-rw- 1 root plugdev 81, 10 Jun 11 16:00 /dev/video5

The iio device can be viewed.

sunplus@ubuntu:/sys/bus/iio/devices$ ll
total 0
drwxr-xr-x 2 root root 0 Jun 11 15:58 ./
drwxr-xr-x 4 root root 0 Jun 11 15:58 ../
lrwxrwxrwx 1 root root 0 Jun 11 15:58 iio:device0 -> ../../../devices/platform/f8802f00.adc/iio:device0/
lrwxrwxrwx 1 root root 0 Jun 11 16:00 iio:device1 -> ../../../devices/platform/usb3drd0/f80a1000.dwc3/xhci-hcd.1.auto/usb4/4-1/4-1:1.5/0003:8086:0B3A.0004/HID-SENSOR-200073.3.auto/iio:device1/
lrwxrwxrwx 1 root root 0 Jun 11 16:00 iio:device2 -> ../../../devices/platform/usb3drd0/f80a1000.dwc3/xhci-hcd.1.auto/usb4/4-1/4-1:1.5/0003:8086:0B3A.0004/HID-SENSOR-200076.4.auto/iio:device2/
lrwxrwxrwx 1 root root 0 Jun 11 16:00 trigger0 -> ../../../devices/platform/usb3drd0/f80a1000.dwc3/xhci-hcd.1.auto/usb4/4-1/4-1:1.5/0003:8086:0B3A.0004/HID-SENSOR-200073.3.auto/trigger0/
lrwxrwxrwx 1 root root 0 Jun 11 16:00 trigger1 -> ../../../devices/platform/usb3drd0/f80a1000.dwc3/xhci-hcd.1.auto/usb4/4-1/4-1:1.5/0003:8086:0B3A.0004/HID-SENSOR-200076.4.auto/trigger1/

iio:device1 is the device for Accelerometer of realsense D435i. The device sysfs content is below:

sunplus@ubuntu:/sys/bus/iio/devices/iio:device1$ ll
total 0
drwxrwxrwx 6 root root    0 Jun 11 16:00 ./
drwxrwxrwx 5 root root    0 Jun 11 16:00 ../
drwxrwxrwx 2 root root    0 Jun 11 16:00 buffer/
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 current_timestamp_clock*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 dev*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_hysteresis*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_offset*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_sampling_frequency*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_scale*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_x_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_y_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_accel_z_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 name*
drwxrwxrwx 2 root root    0 Jun 11 16:00 power/
drwxrwxrwx 2 root root    0 Jun 11 16:00 scan_elements/
lrwxrwxrwx 1 root root    0 Jun 11 16:00 subsystem -> ../../../../../../../../../../../bus/iio/
drwxrwxrwx 2 root root    0 Jun 11 16:00 trigger/
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 uevent*

You can read sensor data by cat command.

sunplus@ubuntu:/sys/bus/iio/devices/iio:device1$ cat in_accel_x_raw
11
sunplus@ubuntu:/sys/bus/iio/devices/iio:device1$ cat in_accel_y_raw
-947
sunplus@ubuntu:/sys/bus/iio/devices/iio:device1$ cat in_accel_z_raw
-142

iio:device2 is the device for Gyroscope of realsense D435i. The device sysfs content is below:

sunplus@ubuntu:/sys/bus/iio/devices/iio:device2$ ll
total 0
drwxrwxrwx 6 root root    0 Jun 11 16:00 ./
drwxrwxrwx 5 root root    0 Jun 11 16:00 ../
drwxrwxrwx 2 root root    0 Jun 11 16:00 buffer/
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 current_timestamp_clock*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 dev*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_hysteresis*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_offset*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_sampling_frequency*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_scale*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_x_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_y_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 in_anglvel_z_raw*
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 name*
drwxrwxrwx 2 root root    0 Jun 11 16:00 power/
drwxrwxrwx 2 root root    0 Jun 11 16:00 scan_elements/
lrwxrwxrwx 1 root root    0 Jun 11 16:00 subsystem -> ../../../../../../../../../../../bus/iio/
drwxrwxrwx 2 root root    0 Jun 11 16:00 trigger/
-rwxrwxrwx 1 root root 4096 Jun 11 16:00 uevent*

You can read sensor data by cat command.

sunplus@ubuntu:/sys/bus/iio/devices/iio:device2$ cat in_anglvel_x_raw
3967
sunplus@ubuntu:/sys/bus/iio/devices/iio:device2$ cat in_anglvel_y_raw
0
sunplus@ubuntu:/sys/bus/iio/devices/iio:device2$ cat in_anglvel_z_raw
610

3. Build and Install Intel Realsense SDK tool

3.1 Install dependencies

Make Ubuntu up-to-date

sudo apt-get update

Install the core packages required to build librealsense2 binaries and the affected kernel modules

sudo apt-get install libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev

Install build tools

sudo apt-get install git wget cmake build-essential

Prepare Linux Backend and the Dev. Environment

Unplug any connected Intel RealSense camera and run:

sudo apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev at

3.2 Clone the librealsense2 repo and check out v2.55.1

git clone https://github.com/IntelRealSense/librealsense.git
git checkout v2.55.1

3.3 Run Intel Realsense permissions script from librealsense2 root directory:

./scripts/setup_udev_rules.sh

Notice: You can always remove permissions by running: ./scripts/setup_udev_rules.sh --uninstall

3.4 Modify the path by adding the following line to the .bashrc file:

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Apply the change:

source ~/.bashrc

3.5 Build and Install librealsense2 SDK

Navigate to librealsense2 root directory and run:

mkdir build && cd build

Run cmake configure step:

cmake .. -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=Release

Linux build configuration is presently configured to use the V4L2 backend by default.

Note: Don’t use “-DFORCE_RSUSB_BACKEND=true” or “-DFORCE_LIBUVC=true” build option, or you can’t get gyroscope and accelerometer sensor data by using realsense API.

Recompile and install librealsense2 binaries:

sudo make uninstall && make clean && make -j4 && sudo make install

3.6 Install RealSense SDK pyrealsense2 Python bindings for librealsense:

cd ~/librealsense/build
cmake .. -DBUILD_PYTHON_BINDINGS=bool:true -DPYTHON_EXECUTABLE=$(which python3)
make -j4
sudo make install

Modify the path by adding the following line to the .bashrc file:

export PYTHONPATH=$PYTHONPATH:/usr/local/lib