In this guide, we will step-by-step guide you to use video camera IMX708 on SP7350 platforms.

Table of Contents

1 IMX708 specification

Back-illuminated and stacked CMOS 12-megapixel image sensor
Pixel Size: 1.4 x 1.4 μm
HorizoHorizontal/vertical: 4608 × 2592 pixels
Sensor size: 7.4mm sensor diagonal
Optical size: 1/2.43"
Maximum exposure times (seconds): 112
HDR mode (up to 3 megapixel output)
video modes: 2304 × 1296p56, 2304 × 1296p30 HDR, 1536 × 864p120

2 Setup

2.1 Enable IMX708 in kernel configuration

1 Enter kernel configuration

Code Block

language	bash

cd ./linux/kernel
make menuconfig

2 Selects submenus: Device Drivers>Multimedia support>Media ancillary drivers>Camera sensor devices

Image Removed

Image Added

3 Select Sony IMX708 sensor support and pressing <Y> inlucdes. Then save and exit.

2.2 Modify device tree source

Device-tree source files of SP7350 are located in linux/kernel/arch/arm64/boot/dts/sunplus/.

Here lists all files for SP7350 boards:

Boards	Device-tree source files
C3V-W EVB	sp7350-ev.dts

Modify device tree source file to define IMX708 source for driver

2.2.1 Using SP7350 MIPI-CSI-RX4

Open IMX708_IN4 macro to add IMX708 node and link mipicsirx4.

2.2.2 Using SP7350 other MIPI-CSI-RX

Define IMX708 node in i2c node:

Code Block

i2c {
    #address-cells = <1>;
    #size-cells = <0>;

    imx708: imx708@1a {
		compatible = "sony,imx708";
		reg = <0x1a>;
		clocks = <&imx708_clk>;
		clock-names = "inclk";
		vana1-supply = <&imx708_vana1_2v8>;	/* 2.8v */
		vana2-supply = <&imx708_vana2_1v8>;/* 1.8v */
		vdig-supply = <&imx708_vdig_1v1>;	/* 1.1v */
		vddl-supply = <&imx708_vddl_1v8>;	/* 1.8v */

        port {
            imx708_ep: endpoint {
                remote-endpoint = <&csi_ep>;
				data-lanes = <1 2>;
				clock-noncontinuous;
				link-frequencies = /bits/ 64 <450000000>;
            };
        };
    };
};

port: Should contain one endpoint sub-node used to model connection to the video receiver according to the specification defined in Documentation/devicetree/bindings/media/video-interfaces.txt.

Video receiver endpoint sub-node should defined in mipicsirx node. For example:

Code Block
mipicsirx: csirx@f8005480 { port@0 { reg = <0>; /* MIPI CSI-2 bus endpoint */ csi_ep: endpoint { remote-endpoint = <&imx708_ep>; bus-type = <4>; clock-lanes = <0>; data-lanes = <1 2>; }; };

2.3 Compile and burn

https://sunplus.atlassian.net/wiki/spaces/C3/pages/1988034774/A+guide+to+downloading+and+compiling+SP7350+code#7.-Build-code

2.4 Connect sensor module

Using C3V-W board MIPI-CSI-RX4, as follows：

2.5 Check

After the burning is completed, start the board. The system should print the following information through the serial port.

Code Block

[    2.121010] i2c /dev entries driver
[    2.133944] imx708 0-001a: camera module ID 0x0301
[    2.173080] sp-csi2 f8005480.csirx: 2 lanes found
[    2.173479] sp-vin f8005980.video: assigned reserved memory node mipicsirx@4f600000
[    2.177650] imx708 0-001a: Consider updating driver imx708 to match on endpoints
[    2.185261] sp-vin f8005980.video: Device VIN8 registered as video0
[    2.191230] sp-vin f8005980.video: SP VIN driver probed

After the system startup is completed, the video device can be viewed.

Code Block
root@dbv3:~# ls /dev/video0 /dev/video0

3 Test

3.1 V4l2-ctl command

This command is used to control V4L2 devices from the command line. It allows you to query device capabilities, set device parameters, and capture video and still image.

Use v4l2-ctl --all --device /dev/video0 command getting device all information.

Code Block

root@dbv3:~# v4l2-ctl --all --device /dev/video0
Driver Info:
	Driver name      : sp_vin
	Card type        : SP_VIN
	Bus info         : platform:f8005980.video
	Driver version   : 5.10.201
	Capabilities     : 0xa5200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x25200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : sp_vin
	Model            : sunplus,sp7350-vin
	Serial           : 
	Bus info         : platform:f8005980.video
	Media version    : 5.10.201
	Hardware revision: 0x00000000 (0)
	Driver version   : 5.10.201
Entity Info:
	ID               : 16
	Name             : VIN8 output
	Type             : V4L2 I/O
	Pad              : 0: Sink
Priority: 2
Video input : 0 (VIN8 output: ok)
Format Video Capture:
	Width/Height      : 1536/864
	Pixel Format      : 'RG10' (10-bit Bayer RGRG/GBGB)
	Field             : None
	Bytes per Line    : 3072
	Size Image        : 2654208
	Colorspace        : Raw
	Transfer Function : None
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Full Range
	Flags             : 

User Controls

                       exposure 0x00980911 (int)    : min=1 max=2602 step=1 default=1600 value=1600
                horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
                  vertical_flip 0x00980915 (bool)   : default=0 value=0 flags=modify-layout
                alpha_component 0x00980929 (int)    : min=0 max=255 step=1 default=255 value=255

Camera Controls

             wide_dynamic_range 0x009a0915 (bool)   : default=0 value=0 flags=modify-layout

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=40 max=8387616 step=1 default=2755 value=58
            horizontal_blanking 0x009e0902 (int)    : min=3680 max=3680 step=1 default=3680 value=3680 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=112 max=960 step=1 default=112 value=112
                red_pixel_value 0x009e0904 (int)    : min=0 max=4095 step=1 default=4095 value=4095
          green_red_pixel_value 0x009e0905 (int)    : min=0 max=4095 step=1 default=4095 value=4095
               blue_pixel_value 0x009e0906 (int)    : min=0 max=4095 step=1 default=4095 value=4095
         green_blue_pixel_value 0x009e0907 (int)    : min=0 max=4095 step=1 default=4095 value=4095
                   notify_gains 0x009e0909 (u32)    : min=1 max=65535 step=1 default=256 [4] flags=has-payload

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=0 default=0 value=0 flags=read-only
                     pixel_rate 0x009f0902 (int64)  : min=566400000 max=566400000 step=1 default=566400000 value=566400000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0
                   digital_gain 0x009f0905 (int)    : min=256 max=65535 step=1 default=256 value=256

3.2 Capture image

3.2.1 Config video device

Use v4l2-ctl -d /dev/video0 --set-ctrl params=value setting device controls.

set exposure(exposure 0x00980911 (int) : min=1 max=2602 step=1 default=1600 value=1600):

Code Block
v4l2-ctl -d /dev/video0 --set-ctrl exposure=2000

set analogue_gain(analogue_gain 0x009e0903 (int): min=112 max=960 step=1 default=112 value=112)

Code Block
v4l2-ctl -d /dev/video0 --set-ctrl analogue_gain=800

set digital_gain(digital_gain 0x009f0905 (int) : min=256 max=65535 step=1 default=256 value=256)

Code Block
v4l2-ctl -d /dev/video0 --set-ctrl digital_gain=550

Change image brightness by changing the exposure, analogue_gain and digital_gain.

3.2.2 Capture

Use v4l2-ctl --list-formats-ext --device /dev/video0 querying device video format.

Code Block

root@dbv3:~# v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
		Size: Discrete 4608x2592
		Size: Discrete 2304x1296
		Size: Discrete 1536x864
	[1]: 'pRAA' (10-bit Bayer RGRG/GBGB Packed)
		Size: Discrete 4608x2592
		Size: Discrete 2304x1296
		Size: Discrete 1536x864

Use v4l2-ctl --stream capture image.

Code Block
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2304,height=1296,pixelformat=pRAA --stream-mmap=3 --stream-to=/home/root/imx708.raw --stream-skip=9 --stream-count=1

--set-fmt-video=width=2304,height=1296,pixelformat=pRAA set image format
--stream-count <count> stream <count> buffers. The default is to keep streaming forever. This count does not include the number of initial skipped buffers as is passed by --stream-skip.
--stream-skip <count> skip the first <count> buffers. The default is 0.
--stream-to <file> stream to this file. The default is to discard the data. If <file> is '-', then the data is written to stdout and the --silent option is turned on automatically.
--stream-mmap <count> capture video using mmap() [VIDIOC_(D)QBUF] count: the number of buffers to allocate. The default is 3.

After executing the capture command, the stream data is written to /home/root/imx708.raw.

3.3 Display image raw data

3.3.1 Download and install raw image viewer

Download raw image viewer from PixelViewer | Carina Studio. And PixelViewer is an open source project based cross-platform image which viewer supports reading raw Luminance/YUV/RGB/ARGB/Bayer pixels data from file and rendering it.

3.3.2 Use pixel viewer open image

Config render params by 3.2.2 v4l2-ctl capture command

3.3 HDR

Enable HDR

Code Block
v4l2-ctl -d /dev/video0 --set-ctrl wide_dynamic_range=1

capture image

Code Block
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2304,height=1296,pixelformat=pRAA --stream-mmap=3 --stream-to=/home/root/imx708_hdr.raw --stream-skip=9 --stream-count=1

image render：

comparison：

By comparing the brightness of the light position, it can be seen that HDR improves the image effect。

Versions Compared

Old Version 2

New Version 3

Key

1 IMX708 specification

2 Setup

2.1 Enable IMX708 in kernel configuration

2.2 Modify device tree source

2.2.1 Using SP7350 MIPI-CSI-RX4

2.2.2 Using SP7350 other MIPI-CSI-RX

2.3 Compile and burn

2.4 Connect sensor module

2.5 Check

3 Test

3.1 V4l2-ctl command

3.2 Capture image

3.2.1 Config video device

3.2.2 Capture

3.3 Display image raw data

3.3.1 Download and install raw image viewer

3.3.2 Use pixel viewer open image

3.3 HDR

Page Comparison

Versions Compared

Old Version 2

New Version 3

Key

1 IMX708 specification

2 Setup

2.1 Enable IMX708 in kernel configuration

2.2 Modify device tree source

2.2.1 Using SP7350 MIPI-CSI-RX4

2.2.2 Using SP7350 other MIPI-CSI-RX

2.3 Compile and burn

2.4 Connect sensor module

2.5 Check

3 Test

3.1 V4l2-ctl command

3.2 Capture image

3.2.1 Config video device

3.2.2 Capture

3.3 Display image raw data

3.3.1 Download and install raw image viewer

3.3.2 Use pixel viewer open image

3.3 HDR