Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

1 OV5640 Specification

  • 5 megapixel (2592x1944) camera

  • Pixel Size: 1.4 x 1.4 μm

  • Optical size: 1/4"

  • Max video resolution: 2592x1944@15fps

  • Max frame rate: QVGA@120fps

  • Maximum exposure times (seconds): 0.97

  • Selectable video resolutions: 2592x1944@15fps 1080p@30fps, 720p@60fps, 480p@90fps

  • automatic image control functions: automatic exposure control (AEC), automatic white balance (AWB), automatic band filter (ABF), automatic 50/60 Hz luminance detection, and automatic black level calibration (ABLC)

  • support for output formats: RAW RGB, RGB565/555/444, CCIR656, YUV422/420, YCbCr422, and compression

2 Setup

2.1 Enable OV5640 in Linux Kernel Configuration

1 Enter the kernel configuration by running the following command in the top directory of project.

make kconfig

2 Selects sub-menus: Device Drivers>Multimedia support>Media ancillary drivers>Camera sensor devices

image-20240322-095304.png

3 Select “OmniVision OV5640 sensor support” and press <Y>. Then save and exit.

2.2 Modify Device-tree Source

Device-tree source files of SP7350 are located in linux/kernel/arch/arm64/boot/dts/sunplus/.

Here lists all files for SP7350 boards:

Boards

Device-tree source files

C3V-W EVB

sp7350-ev.dts

Modify device tree source file to define OV5640 source for driver

2.2.1 Using SP7350 MIPI/CSI-RX4

Open OV5640_IN4 macro to add OV5640 node and link mipicsirx4.

2.2.2 Using SP7350 Other MIPI/CSI-RX

Define OV5640 node in i2c node:

i2c {
    #address-cells = <1>;
    #size-cells = <0>;

    ov5640: ov5640@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		clocks = <&ov5640_clk>;
		clock-names = "xclk";
		AVDD-supply = <&ov5640_avdd_2p8v>;
		DOVDD-supply = <&ov5640_dovdd_1p8v>;
		DVDD-supply = <&ov5640_dvdd_1p5v>;

		port {
			ov5640_out: endpoint {
				remote-endpoint = <&csi1_ep1>;
				clock-lanes = <0>;
				data-lanes = <1 2>;
			};
		};
	};
};

port: Should contain one endpoint sub-node used to model connection to the video receiver according to the specification defined in Documentation/devicetree/bindings/media/video-interfaces.txt.

Video receiver endpoint sub-node should defined in mipicsirx node. For example:

mipicsirx: csirx@f8005480 {
  port@0 {
	reg = <0>;

	/* MIPI CSI-2 bus endpoint */
	csi1_ep1: endpoint {
		remote-endpoint = <&ov5640_out>;
		bus-type = <4>;
		clock-lanes = <0>;
		data-lanes = <1 2>;
	};
  };

2.3 Compile and Burn

https://sunplus.atlassian.net/wiki/spaces/C3/pages/1988034774/A+guide+to+downloading+and+compiling+SP7350+code#7.-Build-code

2.4 Connect Sensor Module

Using C3V-W board MIPI/CSI-RX4, as follows：

image-20240322-100742.png

2.5 Check

After the burning is completed, start the board. The system should print the following information through the serial port.

[    1.905389] i2c /dev entries driver
[    2.857626] sp-csi2 f8005480.csirx: 2 lanes found
[    2.857963] sp-vin f8005980.video: assigned reserved memory node mipicsirx@4f400000
[    2.862233] ov5640 0-003c: Consider updating driver ov5640 to match on endpoints
[    2.869821] sp-vin f8005980.video: Device VIN8 registered as video0
[    2.875882] sp-vin f8005980.video: SP VIN driver probed

After the system startup is completed, the video device can be viewed.

root@dbv3:~# ls /dev/video0
/dev/video0

3 Test

3.1 v4l2-ctl Command

This command is used to control V4L2 devices from the command line. It allows you to query device capabilities, set device parameters, and capture video and still image.

Use v4l2-ctl --all --device /dev/video0 command getting device all information.

root@dbv3:~# v4l2-ctl --all --device /dev/video0
Driver Info:
	Driver name      : sp_vin
	Card type        : SP_VIN
	Bus info         : platform:f8005980.video
	Driver version   : 5.10.201
	Capabilities     : 0xa5200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x25200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : sp_vin
	Model            : sunplus,sp7350-vin
	Serial           : 
	Bus info         : platform:f8005980.video
	Media version    : 5.10.201
	Hardware revision: 0x00000000 (0)
	Driver version   : 5.10.201
Entity Info:
	ID               : 15
	Name             : VIN8 output
	Type             : V4L2 I/O
	Pad              : 0: Sink
Priority: 2
Video input : 0 (VIN8 output: ok)
Format Video Capture:
	Width/Height      : 1280/720
	Pixel Format      : 'UYVY' (UYVY 4:2:2)
	Field             : None
	Bytes per Line    : 2560
	Size Image        : 1843200
	Colorspace        : sRGB
	Transfer Function : sRGB
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Full Range
	Flags             : 

User Controls

                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=0 value=0 flags=slider
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=64 value=64 flags=slider
                            hue 0x00980903 (int)    : min=0 max=359 step=1 default=0 value=0 flags=slider
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1 flags=update
                    red_balance 0x0098090e (int)    : min=0 max=4095 step=1 default=0 value=0 flags=inactive, slider
                   blue_balance 0x0098090f (int)    : min=0 max=4095 step=1 default=0 value=0 flags=inactive, slider
                       exposure 0x00980911 (int)    : min=0 max=1276 step=1 default=560 value=0 flags=inactive, volatile
                 gain_automatic 0x00980912 (bool)   : default=1 value=1 flags=update
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=3 default=1 value=1
                alpha_component 0x00980929 (int)    : min=0 max=255 step=1 default=255 value=255

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=1 default=0 value=0 flags=update
     focus_automatic_continuous 0x009a090c (bool)   : default=0 value=1

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=24 max=2655 step=1 default=560 value=560
            horizontal_blanking 0x009e0902 (int)    : min=320 max=320 step=1 default=320 value=320 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=0 max=1023 step=1 default=0 value=0 flags=inactive, volatile

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=22 default=13 value=18 flags=read-only
                     pixel_rate 0x009f0902 (int64)  : min=48000000 max=168000000 step=1 default=48000000 value=62000000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0

3.2 Capture Image

3.2.1 Config Video Device

Use v4l2-ctl -d /dev/video0 --set-ctrl params=value setting device controls.

If your OV5640 module supports autofocus, you can execute the following command to enable it.

v4l2-ctl -d /dev/video0 --set-ctrl focus_automatic_continuous=1

3.2.2 Capture

Use v4l2-ctl --list-formats-ext --device /dev/video0 querying device video format.

root@dbv3:~# v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'UYVY' (UYVY 4:2:2)
		Size: Discrete 640x480
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 720x480
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 720x576
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 2592x1944
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
	[2]: 'BA81' (8-bit Bayer BGBG/GRGR)
		Size: Discrete 1280x720
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 2592x1944
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.017s (60.000 fps)

Use v4l2-ctl --stream capture UYVY format image.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=UYVY --stream-mmap=3 --stream-to=/home/root/OV5640_UYVY_720P.yuv --stream-skip=9 --stream-count=1

  • --set-fmt-video=width=1280,height=720,pixelformat=UYVY set image format

  • --stream-count <count> stream <count> buffers. The default is to keep streaming forever. This count does not include the number of initial skipped buffers as is passed by --stream-skip.

  • --stream-skip <count> skip the first <count> buffers. The default is 0.

  • --stream-to <file> stream to this file. The default is to discard the data. If <file> is '-', then the data is written to stdout and the --silent option is turned on automatically.

  • --stream-mmap <count> capture video using mmap() [VIDIOC_(D)QBUF] count: the number of buffers to allocate. The default is 3.

After executing the capture command, the stream data is written to /home/root/OV5640_UYVY_720P.yuv.

3.3 Display Image Raw Data

3.3.1 Download and Install Raw Image Viewer

Download raw image viewer from PixelViewer | Carina Studio. And PixelViewer is an open source project based cross-platform image which viewer supports reading raw Luminance/YUV/RGB/ARGB/Bayer pixels data from file and rendering it.

3.3.2 Use PixelViewer to Open Image

Config render params by 3.2.2 v4l2-ctl capture command

image-20240322-102852.png

4 Some considerations for storing camera video data in a file system

Writing video data from V4L2 video buffer to the file system involves two issues：

  • CPU read memory speed

  • File system write speed

In the sp7350 environment, the file system write speed during testing can be resolved by writing data to /tmp（RAM).

There are some issues with the CPU reading speed when reading data generated by Camera, which will be explained in detail below：

Three memory management models(vb2_mem_ops) have been implemented under the V4L2 video buffer 2 framework, namely dma-contiguous, dma-scatter gas, and vmalloc. According to the hardware characteristics of sp7350, the V4L2 buffer model used to receive MIPICSI-RX data can only be defined as dma-contiguous. The DMA mappings type requested by dma-contiguous is Consistent DMA mappings.

Consistent DMA mappings which are usually mapped at driver initialization, unmapped at the end and for which the hardware should guarantee that the device and the CPU can access the data in parallel and will see updates made by each other without any explicit software flushing(This function is usually achieved by disabling the CPU cache).

When the CPU reads data from the V4L2 video buffer and writes it to a file, it cannot use CPU cache to read the contents of the V4L2 video buffer. The situation will cause an increase in the required number of reads and CPU loading when reading the same data.

4.1 Why remap V4L2 video buffer dma memory

By remapping the DMA buffer, the CPU can use CPU cache to read V4L2 video buffer, improving CPU read performance and reducing CPU loading.

4.2 How to remap V4L2 video buffer dma

After system startup, execute the following command to enable remapping V4L2 video buffer.(Default disbale)

echo Y > /sys/module/videobuf2_dma_contig/parameters/dmaremap

Execute the following command to disable remapping V4L2 video buffer:

echo N > /sys/module/videobuf2_dma_contig/parameters/dmaremap

4.3 Test report

UYVY 1920*1080@30FPS (bitrate 124.416MB/s):

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --set-parm=30 --stream-mmap=10 --stream-to=/tmp/OV5640_UYVY_1080P_001.raw --stream-skip=3 --stream-count=180

CPU Loading

FPS

Enable remap

32%

30 fps

Disable remap

100%

15 fps

UYVY 1280*720@30FPS (bitrate 55.296 MB/s):

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=UYVY --set-parm=30 --stream-mmap=10 --stream-to=/tmp/OV5640_UYVY_720P_001.raw --stream-skip=3 --stream-count=180

CPU Loading

FPS

Enable remap

14.6%

30 fps

Disable remap

88.1%

30 fps

UYVY 640*480@30FPS (bitrate 18.432MB/s):

v4l2-ctl -d /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=UYVY --set-parm=30 --stream-mmap=10 --stream-to=/tmp/OV5640_UYVY_480P_001.raw --stream-skip=3 --stream-count=180

CPU Loading

FPS

Enable remap

5.2%

30 fps

Disable remap

34.7%

30 fps

5 Performance testing of OV5640 on SP7350

rootfs：Yocto

tools：v4l2-ctl

viewer： YUView

format： YUV422（UYVY）

5.1 Do not write camera data to rootfs

Test command:

nohup v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=UYVY --set-parm=60 --stream-mmap=10 --stream-count=300 > my_script.log 2>&1 &
top -d 0.5 | grep "v4l2-ctl"

Result:

Frame size/Frame rate/Format

bitrate

CPU loading

memory

FPS

drop frame

1920*1080@30FPS UYVY

124.416MB/s

1.9%

0.0

30

NO

1920*1080@15FPS UYVY

62.208MB/s

1.9%

0.0

15

NO

1280*720@60FPS UYVY

110.592MB/s

2.0%

0.0

60

NO

1280*720@30FPS UYVY

55.296MB/s

1.9%

0.0

30

NO

1280*720@15FPS UYVY

27.648MB/s

2.0%

0.0

15

NO

640*480@60FPS UYVY

36.864MB/s

1.9%

0.0

60

NO

640*480@30FPS UYVY

18.432MB/s

1.9%

0.0

30

NO

640*480@15FPS UYVY

9.216MB/s

1.9%

0.0

15

NO

5.2 Write camera data to rootfs

Test command:

echo Y > /sys/module/videobuf2_dma_contig/parameters/dmaremap
nohup v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --set-parm=30 --stream-mmap=10 --stream-count=300 --stream-to=/tmp/OV5640_UYVY_1080P_30FPS.yuv > my_script.log 2>&1 &
top -d 0.5 | grep "v4l2-ctl"

Result:

Frame size/Frame rate/Format

bitrate

CPU loading

memory

FPS

Drop frame

Video

1920*1080@30FPS UYVY

124.416MB/s

32.7%

0.0

30

NO

normal

1920*1080@15FPS UYVY

62.208MB/s

17.0%

0.0

15

NO

normal

1280*720@60FPS UYVY

110.592MB/s

30.2%

0.0

60

NO

video is dark.

1280*720@30FPS UYVY

55.296MB/s

15.1%

0.0

30

NO

normal

1280*720@15FPS UYVY

27.648MB/s

7.5%

0.0

15

NO

normal

640*480@60FPS UYVY

36.864MB/s

11.3%

0.0

60

NO

video is pinkish

640*480@30FPS UYVY

18.432MB/s

5.7%

0.0

30

NO

normal

640*480@15FPS UYVY

9.216MB/s

2.6%

0.0

15

NO

normal

  • No labels