Content Comparison

...

Introduction

This document is about the usage of hardware video codec used with V4L2 framework on the C3V platform.

Panel

panelIconId	atlassian-info
panelIcon	:info:
bgColor	#FFBDAD#FFFAE6

Before use, please ensure you already know how to use compile the buildroot and compile the C3V projectBuildroot and Ubuntu system for C3V platform.

Environment

This feature needs to make some changes to GStreamer and FFmpeg source code to adapt to the v4l2 codec interface and The kernel needs to do some config to support the video codec V4L2 driver.

Kernel Suport

Run the command make kconfig to configure the kernel driver.

video hardware codec driver

Device Drivers > Multimedia support > Media drivers > Video hardware codec

...

Info
Currently, most boards already support this by default, you can follow the above method to check if there is enabled.

Get the Patches and Build

To use the video codec v4l2 framework in GStreamer and FFmpeg, we need to make some changes to the source code to adapt it. And we will provide the patches for the changes. And the kernel also needs to do some config to support the video codec V4L2 driver.

...

The C3V project source code and the patches of the GStreamer and FFmpeg can be obtained from GitHub . If your GitHub has not updated the patches yet, you can also use the file provided below: View filenamewith the C3V source code.

For the buildroot system:
- GStreamer patches: “<code_root>/linux/rootfs/initramfs/buildroot/package/gstreamer1/gst1-plugins-good

...

View file

name	ffmpeg-4.4.2.diff

Config

Apply the patches

Here we assume that you have already configured GStreamer and FFMPEG in the buildroot and have built it successfully.

GStreamer

...

Code Block
patch -p1 < gst1-plugins-good-1.20.1.diff

Then rebuild the plugin:

...

language	bash

- ”.
- FFmpeg patches “<code_root>/linux/rootfs/initramfs/buildroot/package/ffmpeg” .

After make config, use make bconfig to config the packages and use make buildrootto compile them. The patches will be applied automatically during buildroot building.

If you change the source code and want to rebuild the packages, just run the above two commands to rebuild the buildroot. The updated libs will be installed to the rootfs directory automatically. You can run make to pack it to the ISP BIN file, or you can just copy the libs to the C3V platform for using.

Alterbatuvely, run make <pkg-name>-rebuild under “<code_root>/linux/rootfs/initramfs/buildroot” to recompile the target package only, such as

Code Block
make ffmpeg-rebuild make gst1-plugins-good-rebuild

The modified library is libgstvideo4linux2.so.

FFmpeg

Apply the ffmpeg-4.4.2.diff to <buildroot>/output/build/ffmpeg-4.4.2/ by command:

Code Block

language	bash

patch -p1
< ffmpeg-4.4.2.diff

Then rebuild the FFmpeg:

Code Block

language	bash

make ffmpeggstreamer1-rebuild

The

...

Panel

panelIconId	atlassian-info
panelIcon	:info:
bgColor	#E6FCFF

If can not find the directory in the buildroot, please check the package on the menuconfig and reconfigure it.

After the build, copy the modified libraries to the C3V master rootfs.

Kernel support

Run the command make kconfig on the C3V master to configure the kernel.

video hardware codec driver

Device Drivers > Multimedia support > Media drivers > Video hardware codec

...

“<pkg-name>” can be find under the path “<code_root>/linux/rootfs/initramfs/buildroot/package”. In this way, the updated libs will not be installed to the rootfs directory automatically, you need copy it the by yourself if you want to pack them to the ISP BIN.

For the Ubuntu system:
- Please refer to Build and install specified GStreamer on the Ubuntu for GStreamer.
- Please refer to Build and install specified FFmpeg on the Ubuntu for FFmpeg.

Use

Here just introduce the basic usage with default parameters for H.264 and JPEG. If you need to change the codec parameters, please follow the usage of GStreamer and FFmpeg.

V4L2 Sample

Test sample:

View file

name	v4l2_vsi_sample

This is a V4L2 video codec test sample to test the codec functions without the media framework such as the GStreamer and FFmpeg.

The test needs a YUV source file, you can use the GStreamer to generate a fake one, or you can use the V4L2 utils to get it from the sensor:

Code Block

language	text

gst-launch-1.0 -v videotestsrc horizontal-speed=15 num_buffers=100 ! "video/x-raw,width=1920,height=1080,format=NV12,framerate=30/1" ! filesink location=1080p.yuv

...

H.264 Encode

Code Block
./v4l2_vsi_sample ifile --key 0 --name 1080p.yuv --size 1920 1080 --framenum 100 --fmt nv12 encoder --level 11 --fmt h264 --key 1 --source 0 ofile --key 2 --source 1 --name stream.h264

The command will encode the file 1080p.yuv to a file called stream.h264.

...

The file can be played by PC tools like PotPlayer and VLC media player.

H.264 Decode

Code Block
./v4l2_vsi_sample parser --key 0 --name stream.h264 --fmt h264 --framenum 100 decoder --key 1 --source 0 --size 1920 1080 ofile --fmt nv12 --source 1 --name test.yuv

The command will decode the file stream.h264 to a file called test.yuv.

...

The file can be played by PC tools like Elecard YUV viewer and YuvEye.

Panel

panelIconId	atlassian-info
panelIcon	:info:
bgColor	#DEEBFF

Due to the driver's 16-byte alignment of the YUV, the height of the YUV output from the 1080p decode is 1088. However, the GStreamer and FFmpeg have undergone internal processing, so the final output height is still 1080.

GStreamer Command Line

H.264 Encode

Fake live stream:

Code Block

gst-launch-1.0 -v videotestsrc horizontal-speed=15 num_buffers=100  ! "video/x-raw,width=1920,height=1080,format=NV12,framerate=30/1" ! timeoverlay ! v4l2h264enc ! "video/x-h264,level=(string)4" ! filesink location=gstv4l2h264enc.h264

...

Code Block

gst-launch-1.0 -v filesrc location=1080p.yuv ! rawvideoparse width=1920 height=1080 format=nv12 framerate=60/1 colorimetry=bt601 ! v4l2h264enc ! "video/x-h264,level=(string)4" ! filesink location=gstv4l2h264enc2.h264

...

...
The level of the v4l2h264enc must be set to 4 or 5. Otherwise, the encoder will not work properly.

H.264 Decode

Code Block
gst-launch-1.0 -v filesrc location=stream.h264 ! h264parse ! v4l2h264dec ! filesink location=gstv4l2h264dec.yuv

...

JPEG Encode

Code Block
gst-launch-1.0 -v videotestsrc num_buffers=100 horizontal-speed=15 ! "video/x-raw,width=1920,height=1080,format=NV12" ! v4l2jpegenc ! filesink location=1080p.jpg

...

JPEG Decode

Code Block
gst-launch-1.0 -v filesrc location=1080p.jpg ! jpegparse ! v4l2jpegdec ! "video/x-raw,colorimetry=bt601" ! filesink location=og_1080p.yuv

...

FFmpeg Command Line

H.264 Encode

Code Block
ffmpeg -benchmark -y -pix_fmt nv12 -s 1920x1080 -i 1080p.yuv -b:v 5242880 -vcodec h264_v4l2m2m -vframes 100 ffout.h264

...

H.264 Decode

Code Block
ffmpeg -benchmark -y -r 25 -vcodec h264_v4l2m2m -i ffout.h264 -vframes 100 ffout.yuv

...

JPEG Encode

Code Block
ffmpeg -benchmark -y -pix_fmt nv12 -s 1920x1080 -i 1080p.yuv -vcodec jpeg_v4l2m2m -vframes 5 ffout_%03d.jpg

...

JPEG Decode

Code Block
ffmpeg -benchmark -y -vcodec jpeg_v4l2m2m -i ffout_001.jpg -vframes 1 ffout_001.yuv

...

GStreamer Coding

Only sample code is provided here, please make sure you already know how to code with GStreamer.

H.264 Encode

Code Block
GstElement h264_enc = gst_element_factory_make("v4l2h264enc", NULL); g_assert(h264_enc); g_object_set(h264_enc, "gop-size", 15, "bitrate", 1024 1024 * 3, NULL);

Normally, the filter “h264parse” needs to follow the encoder to adpat to different application scenarios, such as MP4 recoding or network transmission.

H.264 Decode

Code Block
GstElement *h264_dec = gst_element_factory_make("v4l2h264dec", NULL); g_assert(h264_dec);

JPEG Encode

Code Block
GstElement *jpeg_enc = gst_element_factory_make("v4l2jpegenc", NULL); g_assert(jpeg_enc ); g_object_set(jpeg_enc, "qp", 5, NULL); //0~10

JPEG Decode

Code Block
GstElement *jpeg_dec = gst_element_factory_make("v4l2jpegdec", NULL); g_assert(jpeg_dec);

FFmpeg Coding

Only sample code is provided here, please make sure you already know how to code with FFmpeg.

H.264 Encode

Code Block

AVCodec *pCodec = avcodec_find_encoder_by_name("h264_v4l2m2m");
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);

pCodecCtx->codec_id = AV_CODEC_ID_H264;
pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
pCodecCtx->pix_fmt = pix_fmt;
pCodecCtx->width = width;
pCodecCtx->height = height;
pCodecCtx->framerate.num = fps;
pCodecCtx->framerate.den = 1;
pCodecCtx->time_base.num = time_base.num;
pCodecCtx->time_base.den = time_base.den;
pCodecCtx->gop_size = gop;
pCodecCtx->bit_rate = bitrate;

av_opt_set(pCodecCtx->priv_data, "spspps_to_idr", 1, 0); //Add spspps to each idr header

H.264 Decode

Code Block

AVCodec *pCodec = avcodec_find_decoder_by_name("h264_v4l2m2m");
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);

pCodecCtx->codec_id = AV_CODEC_ID_H264;
pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
pCodecCtx->width = width;
pCodecCtx->height = height;
pCodecCtx->time_base.num = time_base.num;
pCodecCtx->time_base.den = time_base.den;
pCodecCtx->pkt_timebase = pCodecCtx->time_base;

JPEG Encode

Code Block

AVCodec *pCodec = avcodec_find_encoder_by_name("jpeg_v4l2m2m");
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);

pCodecCtx->codec_id = AV_CODEC_ID_MJPEG;
pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
pCodecCtx->pix_fmt = pix_fmt;
pCodecCtx->width = width;
pCodecCtx->height = height;

av_opt_set(pCodecCtx->priv_data, "qp", 5, 0); //Set jpeg compress level

JPEG Decode

Code Block

AVCodec *pCodec = avcodec_find_decoder_by_name("jpeg_v4l2m2m");
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);

pCodecCtx->codec_id = AV_CODEC_ID_MJPEG;
pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
pCodecCtx->width = width;
pCodecCtx->height = height;

Reference

https://sunplus.atlassian.net/wiki/x/A4CVgw

Version	Old Version 9	New Version Current
Changes made by	akira.yu	wang.zhou
Saved on	Aug 28, 2024	Feb 07, 2025

Versions Compared

Key

Introduction

Environment

Kernel Suport

video hardware codec driver

Get the Patches and Build

Config

Apply the patches

FFmpeg

Kernel support

Use

V4L2 Sample

H.264 Encode

H.264 Decode

GStreamer Command Line

H.264 Encode

...
The level of the v4l2h264enc must be set to 4 or 5. Otherwise, the encoder will not work properly.

H.264 Decode

JPEG Encode

JPEG Decode

FFmpeg Command Line

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

GStreamer Coding

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

FFmpeg Coding

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

Reference

Content Comparison

Versions Compared

Key

<span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-0">[data-colorid=</span>

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-0">wyouwl2q7k</span><span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-1">]{color:</span>

<span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-2">#006644} html[data-color-mode=dark] [data-colorid=</span>

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-2">y4lng73sd2</span><span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-4">]{color:</span>

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-4">y4lng73sd2</span><span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-6">]{color:</span>

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-6">swcnz2njn4</span><span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-8">]{color:</span>

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-8">swcnz2njn4</span><span class="diff-html-changed" data-a11y-before="Start of changed content" data-a11y-after="End of changed content" id="changed-diff-10">]{color:</span>

Environment

Kernel Suport

video hardware codec driver

Get the Patches and Build

Config

Apply the patches

FFmpeg

Kernel support

Use

V4L2 Sample

H.264 Encode

H.264 Decode

GStreamer Command Line

H.264 Encode

...The level of the v4l2h264enc must be set to 4 or 5. Otherwise, the encoder will not work properly.

H.264 Decode

JPEG Encode

JPEG Decode

FFmpeg Command Line

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

GStreamer Coding

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

FFmpeg Coding

H.264 Encode

H.264 Decode

JPEG Encode

JPEG Decode

Reference

...
The level of the v4l2h264enc must be set to 4 or 5. Otherwise, the encoder will not work properly.