Introduction
This document is about the video codec used with V4L2 on the C3V platform.
Before use, please ensure you already know how to use the buildroot and compile the C3V project.
Environment
This feature needs to make some changes to GStreamer and FFmpeg source code to adapt to the v4l2 codec interface and we will provide the patches for the changes. And the kernel also needs to do some config to support the video codec V4L2 driver.
Get the patches
The C3V project source code and the patches of the GStreamer and FFmpeg can be obtained from GitHub. If your GitHub has not updated the patches yet, you can also use the file provided below:
Config
Apply the patches
Here we assume that you have already configured GStreamer and FFMPEG in the buildroot and have built it successfully.
GStreamer
Apply the gst1-plugins-good-1.20.1.diff to <buildroot>/output/build/gst1-plugins-good-1.20.1/ by the command:
patch -p1 < gst1-plugins-good-1.20.1.diff
Then rebuild the plugin:
make gst1-plugins-good-rebuild
The modified library is libgstvideo4linux2.so.
FFmpeg
Apply the ffmpeg-4.4.2.diff to <buildroot>/output/build/ffmpeg-4.4.2/ by command:
patch -p1 < ffmpeg-4.4.2.diff
Then rebuild the FFmpeg:
make ffmpeg-rebuild
The modified library is libavcodec.so*.
If can not find the directory in the buildroot, please check the package on the menuconfig and reconfigure it.
After the build, copy the modified libraries to the C3V master rootfs.
Kernel support
Run the command make kconfig on the C3V master to configure the kernel.
video hardware codec driver
Device Drivers > Multimedia support > Media drivers > Video hardware codec
Then rebuild the C3V master and boot the platform.
Use
Here just introduce the basic usage with default parameters for H.264 and JPEG. If you need to change the codec parameters, please follow the usage of GStreamer and FFmpeg.
V4L2 Sample
Test sample:
This test needs a YUV source file, you can use the GStreamer to generate one:
gst-launch-1.0 -v videotestsrc horizontal-speed=15 num_buffers=100 ! "video/x-raw,width=1920,height=1080,format=NV12,framerate=30/1" ! filesink location=1080p.yuv
H.264 Encode
./v4l2_vsi_sample ifile --key 0 --name 1080p.yuv --size 1920 1080 --framenum 100 --fmt nv12 encoder --level 11 --fmt h264 --key 1 --source 0 ofile --key 2 --source 1 --name stream.h264
The command will encode the file 1080p.yuv to a file called stream.h264.
The file can be played by PC tools like PotPlayer and VLC media player.
H.264 Decode
./v4l2_vsi_sample parser --key 0 --name stream.h264 --fmt h264 --framenum 100 decoder --key 1 --source 0 --size 1920 1080 ofile --fmt nv12 --source 1 --name test.yuv
The command will decode the file stream.h264 to a file called test.yuv.
The file can be played by PC tools like Elecard YUV viewer and YuvEye.
Due to the driver's 16-byte alignment of the YUV, the height of the YUV output from the 1080p decode is 1088. However, GStreamer and FFmpeg have undergone internal processing, so the final output is still 1080.
GStreamer
H.264 Encode
Fake live stream:
gst-launch-1.0 -v videotestsrc horizontal-speed=15 num_buffers=100 ! "video/x-raw,width=1920,height=1080,format=NV12,framerate=30/1" ! timeoverlay ! v4l2h264enc ! "video/x-h264,level=(string)4" ! filesink location=gstv4l2h264enc.h264
Local file:
gst-launch-1.0 -v filesrc location=1080p.yuv ! rawvideoparse width=1920 height=1080 format=nv12 framerate=60/1 colorimetry=bt601 ! v4l2h264enc ! "video/x-h264,level=(string)4" ! filesink location=gstv4l2h264enc2.h264
The level of the v4l2h264enc must be set to 4 or 5. Otherwise, the encoder will not work properly.
H.264 Decode
gst-launch-1.0 -v filesrc location=stream.h264 ! h264parse ! v4l2h264dec ! filesink location=gstv4l2h264dec.yuv
JPEG Encode
gst-launch-1.0 -v videotestsrc num_buffers=100 horizontal-speed=15 ! "video/x-raw,width=1920,height=1080,format=NV12" ! v4l2jpegenc ! filesink location=1080p.jpg
JPEG Decode
gst-launch-1.0 -v filesrc location=1080p.jpg ! jpegparse ! v4l2jpegdec ! "video/x-raw,colorimetry=bt601" ! filesink location=og_1080p.yuv
FFmpeg
H.264 Encode
ffmpeg -benchmark -y -pix_fmt nv12 -s 1920x1080 -i 1080p.yuv -b:v 5242880 -vcodec h264_v4l2m2m -vframes 100 ffout.h264
H.264 Decode
ffmpeg -benchmark -y -r 25 -vcodec h264_v4l2m2m -i ffout.h264 -vframes 100 ffout.yuv
JPEG Encode
ffmpeg -benchmark -y -pix_fmt nv12 -s 1920x1080 -i 1080p.yuv -vcodec jpeg_v4l2m2m -vframes 5 ffout_%03d.jpg
JPEG Decode
ffmpeg -benchmark -y -vcodec jpeg_v4l2m2m -i ffout_001.jpg -vframes 1 ffout_001.yuv
