We’re excited that StarFive has ported v4l_m2m API to JH7110 SDK from the VisionFive 2 community.
V4L2 (Video for Linux 2) is an API and framework within the Linux kernel designed for video device drivers and management. The Wave511 codec functions as a stateful decoder and is integrated within the Starfive JH7110 SoC. Benefit from community’s work: [v11,0/6] Wave5 codec driver - Patchwork, the wave5 codec driver which uses v4l_m2m API is added to JH7110 SDK. V4L2_m2m is one of the APIs which is favored by many embedded boards.
The driver supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264 and has been tested on the VisionFive 2 board, and it is currently in the experimental version. Under ideal conditions, the driver got better decode performance (about 100% improvement) than OMX APIs. Share this news with users and developers who are interested in V4L2.
Is this going to work with Firefox V4L2 M2M hardware video decoding acceleration?
It should not have problem. Indeed, Firefox also use the same h264_v4l2m2m plugin of FFmpeg.
Where is this hosted for the Starfive SDK? I see that patch series was never merged into mainline, and doesn’t include any devicetree bindings for the JH7110?
My guess would be that https://gitlab.collabora.com/chipsnmedia/kernel/-/commits/wave5-dev is the location of original code. Looking at the image/link in the first post above the ported code was created 2022-12-07.
This series is under a rework process before next version.
We are testing now and will be released soon. They are upstreaming and do not merged into mainline yet.
Yes, you are right. Many thanks to Chips&Media and Collabora, we made some changes based on v11. And that is why we say it is a trail version. I do hope v12 would be OK and merge into mainline soon.
It should be working if there’s nothing wrong… Not tested yet, only tested on rpi 4b
Quick decoding speed and compatibility comparsion with gstreamer
fpsdisplaysink. Trying to only test the decoder, nothing else.
kernel 5.15.2-cwt-3.6.1-1 on Arch
wave5 kmod via v4l2
gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h264parse ! v4l2h264dec ! fpsdisplaysink text-overlay=false sync=false
rendered: 13402, dropped: 0, current: 201.79, average: 187.73
gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h265parse ! v4l2h265dec ! fpsdisplaysink text-overlay=false sync=false
rendered: 126, dropped: 0, current: 251.69, average: 251.69
rendered: 81740, dropped: 0, current: 259.11, average: 259.76 (some warning)
rendered: 5800, dropped: 0, current: 52.69, average: 53.42
wave5 decoder without any display rendering can do hevc 4k at ~54 fps
vdec kmod via omx
gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h264parse ! omxh264dec ! fpsdisplaysink text-overlay=false sync=false
rendered: 13398, dropped: 0, current: 117.16, average: 113.05
gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h265parse ! omxh265dec ! fpsdisplaysink text-overlay=false sync=false
rendered: 17289, dropped: 0, current: 105.94, average: 105.77
Double the fps on hevc 4k but slower on h264…
It was the version that was sent to the kernel mailing list for review back in December. I think they have been busy refactoring since then
wave5 kmod from collabora against 5.15.2 “out of tree”.
There is a call to
v4l2_m2m_set_dst_ignore_streaming which doesn’t exists in 5.15.2
Apply this commit as patch and the module compiles.
insmod without error, but no
/dev/video4 shows up in
kernel tree is at version 6.4.x, so was wondering if “back-porting” would work.
WAVE5 driver landed in linux-next now. Maybe Linux6.8?