StarFive Successfully Ported V4L2 to JH7110 SDK

VisionFive 2 （English Forum)
, ,
1

We’re excited that StarFive has ported v4l_m2m API to JH7110 SDK from the VisionFive 2 community.

V4L2 (Video for Linux 2) is an API and framework within the Linux kernel designed for video device drivers and management. The Wave511 codec functions as a stateful decoder and is integrated within the Starfive JH7110 SoC. Benefit from community’s work: [v11,0/6] Wave5 codec driver - Patchwork, the wave5 codec driver which uses v4l_m2m API is added to JH7110 SDK. V4L2_m2m is one of the APIs which is favored by many embedded boards.

111
111733×471 24.1 KB

The driver supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264 and has been tested on the VisionFive 2 board, and it is currently in the experimental version. Under ideal conditions, the driver got better decode performance (about 100% improvement) than OMX APIs. Share this news with users and developers who are interested in V4L2.

8 Likes
2

Is this going to work with Firefox V4L2 M2M hardware video decoding acceleration?

3

It should not have problem. Indeed, Firefox also use the same h264_v4l2m2m plugin of FFmpeg.

2 Likes
4

Where is this hosted for the Starfive SDK? I see that patch series was never merged into mainline, and doesn’t include any devicetree bindings for the JH7110?

2 Likes
5

My guess would be that https://gitlab.collabora.com/chipsnmedia/kernel/-/commits/wave5-dev is the location of original code. Looking at the image/link in the first post above the ported code was created 2022-12-07.

1 Like
6

This series is under a rework process before next version.

1 Like
7

We are testing now and will be released soon. They are upstreaming and do not merged into mainline yet.

8

Yes, you are right. Many thanks to Chips&Media and Collabora, we made some changes based on v11. And that is why we say it is a trail version. I do hope v12 would be OK and merge into mainline soon.

5 Likes
9

It should be working if there’s nothing wrong… Not tested yet, only tested on rpi 4b

10

Excellent news!

11

Quick decoding speed and compatibility comparsion with gstreamer fpsdisplaysink. Trying to only test the decoder, nothing else.

kernel 5.15.2-cwt-3.6.1-1 on Arch

wave5 kmod via v4l2

h264

gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h264parse ! v4l2h264dec ! fpsdisplaysink text-overlay=false sync=false

  • h264.high.avc1.yuv420p.1280x720.60fps.mkv fails

  • h264.high.yuv420p.1800x1080.25fps.mkv
    rendered: 13402, dropped: 0, current: 201.79, average: 187.73

h265

gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h265parse ! v4l2h265dec ! fpsdisplaysink text-overlay=false sync=false

  • hevc.main.yuv420p.1480x1080.23.98fps.mkv
    rendered: 126, dropped: 0, current: 251.69, average: 251.69

  • hevc.main10.yuv420p10le.1896x816.23.98fps.mkv
    rendered: 81740, dropped: 0, current: 259.11, average: 259.76 (some warning)

  • hevc.main10.yuv420p10le.3832x1596.23.98fps.mkv fails

  • hevc.main10.yuv420p10le.3840x2160.23.98fps.mkv
    rendered: 5800, dropped: 0, current: 52.69, average: 53.42

wave5 decoder without any display rendering can do hevc 4k at ~54 fps

vdec kmod via omx

h264

gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h264parse ! omxh264dec ! fpsdisplaysink text-overlay=false sync=false
  • h264.high.yuv420p.1800x1080.25fps.mkv
    rendered: 13398, dropped: 0, current: 117.16, average: 113.05

h265

gst-launch-1.0 -v filesrc location=filename ! matroskademux ! h265parse ! omxh265dec ! fpsdisplaysink text-overlay=false sync=false
  • hevc.main10.yuv420p10le.3840x2160.23.98fps.mkv
    rendered: 17289, dropped: 0, current: 105.94, average: 105.77

Double the fps on hevc 4k but slower on h264…

2 Likes
12

Can anyone clarify what revision/commit of the wave5 driver went into v3.6.1?

https://github.com/starfive-tech/linux/commits/VF2_v3.6.1/drivers/media/platform/chips-media/wave5

only mentions:

media: add wave5 v4l2 driver from mail list

It should be based off https://gitlab.collabora.com/chipsnmedia/kernel/-/commits/wave5-dev right?

13

It was the version that was sent to the kernel mailing list for review back in December. I think they have been busy refactoring since then

2 Likes
14

Tried building wave5 kmod from collabora against 5.15.2 “out of tree”.

There is a call to v4l2_m2m_set_dst_ignore_streaming which doesn’t exists in 5.15.2

Apply this commit as patch and the module compiles.

https://gitlab.collabora.com/chipsnmedia/kernel/-/commit/ac59eafd5076c4deb3bfe1fb85b3b776586ef3eb.patch

It does insmod without error, but no /dev/video4 shows up in v4l2-ctl --list-devices

kernel tree is at version 6.4.x, so was wondering if “back-porting” would work.

15

Pending merge request to enable wave511, the decoder on vf2

https://gitlab.collabora.com/chipsnmedia/kernel/-/merge_requests/74

2 Likes
16

WAVE5 driver landed in linux-next now. Maybe Linux6.8?

1 Like