VPU accelerated GStreamer Output

VisionFive 2 （English Forum)
1

Hi, I wonder if anybody already managed to get an accelerated OMX GStreamer pipeline via the WAVE511 chip running. In my current Yocto setup (latest revision of Provide support for VisionFive2 by cordlandwehr · Pull Request #382 · riscv/meta-riscv · GitHub) gstreamer-omx is crashing upon calling “OMX_UseEGLImage(…)” in gstomx.c (call stack at the end). My test is “gst-launch-1.0 --gst-debug=3 playbin uri=file:///example.mp4” which seems to correctly init the vdec Kernel module and loading the userspace libraries.
I did also do a short test on the official Debian image, but am also getting the same crash. So my question is mainly, is this rather an integration topic or are there still implementations missing?

Thread 14 “omxh264dec-omxh” received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x3fcbfff180 (LWP 743)]
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x0000003fde68a5d2 in gst_omx_port_allocate_buffers_unlocked (port=port@entry=0x3fe0055010, buffers=buffers@entry=0x0, images=images@entry=0x2aaac74160, n=14) at /usr/src/debug/sf-gst-omx/1.18.5-r0/omx/gstomx.c:2629
#2 0x0000003fde68a84a in gst_omx_port_use_eglimages (port=port@entry=0x3fe0055010, images=images@entry=0x2aaac74160) at /usr/src/debug/sf-gst-omx/1.18.5-r0/omx/gstomx.c:2870
#3 0x0000003fde68f43a in gst_omx_video_dec_allocate_output_buffers (self=self@entry=0x3fe006fbc0) at /usr/src/debug/sf-gst-omx/1.18.5-r0/omx/gstomxvideodec.c:966
#4 0x0000003fde690ad4 in gst_omx_video_dec_reconfigure_output_port (self=0x3fe006fbc0) at /usr/src/debug/sf-gst-omx/1.18.5-r0/omx/gstomxvideodec.c:1619
#5 gst_omx_video_dec_loop (self=0x3fe006fbc0) at /usr/src/debug/sf-gst-omx/1.18.5-r0/omx/gstomxvideodec.c:1805
#6 0x0000003ff7f7f796 in ?? () from /usr/lib/libgstreamer-1.0.so.0

2 Likes
2

I don’t have an answer to your question, but I’m interested in getting involved with your Yocto efforts to learn. I have some experience in both Yocto and GStreamer, but not to this level. I recently received my VF2 4GB and was able to build core-image-minimal from your branch. I’m currently working on getting the board booting when I have available hours.

3

@cordlandwehr please use this patch to try it.

4

Hi @cordlandwehr

Did you ever make any progress with this? I’m hoping to build the wave511/vdec capable ffmpeg but I need to build the kernel module for Debian first and it seems its Makefile is only configured for cross compiling it under buildroot.

Could you please share the Makefile you used to build the wave511 kernel module under Debian please?

Thanks

5

@danboid I got a little bit distracted by starting the Qt6 porting of the KDE layers at about the time when the fix was published… That port finally comes to a stage where it makes sense to look into video decoding again. So hopefully I can give you an answer in about 1-2 weeks.
Regarding building of the kernel module, best have a look at these two files:

The first is the build recipe (in Bitbake syntax). The interesting points/hacks there are going to the subfolder “git/wave511/code” and creating a symlink (yes, horrible hack) “ln -sf . ./wave511”. The latter file is my fork of the Makefile that should work given the previous hack.

3 Likes
6

Hi, a short update: I finally was able to get an MPEG4 video to run very smoothly via HW acceleration on my board. My current setup is based on the VisionFive2 SDK 3.0.4 regarding Kernel, OMX and decoder libs (I will send meta-riscv update patches soon for anybody interested in Yocto). The rest is a very much work-in-progress setup of a Qt6.5 based KDE Plasma session. The video integration uses QtMultimedia, which itself relies on ffmpeg internally.
As soon as I get the Plasma Shell stabilized, patches to meta-kde will follow; looking into GStreamer is on my plan only after that.

7 Likes
7

@cordlandwehr maybe we can collaborate? I’ve been working on an image for the star64 and I’ve packed up all the KDE-gear apps as a meta-layer.

I’ve also updated your layers the latest releases for kf5 and plasma? (All under my name in GitHub. I’ve yet to setup a invent account to open a PR)

2 Likes
8

@Fishwaldo absolutely! I would really like to see more collaboration around the KDE meta layers. At the moment my personal focus is mostly around the Qt6 ports so I specifically would be happy for people helping the Qt5 world in shape.
If you would create an invent account, gladly assign me as reviewer. In the long run I would also like to see the Gear apps integrated in the meta-kde layer as a central place of up-to-date KDE applications. BTW our (as in people in the KDE community working on Yocto) main discussion place is here: https://matrix.to/#/#yocto:kde.org

3 Likes
9

Btw, FIrefox nightly now supports v4l2-m2m now. It should work with the under review mainline wave5 driver.

5 Likes
10

gstreamer has retired gst-omx OpenMAX support.

This “breaks” omx-il (Starfive Openmax IL Shimlayer), as the only user level interface to hardware decoding/encoding.

11

Newest image has switched to v4l2-m2m driver for wave5 decoder.

1 Like
12

Documenting error and solution that took me hours…

Error: when running encoder/decoder via omx/ffmpeg/gstreamer

“EncoderThread 805 VPU_EncGetInitialInfo failed Error code is 0x1”

Solution: remove and re-add the venc / vdec kernel modules.

13

@leuc You mean like this?

V4l2

By default, v4l2 is blacklisted module. To load the module, refer to below steps.
    modprobe -r  venc vdec jpu
    modprobe wave5

https://rvspace.org/en/project/VisionFive2_Debian_Wiki_202308_Release

1 Like
14

No, in this case the kmod driver had apparently died. This specific error occurs when the encoder api is not longer initializing correctly.
rmmod venc ; rmmod vdec ; rmmod jpu ; modprobe venc ; modprobe vdec ; modprobe jpu