DRM format support

Hi,

Is there a list of DRM pixel formats that the display controller can really handle?

Currently the following formats are exposed by the kernel:

  • RGB: RGB565, XRGB8888, ARGB8888, ABGR8888
  • Packed YCbCr: UYVY, VYUY, YUYV, YVYU
  • Semiplanar YCbCr: NV12, NV21
  • Planar YCbCr: YUV420

But:

  • NV12: chrominance plane is off (I believe the driver discards the plane offset and object ID).
  • NV21: fails completely to create a frame buffer
  • YUV420: chrominance planes are off (same problem as NV12)
  • ABGR8888: red and blue components are swapped.

YUYV, UYVY, YVYU and ARGB8888 seem to work fine. I was unable to test RGB565, XRGB8888 and VYUY so far.

Now I’m wondering if those issue are caused by driver bugs or if they are hardware limitations?

Thank you very much for your willingness to help us improve the program functions.

To answer your question, first of all, XRGB8888, ARGB8888, ABGR8888 formats are not supported. In the process of our verification, VYUY can also work, we have not found such a situation for NV12/NV21/YUV420, please provide the testing equipment and display, as well as the recurrence method, so that we can locate the problem more quickly.

In addition, attach all the formats we currently support:
Formats: RG16 UYVY VYUY YUYV YVYU YU12 NV21 NV12 XR24 AR24 AB24

Hi,

I think there is some name confusion here. ARGB8888 is the name for FourCC “AR24”, and so on (see <drm/drm_fourcc.h>). The drm_info tool uses the full names, not the FourCC’s.

The list of formats I wrote down is that exposed by the current version (as of yesterday) of the StarFive DRM driver

we have not found such a situation for NV12/NV21/YUV420, please provide the testing equipment and display

For YUV sources, anything goes, but the simplest option is the vivid V4L2 test driver, which supports a broad range of pixel formats.

I have an LG TV set for output via HDMI, but I doubt that really matters.

It seems to me that the DRM driver is ignoring the plane parameters, handles[], pitches[] and offsets[] from struct drm_mode_fb_cmd2 in ioctl(DRM_IOCTL_MODE_ADDFB2). For single plane formats, this is most inconsequential as offsets is pretty much always 0, but for multiple planes, this leads to misaligned colors.

Br

1 Like

For the user-space software side, I am currently using VLC. I made some build guidelines here: https://code.videolan.org/Courmisch/vlc/-/wikis/StarFive-JH7100#natively

For instance, to test YUV420:

./vlc -Idummy -vv v4l2:///dev/video0:chroma=I420:width=1920:height=1080:fps=1/5

It should be fine to use gstreamer, mpv or other similar tools as well.

1 Like

We haven’t tested VLC. You can try the following instructions:
modetest -M starfive -D 0 -a -s 34@32:1920x1080 -P 31@32:1920x1080@NV21 -Ftiles

Hi Rémi, you can use modetest -M starfive to checkout the supported formats. it’s including:

UYVY VYUY YUYV YVYU YU12 NV21 NV12 AR24 AB24 RA24 BA24 RG16

by the way, please input modetest -M starfive -D 0 -a -s 34@32:1920x1080 -P 31@32:1920x1080@RG16 -Fsmpte to test RGB565

@remid do you have ported VLC on Fedora image? and natively compile on board, right? that’s cool , I would like to try it.

moreover, we have supported vpu(enc/dec) based on openmax, please checkout
https://github.com/starfive-tech/freelight-u-sdk/releases/tag/SDK_v2.0.0

for plane question, yes, JH7100 only support one simple plane. however, our next generation SoC JH7110 will support multiple planes.

Hi,

To be honest, RGB565 is mostly useless for video use-cases in general. And in the case of the JH7100, there is neither GPU nor vectors, so it is not realistic computationally to convert the YUV planes from the video decoder into RGB colour space. We really have to pass the pictures straight from decoder to display controller.

The most two needed formats will be:

  • YUV420 (a.k.a. DRM “YU12”) and
  • the formats output by the VPU (I suppose NV12 or NV21?).

The build instructions there should work on Fedora, but you need to adjust the dependency package list:

It takes a while, so be sure to go grab a cup of tea or coffee…

Actually… if you use the smpte pattern instead of tiles, you get the standard test card.

AB24 and NV21 render wrong colours.

Thanks to this discovery, we will fix this display format difference as soon as possible.
NV21 rebder wrong is a known issue, There will be a version update recently

What are the most common formats for VLC?

Whence decoding video on the CPU, planar YUV 4:2:0 (a.k.a. YU12 a.k.a. I420) is by far the most common. However, that probably will not work without the RISC-V V extension.

So then it depends what format the hardware decoder wants to use. It can be YU12, NV12, or even a tiled format, as long as the VPU hardware and the display controller hardware can agree on the format.

So I tried that but the vdec.ko device driver fails to load:

[  221.664928] vdec: loading out-of-tree module taints kernel.
[  221.667530] vdec 118f0000.vpu_dec: coherent device 0 dev->dma_coherent 0
[  221.667703] vdec 118f0000.vpu_dec: device init.
[  221.667809] SUCCESS alloc_chrdev_region
[  221.667894] vdec 118f0000.vpu_dec: vdec_axi
[  221.667972] vdec 118f0000.vpu_dec: failed to get vdec clock
[  221.668089] vdec: probe of 118f0000.vpu_dec failed with error -2

I think the Device Tree node is incomplete. The documentation (in Chinese) seems to have a more complete DT entry, but it seems for some old FPGA simulation, so I’m not sure what to do…

In principles, VLC already has OpenMAX IL support. All that’s needed is the name of the library to load, since unfortunately OpenMAX did not standardise it.

There’s also support for using gstreamer decoders, though that’s so far only been used for Texas Instruments’.

It’s also possible to write a new decoder directly to the VPU API.

With that said, I have not actually tried the Freelight SDK. It seems that the VPU drivers are managing device registers, memory buffers and interruptions in user space. Normally the kernel takes care of it. User mode is inadequate for VLC use cases as it provides no user isolation nor protection for memory, other processes and kernel in case of problem in the VLC process.

So overall, it’s not clear to me what API should be used as of yet.