cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Observer
Observer
1,651 Views
Registered: ‎08-14-2018

GStreamer v4l2 to kms (DMA buffer import fails)

I am using a ZCU104 board and meta-xilinx release 2019.1.

The gstreamer pipeline which i would like to run looks like:

gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=5/1, format=NV12 ! tee ! kmssink

Where /dev/video1 is my v4l2 video capture device.

So the v4l2src allocates the dma buffer pool and the kmssink should import the dma buffers.

However I get the following errors:

...
0:00:00.702172717 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1593:gst_kms_sink_import_dmabuf:<kmssink0> Found a dmabuf with 2 planes and 1 memories 0:00:00.702255447 5151 0x55be3888b0 LOG kmssink gstkmssink.c:1641:gst_kms_sink_import_dmabuf:<kmssink0> found these prime ids: 24, 24, 0, 0 0:00:00.702944971 5151 0x55be3888b0 ERROR kmsallocator gstkmsallocator.c:596:gst_kms_allocator_dmabuf_import:<KMSMemory::allocator> Failed to import prime fd 24: Operation not permitted (-1) 0:00:00.702980502 5151 0x55be3888b0 INFO GST_PERFORMANCE gstkmssink.c:1744:gst_kms_sink_get_input_buffer:<kmssink0> frame copy 0:00:00.760318267 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:261:gst_kms_allocator_memory_create:<KMSMemory::allocator> Created BO plane 0 with stride 2048 and offset 0 0:00:00.760394818 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:261:gst_kms_allocator_memory_create:<KMSMemory::allocator> Created BO plane 1 with stride 2048 and offset 2211840 0:00:00.763235686 5151 0x55be3888b0 LOG kmsallocator gstkmsallocator.c:514:gst_kms_memory_add_fb:<KMSMemory::allocator> [0] pitch 1920, offset 0 0:00:00.763305786 5151 0x55be3888b0 LOG kmsallocator gstkmsallocator.c:514:gst_kms_memory_add_fb:<KMSMemory::allocator> [1] pitch 1920, offset 2211840 0:00:00.763340386 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:518:gst_kms_memory_add_fb:<KMSMemory::allocator> bo handles: 2, 2, 0, 0 0:00:00.763427307 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:528:gst_kms_memory_add_fb:<KMSMemory::allocator> Create FB 48 0:00:00.763460557 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1804:gst_kms_sink_show_frame:<kmssink0> displaying fb 48 0:00:00.763505118 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1864:gst_kms_sink_show_frame:<kmssink0> drmModeSetPlane at (0,0) 1920x1080 sourcing at (0,0) 1920x1080 0:00:00.774469307 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:112:gst_kms_memory_remove_fb:<KMSMemory::allocator> removing fb id 47 0:00:00.775066781 5151 0x55be3888b0 INFO v4l2src gstv4l2src.c:1189:gst_v4l2src_create:<v4l2src0> sync to 0:00:00.400000000 out ts 0:00:00.227828314 0:00:00.902461033 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1593:gst_kms_sink_import_dmabuf:<kmssink0> Found a dmabuf with 2 planes and 1 memories 0:00:00.902543024 5151 0x55be3888b0 LOG kmssink gstkmssink.c:1641:gst_kms_sink_import_dmabuf:<kmssink0> found these prime ids: 26, 26, 0, 0 0:00:00.903223458 5151 0x55be3888b0 ERROR kmsallocator gstkmsallocator.c:596:gst_kms_allocator_dmabuf_import:<KMSMemory::allocator> Failed to import prime fd 26: Operation not permitted (-1) 0:00:00.903255338 5151 0x55be3888b0 INFO GST_PERFORMANCE gstkmssink.c:1744:gst_kms_sink_get_input_buffer:<kmssink0> frame copy 0:00:00.958484711 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:261:gst_kms_allocator_memory_create:<KMSMemory::allocator> Created BO plane 0 with stride 2048 and offset 0 0:00:00.958548351 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:261:gst_kms_allocator_memory_create:<KMSMemory::allocator> Created BO plane 1 with stride 2048 and offset 2211840 0:00:00.961004667 5151 0x55be3888b0 LOG kmsallocator gstkmsallocator.c:514:gst_kms_memory_add_fb:<KMSMemory::allocator> [0] pitch 1920, offset 0 0:00:00.961840212 5151 0x55be3888b0 LOG kmsallocator gstkmsallocator.c:514:gst_kms_memory_add_fb:<KMSMemory::allocator> [1] pitch 1920, offset 2211840 0:00:00.961878192 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:518:gst_kms_memory_add_fb:<KMSMemory::allocator> bo handles: 1, 1, 0, 0 0:00:00.961955453 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:528:gst_kms_memory_add_fb:<KMSMemory::allocator> Create FB 47 0:00:00.961986653 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1804:gst_kms_sink_show_frame:<kmssink0> displaying fb 47 0:00:00.962035663 5151 0x55be3888b0 TRACE kmssink gstkmssink.c:1864:gst_kms_sink_show_frame:<kmssink0> drmModeSetPlane at (0,0) 1920x1080 sourcing at (0,0) 1920x1080 0:00:00.974535293 5151 0x55be3888b0 DEBUG kmsallocator gstkmsallocator.c:112:gst_kms_memory_remove_fb:<KMSMemory::allocator> removing fb id 48 0:00:00.975186207 5151 0x55be3888b0 INFO v4l2src gstv4l2src.c:1189:gst_v4l2src_create:<v4l2src0> sync to 0:00:00.600000000 out ts 0:00:00.4278326
....

The dma_buffer_import seems to fail and a copy of the buffer is made.

I peeked into the source code of the kmssink and the `drmPrimeFDToHandle` function in `gst_kms_allocator_dmabuf_importhttps://github.com/GStreamer/gst-plugins-bad/blob/1.14/sys/kms/gstkmsallocator.c#L534 is the function that fails. And that's the reason a copy is made, however this results only in a green screen for the NV12 format.

The pipeline keeps on running fine.

I tried another format e.g. UYVY and the behaviour is the same except that instead of a green screen i get the acutal picture. (DMAimport failure and copy are still there)

Now I am required to use NV12 format and I would prefer to do so with zero-copy.

Any suggestions to

  1. What goes wrong with the DMA import? "Failed to import prime fd 26: Operation not permitted (-1)"
  2. Why does the copy of the NV12 buffer result in a green screen? (I also saved the same buffers into a video file and the video file is just fine. So the input buffers are not empty).

would be very well appreciated.

Cheers, David

 

P.S. I also tried the io-mode=5 (the kmssink allocating the buffer pool)

gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=5 ! video/x-raw, width=1920, height=1080, framerate=5/1, format=NV12 ! kmssink

which works fine. However i need a `tee` element and hence the upstream allocation of the buffer pool. With the later approach I realised that the stride of the buffers allocated by the kmssink is 2048 opposed to the stride of the v4l2src buffers which is 1920. However I don't know how I could influence the stride of the v4l2src buffers. Are there some requirements of the kms driver regarding memory alignment / stride of the imported buffers?

0 Kudos
8 Replies
Highlighted
Moderator
Moderator
1,530 Views
Registered: ‎11-09-2015

HI @truessd 

Are you using a Video Mixer in your design? I believe this is required for the KMS pipeline to work.

You might want to refer to the ZCU102 (ZynqUS+ Base) TRD or ZCU106 TRD for reference design using Gstreamer pipelines.

Regards


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Highlighted
Observer
Observer
1,524 Views
Registered: ‎08-14-2018

Hi @florentw 

Thanks for your answer. We are not using the Video Mixer, but the Zynqmp DisplayPort core, the same as in the TRD.

Should this not be the same for the KMS/DRM driver?

Kind regards

 

0 Kudos
Highlighted
Moderator
Moderator
1,512 Views
Registered: ‎11-09-2015

HI @truessd 

You are right, I forgot that the Displayport was also able to do mixing.

I have a doubt about the 420 support of the DP controller. Looking at the figure 33-2 in UG1085, it seems that 420 is only supported through the live input or internal pattern generator:

DP.JPG

If you are using the DPDMA, then you need to be in YUV444.

This might be why you are getting the error in both cases. But am not exactly sure why you get an output with YUYV. I would expect that it fails as well to dispaly


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Highlighted
Observer
Observer
1,498 Views
Registered: ‎08-14-2018

@florentw 

So we thought the two arrows pointing down from "DisplayPort DMA" indicate that both the "video" and "graphics" side can be used.

So why does using "DMABuf export" work with 420, but "DMABuf import" fails with the same format? I would expect that neither should work if the video format is wrong. The import also fails when I use RGB. So the DMABuf import seems to fail whatever format I use.

E.g. 

gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=5/1, format=RGB ! kmssink
0:00:00.764548706  6327   0x558f3b08c0 ERROR           kmsallocator gstkmsallocator.c:596:gst_kms_allocator_dmabuf_import:<KMSMemory::allocator> Failed to import prime fd 24: Operation not permitted (-1)
0:00:00.764587486  6327   0x558f3b08c0 INFO         GST_PERFORMANCE gstkmssink.c:1744:gst_kms_sink_get_input_buffer:<kmssink0> frame copy

 

Also if I query the kms/drm driver with "modetest" i get

Planes:
id	crtc	fb	CRTC x,y	x,y	gamma size	possible crtcs
35	0	0	0,0		0,0	0       	0x00000001
  formats: VYUY UYVY YUYV YVYU YU16 YV16 YU24 YV24 NV16 NV61 GREY Y10  BG24 RG24 XB24 XR24 XB30 XR30 YU12 YV12 NV12 NV21 XV15 XV20

as supported formats. Is this information I get from the driver wrong?

 

0 Kudos
Highlighted
Moderator
Moderator
1,473 Views
Registered: ‎11-09-2015

HI @truessd 

You are right sorry I overlooked the figure 33-2 and missed the path from the DPDMA to the Video interface


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Highlighted
Observer
Observer
1,066 Views
Registered: ‎08-14-2018

The issue came from the DisplayPort DMA having a requirement that the stride is 256 aligned, which was quite hard to find. Using a custom v4l2 driver that enforces this alignment solved the issue. 

Highlighted
Contributor
Contributor
1,008 Views
Registered: ‎12-21-2018


@truessd wrote:

The issue came from the DisplayPort DMA having a requirement that the stride is 256 aligned, which was quite hard to find. Using a custom v4l2 driver that enforces this alignment solved the issue. 


Hi, I have exactly the same need. May i get more details ? have you patched the xilinx framebuffer driver ?

0 Kudos
Highlighted
Observer
Observer
865 Views
Registered: ‎08-14-2018

Hi, sorry for the late reply. We implemented our own v4l2 driver, since the one from xilinx was not fully v4l2 compliant [http://manpages.ubuntu.com/manpages/bionic/man1/v4l2-compliance.1.html] and we had the above problem with the stride alignment. Therefore I can unfortunatelly not send you a patch or anything. But the problem regarding the stride is fairly easy to fix. Just check where the bytesperline value is calculated in the driver and align the resulting value to 256.