Gstreamer launch Video Format: camera -> UYVY, output -> BGR

[Aliyev]

Hello Everybody,

gst-launch-1.0 \
    v4l2src device=/dev/video3 io-mode=dmabuf ! \
    "video/x-raw, width=1920, height=1080, format=UYVY" ! \
    sdxfilter2d filter-preset=4 filter-mode=1 ! queue ! \
    fpsdisplaysink video-sink="kmssink bus-id=b00c0000.v_mix plane-id=29 fullscreen-overlay=true" sync=false text-overlay=false -v

I have a computer vision app running in Zynq U+ and it is operating in BGR format. Hence can I change the above command such that output (DisplayPort) gets BGR image? is the format (UYVY) valid for both? how can I change camera and DP for different video formats?

[kvasantr]

Hello @Aliyev 

Can you explain all the processing elements in your Video pipeline from capture to Display?

If you have capture format as UYVY, you should have necessary elements in your pipeline that would convert it to BGR.

The second important point to mention is, if you are using mixer IP in your pipeline along with PS DP of ZynqMP, it does not offer more than two planes. It offers only 1 overlay plane as graphic plane that too generated by MALI GPU.

In short VIdeo MIxer + PS DP combination is not supported.

with regards

Kunal 

[Aliyev]

Hello @kvasantr 

I have no problem converting formats. The issue is the conversion overhead.

Our image proc pipeline operates on BGR image. Here UYVY->BGR and then BGR->YUV->UYVY conversions take a lot of time which decreaes FPS.

Since camera only supports, I have no choice but take UYVY video in but I dont want to convert BGR back to UYVY as PS DP does support BGR format.

To say the least, how can I write above gst launch command that I get UYVY video in and BGR out? 

Thanks

[Aliyev]

@kvasantr 

To put it i to the perspevtive, the last conversion (YUV->UYVY) on its own decreases the FPS to 17. This means exclusive 50 ms delay, it is insane.

So I definetely need to figure out to set ohtput to BGR.

[watari]

Hi @Aliyev 

Did you make sure your hardware design ?

It depends on your hardware design and GStreamer's pipeline.

So, I recommend to try it.

Best regards,

[kvasantr]

Hello @Aliyev 

Are you doing all this conversion in A53 core? That could be one reason of multiple mem copies and the actual cause of frame drop. What is in your Vivado IP pipeline? That would really help us answer your questions.

[Aliyev]

Hi @kvasantr 

I use the reVision platform and everything is happening on the A53.

What do you mean by "pipeline"? you can assume a simple image processing pipeline with a camera (USB webcam) a kernel (a bunch of OpenCV funcs) and a sink (PS DP). The entire chain is operating on the PS side.

If mem copy and frame drop of YUV2UYVY conversion is the reason for 17 FPS then why the UYVY2BGR does not decrease FPS at all (i.e remains 30 FPS)?

Here is the code for YUV2UYVY

for(int i=0; i<height; i++){
	for(int j=0; j<width; j++){
		dst.at<cv::Vec2b>(i,j)[1] = frame_YUV.at<cv::Vec3b>(i,j)[0];
		if(j%2==0){
		  dst.at<cv::Vec2b>(i,j)[0] = frame_YUV.at<cv::Vec3b>(i,j)[1];
		}
		 else {
		  dst.at<cv::Vec2b>(i,j)[0] = frame_YUV.at<cv::Vec3b>(i,j)[2];
		}
	  }
  }