UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Contributor
Contributor
271 Views
Registered: ‎02-06-2019

VCU demo don't stream 60 fps video.

Jump to solution

Hi all,

I have been trying to stream or record 60 fps camera output after the compression on VCU HW block. My Vivado version 2018.3, camera: See3cam USB3.0, board that ı used: zcu104.

There are several examples in the prebuild zcu104 image. However, these are can not send 60 fps video over ethernet or write in the memory as a .mp4 file. The demo GStreamer pipelines show the metadata as 60 fps on the .ts file or stream over ethernet, but in reality, it just manages the 38 fps.

I connect the zcu104 board to pc via directly ethernet cable and run the “vcu-demo-camera-encode-streamout.sh”  demo. Vlc player just shows the metadata of stream and says 60 fps. However, when I run a custom pipeline which shows the fps on the zcu104’s terraterm screen, it shows 38 fps...

My pipeline =

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, format="UYVY", framerate=60/1 ! tee name=t  \

                t.! queue ! videoconvert ! video/x-raw, format=\(string\)NV12 \

                ! omxh264enc num-slices=8 control-rate=low-latency target-bitrate=25000 prefetch-buffer=true \

                ! h264parse ! mpegtsmux ! rtpmp2tpay \

                ! udpsink host=192.168.0.2 port=50000 max-lateness=-1 qos-dscp=60 sync=false async=false \

                t.! fpsdisplaysink sync=false text-overlay=false -v

 

I faced the same situation when I record the video on the memory and investigate them into the pc. 38fps.

Demo = vcu-demo-camera-encode-file.sh

My pipeline =

gst-launch-1.0 -e v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, format="UYVY", framerate=60/1 ! tee name=t  \

                t.! queue ! videoconvert ! video/x-raw, format=\(string\)NV12 \

                ! omxh264enc num-slices=8 control-rate=low-latency target-bitrate=25000 prefetch-buffer=true \

                ! h264parse ! qtmux ! filesink location/home/root/test-7.mp4 sync=false  \

                t.! fpsdisplaysink sync=false text-overlay=false -v

The demo gsteamer pipeline record video as .ts file which shows the metadata fps value and when you play that file in pot player frame by frame in pc, you can see the frame number jumps not increase one by one. 

And my pipeline recodes the video as .mp4 file which shows the average fps on the metadata.

My first question is how can I get 60fps stream?

----------------------------------------------------------------------------------------------------------------------------------------------------

Additionally, I build a custom platform which has VCU block in it. When I try to use xf-opencv hw functions with vcu, and streaming situations getting even worst. I can get only 8 fps...

My second question is how can I found the bottleneck on the zcu104?

My custom SDx platform's Vivado design:Capture.PNG

All my custom pipeline tries and results can be found in the attachment as txt document. I tried these pipelines in custom platform images and prebuild zcu104 images. 

Thanks in advance,

Yasin Kağan Taşkın

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
Scholar watari
Scholar
216 Views
Registered: ‎06-16-2013

Re: VCU demo don't stream 60 fps video.

Jump to solution

Hi @yasin.kagan 

 

Would you use the following plug-in ?

 

https://github.com/mpekar/gstreamer_timestamp_marking

 

You need to build the binary by yourself.

 

Also, I suspect videoconvert plugin which is calculated by ARM core. (Not hardware acceleration)

I suggest you to use accelerated videoconvert plugin like nvvidconv which is released by nVIDIA for ex. Jetson seriase.

But, unfortunetly, I don't know like this plugin. You need to build hardware by HLS and like this plugin as gstreamer plugin.

Perhaps it might use Chroma Resampler...

 

Best regards,

5 Replies
Scholar watari
Scholar
261 Views
Registered: ‎06-16-2013

Re: VCU demo don't stream 60 fps video.

Jump to solution

Hi @yasin.kagan 

 

Did you confirm the capability of this USB camera whether it can support 60fps or not by ex. v4l2-ctl and gst-device-monitor-1.0 ?

 

Also, I suggest you to use SSD to record your mp4 file.

And, I suggest you to make sure latency with gstreamer_timestamp_marking which is marking plugin.

Here is an example to measure latency between markin and markout.

 

$ export GST_DEBUG=markout:5

$ gst-launch-1.0 -f -e filesrc location=/media/movie.mp4 ! markin ! qtdemux ! h264parse ! markout ! fakesink

 

Best regards,

Contributor
Contributor
234 Views
Registered: ‎02-06-2019

Re: VCU demo don't stream 60 fps video.

Jump to solution

First of all, thanks for quick reply @watari 

See3Cam comes with zcu104 development board and it supports 60fps up to 1080p. I also see on DP monitor.

Capture.PNG

I both try to record video into really fast USB3.0 USB memory stick (150 MB/s write and 300 MB/s read) and I record into the file system (which I believe in RAM memory) then copy the USB memory. Encoded stream only reach 25 Mbps (about 3 MB/s)

I’m actually pretty sure this situation not caused by speed limitation of Ethernet or USB.

When I just add markin and markout elements directly on the pipeline, I get no element “markin” error.

I couldn’t install the gstreamer_timestamp_marking into petalinux, using  “./configure” or “./autogen.sh”. Am I re-build the petalinux image to adding “markin” element?

I don’t think my pipeline was wrong, but I agree with you. Knowing timestamp of each element gives more information about what is going on there.

0 Kudos
Scholar watari
Scholar
217 Views
Registered: ‎06-16-2013

Re: VCU demo don't stream 60 fps video.

Jump to solution

Hi @yasin.kagan 

 

Would you use the following plug-in ?

 

https://github.com/mpekar/gstreamer_timestamp_marking

 

You need to build the binary by yourself.

 

Also, I suspect videoconvert plugin which is calculated by ARM core. (Not hardware acceleration)

I suggest you to use accelerated videoconvert plugin like nvvidconv which is released by nVIDIA for ex. Jetson seriase.

But, unfortunetly, I don't know like this plugin. You need to build hardware by HLS and like this plugin as gstreamer plugin.

Perhaps it might use Chroma Resampler...

 

Best regards,

Contributor
Contributor
192 Views
Registered: ‎02-06-2019

Re: VCU demo don't stream 60 fps video.

Jump to solution

hi @watari

Yes, I try to build binaries into the zcu104 Petalinux using that distribution. But I couldn't manage it to build. I'm still working on it.

When ı try to change chroma resampler (actually video convert) in my pipeline, I see an immediate increase on fps. It might be the issue. However, I am still getting 8 fps when ı add the filter2d HW filter.

I will investigate the issue a little bit more to be make sure this issue is related to chroma resampler.

 

0 Kudos
Moderator
Moderator
127 Views
Registered: ‎11-21-2018

Re: VCU demo don't stream 60 fps video.

Jump to solution

Hi @yasin.kagan 

 

Do you have any update on this?

 

Regards,

Aoife
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos