VCU TRD 2019.2 streaming H264 over ethernet and vi...

MichaelK · ‎06-25-2020

Hello!

I'm new to working with GStreamer and video codecs.
I'm trying to stream the output of the TestPatternGenerator (from ZCU106 with the Multistream VCU TRD 2019.2) with H.264 encoding as RTP-Stream over ethernet to a Windows Host. My GStreamer pipeline looks like this:

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! omxh264enc qp-mode=auto gop-mode=low-delay-p gop-length=60 periodicity-idr=60 b-frames=0 target-bitrate=10000 num-slices=8 control-rate=-1 prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h264, alignment=nal ! h264parse ! rtph264pay ! rtpjitterbuffer ! queue ! udpsink host=192.168.0.1 port=5004 max-bitrate=0 max-lateness=-1 buffer-size=600000000 qos-dscp=60 async=false

And I'm viewing it in VLC with this sdp-File

v=0
m=video 5004 RTP/AVP 96
c=IN IP4 192.168.0.2
a=rtpmap:96 H264/90000
a=framerate=60

The result is a stream that plays for 1 or 2 seconds and then starts stuttering very much.

I tried dumping the video in a mp4-file with:

gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc ! video/x-h264, profile=high, alignment=nal ! queue ! h264parse ! qtmux ! filesink location=test_cf.mp4

(and various other omxh264enc settings, encoding profiles)
I copied that mp4 over to the Windows-Host and tried playing it with VLC but it's even worse than the stream.
Decoding that same mp4 on the board and displaying it over HDMI results in a correctly playing video.

Streaming and recording with H265 encoding works without problems.

Any idea what I'm doing wrong here or what settings i'm missing?

Thanks!

watari · ‎06-25-2020

Hi @MichaelK

How do you connect between Host and your board ? with network hub ? or without network hub ?

And, did you set QoS parameter on MPSoC ?

It was improved bus issue by this parameter.

Also, did you set "prefetch-buffer" as "true" at omxh264enc ?

Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...

Best regards,

MichaelK · ‎06-26-2020

Hello @watari,

thank you for your reply.

How do you connect between Host and your board ? with network hub ? or without network hub ?

I'm connecting to the board with a network switch. But i don't think it's a network issue because copying the mp4 (the one i record on the board with gstreamer) from the board to the windows host and trying to play it there directly doesn't work either. I can decode and play that same file on th board though.

And, did you set QoS parameter on MPSoC ?

How do i set the QoS Paramter? The only thing i used was qos-dscp=60.

Also, did you set "prefetch-buffer" as "true" at omxh264enc ?

Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...

Yes i used prefetch-buffer and control-rate=low-latency, but it doesn't really change anything.

MichaelK · ‎06-26-2020

Hey @watari,

Thank you for your reply.

I'm connected to the board through a network switch.

Also, did you set "prefetch-buffer" as "true" at omxh264enc ?

Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...

Yes i used prefetch-buffer. With low-latency mode you are reffering to control-rate=low-latency? If yes, then i did use it.

Best regards

watari · ‎06-29-2020

Hi @MichaelK

Sorry for my late reply.

> How do i set the QoS Paramter?

Refer attached shell script file. You can understand how to set the QoS parameter.

>Yes i used prefetch-buffer. With low-latency mode you are reffering to control-rate=low-latency? If yes, then i did use it.

Unfortunately, you must describe "alignment = nal" as capability on gstreamer pipeline, too, if you use low-latency or Xilinx low-latency.

Would you try it ?

Best regards,

MichaelK · ‎06-30-2020

Thank you for the script. I will try it and see if the performance improves. I'm still having some trouble with recording and streaming encoded 4k 60 fps from the TPG with the pseudo-random pattern.

My main problem was solved though. Had nothing to do with the pipeline. The main source of error was hardware accelerated decoding in VLC. When i deactivate it the videos run.
Is there any incompatibility between VLC and omxh264enc known? Or is it simply a problem with my PCs hardware?

watari · ‎07-01-2020

Hi @MichaelK

Did you compare streaming data between VLC and omxh264enc by ex. ffmpg ?

As you know, a streaming data which are generated by different alignment are obvious different.

I don't know whether VLC support alinment=nal or not. But I suggest you to make sure it.

[Note]

FYI.

I already confirmed and see streaming data which is generated by omxh265enc, not omxh264enc with VCU TRD 2019.1.

It worked fine in H.265. But it failed in H.264 in my specific environment.

I will investigate it or will try it with VCU TRD 2020.1...

Best regards,

MichaelK · ‎07-03-2020

Hi @watari

I tried both alignments nal and au.

So you see the same difficulties with omxh264enc?

watari · ‎07-05-2020

Hi @MichaelK

I just ask you the following.

- Did you refer the following ? Also follow this setting on your pipe-line ? (low latency or Xilinx low latency)

https://www.xilinx.com/support/documentation/ip_documentation/vcu/v1_2/pg252-vcu.pdf#page=125

Best regards,

MichaelK · ‎07-07-2020

gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc gop-mode= (default or low-delay-p), b-frames=0 control-rate=low-latency ! video/x-h264, alignment=nal ! fakesink

Are you talking about this pipeline? Yes i did.
Did you get the h264-stream in VLC working with it on your system?

Thank you for trying to help me @watari

watari · ‎07-07-2020

Hi @MichaelK

>Did you get the h264-stream in VLC working with it on your system?

No. I haven't gotten the h264 stream in VLC and gstreamer with like this pipeline at 2019.1 on my system yet.

In my case, I suspect some bugs in gstreamer and gstreamer's plugin.

At least, gstreamer 1.14 has some bugs about nal in h264parse plugin.

I'd like to investigate route cause, if I have enough time.

But, I can't do it and I will try it at 2020.1.

FYI.

If you want to use VCU TRD 2019.2, I suggest you to do poting of gstreamer and gstreamer plugin from latest version (1.16).

VCU TRD 2019.2 streaming H264 over ethernet and viewing in VLC