06-25-2020 08:10 AM
Hello!
I'm new to working with GStreamer and video codecs.
I'm trying to stream the output of the TestPatternGenerator (from ZCU106 with the Multistream VCU TRD 2019.2) with H.264 encoding as RTP-Stream over ethernet to a Windows Host. My GStreamer pipeline looks like this:
gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! omxh264enc qp-mode=auto gop-mode=low-delay-p gop-length=60 periodicity-idr=60 b-frames=0 target-bitrate=10000 num-slices=8 control-rate=-1 prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h264, alignment=nal ! h264parse ! rtph264pay ! rtpjitterbuffer ! queue ! udpsink host=192.168.0.1 port=5004 max-bitrate=0 max-lateness=-1 buffer-size=600000000 qos-dscp=60 async=false
And I'm viewing it in VLC with this sdp-File
v=0
m=video 5004 RTP/AVP 96
c=IN IP4 192.168.0.2
a=rtpmap:96 H264/90000
a=framerate=60
The result is a stream that plays for 1 or 2 seconds and then starts stuttering very much.
I tried dumping the video in a mp4-file with:
gst-launch-1.0 -e v4l2src device=/dev/video2 io-mode=4 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc ! video/x-h264, profile=high, alignment=nal ! queue ! h264parse ! qtmux ! filesink location=test_cf.mp4
(and various other omxh264enc settings, encoding profiles)
I copied that mp4 over to the Windows-Host and tried playing it with VLC but it's even worse than the stream.
Decoding that same mp4 on the board and displaying it over HDMI results in a correctly playing video.
Streaming and recording with H265 encoding works without problems.
Any idea what I'm doing wrong here or what settings i'm missing?
Thanks!
06-25-2020 02:15 PM
Hi @MichaelK
How do you connect between Host and your board ? with network hub ? or without network hub ?
And, did you set QoS parameter on MPSoC ?
It was improved bus issue by this parameter.
Also, did you set "prefetch-buffer" as "true" at omxh264enc ?
Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...
Best regards,
06-26-2020 12:01 AM - edited 06-26-2020 12:18 AM
Hello @watari,
thank you for your reply.
How do you connect between Host and your board ? with network hub ? or without network hub ?
I'm connecting to the board with a network switch. But i don't think it's a network issue because copying the mp4 (the one i record on the board with gstreamer) from the board to the windows host and trying to play it there directly doesn't work either. I can decode and play that same file on th board though.
And, did you set QoS parameter on MPSoC ?
How do i set the QoS Paramter? The only thing i used was qos-dscp=60.
Also, did you set "prefetch-buffer" as "true" at omxh264enc ?
Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...
Yes i used prefetch-buffer and control-rate=low-latency, but it doesn't really change anything.
06-26-2020 01:59 AM
Hey @watari,
Thank you for your reply.
I'm connected to the board through a network switch.
Also, did you set "prefetch-buffer" as "true" at omxh264enc ?
Finally, it might be able to get good result if you use low-latency or Xilinx low-latency mode for omxh264enc...
Yes i used prefetch-buffer. With low-latency mode you are reffering to control-rate=low-latency? If yes, then i did use it.
Best regards
06-29-2020 02:54 PM
Hi @MichaelK
Sorry for my late reply.
> How do i set the QoS Paramter?
Refer attached shell script file. You can understand how to set the QoS parameter.
>Yes i used prefetch-buffer. With low-latency mode you are reffering to control-rate=low-latency? If yes, then i did use it.
Unfortunately, you must describe "alignment = nal" as capability on gstreamer pipeline, too, if you use low-latency or Xilinx low-latency.
Would you try it ?
Best regards,
06-30-2020 01:13 AM - edited 06-30-2020 01:14 AM
Thank you for the script. I will try it and see if the performance improves. I'm still having some trouble with recording and streaming encoded 4k 60 fps from the TPG with the pseudo-random pattern.
My main problem was solved though. Had nothing to do with the pipeline. The main source of error was hardware accelerated decoding in VLC. When i deactivate it the videos run.
Is there any incompatibility between VLC and omxh264enc known? Or is it simply a problem with my PCs hardware?
07-01-2020 02:34 PM
Hi @MichaelK
Did you compare streaming data between VLC and omxh264enc by ex. ffmpg ?
As you know, a streaming data which are generated by different alignment are obvious different.
I don't know whether VLC support alinment=nal or not. But I suggest you to make sure it.
[Note]
FYI.
I already confirmed and see streaming data which is generated by omxh265enc, not omxh264enc with VCU TRD 2019.1.
It worked fine in H.265. But it failed in H.264 in my specific environment.
I will investigate it or will try it with VCU TRD 2020.1...
Best regards,
07-03-2020 04:51 AM
Hi @watari
I tried both alignments nal and au.
So you see the same difficulties with omxh264enc?
07-05-2020 03:32 PM
Hi @MichaelK
I just ask you the following.
- Did you refer the following ? Also follow this setting on your pipe-line ? (low latency or Xilinx low latency)
https://www.xilinx.com/support/documentation/ip_documentation/vcu/v1_2/pg252-vcu.pdf#page=125
Best regards,
07-07-2020 02:40 AM - edited 07-07-2020 02:46 AM
gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw, width=1920, height=1080, framerate=60/1, format=NV12 ! omxh264enc gop-mode= (default or low-delay-p), b-frames=0 control-rate=low-latency ! video/x-h264, alignment=nal ! fakesink
Are you talking about this pipeline? Yes i did.
Did you get the h264-stream in VLC working with it on your system?
Thank you for trying to help me @watari
07-07-2020 02:41 PM
Hi @MichaelK
>Did you get the h264-stream in VLC working with it on your system?
No. I haven't gotten the h264 stream in VLC and gstreamer with like this pipeline at 2019.1 on my system yet.
In my case, I suspect some bugs in gstreamer and gstreamer's plugin.
At least, gstreamer 1.14 has some bugs about nal in h264parse plugin.
I'd like to investigate route cause, if I have enough time.
But, I can't do it and I will try it at 2020.1.
FYI.
If you want to use VCU TRD 2019.2, I suggest you to do poting of gstreamer and gstreamer plugin from latest version (1.16).
If you can change TRD version, I suggest you to try latest TRD (2020.1).
Note
I will try like this with latest VCU TRD coming soon.
I will help you, if I can reply your question...
Best regards,