11-20-2018 01:16 PM
The FPGA team here recommended that I ask you for a bit of assistance in using the FPGA board to capture 10 bit video on the ZCU-106, rev. C board.
The FPGA installed is the provided “sdirx_vcu” sample.
I believe that I have effectively worked the sequence down to a couple commands, and I’m unsure where this is going wrong:
xmedia-ctl --verbose --device /dev/media0 \ --set-v4l '"a0030000.smpte_uhdsdi_rx_ss":0 [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":0 [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":1 [fmt: UYVY8_1X16/3840x2160 field:none]' gst-launch-1.0 --messages --tags --toc --verbose \ v4l2src device=/dev/video0 io-mode=dmabuf num-buffers=3000 ! \ capsfilter caps=video/x-raw,format=NV16,framerate=50/1,height=2160,width=3840,bit-depth-luma=10,bit-depth-chroma=10 ! \ omxh265enc gop-length=30 gop-mode=basic low-bandwidth=true target-bitrate=100000 b-frames=0 num-slices=4 control-rate=constant qp-mode=uniform periodicity-idr=30 prefetch-buffer=true filler-data=true latency-mode=normal ! \ queue max-size-bytes=0 ! \ capsfilter caps=video/x-h265 ! \ h265parse ! \ filesink location=UYVY8_1X16.UYVY8_1X16.NV16.format-test.002.12G.h265.mp4
Neither the h264 or h265 encoding seem to work for me.
Have you any suggestions?
Kind Regards, Peimann, Scott Meyer
11-26-2018 06:04 AM
Which VCU TRD are you using? I don't think it supports 10bit until 2018.3. It's better to wait for the 2018.3 VCU TRD.
Please open block design in IPI, and check if all the video IP supports 10bit. If not, you need to modify them, and then re-generate the hdf file.
For stream pipeline, format should be NV12_10LE32 or NV16_10LE32 for 10-bit, not NV16.
11-26-2018 02:35 PM
Thank You @xud,
We appreciate the time you've spent in addressing our problem. Hopefully you have time and can help us understand how readily we can use the Zynq UltraScale+ XCZU7EV-2FFVC1156 MPSoC to develop new, better systems.
We are testing with the Xilinx 2018.2 release.
We appreciate that you confirm use of NV12_10LE32 or NV16_10LE32 as the primary 10-bit formats. These both have appeared in the output from test cases run using GStreamer, although we are (mostly) incapable of using them for useful capture.
Were you able to successfully capture, encode, and store video using a (current) 2018.3 pre-release?
Assuming that you were able to do so, will you please share your working commands with us, so we can build upon them?
As stated in the original message, we used "The FPGA installed is the provided “sdirx_vcu” sample.". Using Vivado against that sample, we observed that 10-bit formats are available at each stage. Our difficulty has been the pipelining of 10-bit video -- a) raw, b) recordings, c) alpha-blending, d) multiple-streams, and e) output through the various sample builds.
Also, the "gst_vcu_app" application, while relatively sophisticated, appears to support only:
Even though there are command inputs which appear capable of affecting these options, the source code which we inspected and modified in our testing was incapable.
Assuming that you were able to use the pre-release 2018.3 to capture and encode 10-bit video from 12Gbit/sec SD input, we would like to know what you had to do:
Lastly, is a current, pre-release 2018.3 available for us to work with? Specifically one which is compatible with the command modifications you suggest?
Sir, we really appreciate your help. You've answered questions and confirmed some of our concerns. Hopefully we can perform verification of the required capabilities of the system using the 2018.3 release, and begin hardware design knowing the Xilinx SoC and what we have to add.
11-29-2018 06:23 AM - edited 11-29-2018 06:24 AM
The release date of 2018.3 is very close. I think it's better to wait till 2018.3.
If it's very urgent to you, please work with your FAE, and see if it's possible to get the EA version.
As per documented in PG252, VCU supports :
. YCbCr 4:2:2, YCbCr 4:2:0, and Y-only (monochrome)
° 8- and 10-bit per color channel
I don't think the restriction is at VCU itself, at ctrl sw level it already supports 10-bit. I only have done some tests with receiving the 10-bit Video file. I haven't tested SDI RX yet.
11-29-2018 03:15 PM
Good Afternoon @xud,
Thank you for getting back to me.
Do you have a targeted release date? If so, we can report to our management, so that they have an indication of when efforts can proceed on the project.
My FAE is on vacation at the moment, and thus was not able to respond to the note that I mailed to him before posting to the forums.
If the software and hardware already support 10 bit, then I would expect the attached file bash script to capture a minute's worth of video.
Failure is the same in two different scenarios:
Time to fail is either ~20 seconds, or 0.25 seconds.
What are we doing incorrectly????
12-05-2018 09:45 AM - edited 12-05-2018 09:48 AM
If you look at the hardware setup for the 2018.2 ZCU106 VCU TRD you will find that the SDI output is 10-bits, but then the PL hardware was modified to drop the 2 LSBs before sending it to the rest of the pipeline. This makes the hardware pipeline to be 8bpc (bits per component) and you will not be able to capture 10bit-data due to this hardware modification.
But with that said, as you found in your other post, you can use videotestsrc to generate input video and you may be able to force videotestsrc to generate 10-bit data for testing.
Unfortunately for release information you will need to contact your local FAE.
12-05-2018 04:14 PM
Thanks for Helping @chrisar,
From what I've been able to observe and learn, the full 10-bit, end-to-end pipeline was not available in the 2018.2 release.
Are you able to demonstrate success (under 2018.3) using the script that I posted as an attachment? [Not the original code that I posted in-line].
Did I do everything correctly in that script, or is there a simple, glaring error.
I believe that the script fails due to an incomplete (or buggy) element in GStreamer, or missing information in the Xilinx drivers, or both. Hopefully the next Xilinx release will allow me to make a ground-up build of the example, and then my script will work.
That said, due to my ignorance using the entire software chain in a repeatable, appropriate manner, I have been trying to avoid building an enhanced 2018.2 release which updates GStreamer up to the latest version, possibly patches elements of the kernel, and will likely introduce actual bugs (by me). Especially as 2018-12-31 is the (nominal) latest end-date for the release, I am willing to wait a short while and focus effort in other ways.
Really, while waiting, I want to bring my Xilinx/PetaLinux/Yocto skills up to snuff, so that I can implement suggested solutions in a safe manner, with minimal effort.
Again, many thanks for your help @chrisar.
12-13-2018 08:49 AM
@peimannthe 2018.3 ZCU106 VCU TRD was just released yesterday. You can find it on the Xilinx Wiki Zynq UltraScale+ MPSoC VCU TRD 2018.3 page. This version does have support for 10-bit for the SDI Rx design. I did not have a chance to try your script on this version.
12-13-2018 09:10 AM
Kudos to you @chrisar,
I'll pull down the update and work with it. It'll likely be Tuesday before I can fully verify functionality.
12-17-2018 05:38 PM
Good Afternoon @chrisar,
I used the 2018.3 image vcu_sdirxtx to re-run this example.
Still fails, as it cannot negotiate.
The output files (from original test and current) are attached in the zip.
Scott M. Peimann
12-18-2018 02:37 PM
We can reproduce the negotiation error on the SDI Rx interface and we are looking into this.
12-19-2018 02:04 PM
I was thinking that this might be related to the version of GStreamer that is built/used by PetaLinux.
On the Xilinx, the version of GStreamer is 1.12.2.
root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19# gst-launch-1.0 --version gst-launch-1.0 version 1.12.2 GStreamer 1.12.2 Unknown package origin root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19#
If I remember, I believe that the current, stable GStreamer version is 1.14.4 (ref: "Recent older news:", https://gstreamer.freedesktop.org).
From what I have read, an amount of greater-than-eight-bit work has been done, thus bringing the tools into the nineteenth century.
Will you guys please try an appropriate update and let me know if that gets us a bit further on?
12-20-2018 07:35 AM
Here is where we are at right now. It turns out that 10-bit support was not completed in the VCU TRD. In the 2018.3 VCU TRD, the hardware portion was put in place, but no work was done on the software side, in order to support 10-bit.
At this point I don’t have an estimate on how much work it will take. I’m hoping it isn’t too much, but I can’t be sure until I have time to dig into it more. It could just be GStreamer, but it could also be driver support.
12-20-2018 08:48 AM
Good Morning @chrisar,
Thanks for getting back to me on this. I appreciate knowing what the status of the components is; it really helps me report and focus my efforts in the "correct" areas.
If there are any straightforward tests that I can do to verify drivers, please let me know.
Kudos for the update keeping me current!
With luck, next year will be good to us both.
07-02-2019 10:24 AM
We are able to capture 10-bit 4:2:2 video using this script against the 2019-1 release:
#!/bin/bash -v time xmedia-ctl --verbose --device /dev/media0 \ --set-v4l '"a0030000.v_smpte_uhdsdi_rx_ss":0 [fmt:UYVY10_1X20/3840x2160 field:none], "a0080000.v_proc_ss":0 [fmt:UYVY10_1X20/3840x2160 field:none], "a0080000.v_proc_ss":1 [fmt:UYVY10_1X20/3840x2160 field:none]' ##--works; element: omx265enc - remove attribute(s): prefetch-buffer time gst-launch-1.0 --gst-debug-level=3 --gst-plugin-spew --messages --verbose --eos-on-shutdown --gst-fatal-warnings \ v4l2src device=/dev/video0 io-mode=dmabuf num-buffers=3000 \ ! capsfilter caps='video/x-raw,format=NV16_10LE32,framerate=50/1,height=2160,width=3840' \ ! omxh265enc gop-length=30 gop-mode=basic low-bandwidth=true target-bitrate=100000 \ b-frames=0 num-slices=4 control-rate=variable qp-mode=uniform periodicity-idr=30 \ ! queue max-size-buffers=5 max-size-bytes=0 \ ! h265parse \ ! qtmux \ ! capsfilter caps='video/quicktime' \ ! filesink location=UYVY10_1X20.UYVY10_1X20.NV16_10LE32.format-test.002.12G.h265.2019-1.mp4
Note that the test was slightly modified to use a variable-bit-rate, rather than a constant-bit-rate; this is not related to the failure.
Also, our observations found that:
However, we have been unable to find proper settings for 4:2:0 video.
What should I use for the xmedia-ctl format specifications, so that they are compatible with NV12_10LE32?
When you have a moment, will you please look into the 4:2:0 encoding problem.
07-04-2019 06:57 AM
I tried your script against the 2019.1 release, but I could not succesfully capture mp4 although I could capture ts. When trying to capture mp4 gst-launch aborts with error "Buffer has no PTS." A quick google shows this is sometimes an error when capturing mp4 using qtmux. Did you ever see this error?
07-08-2019 07:11 AM - edited 07-08-2019 07:15 AM
Good Morning @johne1969,
My apologies for the delay. Your note is came on my National Holiday, and I've been out-of-office.
I recall seeing PTS errors and don't recall exactly what they mean. To understand some of the non-obvious errors from GStreamer, I have taken to searching through the source code to find the root cause. My copy is from the on-line repository, accessible through the site.
As I observed, the following options help with diagnosing problems:
gst-launch-1.0 --gst-debug-level=3 --gst-plugin-spew --messages --verbose --eos-on-shutdown --gst-fatal-warnings
Usually the first warning that I see is the key to the failure.
Also, I used the
pre-build from the 2019.1 release in my last test on this thread.
NB: the Git repository for the GStreamer build is here:
Be certain to actually run the build, so that all of the supporting elements are loaded under the subprojects directory.
The GStreamer build process is a pain, but worth doing.
Your error comes from "subprojects/gst-plugins-good/gst/isomp4/gstqtmux.c", line 4832.
It seems that the stream is trying to add a buffer to the qtmux element, although the exact "cause" would be in this:
if (!GST_BUFFER_PTS_IS_VALID (last_buf)) goto no_pts;
Unfortunately the term "PTS" has me somewhat confused, perhaps "Prior Time Stamp"?
It turns out that "PTS" means "Presentation Time Stamp.".
Look into file
for the macro definition; the message there states "Tests if the pts is known.".
Ok, using "DUCK" to find "PTS", I find in the GStreamer web-site that:
The buffer DTS refers to the timestamp when the buffer should be decoded and is usually monotonically increasing. The buffer PTS refers to the timestamp when the buffer content should be presented to the user and is not always monotonically increasing.
Hope that helps some.
Also, there are quite a few links available through "DUCK", so you may need to take time to read through them.
NB: post was edited to replace indent that was lost during HTML verification by the forum with a "preformatted" box.
07-08-2019 07:46 AM
Hello Again @johne1969,
I see this problem a great deal in my dual-encode tests that I am working with.
Your problem seems concerned with the qtmux element. I'd suggest that you try the matroska mux element to see if it helps.
Keep watching the forums for a bit, as I am trying to write up the problem now and on the Video forum when I have completed.
Just remember, if this was easy someone else would have done it already.
07-08-2019 08:30 AM
Hello Once Again @johne1969,
It looks like the qtmux element dies where the matroskamux element just drops frames. This is an issue that will push all the way back to GStreamer, as we are illuminating a potential corner case. I haven't tried running version diffs on the source-code for the qtmux element.
Will you please create a new thread for your problem, so that this thread can stay focused on my issues with real-time, 10-bit encoding?
When you do, e-Mail the link to me, and I'll do my best to help you.
07-08-2019 09:33 AM