UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Explorer
Explorer
1,799 Views
Registered: ‎10-03-2018

SDI Video to 10 Bit H265 MPEG

Folks,

The FPGA team here recommended that I ask you for a bit of assistance in using the FPGA board to capture 10 bit video on the ZCU-106, rev. C board. 

The FPGA installed is the provided “sdirx_vcu” sample. 

I believe that I have effectively worked the sequence down to a couple commands, and I’m unsure where this is going wrong:

xmedia-ctl --verbose --device /dev/media0   \
     --set-v4l '"a0030000.smpte_uhdsdi_rx_ss":0  [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":0  [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":1 [fmt: UYVY8_1X16/3840x2160 field:none]' 

gst-launch-1.0 --messages --tags --toc --verbose  \
    v4l2src device=/dev/video0 io-mode=dmabuf num-buffers=3000 !  \
    capsfilter caps=video/x-raw,format=NV16,framerate=50/1,height=2160,width=3840,bit-depth-luma=10,bit-depth-chroma=10 !  \
     omxh265enc gop-length=30 gop-mode=basic low-bandwidth=true target-bitrate=100000 b-frames=0 num-slices=4 control-rate=constant qp-mode=uniform periodicity-idr=30 prefetch-buffer=true filler-data=true latency-mode=normal ! \
    queue max-size-bytes=0 !  \
     capsfilter caps=video/x-h265 !  \
     h265parse !  \
     filesink location=UYVY8_1X16.UYVY8_1X16.NV16.format-test.002.12G.h265.mp4  

 

Neither the h264 or h265 encoding seem to work for me. 

Have you any suggestions? 

Kind Regards, Peimann, Scott Meyer
 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
19 Replies
Xilinx Employee
Xilinx Employee
1,731 Views
Registered: ‎08-02-2007

Re: SDI Video to 10 Bit H265 MPEG

Hi @peimann

Which VCU TRD are you using? I don't think it supports 10bit until 2018.3. It's better to wait for the 2018.3 VCU TRD.

Please open block design in IPI, and check if all the video IP supports 10bit. If not, you need to modify them, and then re-generate the hdf file.

For stream pipeline,  format should be NV12_10LE32 or NV16_10LE32 for 10-bit, not NV16.

 

0 Kudos
Explorer
Explorer
1,706 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Thank You @xud,

We appreciate the time you've spent in addressing our problem.  Hopefully you have time and can help us understand how readily we can use the Zynq UltraScale+ XCZU7EV-2FFVC1156 MPSoC to develop new, better systems. 

We are testing with the Xilinx 2018.2 release. 

We appreciate that you confirm use of NV12_10LE32 or NV16_10LE32 as the primary 10-bit formats.  These both have appeared in the output from test cases run using GStreamer, although we are (mostly) incapable of using them for useful capture. 

Were you able to successfully capture, encode, and store video using a (current) 2018.3 pre-release? 

Assuming that you were able to do so, will you please share your working commands with us, so we can build upon them? 

As stated in the original message, we used "The FPGA installed is the provided “sdirx_vcu” sample.".  Using Vivado against that sample, we observed that 10-bit formats are available at each stage.  Our difficulty has been the pipelining of 10-bit video -- a) raw, b) recordings, c) alpha-blending, d) multiple-streams, and e) output through the various sample builds. 

Also, the "gst_vcu_app" application, while relatively sophisticated, appears to support only:

  1. Streaming of 8-bit video data. 
  2. NV12 formatting. 

Even though there are command inputs which appear capable of affecting these options, the source code which we inspected and modified in our testing was incapable. 

Assuming that you were able to use the pre-release 2018.3 to capture and encode 10-bit video from 12Gbit/sec SD input, we would like to know what you had to do:

  1. What was the FPGA build? 
  2. What were the command sequences required?  [including commands such as devmem]. 
  3. Were you capable of recording one-minute long videos using the streaming pipeline that you constructed? 
  4. Were you capable of 4:2:2 chroma subsampling (NV16, NV16_10LE32, &c.), were you restricted to 4:2:0 subsampling, or are other forms available?
  5. Will 12-bit (or greater) colour depths be available? 
  6. Can you give us a schedule of expected releases and the processing capabilities supported and demonstrated in examples therein? 

Lastly, is a current, pre-release 2018.3 available for us to work with?  Specifically one which is compatible with the command modifications you suggest? 

Sir, we really appreciate your help.  You've answered questions and confirmed some of our concerns.  Hopefully we can perform verification of the required capabilities of the system using the 2018.3 release, and begin hardware design knowing the Xilinx SoC and what we have to add. 

Kind Regards,
Peimann

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
1,666 Views
Registered: ‎08-02-2007

Re: SDI Video to 10 Bit H265 MPEG

@peimann

The release date of 2018.3 is very close. I think it's better to wait till 2018.3.

If it's very urgent to you, please work with your FAE, and see if it's possible to get the EA version.

As per documented in PG252, VCU supports :

. YCbCr 4:2:2, YCbCr 4:2:0, and Y-only (monochrome)

° 8- and 10-bit per color channel

I don't think the restriction is at VCU itself, at ctrl sw level it already supports 10-bit. I only have done some tests with receiving the 10-bit Video file. I haven't tested SDI RX yet.

 

 

 

0 Kudos
Explorer
Explorer
1,644 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Afternoon @xud,

Thank you for getting back to me. 

Do you have a targeted release date?  If so, we can report to our management, so that they have an indication of when efforts can proceed on the project. 

My FAE is on vacation at the moment, and thus was not able to respond to the note that I mailed to him before posting to the forums. 

If the software and hardware already support 10 bit, then I would expect the attached file bash script to capture a minute's worth of video. 

  • Note, attached file includes "mp4mux" that was somehow edited out when writing original message. 
  • Note, attached file includes "NV16_10LE32" as stated in your original reply. 

Failure is the same in two different scenarios:

  • GStreamer reports "Failed to enable OMX encoder: Component doesn't have enough hardware resources available to process the channel (0x90001001)"
  • dmesg reports:
    • [29106.563344] Failed on create channel
      [29120.532408] al5e a0100000.al5e: VCU: unavailable resources or wrong configuration

Time to fail is either ~20 seconds, or 0.25 seconds. 

What are we doing incorrectly????

Many Thanks!

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
1,579 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

If you look at the hardware setup for the 2018.2 ZCU106 VCU TRD you will find that the SDI output is 10-bits, but then the PL hardware was modified to drop the 2 LSBs before sending it to the rest of the pipeline.  This makes the hardware pipeline to be 8bpc (bits per component) and you will not be able to capture 10bit-data due to this hardware modification.

But with that said, as you found in your other post, you can use videotestsrc to generate input video and you may be able to force videotestsrc to generate 10-bit data for testing.


Unfortunately for release information you will need to contact your local FAE.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
0 Kudos
Explorer
Explorer
1,560 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Thanks for Helping @chrisar,

From what I've been able to observe and learn, the full 10-bit, end-to-end pipeline was not available in the 2018.2 release. 

Are you able to demonstrate success (under 2018.3) using the script that I posted as an attachment?  [Not the original code that I posted in-line]. 

Did I do everything correctly in that script, or is there a simple, glaring error. 

I believe that the script fails due to an incomplete (or buggy) element in GStreamer, or missing information in the Xilinx drivers, or both.  Hopefully the next Xilinx release will allow me to make a ground-up build of the example, and then my script will work. 

That said, due to my ignorance using the entire software chain in a repeatable, appropriate manner, I have been trying to avoid building an enhanced 2018.2 release which updates GStreamer up to the latest version, possibly patches elements of the kernel, and will likely introduce actual bugs (by me).  Especially as 2018-12-31 is the (nominal) latest end-date for the release, I am willing to wait a short while and focus effort in other ways. 

Really, while waiting, I want to bring my Xilinx/PetaLinux/Yocto skills up to snuff, so that I can implement suggested solutions in a safe manner, with minimal effort. 

Again, many thanks for your help @chrisar

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
1,495 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

@peimannthe 2018.3 ZCU106 VCU TRD was just released yesterday.  You can find it on the Xilinx Wiki Zynq UltraScale+ MPSoC VCU TRD 2018.3 page.  This version does have support for 10-bit for the SDI Rx design.  I did not have a chance to try your script on this version.

 

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
Explorer
Explorer
1,490 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Kudos to you @chrisar,

I'll pull down the update and work with it.  It'll likely be Tuesday before I can fully verify functionality.

Many Thanks!
--Scott Peimann

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Explorer
Explorer
1,443 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Afternoon @chrisar,

I used the 2018.3 image vcu_sdirxtx to re-run this example. 

Still fails, as it cannot negotiate. 

The output files (from original test and current) are attached in the zip. 

Thanks,
Scott M. Peimann

 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
1,426 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

We can reproduce the negotiation error on the SDI Rx interface and we are looking into this.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
0 Kudos
Explorer
Explorer
1,405 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Hello @chrisar,

I was thinking that this might be related to the version of GStreamer that is built/used by PetaLinux

On the Xilinx, the version of GStreamer is 1.12.2. 

root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19# gst-launch-1.0 --version
gst-launch-1.0 version 1.12.2
GStreamer 1.12.2
Unknown package origin
root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19#

 

If I remember, I believe that the current, stable GStreamer version is 1.14.4 (ref: "Recent older news:", https://gstreamer.freedesktop.org).

From what I have read, an amount of greater-than-eight-bit work has been done, thus bringing the tools into the nineteenth century. 

Will you guys please try an appropriate update and let me know if that gets us a bit further on? 

Many Thanks!

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
1,380 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

Here is where we are at right now.  It turns out that 10-bit support was not completed in the VCU TRD.  In the 2018.3 VCU TRD, the hardware portion was put in place, but no work was done on the software side, in order to support 10-bit.

At this point I don’t have an estimate on how much work it will take.  I’m hoping it isn’t too much, but I can’t be sure until I have time to dig into it more.  It could just be GStreamer, but it could also be driver support.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
Explorer
Explorer
1,371 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Morning @chrisar,

Thanks for getting back to me on this.  I appreciate knowing what the status of the components is; it really helps me report and focus my efforts in the "correct" areas. 

If there are any straightforward tests that I can do to verify drivers, please let me know. 

Kudos for the update keeping me current! 

With luck, next year will be good to us both. 

 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Explorer
Explorer
470 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Hello @chrisar,

We are able to capture 10-bit 4:2:2 video using this script against the 2019-1 release:

#!/bin/bash -v

time xmedia-ctl --verbose --device /dev/media0   \
     --set-v4l '"a0030000.v_smpte_uhdsdi_rx_ss":0  [fmt:UYVY10_1X20/3840x2160 field:none],
                "a0080000.v_proc_ss":0  [fmt:UYVY10_1X20/3840x2160 field:none],
                "a0080000.v_proc_ss":1  [fmt:UYVY10_1X20/3840x2160 field:none]'

##--works; element: omx265enc - remove attribute(s): prefetch-buffer
time gst-launch-1.0 --gst-debug-level=3 --gst-plugin-spew --messages --verbose --eos-on-shutdown --gst-fatal-warnings  \
    v4l2src device=/dev/video0 io-mode=dmabuf num-buffers=3000  \
    ! capsfilter caps='video/x-raw,format=NV16_10LE32,framerate=50/1,height=2160,width=3840'  \
    ! omxh265enc gop-length=30 gop-mode=basic low-bandwidth=true target-bitrate=100000  \
                 b-frames=0 num-slices=4 control-rate=variable qp-mode=uniform periodicity-idr=30  \
    ! queue max-size-buffers=5 max-size-bytes=0  \
    ! h265parse  \
    ! qtmux  \
    ! capsfilter caps='video/quicktime'  \
    ! filesink location=UYVY10_1X20.UYVY10_1X20.NV16_10LE32.format-test.002.12G.h265.2019-1.mp4

Note that the test was slightly modified to use a variable-bit-rate, rather than a constant-bit-rate; this is not related to the failure. 

Also, our observations found that:

  1. The omxh265enc attribute "latency-mode" is now obsolete. 
  2. The omxh265enc attribute "prefetch-buffer" causes failure with a GStreamer "insufficient resources" message. 

However, we have been unable to find proper settings for 4:2:0 video.

What should I use for the xmedia-ctl format specifications, so that they are compatible with NV12_10LE32?

When you have a moment, will you please look into the 4:2:0 encoding problem. 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Observer johne1969
Observer
425 Views
Registered: ‎05-07-2018

Re: SDI Video to 10 Bit H265 MPEG

Hi @peimann 

I tried your script against the 2019.1 release, but I could not succesfully capture mp4 although I could capture ts. When trying to capture mp4 gst-launch aborts with error "Buffer has no PTS." A quick google shows this is sometimes an error when capturing mp4 using qtmux. Did you ever see this error?

Thanks

0 Kudos
Explorer
Explorer
372 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Morning @johne1969,

My apologies for the delay.  Your note is came on my National Holiday, and I've been out-of-office. 

I recall seeing PTS errors and don't recall exactly what they mean.  To understand some of the non-obvious errors from GStreamer, I have taken to searching through the source code to find the root cause.  My copy is from the on-line repository, accessible through the site. 

As I observed, the following options help with diagnosing problems:

gst-launch-1.0  --gst-debug-level=3 --gst-plugin-spew --messages --verbose --eos-on-shutdown --gst-fatal-warnings

Usually the first warning that I see is the key to the failure. 

 

Also, I used the

vcu_sdirxtx

pre-build from the 2019.1 release in my last test on this thread. 

 

NB: the Git repository for the GStreamer build is here:

git://anongit.freedesktop.org/gstreamer/gst-build

Be certain to actually run the build, so that all of the supporting elements are loaded under the subprojects directory. 

The GStreamer build process is a pain, but worth doing. 

Your error comes from "subprojects/gst-plugins-good/gst/isomp4/gstqtmux.c", line 4832. 

It seems that the stream is trying to add a buffer to the qtmux element, although the exact "cause" would be in this:

  if (!GST_BUFFER_PTS_IS_VALID (last_buf))
    goto no_pts;

Unfortunately the term "PTS" has me somewhat confused, perhaps "Prior Time Stamp"?

It turns out that "PTS" means "Presentation Time Stamp.". 

Look into file

subprojects/gstreamer/gst/gstbuffer.h

for the macro definition; the message there states "Tests if the pts is known.".

Ok, using "DUCK" to find "PTS", I find in the GStreamer web-site that:

The buffer DTS refers to the timestamp when the buffer should be decoded and is usually monotonically increasing. The buffer PTS refers to the timestamp when the buffer content should be presented to the user and is not always monotonically increasing.

Read this: https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html

Hope that helps some. 

Also, there are quite a few links available through "DUCK", so you may need to take time to read through them. 

Try reading https://stackoverflow.com/questions/42874691/gstreamer-for-android-buffer-has-no-pts

Luck!

 

NB: post was edited to replace indent that was lost during HTML verification by the forum with a "preformatted" box. 

 

 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
Explorer
Explorer
362 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Hello Again @johne1969,

I see this problem a great deal in my dual-encode tests that I am working with. 

Your problem seems concerned with the qtmux element.  I'd suggest that you try the matroska mux element to see if it helps. 

Keep watching the forums for a bit, as I am trying to write up the problem now and on the Video forum when I have completed. 

Cheers!

Just remember, if this was easy someone else would have done it already. 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
Explorer
Explorer
355 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Hello Once Again @johne1969,

It looks like the qtmux element dies where the matroskamux element just drops frames.  This is an issue that will push all the way back to GStreamer, as we are illuminating a potential corner case.  I haven't tried running version diffs on the source-code for the qtmux element. 

Will you please create a new thread for your problem, so that this thread can stay focused on my issues with real-time, 10-bit encoding? 

When you do, e-Mail the link to me, and I'll do my best to help you. 

Thanks!

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
Observer johne1969
Observer
349 Views
Registered: ‎05-07-2018

Re: SDI Video to 10 Bit H265 MPEG

 

Hi @peimann 

 

Thanks for your very detailed replies! I will start a new thread, but will try to understand a little more about what is causing the problem first.

Regards

0 Kudos