UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Participant peimann
Participant
936 Views
Registered: ‎10-03-2018

SDI Video to 10 Bit H265 MPEG

Folks,

The FPGA team here recommended that I ask you for a bit of assistance in using the FPGA board to capture 10 bit video on the ZCU-106, rev. C board. 

The FPGA installed is the provided “sdirx_vcu” sample. 

I believe that I have effectively worked the sequence down to a couple commands, and I’m unsure where this is going wrong:

xmedia-ctl --verbose --device /dev/media0   \
     --set-v4l '"a0030000.smpte_uhdsdi_rx_ss":0  [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":0  [fmt:UYVY8_1X16/3840x2160 field:none], "a0080000.scaler":1 [fmt: UYVY8_1X16/3840x2160 field:none]' 

gst-launch-1.0 --messages --tags --toc --verbose  \
    v4l2src device=/dev/video0 io-mode=dmabuf num-buffers=3000 !  \
    capsfilter caps=video/x-raw,format=NV16,framerate=50/1,height=2160,width=3840,bit-depth-luma=10,bit-depth-chroma=10 !  \
     omxh265enc gop-length=30 gop-mode=basic low-bandwidth=true target-bitrate=100000 b-frames=0 num-slices=4 control-rate=constant qp-mode=uniform periodicity-idr=30 prefetch-buffer=true filler-data=true latency-mode=normal ! \
    queue max-size-bytes=0 !  \
     capsfilter caps=video/x-h265 !  \
     h265parse !  \
     filesink location=UYVY8_1X16.UYVY8_1X16.NV16.format-test.002.12G.h265.mp4  

 

Neither the h264 or h265 encoding seem to work for me. 

Have you any suggestions? 

Kind Regards, Peimann, Scott Meyer
 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
13 Replies
Xilinx Employee
Xilinx Employee
868 Views
Registered: ‎08-02-2007

Re: SDI Video to 10 Bit H265 MPEG

Hi @peimann

Which VCU TRD are you using? I don't think it supports 10bit until 2018.3. It's better to wait for the 2018.3 VCU TRD.

Please open block design in IPI, and check if all the video IP supports 10bit. If not, you need to modify them, and then re-generate the hdf file.

For stream pipeline,  format should be NV12_10LE32 or NV16_10LE32 for 10-bit, not NV16.

 

0 Kudos
Participant peimann
Participant
843 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Thank You @xud,

We appreciate the time you've spent in addressing our problem.  Hopefully you have time and can help us understand how readily we can use the Zynq UltraScale+ XCZU7EV-2FFVC1156 MPSoC to develop new, better systems. 

We are testing with the Xilinx 2018.2 release. 

We appreciate that you confirm use of NV12_10LE32 or NV16_10LE32 as the primary 10-bit formats.  These both have appeared in the output from test cases run using GStreamer, although we are (mostly) incapable of using them for useful capture. 

Were you able to successfully capture, encode, and store video using a (current) 2018.3 pre-release? 

Assuming that you were able to do so, will you please share your working commands with us, so we can build upon them? 

As stated in the original message, we used "The FPGA installed is the provided “sdirx_vcu” sample.".  Using Vivado against that sample, we observed that 10-bit formats are available at each stage.  Our difficulty has been the pipelining of 10-bit video -- a) raw, b) recordings, c) alpha-blending, d) multiple-streams, and e) output through the various sample builds. 

Also, the "gst_vcu_app" application, while relatively sophisticated, appears to support only:

  1. Streaming of 8-bit video data. 
  2. NV12 formatting. 

Even though there are command inputs which appear capable of affecting these options, the source code which we inspected and modified in our testing was incapable. 

Assuming that you were able to use the pre-release 2018.3 to capture and encode 10-bit video from 12Gbit/sec SD input, we would like to know what you had to do:

  1. What was the FPGA build? 
  2. What were the command sequences required?  [including commands such as devmem]. 
  3. Were you capable of recording one-minute long videos using the streaming pipeline that you constructed? 
  4. Were you capable of 4:2:2 chroma subsampling (NV16, NV16_10LE32, &c.), were you restricted to 4:2:0 subsampling, or are other forms available?
  5. Will 12-bit (or greater) colour depths be available? 
  6. Can you give us a schedule of expected releases and the processing capabilities supported and demonstrated in examples therein? 

Lastly, is a current, pre-release 2018.3 available for us to work with?  Specifically one which is compatible with the command modifications you suggest? 

Sir, we really appreciate your help.  You've answered questions and confirmed some of our concerns.  Hopefully we can perform verification of the required capabilities of the system using the 2018.3 release, and begin hardware design knowing the Xilinx SoC and what we have to add. 

Kind Regards,
Peimann

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
803 Views
Registered: ‎08-02-2007

Re: SDI Video to 10 Bit H265 MPEG

@peimann

The release date of 2018.3 is very close. I think it's better to wait till 2018.3.

If it's very urgent to you, please work with your FAE, and see if it's possible to get the EA version.

As per documented in PG252, VCU supports :

. YCbCr 4:2:2, YCbCr 4:2:0, and Y-only (monochrome)

° 8- and 10-bit per color channel

I don't think the restriction is at VCU itself, at ctrl sw level it already supports 10-bit. I only have done some tests with receiving the 10-bit Video file. I haven't tested SDI RX yet.

 

 

 

0 Kudos
Participant peimann
Participant
781 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Afternoon @xud,

Thank you for getting back to me. 

Do you have a targeted release date?  If so, we can report to our management, so that they have an indication of when efforts can proceed on the project. 

My FAE is on vacation at the moment, and thus was not able to respond to the note that I mailed to him before posting to the forums. 

If the software and hardware already support 10 bit, then I would expect the attached file bash script to capture a minute's worth of video. 

  • Note, attached file includes "mp4mux" that was somehow edited out when writing original message. 
  • Note, attached file includes "NV16_10LE32" as stated in your original reply. 

Failure is the same in two different scenarios:

  • GStreamer reports "Failed to enable OMX encoder: Component doesn't have enough hardware resources available to process the channel (0x90001001)"
  • dmesg reports:
    • [29106.563344] Failed on create channel
      [29120.532408] al5e a0100000.al5e: VCU: unavailable resources or wrong configuration

Time to fail is either ~20 seconds, or 0.25 seconds. 

What are we doing incorrectly????

Many Thanks!

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
716 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

If you look at the hardware setup for the 2018.2 ZCU106 VCU TRD you will find that the SDI output is 10-bits, but then the PL hardware was modified to drop the 2 LSBs before sending it to the rest of the pipeline.  This makes the hardware pipeline to be 8bpc (bits per component) and you will not be able to capture 10bit-data due to this hardware modification.

But with that said, as you found in your other post, you can use videotestsrc to generate input video and you may be able to force videotestsrc to generate 10-bit data for testing.


Unfortunately for release information you will need to contact your local FAE.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
0 Kudos
Participant peimann
Participant
697 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Thanks for Helping @chrisar,

From what I've been able to observe and learn, the full 10-bit, end-to-end pipeline was not available in the 2018.2 release. 

Are you able to demonstrate success (under 2018.3) using the script that I posted as an attachment?  [Not the original code that I posted in-line]. 

Did I do everything correctly in that script, or is there a simple, glaring error. 

I believe that the script fails due to an incomplete (or buggy) element in GStreamer, or missing information in the Xilinx drivers, or both.  Hopefully the next Xilinx release will allow me to make a ground-up build of the example, and then my script will work. 

That said, due to my ignorance using the entire software chain in a repeatable, appropriate manner, I have been trying to avoid building an enhanced 2018.2 release which updates GStreamer up to the latest version, possibly patches elements of the kernel, and will likely introduce actual bugs (by me).  Especially as 2018-12-31 is the (nominal) latest end-date for the release, I am willing to wait a short while and focus effort in other ways. 

Really, while waiting, I want to bring my Xilinx/PetaLinux/Yocto skills up to snuff, so that I can implement suggested solutions in a safe manner, with minimal effort. 

Again, many thanks for your help @chrisar

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
632 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

@peimannthe 2018.3 ZCU106 VCU TRD was just released yesterday.  You can find it on the Xilinx Wiki Zynq UltraScale+ MPSoC VCU TRD 2018.3 page.  This version does have support for 10-bit for the SDI Rx design.  I did not have a chance to try your script on this version.

 

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
Participant peimann
Participant
627 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Kudos to you @chrisar,

I'll pull down the update and work with it.  It'll likely be Tuesday before I can fully verify functionality.

Many Thanks!
--Scott Peimann

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Participant peimann
Participant
580 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Afternoon @chrisar,

I used the 2018.3 image vcu_sdirxtx to re-run this example. 

Still fails, as it cannot negotiate. 

The output files (from original test and current) are attached in the zip. 

Thanks,
Scott M. Peimann

 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
561 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

We can reproduce the negotiation error on the SDI Rx interface and we are looking into this.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
0 Kudos
Participant peimann
Participant
540 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Hello @chrisar,

I was thinking that this might be related to the version of GStreamer that is built/used by PetaLinux

On the Xilinx, the version of GStreamer is 1.12.2. 

root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19# gst-launch-1.0 --version
gst-launch-1.0 version 1.12.2
GStreamer 1.12.2
Unknown package origin
root@zcu106_vcu_trd:/media/sata/graphic-madness.000.2018-12-19#

 

If I remember, I believe that the current, stable GStreamer version is 1.14.4 (ref: "Recent older news:", https://gstreamer.freedesktop.org).

From what I have read, an amount of greater-than-eight-bit work has been done, thus bringing the tools into the nineteenth century. 

Will you guys please try an appropriate update and let me know if that gets us a bit further on? 

Many Thanks!

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos
Xilinx Employee
Xilinx Employee
515 Views
Registered: ‎08-01-2007

Re: SDI Video to 10 Bit H265 MPEG

Here is where we are at right now.  It turns out that 10-bit support was not completed in the VCU TRD.  In the 2018.3 VCU TRD, the hardware portion was put in place, but no work was done on the software side, in order to support 10-bit.

At this point I don’t have an estimate on how much work it will take.  I’m hoping it isn’t too much, but I can’t be sure until I have time to dig into it more.  It could just be GStreamer, but it could also be driver support.

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
Participant peimann
Participant
506 Views
Registered: ‎10-03-2018

Re: SDI Video to 10 Bit H265 MPEG

Good Morning @chrisar,

Thanks for getting back to me on this.  I appreciate knowing what the status of the components is; it really helps me report and focus my efforts in the "correct" areas. 

If there are any straightforward tests that I can do to verify drivers, please let me know. 

Kudos for the update keeping me current! 

With luck, next year will be good to us both. 

 

Kind Regards,
Peimann, S. M.
----
Toddlers are the Storm-Troopers of the Great God Entropy.
Physics: Not Just a Good Idea, It's THE LAW.
0 Kudos