NV16 Video Pass Through in sdirx_vcu_sdtrx

[peimann]

Good Evening Folks,

Attached are a script and output file showing my system's failure to pass 3G and 12G video through using NV16 (4:2:2). 

You will see where I have commented out where NV12 (4:2:0) "works". 

What am I doing wrong? 

• Wrong format specification to xmedia-ctl?
• Incorrect format specification in the capabilities filter?

Any guidance is appreciated.  I have done quite a few iterations to identify what works, and I am at a point where I have eroded into the scalp where my hair used to be. 

Run with one of the following two commands:

./xilinx-question.002.bash 3G
./xilinx-question.002.bash 12G

I am running on a ZCU-106, rev C, under 2018.2 until my download completes, then I will update. 

Thanks,
Scott Peimann

[chrisar]

Sorry for the frustration, but NV16 is not supported in the 2018.1 or 2018.2 versions of the ZCU106 VCU TRD, only NV12 is supported.
If you look at the hardware you will see that the LSBs are removed and the data feeding the Frame Buffer is only 8-bits (NV12).

[peimann]

Good Afternoon @chrisar,

I just tried this with the 2018.3 pre-build vcu_sdirxtx.  No joy. 

Attached are the previous output and current output. 

Can you give me any advise?

Thank You,
Scott M. Peimann

[peimann]

Good Morning @chrisar,

I appreciate your help on this topic.  I must apologise to you, as my previous response was useless, and frankly intellectually void.  Please tell me when I waste your time with something like that. 

My previous post was vague about what I wanted to communicate - what I should have written is that we are not capable of invoking NV16 (4:2:2 8 bit) pass-through using the 2018.3 pre-build vcu_sdirxtx. 

I understand that ten bit video is supposed to be fully functional on the ZCU-106 hardware under 2018.3, however the Linux drivers, Linux utilities, and vcu_gst_app have not been brought up to a tested level of 10 bit functionality. 

The problem/goal which we are discussing here is to demonstrate simple, correct behaviour of the system:

1. Accept 12G (or 3G) video input, which I understand by standards is normally 4:2:2 10 bits. 
2. Pass 12G video through the vcu_sdirxtx FPGA as 4:2:2 8 bits. 
3. Emit the pass-through video as 12G SDI, again 4:2:2 10 bits. 

Under this test, I would expect to see the data stream transmit from the FPGA differing from the input stream by two bits. 

When slightly modified, the bash file, xilinx-question.002.bash will run NV12 pass-through of both 3G and 12G video.  Using an external bit-stream analyser to inspect the input and output streams, I see colour transforms of single-colour input, which I have documented for one of our optics people to confirm.  If he sees these as problematic transforms, it will be a new topic. 

My working NV12 (4:2:0) pass-through is attached as part of this post as xilinx-question.002.NV12.bash.  Also attached is a copy of the execution output.  To use this script, it requires on argument, 3G or 12G, to specify the SDI input.  You will likely have to configure the autostart.sh script, so that modetest is compatible.  See my post here, wherein I ask about modetest. 

Invocation of SDI pass-through using NV12 on FPGA:

./xilinx-question.002.NV12.bash 3G
./xilinx-question.002.NV12.bash 12G

So, how should I modify the script to give NV16 pass-through? 

The two locations marked "DO WE CHANGE HERE FOR NV12 versus NV16?" are where I believe that the script should be modified.  Unfortunately neither, or both together, of the commented out changes work. 

Apparently I am doing something incorrectly, I am unable to figure out what it might be. 

@chrisar, again, I apologise for my previous post on this topic, your time is valuable to me, wasting it is unacceptable. 

Hopefully this post is complete enough that you have the "right" information to help me with this problem. If you need more information, I will get it for you. 

Thank You,
Scott Peimann

---

ref: autostart.sh - https://forums.xilinx.com/t5/Embedded-Linux/Invocation-of-modetest-In-media-card-autostart-sh-Fail-on-ZCU/td-p/922468

[hgtcs]

@peimann 

hello，

I'm sorry to trouble you, but I wondered if you have solved this problem.

I also meet this problem. I hope you can give me some guidance.

Thank you very much,

hgtcs

[peimann]

Good Morning @hgtcs,

I am still waiting on the 2019.1 release, so that I can re-test this problem. 

There are several issues with the 2018.3 Vivado release, and within PetaLinux, as well. 

As this problem is constructed using elements of GStreamer [see: https://gstreamer.freedesktop.org/], I am limited by the capabilities of the supported version.  The support in version 1.12, which distributes in the pre-built image that I was testing with.  That particular version of GStreamer does not support NV16 and 10-bit in a comprehensive fashion.  The current release candidate (v1.15.90, presumably released as v1.16) has a great deal of work targeting NV16/10-bit (i.e. NV16_10LE32, &c.). 

Although not directly affecting this problem, some of the OMX plug-in code specific to the Xilinx [see: https://github.com/GStreamer/gst-omx] did not fully support NV16 and 10-bit at the time when I wrote the original post.  Recent efforts on GStreamer show enhancement to the Xilinx elements, which is an encouraging sign. 

Not only GStreamer, but there are device drivers specific to Xilinx, which may not have fully, correctly, or consistently supported NV16 and 10-bit.  When I read through some of the driver code, it appeared that NV16/10-bit was there, but the implementation had some consistency issues that gave doubt.  I have not been tracking against changes to the Xilinx drivers, although my belief is that @chrisar can address this in detail. 

My belief is that these both of these elements must be in the 2019.1 distribution for NV16 and 10-bit to work properly.  I suppose that one might solve this problem by writing device-control software which directly addresses the FPGA elements through their registers, however that obviates the much of the value of the Linux "distro". 

Thus, when the new distribution is out, I will re-evaluate using the test code for this post, as well as several other, related posts.  Given that some of the changes I saw when browsing the repository were specifically for Xilinx based encoders/decoders, we may well see very good things this next release.  My hopes are up, however GStreamer is still showing as release-candidate as of this morning. 

Once I can evaluate, I will be updating all of my outstanding posts. 

Good Luck!

[hgtcs]

Good Morning @peimann,

Thanks for your reply and look forward to your test.

Good Luck!

hgtcs

[peimann]

Good Afternoon @chrisar and @hgtcs,

It seems that this only works for NV12, even though it seems as if it aught work for NV16 in 2019.1, as well. 

Apparently there was a driver update for DRM/KMS, as I have to modify my invocation of modetest to requires use of connector 37, rather than 36 [as in 2018.3]:

sleep 7d | /usr/bin/modetest -M xlnx -s 37:3840x2160-60@YUYV  -w 37:sdi_mode:5 -w 37:sdi_data_stream:8 -w 37:is_frac:0 &

Here is a copy of my script - note three change points for NV16 versus NV12: 

root@zcu106_vcu_trd:/media/sata/xilinx-question.002.2019-07-02# cat ./xilinx-question.002.2019-1.bash
#!/bin/bash
##
##  xilinx-question.002.bash
##    - 3G versus 12G video copy through
##

  ##--fixed input configuration
  ##--assumes that no significant difference betwixt HD and UHD exists...

  declare  -ri  IN_HEIGHT=2160
  declare  -ri  IN_WIDTH=3840

  declare  -r   IN_WxH="${IN_WIDTH}x${IN_HEIGHT}"

  declare  -ri  OUT_HEIGHT=${IN_HEIGHT}
  declare  -ri  OUT_WIDTH=${IN_WIDTH}


  ##--Use KMS to test that the driver is in good shape
  ##    - KMS: Kernel Mode Setting

  if  kmstest -M xlnx;  then
    printf  '\n\n  ==> success: kmstest \n'
    sleep   1
  else
    printf  '\n\n  ==> failed to test health of video output.  ABORT! <== \n\n\n'
    exit    1
  fi


  ##--configuration of the media control mess

  declare  -r  XM_SOURCE_MB_FORMAT="[fmt:UYVY10_1X20/${IN_WxH} field:none]"


  ######--DO WE CHANGE HERE FOR NV12 versus NV16?

  #eclare  -r  XM_SCALER_MB_FORMAT="[fmt:VYYUYY8_1X24/${IN_WxH} field:none]"  ## works, NV12
  declare  -r  XM_SCALER_MB_FORMAT="[fmt:UYVY8_1X16/${IN_WxH} field:none]"    ## suitable for NV16?


  ##--do the media control setup

  if  xmedia-ctl --verbose  \
        --set-v4l "5:0 ${XM_SOURCE_MB_FORMAT}, 7:0 ${XM_SOURCE_MB_FORMAT}, 7:1 ${XM_SCALER_MB_FORMAT}"
  then
    printf  '\n\n  ==> success: xmedia-ctl \n'
    sleep   1
  else
    printf  '\n\n  ==> failed to configure input.  ABORT! <== \n\n\n'
    exit    1
  fi


  ##--configuration of the GST mess

  declare  -ra  GST_OPTION=('--eos-on-shutdown')
  declare  -ra  GST_DEBUG=('--messages' '--tags' '--toc' '--verbose')

  declare  -ra  GST_V4L2src=('device=/dev/video0' 'io-mode=dmabuf')

  declare  -ra  GST_QUEUE=('max-size-bytes=0')

  #eclare  -ra  GST_CAPSFILTER=("caps=video/x-raw,format=NV12,framerate=60/1,height=${OUT_HEIGHT},width=${OUT_WIDTH}")  ## works, NV12
  declare  -ra  GST_CAPSFILTER=("caps=video/x-raw,format=NV16,framerate=60/1,height=${OUT_HEIGHT},width=${OUT_WIDTH}")  ## suitable for NV16?

  #eclare  -ra  GST_KMSSINK=('driver-name=xlnx')                 ## works, NV12
  declare  -ra  GST_KMSSINK=('driver-name=xlnx' 'plane-id=34')   ## suitable for NV16?


  ##--do the GST deed
  gst-launch-1.0  "${GST_OPTION[@]}"  "${GST_DEBUG[@]}"  \
      v4l2src       "${GST_V4L2SRC[@]}"   \
      ! queue       "${GST_QUEUE[@]}"     \
      ! capsfilter  "${GST_CAPSFILTER}"   \
      ! kmssink     "${GST_KMSSINK[@]}"


##
##--end of file
##
root@zcu106_vcu_trd:/media/sata/xilinx-question.002.2019-07-02#

From out-of band conversation we know that output of 10-bit does not yet work correctly under GStreamer, so there was no attempt made for formats NV12_10LE32 and NV16_10LE32.

Chris - I have two questions for you:

1. How would I best modify this script to pass NV16 from SDI to SDI? 
2. Do you expect additional SDI output changes under 2019.2?