UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
501 Views
Registered: ‎06-07-2019

Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Hi,

I am using the VCU on the ZCU104 board. I have upload the images contained in the BSP for the ZCU104 (https://www.xilinx.com/member/forms/download/xef.html?filename=xilinx-zcu104-v2019.1-final.bsp) using the software PetaLinux 2019.1. I have installed the software and send the binaries following the instructions fo the 2 documents below:

pg252 - H.264/H.265 Video Codec Unit v1.2

ug1144 - petalinux tools reference guide

I have successfully used several of the example given in this BSP. I have also used a picture that I have transformed into a binary format, more specifically, I have transformed this picture in Y800 (8 bits per pixels, only luma component). I have encoded it then decoded it with the VCU to see any difference between the two pictures side by side. It worked (both compression and decompression and I have been able to see the two pictures). I used these command:

 

ctrlsw_encoder -i myImage.bin -o myImage.bin.yuv --input-width 1016 --input-height 256 --chroma-mode CHROMA_MONO --input-format Y800
ctrlsw_decoder -i myImage.bin.yuv -o myImage.bin.yuv.bin

Now I am trying to do the same with a 10 bits picture but it does not work. The document "pg252 - H.264/H.265 Video Codec Unit v1.2" says that the format Y010 is for "Input files (that) contains monochrome 10-bit video samples each stored in a 16-bit word". I have transformed my picture according to these specifications, then gave it to the encoder using the following command:

 

 

ctrlsw_encoder -i myImage10bits.bin -o myImage10bits.bin.yuv --input-width 1016 --input-height 256 --chroma-mode CHROMA_MONO --input-format Y010 --ip-bitdepth 10

First of all, if I don't precise the "--ip-bitdepth 10" then the encoder output an error:

 

 

Allegro DVT2 - AVC/HEVC Encoder Reference Software v1.0.44 - Copyright (C) 2019
Confidential material

!! Warning : Adapting profile to support bitdepth and chroma mode
No conversion known from Y010
ctrlsw_encoder: lib_conv_yuv/AL_NvxConvert.cpp:114: void convertToY800(const AL_TBuffer*, TFourCC, AL_TBuffer*): Assertion `0' failed.
Aborted

When, I precise the "--ip-bitdepth 10" then the encoder seems to succeed the compression:

 

Allegro DVT2 - AVC/HEVC Encoder Reference Software v1.0.44 - Copyright (C) 2019
Confidential material

!! Warning : Adapting profile to support bitdepth and chroma mode
Encoding picture #0 - Flushing...
Achieved bitrate = 34261.4375 Kbps

1 pictures encoded. Average FrameRate = 200.0000 Fps

Then I decode the image using the decoder:

 

ctrlsw_decoder -i myImage10bits.bin.yuv -o myImage10bits.bin.yuv.bin

It outputs this:

 

 

Allegro DVT2 - AVC/HEVC Decoder Reference Software v1.0.44 - Copyright (C) 2019
Confidential material

Resolution: 1016x256
FourCC: XV10
Profile: 4
Level: 51
Bitdepth: 10
Sequence picture: progressive
Buffers needed: 19 of size 393216

Displayed picture #0 - Complete

Decoded time = 0.0310 s; Decoding FrameRate ~ 32.2581 Fps; Frame(s) conceal = 0

So, it does seem to work, but the output format is XV10 and not anymore Y010. I have not found any information about this format in the documentation pg252 of the VCU (or anywhere else that provides me a understanding of this format). The problem is that, I can not display my picture like I used to do with an 8 bits picture.

Even if I used the option :

--rec-format Y010

I do not succeed to have the wanted output format.

What am I missing ? Do you have information about the XV10 format ?

I would be grateful for any help provided.

Tags (1)
0 Kudos
1 Solution

Accepted Solutions
428 Views
Registered: ‎06-07-2019

Re: Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Hi, 

So, I have investigated a bit more,

The problem was that when you give to the VCU a monochrome picture (10 bits per pixels) in the binary format Y010, it will compress your picture. When you decompress your picture, it will output the binary in a XV10 format which I was not able to read. 

Solution/explanation:

I believe that Y010 and XV10 are the same binary format : "Input files (that) contains monochrome 10-bit video samples each stored in a 16-bit word".

However, what I did not understood was the fact that, the 16-bit representations of the Y010 use "little-endian representation" for each pixel. Meaning that, you will store each pixel in a 16-bit structure such that: the 10 bits value of a pixel are the Most Significant Bits and the 6 Least Significant Bits are set to zero.

My problem was that I did the opposite and used a "big-endian representation" (6 Most Significant Bits are 0 and 10 Least Significant Bits are the pixel value). This  detail was not precised in the documentation pg252 - H.264/H.265 Video Codec Unit v1.2.

Once the encoder was given the good binary representation Y010, if you compress your picture then decompress it, it will still output the XV10 format instead of the Y010, but it does not seem to be a problem, as I have successfully decoded the picture as if it was a Y010 format.

One last point about the binary file generated by the decoder: the 6 Most Significant Bits will not always be set to zero but instead it will be some random bits.

Hope it helps some of you

bye

4 Replies
429 Views
Registered: ‎06-07-2019

Re: Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Hi, 

So, I have investigated a bit more,

The problem was that when you give to the VCU a monochrome picture (10 bits per pixels) in the binary format Y010, it will compress your picture. When you decompress your picture, it will output the binary in a XV10 format which I was not able to read. 

Solution/explanation:

I believe that Y010 and XV10 are the same binary format : "Input files (that) contains monochrome 10-bit video samples each stored in a 16-bit word".

However, what I did not understood was the fact that, the 16-bit representations of the Y010 use "little-endian representation" for each pixel. Meaning that, you will store each pixel in a 16-bit structure such that: the 10 bits value of a pixel are the Most Significant Bits and the 6 Least Significant Bits are set to zero.

My problem was that I did the opposite and used a "big-endian representation" (6 Most Significant Bits are 0 and 10 Least Significant Bits are the pixel value). This  detail was not precised in the documentation pg252 - H.264/H.265 Video Codec Unit v1.2.

Once the encoder was given the good binary representation Y010, if you compress your picture then decompress it, it will still output the XV10 format instead of the Y010, but it does not seem to be a problem, as I have successfully decoded the picture as if it was a Y010 format.

One last point about the binary file generated by the decoder: the 6 Most Significant Bits will not always be set to zero but instead it will be some random bits.

Hope it helps some of you

bye

379 Views
Registered: ‎06-07-2019

Re: Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Hi again, 

I am currently writing a report about my work (including the issue discussing here, I mean... a one man discussion). I have realized that what I said on my previous post may be confusing, so let me give an example about the conversion of a luma pixel (coded on 10 bits) into Y010 format:

Let say you have a value of a luma pixel which is 767. In binairy, you will write this 1011111111

After conversion in Y010 format, you will end up with these 16-bits in memory : 1011111111 000000

The first 10 bits (most significant bits) are your value 767 in binairy in a Big Endian form and the six zeros that are padded at the end (least significant bits). 

So the value of the pixel is in encoded in Big Endian, and the 6 zeros are padded at the end, which leads to :

Y010 = "Input files contains monochrome 10-bit video samples each stored in a 16-bit word. In memory the first 10bits are the luma value (in a Big Endian representation) then the last 6 least significant bits are zeros." 

I wrongly used the term little endian in my previous post (english is not my first language)

Bye.

Xilinx Employee
Xilinx Employee
193 Views
Registered: ‎08-01-2007

Re: Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Let me try to clarify the 10-bit Y-Only formats.

Video formats can be confusing.  Often the same format can have different names depending on what software layer you are looking at, or if you are talking about the format supported by the hardware.  And the formats supported by the VCU are no exception to this.

The Zynq UltraScale+ MPSoC VCU Control Software is based on the fourcc formats.   But there were some formats supported by the Zynq UltraScale+ MPSoC VCU that didn't have existing format codes, so the Zynq UltraScale+ MPSoC VCU Control Software has extended the formats list to address these missing formats.

The Y010 listed in PG252 is actually the same at Y10 in the V4L2 Pixel Formats (pixfmt-y10). This is a 16-bit pixel format where the MSBs are typically zero, but could be are really a don't care.  The reason that it is called Y010 is that the Zynq UltraScale+ MPSoC VCU software is based on the fourcc codes and fourcc requires 4 character format definitions.

 

Format: V4L2_PIX_FMT_Y10; Y10; Y010
16-bit word, 10-bit data with 6-bit MSB padding (6-10) (15:10) (9:0) X Y0

 

 

You should also keep in mind that the Zynq UltraScale+ MPSoC VCU example applications (ctrlsw_encoder and ctrlsw_decoder) actually include some conversion APIs that allow for the software conversion to and from a short list of formats to the native format supported by the Zynq UltraScale+ MPSoC VCU.

The native format supported for Y-Only is XY10 (V4L2 and FOURCC) or XV10 as it is called in the Zynq UltraScale+ MPSoC VCU Control Software.  The XV10 format is a packed 32-bit format.

 

Format: V4L2_PIX_FMT_XY10; XY10; XV10
32-bit word, 10-bit sample, 10-bit sample and 10-bit sample with 2-bit MSB padding (2-10-10-10) (31:30) (29:20) (19:10) (9:0) X Y2 Y1 Y0

 I hope that this helps to clear up the confusion on the Y-Only formats.


Additional Ref:
Xilinx Video IP Formats ( xvip_video_format xvip_video_formats[] )

Chris
Video Design Hub | Embedded SW Support

---------------------------------------------------------------------------
Don’t forget to Reply, Kudo, and Accept as Solution.
---------------------------------------------------------------------------
144 Views
Registered: ‎06-07-2019

Re: Why does the VCU Control Software Encoder does not output the format specified ?

Jump to solution

Hi, thank you for your answer. I have understood where my error was.

First, I point out that 767 is written 10111 11111 in binary format (2 MSB are in red).

So, earlier in this post I said the Y010 format might be:

16-bit word, 10-bit data with 6-bit LSB padding (6-10)
(15:6) (5:0)
    Y0     X
For example, with pixel value of 767: 10111 11111 000000

When the conversion was done, I was writing into the memory for the VCU the 16bits this way: 10111111  11000000.

Indeed, it was the only format that worked for me (I was able to compress then decompress the image and print it). The quality was not great (worth than with the 8bits format Y800).

As you stated that the real format is:

16-bit word, 10-bit data with 6-bit MSB padding (6-10)
(15:10) (9:0)
     X    Y0
For example, with pixel value of 767: 000000 10111 11111

This is a format I had tried and it did not work. Indeed, I was writing into the memory the 16bits “directly”: 000000 10111 11111 which do not work!

The way it should be done is: the 16-bits of data have to be written by 8bits packets starting with the packets of LSB. For example, the 767 pixel value should be written in memory: 11111111 then 00000010.

 

This is an 8bits Little Endian representation of a number.

The XV10 format works the same way; each 32-bits of data have to be written by 8bits packets starting with the packets of LSB: 8bits Little Endianrepresentation of a number.

 

Maybe this should be written in the documentation (pg252), but it also seems that it is a very usual way of representing a number (but I lack of experience).

 

For those who wonder how the wrong format could work with the VCU and just output a result of bad quality (some blurred image) and not some random bunch of binary file, here is the explanation: if I send to the VCU the pixel value 380 = 01011 11100:

  • in the “true” format, it would be 01111100 00000001.
  • in the “wrong” format it would be: 01011111 00000000 which will be interpreted as 95 by the VCU.

But 95 = 380/4, so the binary representation is about the same as 767 except that the bit are shift on the right, the 2 LSB of 767 have been erased (some details are lost, the image is a little blurred). So the VCU still compress the most important bit of the 767 binary representation and output an image which is visually acceptable.

 

Thank you for the inforamtion and all your assistance. I hope this will help other people.