cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Video Series 29 – Understanding Video Scaling – Example with VPSS IP

Moderator
Moderator
4 8 2,134

What is Video Scaling?

 

Video Scaling is the process of changing the resolution (size) of a video stream. You can do this in 2 directions: up-scaling and down-scaling (or zoom in and zoom out if you prefer).

How do we do video scaling?

 

Let’s talk about the up-scaling operation. We can take as our example a video input with a resolution of 720p (1280x720). This means that each frame is made up of 720 rows of 1280 pixels, which means 961,600 pixels per frame.

If we have a video output with a resolution of 1080p (1920x1080), each output frame is made up of 1080 rows of 1920 pixels or 2,073,600.

This means that the output would have twice as many pixels as the input.

So where are the new pixels coming from?

 

We need to create the new pixels by estimating their value then adding them to the output frame.

Most algorithms scale one side, either vertically or horizontally, then the other.

For example, the scaler used in the Xilinx Video Processing IP (PG231) starts by scaling the frames vertically, then scales them horizontally.

 

1.jpg

 

Below is a quick example of what it would look like.

Imagine a 3x3 image (9 pixels) that you want to scale to 5x5 (25 pixels). This means that we need to recreate 16 pixels. Note that I am taking a really simplified case here.

 

2.jpg

 

If we start by doing vertical scaling this mean that we need to create 2 lines in-between the existing lines, or 6 pixels (where I have added the crosses below).

 

3.jpg

 

The simplest algorithm would be to take the mean value of each component (Red, Green, Blue) of the pixels above and below the one we are adding.

For example, let's take the first pixel, which is between the red and the green pixels.

A red pixel means the following values for its components (if 8 bits/per component): Red = 255, Green = 0 and Blue = 0.

And for the green pixel: Red = 0, Green = 255, Blue = 0.

Thus, if we take the mean value for each component of the new pixel we would get: Red = 127, Green = 127, Blue = 0 (a kind of yellow-green color).

If we do the same for all of the pixels this is what we would get after the upscaling:

 

4.jpg

 

I know what you are thinking. It does not look pretty.

But this is only a very simple algorithm, and it is not really common to find pictures with adjacent pixels so different from each other.

Now at least you understand the principle.

To get the 9x9 image you would need to repeat the process horizontally to create 10 new pixels (2 columns of 5 pixels).

What if we need to recreate more pixels than just one between 2 pixels?

 

In this case you would need to estimate the position of your output pixels compared to the input ones and then compute the coefficient according to this position (the closer an output pixel is to an input pixel, the more it will look like it).

The image below shows a horizontal scaling of 1.2x the input. The circles are the input pixels and the cross the output pixels:

scale.JPG

Note that only 2 pixels of the line will get their value copied from the input as the crosses and circles are overlapping only in the corners.

The same principle applies for down-scaling.

 


Example of a scaler application with the VPSS (Scaler Only)

 

The attached example shows the Video Processing Subsystem configured as a Scaler only.

The design is based on the Video Series 28: Using the VPSS IP in Color Space Converter mode with the AXI4-Lite interface enabled on the clocking wizard to support multiple output resolutions (see Video Series 22: Supporting multiple video resolutions on ZC702 HDMI for details)

Generate the Design

  1. Download the tutorial files and unzip the folder

  2. Open Vivado 2019.1

  3. In the Tcl console, cd into the unzipped directory
cd XVES_0029/hw
  1. In the Tcl console, source the script Tcl
source ./create_proj.tcl
  1. Generate the BD output products then run Synthesis and Implementation and Generate a Biststream

  2. Export the Hardware including the bitstream to XVES_0029/sw/sdk_export

  3. Start the Xilinx Software Command Line Tool (XSCT) 2019.1
  • From the windows menu, select the following:

Start > All Programs > Xilinx Design Tools > Xilinx Software Command Line Tool 2019.1

  • From the command line:

Use the command xsct (the environment variables for SDK 2019.1 needs to be set)

  1. In xsct, cd to XVES_0029/sw.

  2. Use the command source create_SW_proj.tcl

  3. Open SDK and select XVES_0029/sw/sdk_workspace as workspace

You can change the input and output resolutions by using the options from the UART console.

 


Liked this Video Series?

  • You can give Kudos using the Kudos button  kudos.JPG
  • Share it on social media using the Share button share.JPG

  • Feel free to comment on this topic or to create a new topic on the forums to ask questions

Want more Video Series?

8 Comments

hi

I have a question in the SDK project about VDMA  configure in XVES_0029.

Configure the  register of the Hsize of VDMA  should be "height*bytePerPixels" or "width*bytePerPixels"?

30ed6ea477073ca8c42cadac39a979a.png

Moderator
Moderator

Hi 994956907@qq.com 

HSIZE means "Horizontal" Size. Thus this is the width that you have horizontally. So HSIZE needs to be programmed with width*bytePerPixels. This is correct (it would not work anyway if it was incorrect)

understand thanks

hi

I have test this project in my board,and I have changed some parameters to fit my board.bdbd

 When I debug this project ,the video is perfect scaling.

normal videonormal video

 But when I chenged the parameter of input/output resolution by uart,the video have changed,the first data coming in is the first pixel is not the first pixel of a frame,the video seems to have shifted to the right.Even if I have reinitialize the application by"init_application(&app_periphs);",the video will shifted either. Can you help me deal with this problem,thanks.

shifted videoshifted video

 

 

Moderator
Moderator

HI 994956907@qq.com 

This is not a big issue. This might be happening from the Test pattern generator.

I see that you have an ILA capturing the reset signal. Make sure the reset signal is happening when you change the resolution.

hi

I did captured the reset signal of "aresetn_io_axis" from VPSS. When I change the resolution or reinitialize the application,the reset signal did happening and below is the signal.

reset signal from VPSSreset signal from VPSS

 

AND I had captured the inout video stream signal of vpss and VDMA. I set the test patten mode be red,and the active video coulde be red/blue/green randomly along with the action of resolution changing. I foud the stream data of VDMA is different with VPSS signal .

same with vpsssame with vpss

 

vdma_mm2s different with VPSSvdma_mm2s different with VPSS

 

vdma_mm2s different with VPSSvdma_mm2s different with VPSS

 

 

Moderator
Moderator

Hi 994956907@qq.com 

This might be too long to do over comments on the blog. Can you create a new topic on the Video board

Thanks

Hi

I have create a new topic on the video board.

Thanks for your support.