cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer
Observer
621 Views
Registered: ‎06-23-2010

Multiple pixel processing

Jump to solution

Hi,

I am developing a image processing core.
For Full HD video, I have used one pixel per clock for image processing core.

But for 4K/8K Video project, I have timing problems.
So I studied about mutiple pixel processing to solve the timing problems.

In some Xilinx IPs, there are about one, two, four, or eight pixel per clock processing.
As a example, in case of two pixel per clock processing, 
I guess that two pixel are processed at one clock.
but  I would like to get some more detail information or application document about the mutiple pixel per clock processing.

Thanks.

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Explorer
Explorer
603 Views
Registered: ‎07-18-2011

Re: Multiple pixel processing

Jump to solution

@k621219 

It is exactly what it sounds like - you're processing more than one pixel at a time.

It is done to trade off resources for increased bandwidth.    If you can't meet timing with one pixel per clock, you can increase to two or four pixels per clock.   While this results in increased throughput for the same clock rate, it also uses more resources, because the processing path is wider.

For example, if you were to try to process 1080p60 RGB video in its native format of 24 bits at 148.5MHz pixel rate in an Artix-7, you wouldn't be able to do it if your axi bus is running at the typical 100MHz rate.     If you instead processed it as 48 bits at the 100MHz clock (a 74.25MHz effective throughput rate for dual-pixel processing), it would meet timing, but would use more resources because the RGB path is now 48 bits wide instead of 24 bits wide.    You can further increase the throughput by processing at four pixels per clock.

Refer to UG934 for information about multiple-pixel-per clock encoding.and data-packing for various video formats.

 

View solution in original post

6 Replies
Highlighted
Explorer
Explorer
604 Views
Registered: ‎07-18-2011

Re: Multiple pixel processing

Jump to solution

@k621219 

It is exactly what it sounds like - you're processing more than one pixel at a time.

It is done to trade off resources for increased bandwidth.    If you can't meet timing with one pixel per clock, you can increase to two or four pixels per clock.   While this results in increased throughput for the same clock rate, it also uses more resources, because the processing path is wider.

For example, if you were to try to process 1080p60 RGB video in its native format of 24 bits at 148.5MHz pixel rate in an Artix-7, you wouldn't be able to do it if your axi bus is running at the typical 100MHz rate.     If you instead processed it as 48 bits at the 100MHz clock (a 74.25MHz effective throughput rate for dual-pixel processing), it would meet timing, but would use more resources because the RGB path is now 48 bits wide instead of 24 bits wide.    You can further increase the throughput by processing at four pixels per clock.

Refer to UG934 for information about multiple-pixel-per clock encoding.and data-packing for various video formats.

 

View solution in original post

Highlighted
Observer
Observer
575 Views
Registered: ‎06-23-2010

Re: Multiple pixel processing

Jump to solution

Hi Reaiken.
Thanks for your answer.

In my design, I don't use AXI4 interface so that I don't need to use multiple-pixel-per clock encoding and data-packing concept.

I am going to modify about 10 more image processing cores. the verilog coding lines of the cores  are not small.
So I think that it might not be simple to modify from one pixel per clock processing into multiple pixel per clock processing for all cores.

I need to understand more about mutiple pixel processing.
I would like to modify the existing cores with reducing development/modification time and reducing FPGA resource.

So I need to get some document/thesis about multiple pixel processsing.
Through internet search, in Vision HDL Toolbox of Matlab, I know that the multiple pixel processsing is used.
But there are no some detail information.

Also I could find  a document for multiple pixel processsing.
And it explains about it at some level.

Real-time implementation of contextual image processing operations for 4K video stream in Zynq UltraScale+ MPSoC, 
M Kowalczyk, D Przewlocka… - 2018 Conference on …, 2018 - ieeexplore.ieee.org

But I could not find more document except for these about the multiple pixel processsing.
So I would like to get more detail information and document about multiple pixel processsing.

Also If you could advice about experience in RTL coding of mutiple pixel processing, it will be helpful to me.

Thanks.

 

0 Kudos
Highlighted
Moderator
Moderator
510 Views
Registered: ‎11-09-2015

Re: Multiple pixel processing

Jump to solution

Hi @k621219 

The processing is the same if you have 1, 2, 4 or 8 pixel per clock. The only difference is that you are doing multiple computation at the same time. So you end up consuming and producing multiple pixel per clock.


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
Highlighted
Voyager
Voyager
483 Views
Registered: ‎03-28-2016

Re: Multiple pixel processing

Jump to solution

@k621219 

As stated above, processing multiple pixels per clock can be fairly simple as long as there is no dependence between pixels.  If each pixel is processed independently from other pixels then you can simply create N independent processing chains to process N pixels at the same time.

Things get more complicated if there is interdependence between pixels such as when filtering.  In that case you still implement N filters to process N pixels per clock, but the incoming data has to be properly routed to multiple filters.

The hardest to deal with is image scaling where pixels can be dropped or reused multiple times depending upon whether you are scaling up or scaling down.

I don't think that you are going to find much in the way of documentation or formal teaching materials.  

To convert your Verilog IP, you might be able to get away with a simple wrapper that handles the N-pixel inputs and outputs and then instantiates N copies of the original code.  Then again you might not.  It's all dependent upon each of the algorithms and how it's written.

Ted Booth | Tech. Lead FPGA Design Engineer | DesignLinx Solutions
https://www.designlinxhs.com
Highlighted
Observer
Observer
409 Views
Registered: ‎06-23-2010

Re: Multiple pixel processing

Jump to solution

Hi tedbooth,

Thanks for your answer. it is helpful to me.

I could understand two cases that you said.

Your answer:

"To convert your Verilog IP, you might be able to get away with a simple wrapper that handles the N-pixel inputs and outputs and then instantiates N copies of the original code.  Then again you might not.  It's all dependent upon each of the algorithms and how it's written."

---->

I could not understand this well. Could you explain again ?

Do you think that I need to modify internal codes of core (with interdependence between pixel) for multiple pixel per clock  ?
In case of pixels of no dependence, I might not need to modify internal codes of cores.
In general, cores have several functions with "interdependence between pixels" and  "no dependence between pixels".
In these cores, do I need to modify internal code of cores ?
Is there a good way to implement multiple pixel per clock processing without modification or with simpe modification of internal code ?

Thanks.

 

 

0 Kudos
Highlighted
Observer
Observer
320 Views
Registered: ‎06-23-2010

Re: Multiple pixel processing

Jump to solution

Thank for all, @reaiken , @florentw , @tedbooth 

I think that all your answer will be very helpful for my new project.

For conversion work of multiple pixel per clock processing, I think that it will take some time , based on your answers and my current study.
I would like to select all your answers as "Solution", but I am sorry that forum system could not allow it.

Thanks again, all.

0 Kudos