cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Voyager
Voyager
2,781 Views
Registered: ‎07-06-2016

Undestanding XAPP1167 sobel filter

Jump to solution

Hello,

 

I'm trying to understand how the XAPP1167 Sobel filter works in order to get a 3x3 pixel window in the next function:

 

 

void sobel_filter_core(YUV_IMAGE& src, YUV_IMAGE& dst)
{
  Y_BUFFER buff_A;
  Y_WINDOW buff_C;

  for(int row = 0; row < rows+1; row++){
    for(int col = 0; col < cols+1; col++){
#pragma HLS loop_flatten off
#pragma HLS dependence variable=&buff_A false
#pragma HLS PIPELINE II = 1

      // Temp values are used to reduce the number of memory reads
      unsigned char temp;
      YUV_PIXEL tempx;

      //Line Buffer fill
      if(col < cols){
          buff_A.shift_down(col);
          temp = buff_A.getval(0,col);
      }

      //There is an offset to accomodate the active pixel region
      //There are only MAX_WIDTH and MAX_HEIGHT valid pixels in the image
      if(col < cols && row < rows){
          YUV_PIXEL new_pix;
          src >> new_pix;
          tempx = new_pix;
          buff_A.insert_bottom(tempx.val[0],col);
      }

      //Shift the processing window to make room for the new column
      buff_C.shift_right();

      //The Sobel processing window only needs to store luminance values
      //rgb2y function computes the luminance from the color pixel
      if(col < cols){
          buff_C.insert(buff_A.getval(2,col),2,0);
          buff_C.insert(temp,1,0);
          buff_C.insert(tempx.val[0],0,0);
      }
      YUV_PIXEL edge;

      //The sobel operator only works on the inner part of the image
      //This design assumes there are no edges on the boundary of the image
      if( row <= 1 || col <= 1 || row > (rows-1) || col > (cols-1)){
          edge.val[0] = 0;
          edge.val[1] = 128;
      } else {
          //Sobel operation on the inner portion of the image
          edge = sobel_operator(&buff_C);
      }

      //The output image is offset from the input to account for the line buffer
      if(row > 0 && col > 0) {
          dst << edge;
      }
    }
  }
}

 

 

 

I'm assuming that the pixels are coming one at time from top left to bottom right and the function will be translated to a HDL process. For another side looks to me that (please correct me if I'm wrong) the function gets one column every clock cycle and saves it in a window type array, if this is right I don't understand how it can get a column if the pixels come one at time....

 

In the UG902 we have that the line array and methods work like this for shifted and inserted values:

 

Untitled.jpg

Having the next portion of code, and thinking in hardware, looks to me that every inserted value at the bottom of each column is go to be drop out or lost at the next line wen the column is shift it down.

 

 

      if(col < cols){
          buff_A.shift_down(col);
          temp = buff_A.getval(0,col);
      }

      //There is an offset to accomodate the active pixel region
      //There are only MAX_WIDTH and MAX_HEIGHT valid pixels in the image
      if(col < cols && row < rows){
          YUV_PIXEL new_pix;
          src >> new_pix;
          tempx = new_pix;
          buff_A.insert_bottom(tempx.val[0],col);
      }

 

Does anyone knows how this works?

 

Thanks. 

 

 

 

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
Highlighted
Scholar
Scholar
4,188 Views
Registered: ‎04-26-2015

Re: Undestanding XAPP1167 sobel filter

Jump to solution

I suspect that your confusion is largely due to some ... interesting ... coding in the HLS video files. Specifically:

 

template<int ROWS, int COLS, typename T> void Window<ROWS, COLS, T>::insert_bottom(T value[COLS]) {
#pragma HLS inline
    insert_top_row(value);
}

insert_bottom actually inserts data into the top row.

 

There are two separate buffers here, a three-line buffer (buff_A) and a 3x3 square buffer (buff_C). Incoming pixels get stored in buff_A so that they can be used when the program reaches the same column of the next line. At each column, the program pulls two pixels out of buff_A (at the same column) and reads one pixel from the stream, which get used to update buff_C.

 

 

From the code:

 

On each cycle, the current column in buff_A gets shifted down (ie each element in that column moves one index higher), so if the data in a column was:

 

761
982
123

then after this shift the data will be:

 

 

761 (first element gets duplicated)
761
982
(123 drops out the bottom)

It also stores the top value (ie 761) in "temp", for later use.

 

A pixel is then loaded from the image and inserted using insert_bottom (which actually inserts it at the top, the first "761" element above). This pixel is stored in "tempx" for later use.

 

Finally, one pixel is read from the bottom line of the line buffer (value 982 in this case). The program now has three pixels from that column available. It shifts buff_C across one space to make room, and then inserts these.

 

 

View solution in original post

0 Kudos
5 Replies
Highlighted
Scholar
Scholar
4,189 Views
Registered: ‎04-26-2015

Re: Undestanding XAPP1167 sobel filter

Jump to solution

I suspect that your confusion is largely due to some ... interesting ... coding in the HLS video files. Specifically:

 

template<int ROWS, int COLS, typename T> void Window<ROWS, COLS, T>::insert_bottom(T value[COLS]) {
#pragma HLS inline
    insert_top_row(value);
}

insert_bottom actually inserts data into the top row.

 

There are two separate buffers here, a three-line buffer (buff_A) and a 3x3 square buffer (buff_C). Incoming pixels get stored in buff_A so that they can be used when the program reaches the same column of the next line. At each column, the program pulls two pixels out of buff_A (at the same column) and reads one pixel from the stream, which get used to update buff_C.

 

 

From the code:

 

On each cycle, the current column in buff_A gets shifted down (ie each element in that column moves one index higher), so if the data in a column was:

 

761
982
123

then after this shift the data will be:

 

 

761 (first element gets duplicated)
761
982
(123 drops out the bottom)

It also stores the top value (ie 761) in "temp", for later use.

 

A pixel is then loaded from the image and inserted using insert_bottom (which actually inserts it at the top, the first "761" element above). This pixel is stored in "tempx" for later use.

 

Finally, one pixel is read from the bottom line of the line buffer (value 982 in this case). The program now has three pixels from that column available. It shifts buff_C across one space to make room, and then inserts these.

 

 

View solution in original post

0 Kudos
Highlighted
Voyager
Voyager
2,737 Views
Registered: ‎06-24-2013

Re: Undestanding XAPP1167 sobel filter

Jump to solution

Hey @joseer,

 

The trick here is in having two different views for the same data:

The array/matrix/image view where you can access pixels at random
The pointer/fifo/stream view where you can only pop or push one value at a time

Now obviously processing the image in a completely random fashion wouldn't be a good idea if you want to keep the amount of involved memory and delay minimal, so a stream approach looks very appealing. Luckily accessing memory in an ordered way (e.g. rows from left to right, columns from top to bottom) is identical to processing a stream.

 

So what is required for a kernel like the Sobel filter? We need access to a small rectangular area of the image to calculate the result and we will move that window over the source image in a stream like fashion from left to right and from top to bottom. With a little buffering, this can be done in a way which is perfectly compatible with a stream interface. Care must be taken not to access any data out of order or more than once, because that would not be possible in a stream.

 

So how does it work in this example? There are three levels of buffering involved. The first level is the line buffer, which basically keeps a history of N lines required for the kernel to operate on. The second level is the window buffer, which holds the matrix where the kernel is applied. And finally the third buffer level are the temp variables to hold new and old input data during buffer updates.

 

The tools will use the described data dependencies to pipeline the steps as optimally as possible, generating a complete streaming solution with the latency required to input the necessary lines and pixels to apply the kernel and output values at the same rate they enter.

 

Hope this clarifies,

Herbert

-------------- Yes, I do this for fun!
Highlighted
Voyager
Voyager
2,723 Views
Registered: ‎07-06-2016

Re: Undestanding XAPP1167 sobel filter

Jump to solution

@hpoetzl@u4223374 Thanks for the replies!, 

That makes more sense now regarding data buffering. 

0 Kudos
Highlighted
Voyager
Voyager
2,717 Views
Registered: ‎06-24-2013

Re: Undestanding XAPP1167 sobel filter

Jump to solution

You're welcome!

 

All the best,

Herbert

-------------- Yes, I do this for fun!
0 Kudos
Highlighted
453 Views
Registered: ‎04-25-2019

Re: Undestanding XAPP1167 sobel filter

Jump to solution

i couldnt understand how this code converts rgb values to grayscale?

0 Kudos