UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Contributor
Contributor
1,761 Views
Registered: ‎01-22-2018

Traverse an image in 3x3 windows in vivado HLS

I have an image of size 640x480. I want to access it window by window such that there are no overlapping pixels (i.e. no running window).

xilinx.jpg

As shown above, I want to store window_1 in an HLS_window and copy it to the destination, and in next iteration, window_2 should be stored in HLS_window and copied to the destination. 

 

I have implemented it using the method given in Xapp1167 (Sobel). But that was running (overlapping) window. How to tackle this problem?

0 Kudos
12 Replies
Observer caccolillo
Observer
1,753 Views
Registered: ‎09-09-2010

Re: Traverse an image in 3x3 windows in vivado HLS

You could add an if branch in which you process the window only when is the right one. For example, reasoning in terms of windows, you consider only one and skip the next two windows and so on. Using a counter managed properly, this could do the trick.

 

0 Kudos
Scholar u4223374
Scholar
1,734 Views
Registered: ‎04-26-2015

Re: Traverse an image in 3x3 windows in vivado HLS

How many cycles will it spend on each window?

 

If you're aiming for one cycle per window, that's going to be tricky - you have to read nine pixels per cycle. If you're aiming for 9+ cycles per window it's pretty easy to do with a couple of line buffers.

0 Kudos
Contributor
Contributor
1,713 Views
Registered: ‎01-22-2018

Re: Traverse an image in 3x3 windows in vivado HLS

9+ cycles per window will do just fine. I am trying line buffer and HLS window.

0 Kudos
1,666 Views
Registered: ‎10-17-2017

Re: Traverse an image in 3x3 windows in vivado HLS

@anuragkush As @caccolillo mentioned, an if branch with an appropriate counter should do the trick for you. Just keep sliding the window in overlapping way but do computations only when you have the desired window. 

0 Kudos
Contributor
Contributor
1,647 Views
Registered: ‎01-22-2018

Re: Traverse an image in 3x3 windows in vivado HLS

Should I fill line buffer first and then window? Can you guide me with steps?
0 Kudos
1,639 Views
Registered: ‎10-17-2017

Re: Traverse an image in 3x3 windows in vivado HLS

@anuragkush - can you share your code here? 

0 Kudos
Contributor
Contributor
1,619 Views
Registered: ‎01-22-2018

Re: Traverse an image in 3x3 windows in vivado HLS

I am posting here as well as attaching it to message.

This modified Sobel xapp1167 code to just copy the image via an overlapping window. 

#include "window_top.h"


MY_PIXEL find_conn(Y_WINDOW *window)
{

MY_PIXEL pixel;

char i;
char j;

for(i=0; i < 3; i++){
for(j = 0; j < 3; j++){

pixel.val[0]=window->getval(i,j);

}
}

return pixel;
}

 

 


void created_window(MY_IMAGE& src, MY_IMAGE& dst, int rows, int cols)
{
Y_BUFFER buff_A;
Y_WINDOW buff_C;

for(int row = 0; row < rows+1; row++){
for(int col = 0; col < cols+1; col++){
#pragma HLS loop_flatten off
#pragma HLS dependence variable=&buff_A false
#pragma HLS PIPELINE II = 1

// Temp values are used to reduce the number of memory reads
unsigned char temp;
MY_PIXEL tempx;

//Line Buffer fill
if(col < cols){
buff_A.shift_down(col);
temp = buff_A.getval(0,col);
}

//There is an offset to accomodate the active pixel region
//There are only MAX_WIDTH and MAX_HEIGHT valid pixels in the image
if(col < cols && row < rows){
MY_PIXEL new_pix;
src >> new_pix;
tempx = new_pix;
buff_A.insert_bottom(tempx.val[0],col);
}

//Shift the processing window to make room for the new column
buff_C.shift_right();

//The Sobel processing window only needs to store luminance values
//rgb2y function computes the luminance from the color pixel
if(col < cols){
buff_C.insert(buff_A.getval(2,col),2,0);
buff_C.insert(temp,1,0);
buff_C.insert(tempx.val[0],0,0);
}
MY_PIXEL conn_obj;

//The sobel operator only works on the inner part of the image
//This design assumes there are no edges on the boundary of the image
if( row <= 1 || col <= 1 || row > (rows-1) || col > (cols-1)){
conn_obj.val[0] = 0;

} else {
//Sobel operation on the inner portion of the image
conn_obj = find_conn(&buff_C);
}

//The output image is offset from the input to account for the line buffer
if(row > 0 && col > 0) {
dst << conn_obj;
}
}
}
}

 

 


void read_image(AXI_STREAM& video_in, AXI_STREAM& video_out, int rows, int cols)
{
//Create AXI streaming interfaces for the core
#pragma HLS INTERFACE axis port=video_in bundle=INPUT_STREAM
#pragma HLS INTERFACE axis port=video_out bundle=OUTPUT_STREAM

#pragma HLS INTERFACE s_axilite port=rows bundle=CONTROL_BUS offset=0x14
#pragma HLS INTERFACE s_axilite port=cols bundle=CONTROL_BUS offset=0x1C
#pragma HLS INTERFACE s_axilite port=return bundle=CONTROL_BUS

#pragma HLS INTERFACE ap_stable port=rows
#pragma HLS INTERFACE ap_stable port=cols


MY_IMAGE img_0(rows, cols);
MY_IMAGE img_1(rows, cols);
#pragma HLS dataflow
hls::AXIvideo2Mat(video_in, img_0);
created_window(img_0, img_1, rows, cols);
hls::Mat2AXIvideo(img_1, video_out);
}

 

0 Kudos
Contributor
Contributor
1,618 Views
Registered: ‎01-22-2018

Re: Traverse an image in 3x3 windows in vivado HLS

@u4223374 @khareashish_trimble  Please suggest a solution. I am open to a fresh code.

0 Kudos
Scholar u4223374
Scholar
1,607 Views
Registered: ‎04-26-2015

Re: Traverse an image in 3x3 windows in vivado HLS

@anuragkush

 

Try this code.

 

#include <ap_int.h>
#include <hls_stream.h>
#include <stdbool.h>

typedef ap_uint<8> pixel_t;

#define IMAGE_HEIGHT 1000
#define IMAGE_WIDTH 1000
#define BLOCK_SIZE 3
#define IMAGE_WIDTH_BLOCKS (IMAGE_WIDTH/BLOCK_SIZE)
#define IMAGE_HEIGHT_BLOCKS (IMAGE_HEIGHT/BLOCK_SIZE)

void readNextRow(pixel_t * dataIn, pixel_t buffer[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE], ap_uint<16> row, bool enable);
void processRow(pixel_t buffer[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE], hls::stream<pixel_t> & dataOut, bool enable);


void top(pixel_t * dataIn, hls::stream<pixel_t> & dataOut) {

#pragma HLS INTERFACE m_axi port=dataIn offset=slave
#pragma HLS INTERFACE s_axilite port=return
#pragma HLS INTERFACE axis port=dataOut

	pixel_t bufferA[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE];
	pixel_t bufferB[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE];
#pragma HLS ARRAY_PARTITION variable=bufferA dim=3 complete
#pragma HLS ARRAY_PARTITION variable=bufferA dim=2 complete
#pragma HLS ARRAY_PARTITION variable=bufferB dim=3 complete
#pragma HLS ARRAY_PARTITION variable=bufferB dim=2 complete



	for (ap_uint<16> by = 0; by < IMAGE_HEIGHT_BLOCKS; by ++) {
		if (by[0]) {
			readNextRow(dataIn,bufferA,by*BLOCK_SIZE,by != (IMAGE_HEIGHT_BLOCKS-1));
			processRow(bufferB,dataOut,by != 0);
		} else {
			readNextRow(dataIn,bufferB,by*BLOCK_SIZE,by != (IMAGE_HEIGHT_BLOCKS-1));
			processRow(bufferA,dataOut,by != 0);
		}
	}
}

void readNextRow(pixel_t * dataIn, pixel_t buffer[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE], ap_uint<16> row, bool enable) {
	if (!enable) {
		return;
	}
	for (int y = 0; y < BLOCK_SIZE; y++) {
		for (int x = 0; x < IMAGE_WIDTH_BLOCKS*BLOCK_SIZE; x++) {
	#pragma HLS PIPELINE II=1
			pixel_t p = dataIn[(row + y)*IMAGE_WIDTH + x];
			buffer[x/BLOCK_SIZE][x % BLOCK_SIZE][y] = p;
		}
	}
}

void processRow(pixel_t buffer[IMAGE_WIDTH_BLOCKS][BLOCK_SIZE][BLOCK_SIZE], hls::stream<pixel_t> & dataOut, bool enable) {
	if (!enable) {
		return;
	}
	ap_uint<12> sum = 0;
	for (int bx = 0; bx < IMAGE_WIDTH_BLOCKS; bx++) {
#pragma HLS PIPELINE II=1
		for (int x = 0; x < BLOCK_SIZE; x++) {
			for (int y = 0; y < BLOCK_SIZE; y++) {
				sum += buffer[bx][x][y];
			}
		}
		dataOut.write(sum/9);
	}
}

I haven't actually run it, but it synthesizes fine. It's also not really optimized - there's definitely space to improve in either performance or resource usage.

 

The readNextRow function takes about 3000 cycles to run (reading three rows of the image, or one row of blocks). Currently processRow finishes in 337 cycles because it's going through one block per cycle, but that could be extended.

944 Views
Registered: ‎10-17-2017

Re: Traverse an image in 3x3 windows in vivado HLS

@anuragkush - I am assuming that your output image is same size as your input image. 

 

And that you want to

 

- run find_conn on your window and write output to dst  only when its in the desired location 

- otherwise you probably want to write a zero ? 

 

 

With these assumptions I would suggest these modifications : 

 

 

if( (row <= 1 || col <= 1 || row > (rows-1) || col > (cols-1) ) && (iColCount < 2) )
{
	conn_obj.val[0] = 0;
				
	iColCount++; 

} 
else 
{
	//Sobel operation on the inner portion of the image
	conn_obj = find_conn(&buff_C);
	iColCount = 0 ; 
				
}

you must define int iColCount = 0  at the top of your row loop. 

 

You might need to do a similar trick if you want to skip rows as well. Also, this code might need some testing, especially for the boundary of the image. 

0 Kudos
Contributor
Contributor
887 Views
Registered: ‎01-22-2018

Re: Traverse an image in 3x3 windows in vivado HLS

Hi, @u4223374 Sorry for the late reply. I was busy with some other project. I found your code very useful but I am having trouble sending an image via test file. Can you please give test file?

0 Kudos
Scholar u4223374
Scholar
878 Views
Registered: ‎04-26-2015

Re: Traverse an image in 3x3 windows in vivado HLS

@anuragkush As I mentioned in the previous post, I didn't test that code (and therefore did not write a testbench for it). However, getting an array of ap_uint<8> from an image should not be hard; the simplest option is to just start with a CV::Mat (eg. from imread) and memcpy the data across to your new array.

0 Kudos