UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Contributor
Contributor
506 Views
Registered: ‎02-24-2019

Simple HLS Image Cropping but no hdmi output

Jump to solution

 

The program sf_c3lr_2_c3d will produce hdmi output  only when  I uncomment //#define USE_DUMMY_OUTPUT

i.e.

//#define USE_DUMMY_OUTPUT       //no hdmi output (bisecting and enlarge frames from VDMA)

#define USE_DUMMY_OUTPUT        //have hdmi output( dump frames from VDMA)

Both ways are using the same program "dummy_cp"

I doubt whether I accidentally corrupt the VDMA image during bisecting process. Please advise.

 

Eli

 

 

 

top.cpp

#include "top.h"

//#define USE_DUMMY_OUTPUT

template <typename T_in, typename T_out, typename Pixel>
void dummy_cp( T_in& img_in, T_out& img_out, int rows, int cols, int new_cols) {

Pixel pixel_out;
Pixel pixel_in;

L_row: for(int row = 0; row < rows; row++) {
    L_col: for(int col = 0; col <cols; col++) {
    #pragma HLS pipeline
    img_in >> pixel_in;
    if (col>=new_cols)
         continue;
    pixel_out = pixel_in;
    img_out << pixel_out;

    }  // L_col
    }   // L_row

}

void sf_c3lr_2_c3d(AXI_STREAM& src_axi, AXI_STREAM& dst_axi)
{
#pragma HLS INTERFACE axis port=src_axi
#pragma HLS INTERFACE axis port=dst_axi

    RGB_SRC_IMAGE  src_rgb(src_rows, src_cols);
    GRAY_SRC_IMAGE  src_gray(src_rows, src_cols);
    GRAY_SRC_IMAGE  src1_gray(src_rows, src_cols);

#pragma HLS dataflow

    RGB_DST_IMAGE  dst_rgb(dst_rows, dst_cols);
    GRAY_SS_IMAGE   ss_l_gray(ss_rows, ss_cols);
    RGB_SS_IMAGE   ss_rgb(ss_rows, ss_cols);

    //hls::Sobel<1,0,3>(src_gray, src_gray1);
    hls::AXIvideo2Mat(src_axi, src_rgb);
    hls::CvtColor<HLS_RGB2GRAY>(src_rgb, src_gray);
    #ifndef USE_DUMMY_OUTPUT
        dummy_cp<GRAY_SRC_IMAGE, GRAY_SS_IMAGE, hls::Scalar<1, unsigned char> >(src_gray, ss_l_gray, src_rows, src_cols, src_cols/2);
        hls::Resize(ss_l_gray, src1_gray, HLS_INTER_LINEAR);
    #else
        dummy_cp<GRAY_SRC_IMAGE, GRAY_SRC_IMAGE, hls::Scalar<1, unsigned char> >(src_gray, src1_gray, src_rows, src_cols, src_cols);

    #endif

    hls::CvtColor<HLS_GRAY2RGB>(src1_gray, dst_rgb);
    hls::Mat2AXIvideo(dst_rgb, dst_axi);

}

Screenshot from 2019-05-10 21-42-38.png

top.h

typedef hls::stream<ap_axiu<24,1,1,1> >  AXI_STREAM;

typedef hls::Mat<SRC_HEIGHT, SRC_WIDTH, HLS_8UC1>   GRAY_SRC_IMAGE;
typedef hls::Mat<SRC_HEIGHT, SRC_WIDTH, HLS_8UC3>   RGB_SRC_IMAGE;

typedef hls::Mat<SS_HEIGHT, SS_WIDTH, HLS_8UC1>   GRAY_SS_IMAGE;
typedef hls::Mat<SS_HEIGHT, SS_WIDTH, HLS_8UC3>   RGB_SS_IMAGE;

typedef hls::Mat<WK_HEIGHT, WK_WIDTH, HLS_8UC1>     GRAY_WK_IMAGE;
typedef hls::Mat<WK_HEIGHT, WK_WIDTH, HLS_16SC1>     GRAY16_WK_IMAGE;
typedef hls::Mat<WK_HEIGHT, WK_WIDTH, HLS_8UC3>     RGB_WK_IMAGE;

typedef hls::Mat<DST_HEIGHT, DST_WIDTH, HLS_8UC1>   GRAY_DST_IMAGE;
typedef hls::Mat<DST_HEIGHT, DST_WIDTH, HLS_8UC3>   RGB_DST_IMAGE;

void sf_c3lr_2_c3d(AXI_STREAM& src_axi, AXI_STREAM& dst_axi);  

 

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Contributor
Contributor
402 Views
Registered: ‎02-24-2019

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Florent,

By strictly following your examples, my problem on cropping is solved.

Many Many thanks.

However, even though I can do image_splitting now but I still cannot do image_stitching in a dataflow enviroment.

1) image_splitting (one input -> multiple output)

func( hls::Mat& img_in, hls::Mat& img1_out, hls::Mat& img2_out)

solved: by splitting one function into 2 functions

2) image_stitching (multiple inputs -> one output)

func( hls::Mat& img1_in, hls::Mat& img2_in, hls::Mat& img_out) 

I can't do as simple as stitching two images by vertical stacking.

Any advices?

Eli

 

 

Many thanks.

Eli

 

0 Kudos
9 Replies
Moderator
Moderator
460 Views
Registered: ‎11-09-2015

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

HI @eewse ,

My guess it that you are not reconfiguring the Video Timing Controller and the output video clock when you are cropping the picture. This is why the HDMI output is invalid (the video timing and the clock do not match the number of pixels you are outputting).

You might want to have a look at my Video Series 22: Supporting multiple video resolutions on ZC702 HDMI , it might help.

 


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Contributor
Contributor
448 Views
Registered: ‎02-24-2019

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

HI Florent,

I followed your advise and modified the program so that the input and output mat size are the same. Still the control code works but the code with crop and merge does not produce any output. Because the Mat file size is only 640x480, time slack is not a problem.  Attempts have been made to cosim the program but no luck.  It takes forever to do the simulation. I wonder whether there is any special difficulties on doing cropping. If not, why hls does not provide us with a function to do croppping. 

Eli

#include "top.h"
#define NO_PREFILTER

#define DEBUG_001

template <typename T_in, typename T_out, typename Pixel>
void merge_tb( T_in& img_t_in, T_in& img_b_in, T_out& img_out, int rows, int cols) {

int new_cols=cols/2;
Pixel pixel_out;
Pixel pixel_t_in;
Pixel pixel_b_in;
int half_rows=rows/2;
#pragma HLS dataflow
L_row: for(int row = 0; row < rows; row++) {
    L_col: for(int col = 0; col <cols; col++) {
#pragma HLS LOOP_FLATTEN OFF
#pragma HLS DEPENDENCE array inter false
#pragma HLS PIPELINE

        if (row < half_rows) {
        img_t_in >> pixel_t_in;
        img_out << pixel_t_in;
    }
    else {
        img_b_in >> pixel_b_in;
        img_out << pixel_b_in;
    }
  }  // L_col
  }   // L_row

}


template <typename T_in, typename T_out, typename Pixel>
void split_lr( T_in& img_in, T_out& img_l, T_out& img_r, int rows, int cols) {
int new_cols=cols/2;
Pixel pixel_out;
Pixel pixel_in;
#pragma HLS dataflow
sL_row: for(int row = 0; row < rows; row++) {
    sL_col: for(int col = 0; col <cols; col++) {
#pragma HLS LOOP_FLATTEN OFF
#pragma HLS DEPENDENCE array inter false
#pragma HLS PIPELINE
    img_in >> pixel_in;
    pixel_out = pixel_in;
    if (col < new_cols)
        img_l << pixel_out;
    else
        img_r << pixel_out;
  }  // sL_col
  }   // sL_row

}

void sf_c3lr_to_c1tb(AXI_STREAM& src_axi, AXI_STREAM& dst_axi)
{
#pragma HLS INTERFACE axis port=src_axi
#pragma HLS INTERFACE axis port=dst_l_axi
#pragma HLS INTERFACE axis port=dst_r_axi

    RGB_SRC_IMAGE  src_rgb(src_rows, src_cols);
    RGB_SRC_IMAGE  src2_rgb(src_rows, src_cols);
    GRAY_SRC_IMAGE  src_gray(src_rows, src_cols);
    GRAY_SRC_IMAGE  src2_gray(src_rows, src_cols);
    GRAY_SRC_IMAGE  src_l_gray(src_rows, src_cols);
    GRAY_SRC_IMAGE  src_r_gray(src_rows, src_cols);

    GRAY_WK_IMAGE  wk_l_gray(wk_rows, wk_cols);
    GRAY_WK_IMAGE  wk_r_gray(wk_rows, wk_cols);

    RGB_DST_IMAGE  dst_l_rgb(dst_rows, dst_cols);
    RGB_DST_IMAGE  dst_r_rgb(dst_rows, dst_cols);

    GRAY_DST_IMAGE  dst_gray(dst_rows, dst_cols);

    GRAY_SS_IMAGE   ss_l_gray(ss_rows, ss_cols);
    GRAY_SS_IMAGE   ss_r_gray(ss_rows, ss_cols);

#pragma HLS dataflow
#ifndef DEBUG_001
    hls::AXIvideo2Mat(src_axi, src_rgb);
    hls::CvtColor<HLS_RGB2GRAY>(src_rgb, src_gray);
    split_lr<GRAY_SRC_IMAGE, GRAY_SS_IMAGE, GRAY_PIXEL>(src_gray, ss_l_gray, ss_r_gray, src_rows, src_cols);
    hls::Resize(ss_l_gray, wk_l_gray, HLS_INTER_LINEAR);
    hls::Resize(ss_r_gray, wk_r_gray, HLS_INTER_LINEAR);
    merge_tb<GRAY_WK_IMAGE, GRAY_DST_IMAGE, GRAY_PIXEL>(wk_l_gray, wk_r_gray, dst_gray, dst_rows, dst_cols);
    hls::CvtColor<HLS_GRAY2RGB>(dst_gray, dst_rgb);
    hls::Mat2AXIvideo(dst_rgb, dst_axi);
#else
    hls::AXIvideo2Mat(src_axi, src_rgb);
    hls::CvtColor<HLS_RGB2GRAY>(src_rgb, src_gray);
    hls::Sobel<1,0,3>(src_gray, src_l_gray);
    hls::CvtColor<HLS_GRAY2RGB>(src_l_gray, dst_rgb);
    hls::Mat2AXIvideo(dst_rgb, dst_axi);
#endif
}
}

 

0 Kudos
Moderator
Moderator
446 Views
Registered: ‎11-09-2015

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Hi @eewse ,

You might want to have a look at my Video Beginner Series 17: Create a Video Crop IP using HLS (part 1) and Video Beginner Series 18: Create a Video Crop IP using HLS (part 2).

While I have never integrated them in HW, the simulation in vivado looked correct ;)


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Contributor
Contributor
435 Views
Registered: ‎02-24-2019

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Florent,

Thank you for your reply.

Actually, I have read these two tutorials when I wrote the above programs. Maybe I should separate the cropping progam out and make it an independent ip.(will it be better? I doubt) Maybe I should work directly on axi stream level rather than on hls:Mat level when doing the cropping.(will it be faster and smoother? I need to handle TLAST and TUSER on my own). Maybe there are dataflow scheduling problem. As I can't make the HLS program complete the cosim. I don't know how to pursue.

As I am very new to HLS, I honestly don't know how to start with.

 

0 Kudos
Moderator
Moderator
422 Views
Registered: ‎11-09-2015

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Hi @eewse ,

I am not an HLS expert so I will not be able to answer your question. The only thing I know is that the code I wrote seems to be working well. Thus if you want to do a corpping function this is were you need to start.

Try to integrate it into your vivado project. If you can make it work, then you can modify the code for your own improvements


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Moderator
Moderator
416 Views
Registered: ‎05-31-2017

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Hi @eewse ,

As you are facing issues while running CoSim, please check the suggestions mentioned at page 186 of UG 902 under the topic "Debugging C/RTL Cosimulation" and see if it helps.

0 Kudos
Highlighted
Contributor
Contributor
403 Views
Registered: ‎02-24-2019

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Florent,

By strictly following your examples, my problem on cropping is solved.

Many Many thanks.

However, even though I can do image_splitting now but I still cannot do image_stitching in a dataflow enviroment.

1) image_splitting (one input -> multiple output)

func( hls::Mat& img_in, hls::Mat& img1_out, hls::Mat& img2_out)

solved: by splitting one function into 2 functions

2) image_stitching (multiple inputs -> one output)

func( hls::Mat& img1_in, hls::Mat& img2_in, hls::Mat& img_out) 

I can't do as simple as stitching two images by vertical stacking.

Any advices?

Eli

 

 

Many thanks.

Eli

 

0 Kudos
Contributor
Contributor
381 Views
Registered: ‎02-24-2019

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

I break down the programs into four blocks. However, irrespective of whatever I could do, I could not avoid to route and combine  two axi4-streams to ouput one streams. In dataflow environment, is it impossible to handle "two inputs and one output"?

Screenshot from 2019-05-17 19-24-35.png

0 Kudos
Moderator
Moderator
350 Views
Registered: ‎11-09-2015

Re: Simple HLS Image Cropping but no hdmi output

Jump to solution

Hi @eewse ,

Sorry about the delay. Were you able to make any progress on you streams merging?

I have no experience in this but I would say this is possible. But you need to make sure there are data on both streams else you might stall everything.


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos