cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
kiniman_2000
Observer
Observer
1,888 Views
Registered: ‎10-21-2018

OpenCV with Vivado HLS

Jump to solution

Hello,

I am almost starting with FPGA, so I guess that the next question will be very easy to be solved but I haven´t found an answer:

I know that there are OpenCV functions that have its equivalent function like cv::sobel and hls::sobel and others, so I can use them directly on my project. My doubt is if I want to synthesize any other function of OpenCV library (without equivalent function)  like the simple cv::rectangle (draw a rectangle), how can I include this library in my project? Is it already installed with Vivado? 

or do I have to download the OpenCV library and copy the files I need in my Vivado project? If this is the way, could you please assist me in it?

I am using windows 7 and Vivado HLS 2018.3.

Thank you in advance.

0 Kudos
1 Solution

Accepted Solutions
u4223374
Advisor
Advisor
1,822 Views
Registered: ‎04-26-2015

@kiniman_2000 


@kiniman_2000 wrote:

 

I knew that not all the functions can´t be synthesized, but I expected to go in depth in detectMultiScale function and try to synthesize some part of the code doing minimum changes, not forgetting that the goal is the study of the automatic tools. It means that it makes no sense if I develop complete new functions in order to be synthesized. 

Even without having a close look at the algorithm, I think this can be answered pretty easily:

How effective are the automatic tools if you take code designed for (and heavily optimized for) a modern CPU, make minimal code changes, and implement it for an FPGA? Not effective at all.

In some cases (particularly older algorithms, I find, which tend to rely less on dynamic allocation of large amounts of memory simply due to the limitations of systems at the time) you can use the same algorithm but rewrite it in a HLS-friendly way. However, this isn't "take some existing code and fix the few HLS-unfriendly parts" - it's "read the original paper to understand the algorithm, plan the dataflow for that algorithm in a way that will work with HLS, and implement that". In other cases, the algorithm is simply unsuitable for FPGA implementation - although splitting it into CPU-friendly bits (high-level processing) and FPGA-friendly bits (refining massive amounts of image data) can work well.

 

 

 

View solution in original post

0 Kudos
8 Replies
u4223374
Advisor
Advisor
1,881 Views
Registered: ‎04-26-2015

The short answer is that you can't. Very nearly everything in OpenCV relies heavily on dynamic memory allocation and random access to a framebuffer. HLS can't do dynamic memory allocation in synthesis, and HLS video processing is almost invariably done with a streamed image (no direct access to a framebuffer).

 

For something simple like drawing a rectangle, you can write an equivalent function yourself - either with an AXI Master to write to a framebuffer, or via streaming. For more complex functions, you're likely to find that the algorithm used in OpenCV is unsuitable for FPGA implementation, and a better approach is to research an alternative algorithm.

kiniman_2000
Observer
Observer
1,873 Views
Registered: ‎10-21-2018

Thank you for your reply. I understand...but I will explain my project if you can tell me what you think about it, if it´s feasible.

I have to do a study about how automatic tools (Vivado HLS or else) can be used to accelerate an algorithm for deployment in an FPGA. Specifically, I should accelerate the face detection algorithm of Viola Jones. After, I have to compare the efficiency of using just the ARM and using ARM+Hardware logic. Below a summary of the code:

face_cascade = cv.CascadeClassifier('haarcascade_frontalface_default.xml')
img = cv.imread('sachin.jpg')
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
     cv.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
I knew that not all the functions can´t be synthesized, but I expected to go in depth in detectMultiScale function and try to synthesize some part of the code doing minimum changes, not forgetting that the goal is the study of the automatic tools. It means that it makes no sense if I develop complete new functions in order to be synthesized. 
Any opinion from an experienced point of view will be much appreciated.
0 Kudos
nmoeller
Xilinx Employee
Xilinx Employee
1,844 Views
Registered: ‎09-05-2018

Hey @kiniman_2000,

The code you have provided looks like testbench code, which is not synthesized, so you should be good to use that code as is, in main(). It's just being compiled and run like C.

OpenCV is included with HLS 2018.3, so you should be able to just add #include <opencv2/opencv.hpp> to your header and be off to the rases.

For synthesis, you will define a top funciton using HLS. That will probably be detectMultiScale() or a subfunction thereof. Only the code inside of that needs to be modified to be synthesizeable.

Nicholas Moellers

Xilinx Worldwide Technical Support
0 Kudos
frederic
Xilinx Employee
Xilinx Employee
1,835 Views
Registered: ‎04-14-2013

For Vivado HLS take a look at xfOpenCV: https://github.com/Xilinx/xfopencv

Perhaps looking at SDAccel with OpenCV kernels would be better for your project.

0 Kudos
u4223374
Advisor
Advisor
1,823 Views
Registered: ‎04-26-2015

@kiniman_2000 


@kiniman_2000 wrote:

 

I knew that not all the functions can´t be synthesized, but I expected to go in depth in detectMultiScale function and try to synthesize some part of the code doing minimum changes, not forgetting that the goal is the study of the automatic tools. It means that it makes no sense if I develop complete new functions in order to be synthesized. 

Even without having a close look at the algorithm, I think this can be answered pretty easily:

How effective are the automatic tools if you take code designed for (and heavily optimized for) a modern CPU, make minimal code changes, and implement it for an FPGA? Not effective at all.

In some cases (particularly older algorithms, I find, which tend to rely less on dynamic allocation of large amounts of memory simply due to the limitations of systems at the time) you can use the same algorithm but rewrite it in a HLS-friendly way. However, this isn't "take some existing code and fix the few HLS-unfriendly parts" - it's "read the original paper to understand the algorithm, plan the dataflow for that algorithm in a way that will work with HLS, and implement that". In other cases, the algorithm is simply unsuitable for FPGA implementation - although splitting it into CPU-friendly bits (high-level processing) and FPGA-friendly bits (refining massive amounts of image data) can work well.

 

 

 

View solution in original post

0 Kudos
kiniman_2000
Observer
Observer
1,800 Views
Registered: ‎10-21-2018

I understand it...I will try to adapt the code that needs minimum changes and split the others between CPU and PL. Thank you very much for your help!

0 Kudos
kiniman_2000
Observer
Observer
1,799 Views
Registered: ‎10-21-2018
I will explore the SDAccel option too. Thank you
0 Kudos
kiniman_2000
Observer
Observer
1,799 Views
Registered: ‎10-21-2018
Thanks
0 Kudos