cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Moderator
Moderator
1,988 Views
Registered: ‎10-04-2011

Using Vitis Vision Libraries and OpenCV

Background:
The Vitis Vision Libraries are an accelerated library of OpenCV and Vision functions for implementation in the Vitis environment. These libraries provide a L1 directory which contains Vitis HLS kernels as well as example designs that highlight the kernel behavior. The example designs use OpenCV for the testbench functionality, and as such, require an existing OpenCV installation. In order to accommodate a wide variety of user environments, Xilinx no longer provides a pre-installed version of OpenCV with the Vivado/Vitis tools as of the 2020.1 release. While OpenCV is not required for Vitis implementation of the Vision libraries, it is required to run the example designs, and may be desirable to use in user testbench verification. 

This post describes the process for creating a standalone Vitis HLS TCL project file and augments the information found in the Getting Started with HLS section of the Vision documentation which is located here:
https://xilinx.github.io/Vitis_Libraries/vision/2020.1/overview.html#getting-started-with-hls


Flow:
This AR describes the process to create a HLS TCL script with runs the resize example in the L1 Vision directory in Windows which utilizes a user installed OpenCV library. 

To utilize the examples designs, or reference the OpenCV libraries in a user testbench, the following steps must be taken:
- Install the OpenCV tools version 3.x 
- Setup the environment to reference the OpenCV installation 
- Create a TCL script for Vitis HLS execution which references the OpenCV libraries

Install OpenCV:
The installation of OpenCV and setting of the environment is beyond the scope of this article. OpenCV is a open-source distribution located at the following location:
https://opencv.org/


Note: The 2020.1 Vitis Vision library was verified using version 3.3 of the OpenCV library. Any version more recent than that will work, however, version 4.x may have library function changes relative to the 3.x version, and as such may require the example design testbenches to be modified. For this reason, it is recommended that example designs be run using OpenCV version 3.x.

Important: The OpenCV libraries only provide testbench functionality, and are not required and do not affect the implementation of the Vision kernels in any way. 

Environment Setup:
The environment setup requirements for the acceleration platform makefile flow are described in the L1 Kernel github page located at the following location:
https://github.com/Xilinx/Vitis_Libraries/tree/master/vision/L1

This AR describes a process outside the acceleration platform makefile flow which will use the environment settings similar to the makefile flow process for consistency. 

Note: The LD_LIBRARY_PATH environment variable and OpenCV PATH information must be properly set in the User's environment for this script, and the Vitis Vision example designs, to work properly. In addition, the paths to the OpenCV include libraries and binaries must be included in the environment variables of the system. Failure to do so will result in library include errors during simulation.

An example Windows environment variable settings is shown below, and will vary for each user depending on the installation directory of the OpenCV and compiler tools. 

opencv_windows_environment.jpg


Vitis HLS TCL Script:

A TCL script to run the L1 resize example design outside the makefile flow is provided with this AR. This script is based on the Windows environment. The script is based on the following environment settings:
- OpenCV version 3.4.11
- OpenCV include directory : C:/Data/OpenCV/build_win64/install/include
- OpenCV library directory : C:/Data/OpenCV/build_win64/install/x64/mingw/lib
- Vitis Vision Directory : C:/Data/Vitis_Libraries/Vitis_Libraries-master/vision/

To run the Vitis HLS script, perform the following actions:
- place the script in the <path>/vision/L1/example/resize directory
- open the Vitis HLS command line shell and navigate to the <path>/vision/L1/example/resize directory
- run the following command: vitis_hls -f run_hls_standalone.tcl

The TCL script file contains the following sections

  • Variable declarations representing the OpenCV and project environment
  • Project creation commands
  • Design file addition with Vitis Vision library include paths
  • Testbench file addition with OpenCV and Vitis Vision library include paths
  • C simulation with OpenCV linker references
  • Vitis HLS IP Synthesis
  • RTL CoSimulation with OpenCV linker references
  • Exportation of IP

Variable Declarations:

The first part of the variable declaration section declares variables which replicate the environment variables of the makefile flow and the settings.tcl file that flow produces. These variables point to the Vitis Vision Includes, the OpenCV header files, and the OpenCV pre-compiled libraries. These locations may vary based on the install paths of the user's system.

set XF_PROJ_ROOT "C:/Data/Vitis_Libraries/Vitis_Libraries-master/vision"   # Vitis Vision Libary include directory
set OPENCV_INCLUDE "C:/Data/OpenCV/build_win64/install/include"            # OpenCV header file directory
set OPENCV_LIB "C:/Data/OpenCV/build_win64/install/x64/mingw/lib"          # OpenCV compiled library directory

The next variable declaration section assists in creating the Vitis HLS project and helps to make the script portable:

set PROJ_DIR "$XF_PROJ_ROOT/L1/examples/resize"
set SOURCE_DIR "$PROJ_DIR/"
set PROJ_NAME "hls_example"
set PROJ_TOP "resize_accel"
set SOLUTION_NAME "sol1"
set SOLUTION_PART "xcvu11p-flgb2104-1-e"
set SOLUTION_CLKP 5

Finally, the last section declares variables that represent the include and simulation flags required by HLS to reference and use the libraries. Using variables instead of direct code helps in understanding how these options are used. 

set VISION_INC_FLAGS "-I$XF_PROJ_ROOT/L1/include -std=c++0x"    # Vitis Vision Include path, and C++ 11 settings
set OPENCV_INC_FLAGS "-I$OPENCV_INCLUDE"                        # OpenCV include directory reference
set OPENCV_LIB_FLAGS "-L $OPENCV_LIB"                           # OpenCV library reference

Note: In Windows, the library references must include the version number. This example uses a OpenCV 3.4.11 installation. The precise include format will depend on the user's installation and may be different than listed below. 

set OPENCV_LIB_REF "-lopencv_imgcodecs3411 -lopencv_imgproc3411 -lopencv_core3411 -lopencv_highgui3411 -lopencv_flann3411 -lopencv_features2d3411"


The equivalent Linux include statement does not use the version number and is given as follows:

set OPENCV_LIB_REF "-lopencv_imgcodecs -lopencv_imgproc -lopencv_core -lopencv_highgui -lopencv_flann -lopencv_features2d"

Note: The exact library references required will depend on the OpenCV functions used in the testbench. Please reference the OpenCV documentation to determine the required libraries for inclusion.


Project Creation Part I:

The project creation section is straightforward and creates a new project directory and project file

open_project -reset $PROJ_NAME


Design File Include:

The design files are added to the design in this section. This command:

- references the individual HLS kernel file : add_files "${PROJ_DIR}/xf_resize_accel.cpp"
- references the vision libraries and project specific include directories for synthesis: -cflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build
- references the vision libraries and project specific include directories for C Simulation:  -csimflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build"
 
The complete command is shown below:
add_files "${PROJ_DIR}/xf_resize_accel.cpp" -cflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build" -csimflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build"

Testbench File Include:

The testbench files are added to the design in this section. The command:

- references the testbench file: add_files -tb "${PROJ_DIR}/xf_resize_tb.cpp"
- references the vision libraries and project specific include directories: -cflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"
- references the vision libraries and project specific include directories for C Simulation:  -csimflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"

Note the addition of the ${VISION_INC_FLAGS} variable in the testbench flags versus the design file flags. This setting references the OpenCV include files. 
 
The complete command is shown below:
add_files -tb "${PROJ_DIR}/xf_resize_tb.cpp" -cflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build" -csimflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"


Project Creation Part II:

Now that the C source files have been added, the final step of project creation is executed. These commands set the top level function of the HLS IP, and open the desired project solution. 

set_top $PROJ_TOP                        # set top level file of HLS IP
open_solution -reset $SOLUTION_NAME      # create the project solution
set_part $SOLUTION_PART                  # set the solution part
create_clock -period $SOLUTION_CLKP      # set the project target clock period


C Simulation:

This section executes the C simulation stage of the HLS flow by sending the HLS IP and Testbench designs to the compiler for compilation and execution. This command is used to set the compiler linker flags and testbench files and:

- References the OpenCV include and pre-compiled library directory : -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}"
- Includes an image for the verification testbench as an argument to the main function: -argv "${XF_PROJ_ROOT}/data/128x128.png"
 
The complete command is shown below:
csim_design -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}" -argv "${XF_PROJ_ROOT}/data/128x128.png"


C to RTL Synthesis:

This section executes the Vitis HLS C to RTL synthesis stage. No flags or options are required for this stage. 

csynth_design


RTL CoSimulation:

This section executes the RTL CoSimulation of the Vitis HLS IP after synthesis. The command is similar to the C Simulation command and is used to set the compiler linker flags and testbench files and:

- References the OpenCV include and pre-compiled library directory : -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}"
- Includes an image for the verification testbench: -argv "${XF_PROJ_ROOT}/data/128x128.png"
 
The complete command is shown below:
cosim_design -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}" -argv "${XF_PROJ_ROOT}/data/128x128.png"


Exportation: 

The final stage of the Vitis HLS flow is the exportation of the design. There are several options for this stage which are described in the HLS User's Guide. This example exports the design for RTL and runs Vivado Synthesis to obtain accurate resource utilization and estimated timing results.

export_design -flow syn -rtl verilog

6 Replies
Contributor
Contributor
1,932 Views
Registered: ‎06-10-2018

On Ubuntu 18.04 there is a bug with COSIM with respect to gmp.h.

This file location has to be added to the flags otherwise it fails.

 

0 Kudos
Reply
1,248 Views
Registered: ‎07-05-2019

I follow the above-mentioned steps to create the resize project in Vitis 2020.2 on Windows. But I am facing a few errors,

Makefile.rules:392: recipe for target 'csim.exe' failed
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_imgcodecs3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_imgproc3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_core3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_highgui3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_flann3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_features2d3411

I have attached a few screenshots for a better understanding of the issue and the steps taken..i have also attached the tcl file.

I could not find much help online regarding the error thrown so am posting here.

Please let me know if something is still missing.

image (27).png

image (28).png

image (29).png

image (30).png

image (31).png

image (32).png

0 Kudos
Reply
Moderator
Moderator
1,107 Views
Registered: ‎10-04-2011

I think what may be missing here is the LD_LIBRARY_PATH variable which should point to the compiled libraries. I show that in the environment variables image in the previous post of mine. Can you see if that resolves this then?


Thank you,
Scott

 

0 Kudos
Reply
988 Views
Registered: ‎07-05-2019

Hey Scott

Firstly, thank you for your time and support.

I added the LD_LIBRARY_PATH as you suggest. But the same issue is thrown.

From what I understand the path to these -lopencv_imgcodecs3411.. are not correct. If I try to trace the path as shown in the console...I don't see these files there. There are no files with those explicit names though I am not sure with what names is it suppose to reflected. Also, I do not see any other OpenCV file mentioned.

On the other hand when I open my opencv folder bin..I see related files that might be the path actually needed.

I have attached the screenshots below. Let me know what do you think.

image (42).png
image (41).png
image (46).png
image (45).png
image (44).png
image (43).png
0 Kudos
Reply
Observer
Observer
855 Views
Registered: ‎11-20-2020

I spent a week of time configurating OpenCV to work with Vitis HLS 2020 a few weeks back then. The process was frustrating but I finally made it with the help of this guide. 

My guess is that you have encountered some problems with your OpenCV build. When I built it with CMAKE, the directory to the .bin folder is D:\opencv-4.2.0_mingw\opencv-4.2.0\build\install\x64\mingw\bin. From your images, the install folder is missing. I guess you may have built OpenCV with customized directory, or the build process simply went wrong. On the other hand, upon successful build, the .bin folder should include around 15 .dll files, including those that you are now missing (if you did not uncheck any package during the build). The image below shows what my .bin folder looks like for OpenCV 4.2.0. I did not check or uncheck anything during the build. 

opencv.PNG

Note: I think you are using Visual Studio to build OpenCV. I did the same thing at the beginning. The build was successful but it does not work well with Vitis HLS (since they use different compilers). You may want to build it with MinGW. I followed the guild in this website to build OpenCV 4.2.0 with MinGW. It should also work with OpenCV 3.4.11.

0 Kudos
Reply
569 Views
Registered: ‎07-05-2019

 Hey Jeff 

This was really helpful. Thank you!!

I wanted to reply you long back but am guessing due to winter holidays or some reason the reply tab was inactive!

0 Kudos
Reply