Background:
The Vitis Vision Libraries are an accelerated library of OpenCV and Vision functions for implementation in the Vitis environment. These libraries provide a L1 directory which contains Vitis HLS kernels as well as example designs that highlight the kernel behavior. The example designs use OpenCV for the testbench functionality, and as such, require an existing OpenCV installation. In order to accommodate a wide variety of user environments, Xilinx no longer provides a pre-installed version of OpenCV with the Vivado/Vitis tools as of the 2020.1 release. While OpenCV is not required for Vitis implementation of the Vision libraries, it is required to run the example designs, and may be desirable to use in user testbench verification.
This post describes the process for creating a standalone Vitis HLS TCL project file and augments the information found in the Getting Started with HLS section of the Vision documentation which is located here:
https://xilinx.github.io/Vitis_Libraries/vision/2020.1/overview.html#getting-started-with-hls
Flow:
This AR describes the process to create a HLS TCL script with runs the resize example in the L1 Vision directory in Windows which utilizes a user installed OpenCV library.
To utilize the examples designs, or reference the OpenCV libraries in a user testbench, the following steps must be taken:
- Install the OpenCV tools version 3.x
- Setup the environment to reference the OpenCV installation
- Create a TCL script for Vitis HLS execution which references the OpenCV libraries
Install OpenCV:
The installation of OpenCV and setting of the environment is beyond the scope of this article. OpenCV is a open-source distribution located at the following location:
https://opencv.org/
Note: The 2020.1 Vitis Vision library was verified using version 3.3 of the OpenCV library. Any version more recent than that will work, however, version 4.x may have library function changes relative to the 3.x version, and as such may require the example design testbenches to be modified. For this reason, it is recommended that example designs be run using OpenCV version 3.x.
Important: The OpenCV libraries only provide testbench functionality, and are not required and do not affect the implementation of the Vision kernels in any way.
Environment Setup:
The environment setup requirements for the acceleration platform makefile flow are described in the L1 Kernel github page located at the following location:
https://github.com/Xilinx/Vitis_Libraries/tree/master/vision/L1
This AR describes a process outside the acceleration platform makefile flow which will use the environment settings similar to the makefile flow process for consistency.
Note: The LD_LIBRARY_PATH environment variable and OpenCV PATH information must be properly set in the User's environment for this script, and the Vitis Vision example designs, to work properly. In addition, the paths to the OpenCV include libraries and binaries must be included in the environment variables of the system. Failure to do so will result in library include errors during simulation.
An example Windows environment variable settings is shown below, and will vary for each user depending on the installation directory of the OpenCV and compiler tools.
Vitis HLS TCL Script:
A TCL script to run the L1 resize example design outside the makefile flow is provided with this AR. This script is based on the Windows environment. The script is based on the following environment settings:
- OpenCV version 3.4.11
- OpenCV include directory : C:/Data/OpenCV/build_win64/install/include
- OpenCV library directory : C:/Data/OpenCV/build_win64/install/x64/mingw/lib
- Vitis Vision Directory : C:/Data/Vitis_Libraries/Vitis_Libraries-master/vision/
To run the Vitis HLS script, perform the following actions:
- place the script in the <path>/vision/L1/example/resize directory
- open the Vitis HLS command line shell and navigate to the <path>/vision/L1/example/resize directory
- run the following command: vitis_hls -f run_hls_standalone.tcl
The TCL script file contains the following sections
- Variable declarations representing the OpenCV and project environment
- Project creation commands
- Design file addition with Vitis Vision library include paths
- Testbench file addition with OpenCV and Vitis Vision library include paths
- C simulation with OpenCV linker references
- Vitis HLS IP Synthesis
- RTL CoSimulation with OpenCV linker references
- Exportation of IP
Variable Declarations:
The first part of the variable declaration section declares variables which replicate the environment variables of the makefile flow and the settings.tcl file that flow produces. These variables point to the Vitis Vision Includes, the OpenCV header files, and the OpenCV pre-compiled libraries. These locations may vary based on the install paths of the user's system.
set XF_PROJ_ROOT "C:/Data/Vitis_Libraries/Vitis_Libraries-master/vision" # Vitis Vision Libary include directoryset OPENCV_INCLUDE "C:/Data/OpenCV/build_win64/install/include" # OpenCV header file directoryset OPENCV_LIB "C:/Data/OpenCV/build_win64/install/x64/mingw/lib" # OpenCV compiled library directoryThe next variable declaration section assists in creating the Vitis HLS project and helps to make the script portable:
set PROJ_DIR "$XF_PROJ_ROOT/L1/examples/resize"
set SOURCE_DIR "$PROJ_DIR/"
set PROJ_NAME "hls_example"
set PROJ_TOP "resize_accel"
set SOLUTION_NAME "sol1"
set SOLUTION_PART "xcvu11p-flgb2104-1-e"
set SOLUTION_CLKP 5
Finally, the last section declares variables that represent the include and simulation flags required by HLS to reference and use the libraries. Using variables instead of direct code helps in understanding how these options are used.
set VISION_INC_FLAGS "-I$XF_PROJ_ROOT/L1/include -std=c++0x" # Vitis Vision Include path, and C++ 11 settingsset OPENCV_INC_FLAGS "-I$OPENCV_INCLUDE" # OpenCV include directory referenceset OPENCV_LIB_FLAGS "-L $OPENCV_LIB" # OpenCV library reference
Note: In Windows, the library references must include the version number. This example uses a OpenCV 3.4.11 installation. The precise include format will depend on the user's installation and may be different than listed below.
set OPENCV_LIB_REF "-lopencv_imgcodecs3411 -lopencv_imgproc3411 -lopencv_core3411 -lopencv_highgui3411 -lopencv_flann3411 -lopencv_features2d3411"The equivalent Linux include statement does not use the version number and is given as follows:
set OPENCV_LIB_REF "-lopencv_imgcodecs -lopencv_imgproc -lopencv_core -lopencv_highgui -lopencv_flann -lopencv_features2d"Note: The exact library references required will depend on the OpenCV functions used in the testbench. Please reference the OpenCV documentation to determine the required libraries for inclusion.
Project Creation Part I:
The project creation section is straightforward and creates a new project directory and project file
open_project -reset $PROJ_NAMEDesign File Include:
The design files are added to the design in this section. This command:
- references the individual HLS kernel file :
add_files "${PROJ_DIR}/xf_resize_accel.cpp"- references the vision libraries and project specific include directories for synthesis:
-cflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build- references the vision libraries and project specific include directories for C Simulation:
-csimflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build"The complete command is shown below:
Testbench File Include:
The testbench files are added to the design in this section. The command:
- references the testbench file:
- references the vision libraries and project specific include directories:
- references the vision libraries and project specific include directories for C Simulation:
Note the addition of the
add_files "${PROJ_DIR}/xf_resize_accel.cpp" -cflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build" -csimflags "${VISION_INC_FLAGS} -I${PROJ_DIR}/build"Testbench File Include:
The testbench files are added to the design in this section. The command:
- references the testbench file:
add_files -tb "${PROJ_DIR}/xf_resize_tb.cpp"- references the vision libraries and project specific include directories:
-cflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"- references the vision libraries and project specific include directories for C Simulation:
-csimflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"Note the addition of the
${VISION_INC_FLAGS} variable in the testbench flags versus the design file flags. This setting references the OpenCV include files. The complete command is shown below:
Project Creation Part II:
Now that the C source files have been added, the final step of project creation is executed. These commands set the top level function of the HLS IP, and open the desired project solution.
C Simulation:
This section executes the C simulation stage of the HLS flow by sending the HLS IP and Testbench designs to the compiler for compilation and execution. This command is used to set the compiler linker flags and testbench files and:
- References the OpenCV include and pre-compiled library directory :
- Includes an image for the verification testbench as an argument to the main function:
add_files -tb "${PROJ_DIR}/xf_resize_tb.cpp" -cflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build" -csimflags "${OPENCV_INC_FLAGS} ${VISION_INC_FLAGS} -I${PROJ_DIR}/build"Project Creation Part II:
Now that the C source files have been added, the final step of project creation is executed. These commands set the top level function of the HLS IP, and open the desired project solution.
set_top $PROJ_TOP # set top level file of HLS IPopen_solution -reset $SOLUTION_NAME # create the project solutionset_part $SOLUTION_PART # set the solution partcreate_clock -period $SOLUTION_CLKP # set the project target clock periodC Simulation:
This section executes the C simulation stage of the HLS flow by sending the HLS IP and Testbench designs to the compiler for compilation and execution. This command is used to set the compiler linker flags and testbench files and:
- References the OpenCV include and pre-compiled library directory :
-ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}"- Includes an image for the verification testbench as an argument to the main function:
-argv "${XF_PROJ_ROOT}/data/128x128.png"The complete command is shown below:
C to RTL Synthesis:
This section executes the Vitis HLS C to RTL synthesis stage. No flags or options are required for this stage.
RTL CoSimulation:
This section executes the RTL CoSimulation of the Vitis HLS IP after synthesis. The command is similar to the C Simulation command and is used to set the compiler linker flags and testbench files and:
- References the OpenCV include and pre-compiled library directory :
- Includes an image for the verification testbench:
csim_design -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}" -argv "${XF_PROJ_ROOT}/data/128x128.png"C to RTL Synthesis:
This section executes the Vitis HLS C to RTL synthesis stage. No flags or options are required for this stage.
csynth_designRTL CoSimulation:
This section executes the RTL CoSimulation of the Vitis HLS IP after synthesis. The command is similar to the C Simulation command and is used to set the compiler linker flags and testbench files and:
- References the OpenCV include and pre-compiled library directory :
-ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}"- Includes an image for the verification testbench:
-argv "${XF_PROJ_ROOT}/data/128x128.png"The complete command is shown below:
cosim_design -ldflags "-L ${OPENCV_LIB} ${OPENCV_LIB_REF}" -argv "${XF_PROJ_ROOT}/data/128x128.png"Exportation:
The final stage of the Vitis HLS flow is the exportation of the design. There are several options for this stage which are described in the HLS User's Guide. This example exports the design for RTL and runs Vivado Synthesis to obtain accurate resource utilization and estimated timing results.export_design -flow syn -rtl verilog
I follow the above-mentioned steps to create the resize project in Vitis 2020.2 on Windows. But I am facing a few errors,
Makefile.rules:392: recipe for target 'csim.exe' failed
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_imgcodecs3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_imgproc3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_core3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_highgui3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_flann3411
C:/Xilinx/Vitis_HLS/2020.2/tps/win64/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopencv_features2d3411
I have attached a few screenshots for a better understanding of the issue and the steps taken..i have also attached the tcl file.
I could not find much help online regarding the error thrown so am posting here.
Please let me know if something is still missing.
.png)
.png)
.png)
.png)
.png)
.png)
I spent a week of time configurating OpenCV to work with Vitis HLS 2020 a few weeks back then. The process was frustrating but I finally made it with the help of this guide.
My guess is that you have encountered some problems with your OpenCV build. When I built it with CMAKE, the directory to the .bin folder is D:\opencv-4.2.0_mingw\opencv-4.2.0\build\install\x64\mingw\bin. From your images, the install folder is missing. I guess you may have built OpenCV with customized directory, or the build process simply went wrong. On the other hand, upon successful build, the .bin folder should include around 15 .dll files, including those that you are now missing (if you did not uncheck any package during the build). The image below shows what my .bin folder looks like for OpenCV 4.2.0. I did not check or uncheck anything during the build.
Note: I think you are using Visual Studio to build OpenCV. I did the same thing at the beginning. The build was successful but it does not work well with Vitis HLS (since they use different compilers). You may want to build it with MinGW. I followed the guild in this website to build OpenCV 4.2.0 with MinGW. It should also work with OpenCV 3.4.11.