Working Block Design Example for QDMA IP

[vishy]

Vivado: 2020.1
Board: Zynq Ultrascale+ (ZCU106)

I have managed to open and implement an IP Example Design for QDMA IP (IP Catalog -> QDMA for PCIe -> Open IP Example Design). The design boots perfectly fine and I am able to transfer data in both directions (card-to-host and host-to-card) using DPDK PMD driver on Linux x86 host.

However, the example design is not perfectly suited to my target application. For example, it is using BRAM memory and I would prefer to use DDR4 memory. This means that I need to modify the design somehow.

The problem here for me is the following:

1. QDMA IP Example design is using complex additional user logic (written in SystemVerilog) that is not part of the QDMA IP itself. This can easily be seen by inspecting the source files of the example design in Vivado. Without that part of the design, driver running on x86 host is not able to register the Xilinx PCI device.
2. QDMA IP Example Design in Block Design representation (IP integrator) is not available. Therefore, I am not able to use IP Integrator design flow (simply adding/deleting IP blocks) to edit the design to suit my needs.
3. My SystemVerilog skills are very poor, which makes the idea of playing directly with the SystemVerilog source files to introduce adjustments fairly unfeasible.

Current approach:

Before being aware of the point (1.) written above, I have tried to build a simple Block Design myself, consisting of DDR4 memory controller and QDMA IP. The design can be successfully implemented on the board, but the Xilinx PCI device is not recognized by the host (which can be simply verified by running lspci | grep 'Xilinx' in terminal).

Could you please suggest a solution? Is there a way to construct a minimum working Block Design example for QDMA IP, that would be able to communicate with the host using the Xilinx's recommended drivers like DPDK?

[dsakjl · Accepted solution]

Hi vishy,

I suggest you to check the following Xilinx page:

https://www.xilinx.com/support/answers/72352.html

Essentially the QDMA IP has two TCL parameters that can help you:

1) CONFIG.xlnx_ddr_ex : set this and generate example design to get an example
design with DDR4 instantiation;

2) CONFIG.example_design_type : if you set this parameter to IPI you can get
an example design based on a block design.

Hope this help you.

[vishy]

Thank you so much for your answer, this really seems to be something I have been looking for!
The only issue is, it does not seem to work out-of-the-box for Vivado 2020.1. (Just for the record, my Vivado 2020.1 is installed on a Linux Ubuntu 18.04 host.)

After doing the following:

IP Catalogue -> Customize and add QDMA IP -> 
Run in tcl console:
set_property -dict [list CONFIG.example_design_type {IPI}] [get_ips qdma_0]
set_property -dict [list CONFIG.xlnx_ddr_ex {true}] [get_ips qdma_0]
-> Right click on the QDMA IP -> Open IP Example Design

New Vivado window opens up and starts instantiating the example block design, but fails with 2 errors and the following output in the tcl console:

INFO: [open_example_project] Creating new example project...
INFO: [IP_Flow 19-234] Refreshing IP repositories
INFO: [IP_Flow 19-1704] No user IP repositories specified
INFO: [IP_Flow 19-2313] Loaded Vivado IP repository '/tools/Xilinx/Vivado/2020.1/data/ip'.
create_project: Time (s): cpu = 00:00:11 ; elapsed = 00:00:07 . Memory (MB): peak = 7013.918 ; gain = 56.082 ; free physical = 115 ; free virtual = 30482
INFO: [open_example_project] Importing original IP ...
INFO: [open_example_project] Generating the example project IP ...
INFO: [open_example_project] Adding example synthesis HDL files ...
INFO: [open_example_project] Adding example synthesis miscellaneous files ...
INFO: [open_example_project] Adding example XDC files ...
INFO: [open_example_project] Adding simulation HDL files ...
INFO: [open_example_project] Adding simulation miscellaneous files ...
INFO: [open_example_project] Sourcing example extension scripts ...

ERROR: [BD_TCL-109] This script was generated using Vivado <2018.3> and is being run in <2020.1> of Vivado. Please run the script in Vivado <2018.3> then open the design in Vivado <2020.1>. Upgrade the design by running "Tools => Report => Report IP Status...", then run write_bd_tcl to create an updated script.
CRITICAL WARNING: [filemgmt 20-742] The top module "xilinx_qdma_pcie_ep" specified for this project can not be validated. The current project is using automatic hierarchy update mode, and hence a new suitable replacement top will be automatically selected. If this is not desired, please change the hierarchy update mode to one of the manual compile order modes first, and then set top to any desired value.
Resolution: To switch to manual update order go to the Sources view, right-click on any node in the hierarchy and in the context menu select: 'Hierarchy Update' option 'No Update' or run the following Tcl Command: set_property source_mgmt_mode None [current_project] (which is the Manual Compile Order mode).
CRITICAL WARNING: [filemgmt 20-730] Could not find a top module in the fileset sources_1.
Resolution: With the gui up, review the source files in the Sources window. Use Add Sources to add any needed sources. If the files are disabled, enable them. You can also select the file and choose Set Used In from the pop-up menu. Review if they are being used at the proper points of the flow.
CRITICAL WARNING: [filemgmt 20-730] Could not find a top module in the fileset sim_1.
Resolution: With the gui up, review the source files in the Sources window. Use Add Sources to add any needed sources. If the files are disabled, enable them. You can also select the file and choose Set Used In from the pop-up menu. Review if they are being used at the proper points of the flow.
WARNING: [Vivado 12-818] No files matched '*'
INFO: [open_example_project] Rebuilding all the top level IPs ...
ERROR: [Common 17-161] Invalid option value '' specified for 'objects'.
update_compile_order -fileset sources_1

So, I am planning to try to:
1.) Download & Install Vivado 2018.3
2.) Open Vivado 2018.3 & Repeat the steps reported above to generate IPI QDMA Example Design
3.) After (hopefully) successful generation of a IPI Example Project in Vivado 2018.3, open the generated project with Vivado 2020.1
4.) Then find a way to perform the following in the Vivado 2020.1:(...) Upgrade the design by running "Tools => Report => Report IP Status...", then run write_bd_tcl to create an updated script

Is this the right approach? Is there a simpler solution than downloading the entire Vivado 2018.3 edition?

[dsakjl]

Hi,

I use both options successfully on Vivado 2019.2 but one at a time.

I suggest you to try them separately.

Cheers.

[vishy]

So, to wrap up the discussion:

First of all, I have to say that the option to generate DDR example design for ZCU106 board seems to be unavailable. In https://www.xilinx.com/support/answers/72352.html, it is specifically written that the DDR example design is available for VCU1525, AU200, AU250 and AU280. It seems that the list never really expanded beyond those 4 boards.

When it comes to generating working IPI QDMA Example Design, this is what I can report for the ones that are interested:

First I have downloaded Vivado 2018.3, as instructed by the error message output, to open the QDMA example design in that particular version of Vivado, and to subsequently update the design in Vivado 2020.1. However, after trying to open an IPI Example Design, I have received the following:

After upgrading this design in Vivado 2020.1, I can report that the implemented design was not functional.

Later, I went on to install Vivado 2019.2. In Vivado 2020.1, I have worked with QDMA IP v4, while in Vivado 2019.2, there is only QDMA IP v3 available. In this particular version of Vivado, I have managed to successfully produce IPI Design Example of QDMA IP v3 (however, without DDR memory).

So, that settles it. For ZCU106 boards, it seems that there is no option to generate an Example Design utilizing DDR memory instead of the default BRAM. Also, it seems that in Vivado 2020.1 there is no way to get a working IPI example design for QDMA IP v4.