QDMA dmaperf

marcovalerioa · ‎02-28-2020

I need to test the QDMA IP on a board from Vadatech.

I have generated the firmware using the QDMA example provided with Vivado (Release 2018.2), specifying the device on my board (xcvu13p). I have changed the constraint file xilinx_qdma_pcie_x0y1.xdc in order to specify the appropriate pin locations for my board.

I have downloaded the drivers and other tools from the Git repository: https://github.com/Xilinx/dma_ip_drivers and I have compiled and installed them following the instructions found in https://xilinx.github.io/dma_ip_drivers/2019.1/linux-kernel/html/index.html#

It seems that the board is correctly found on the PCIe bus, according to the lspci command:

-----------------------------------------------------------------------------------------------

eldes@eldes-TR-E5x-02:~$ lspci | grep Xilinx
06:00.0 Signal processing controller: Xilinx Corporation Device 9034

------------------------------------------------------------------------------------------------

however when I try to use the dmaperf tool it seems that no transactions on the bus occurs.

The following is one of the configuration files that I have used:

------------------------------------------------------------------------------------------------

eldes@eldes-TR-E5x-02:~/TestDMA00/dma_ip_drivers-master/QDMA/linux-kernel/tools$ more bi_mm_1_4_64
mode=mm
dir=bi
pf_range=0:0
q_range=0:3
flags=
cmpl_status_acc=5
dump_en=0
tmr_idx=5
cntr_idx=6
trig_mode=cntr_tmr
pfetch_en=0
cmptsz=0
rngidx=5
runtime=30
num_threads=4
num_pkt=64
pkt_sz=64
pci_bus=06
pci_device=00

------------------------------------------------------------------------------------------------

And the output of the dmaperf and dmactl stat tools:

------------------------------------------------------------------------------------------------

eldes@eldes-TR-E5x-02:~/TestDMA00/dma_ip_drivers-master/QDMA/linux-kernel/tools$ sudo dmaperf -c bi_mm_1_4_64
dmactl qdma06000 q add idx 0 mode mm dir h2c
qdma06000-MM-0 H2C added.
Added 1 Queues.
dmactl qdma06000 q start idx 0 dir h2c idx_ringsz 5
1 Queues started, idx 0 ~ 0.
dmactl qdma06000 q add idx 0 mode mm dir c2h
qdma06000-MM-0 C2H added.
Added 1 Queues.
dmactl qdma06000 q start idx 0 dir c2h idx_ringsz 5
1 Queues started, idx 0 ~ 0.
dmactl qdma06000 q add idx 1 mode mm dir h2c
qdma06000-MM-1 H2C added.
Added 1 Queues.
dmactl qdma06000 q start idx 1 dir h2c idx_ringsz 5
1 Queues started, idx 1 ~ 1.
dmactl qdma06000 q add idx 1 mode mm dir c2h
qdma06000-MM-1 C2H added.
Added 1 Queues.
dmactl qdma06000 q start idx 1 dir c2h idx_ringsz 5
1 Queues started, idx 1 ~ 1.
dmactl qdma06000 q add idx 2 mode mm dir h2c
qdma06000-MM-2 H2C added.
Added 1 Queues.
dmactl qdma06000 q start idx 2 dir h2c idx_ringsz 5
1 Queues started, idx 2 ~ 2.
dmactl qdma06000 q add idx 2 mode mm dir c2h
qdma06000-MM-2 C2H added.
Added 1 Queues.
dmactl qdma06000 q start idx 2 dir c2h idx_ringsz 5
1 Queues started, idx 2 ~ 2.
dmactl qdma06000 q add idx 3 mode mm dir h2c
qdma06000-MM-3 H2C added.
Added 1 Queues.
dmactl qdma06000 q start idx 3 dir h2c idx_ringsz 5
1 Queues started, idx 3 ~ 3.
dmactl qdma06000 q add idx 3 mode mm dir c2h
qdma06000-MM-3 C2H added.
Added 1 Queues.
dmactl qdma06000 q start idx 3 dir c2h idx_ringsz 5
1 Queues started, idx 3 ~ 3.
dmautils(32) threads
dmactl qdma06000 q stop idx 0 dir h2c
dmactl qdma06000 q stop idx 0 dir c2h
dmactl qdma06000 q stop idx 3 dir c2h
dmactl qdma06000 q stop idx 2 dir h2c
dmactl qdma06000 q stop idx 3 dir h2c
dmactl qdma06000 q stop idx 2 dir c2h
dmactl qdma06000 q stop idx 1 dir c2h
dmactl qdma06000 q stop idx 1 dir h2c
Stopped Queues 0 -> 0.
Stopped Queues 3 -> 3.
Stopped Queues 3 -> 3.
Stopped Queues 2 -> 2.
Stopped Queues 0 -> 0.
Stopped Queues 2 -> 2.
Stopped Queues 1 -> 1.
Stopped Queues 1 -> 1.
dmactl qdma06000 q del idx 0 dir h2c
Deleted Queues 0 -> 0.
dmactl qdma06000 q del idx 0 dir c2h
Deleted Queues 0 -> 0.
dmactl qdma06000 q del idx 1 dir h2c
Deleted Queues 1 -> 1.
dmactl qdma06000 q del idx 1 dir c2h
Deleted Queues 1 -> 1.
dmactl qdma06000 q del idx 2 dir h2c
Deleted Queues 2 -> 2.
dmactl qdma06000 q del idx 2 dir c2h
Deleted Queues 2 -> 2.
dmactl qdma06000 q del idx 3 dir h2c
Deleted Queues 3 -> 3.
dmactl qdma06000 q del idx 3 dir c2h
Deleted Queues 3 -> 3.
No IOs happened

eldes@eldes-TR-E5x-02:~/TestDMA00/dma_ip_drivers-master/QDMA/linux-kernel/tools$ dmactl qdma06000 stat
qdma06000:statistics
Total MM H2C packets processed = 0
Total MM C2H packets processed = 0
Total ST H2C packets processed = 0
Total ST C2H packets processed = 0

------------------------------------------------------------------------------------------------

Could someone give me a clue about what is going on?

Many thanks

Marco Arbolino

marcovalerioa · ‎03-04-2020

I have repeated the test changing the configuration file for dmaperf:

mode=mm
dir=h2c
pf_range=0:0
q_range=0:0
flags=
cmpl_status_acc=5
dump_en=1
tmr_idx=5
cntr_idx=6
trig_mode=cntr_tmr
pfetch_en=0
cmptsz=0
rngidx=5
runtime=10
num_threads=1
num_pkt=64
pkt_sz=64
pci_bus=06
pci_device=00

The main change is that I have enabled the dump of registers, that I have attached to this post.

It seems to me that some of the registers in BAR0 indicate that some kind of error occurs (e.g. Registers 0x248, 0x264, and 0x26c), but I cannot understand the nature of the error.

Could someone help me in understanding what occurs?

many thanks

Marco Arbolino

marcovalerioa · ‎03-11-2020

Trying to solve my problem I moved to Vivado 2018.3 and found a different behaviour:

1) Add a QDMA IP in a block design

2) Set the Mode to "Advanced" in the "Basic" tab

3) The default content of the "PCIe:Misc" tab is as shown in attached figure MSI0.png

4) Uncheck the "Enable PF0 MSI-X Capability Structure" flag, the content of the "PCIe:Misc" tab changes as shown in attached figure MSI1.png

5) Check the "Enable PF0 MSI-X Capability Structure" flag, the content of the "PCIe:Misc" tab changes as shown in attached figure MSI2.png showing an error, however none of the values can be changed.