UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Adventurer
Adventurer
789 Views
Registered: ‎03-22-2016

XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

With the original xdma driver (Xilinx_Answer_65444_Linux_Files.zip dated 6/8/2017), the ISR is only called once at the end of each transfer. In the ISR routine, the ch_irq register indicates that the interrupt was caused by a channel. I believe this is the correct behavior.

 

kernel: [ 4840.803789] xdma_isr():(irq=35) <<<< INTERRUPT SERVICE ROUTINE
kernel: [ 4840.803796] xdma_isr():ch_irq = 0x00000002
kernel: [ 4840.803804] xdma_isr():user_irq = 0x00000000

 

With the same design in the FPGA, the updated xdma driver (Xilinx_Answer_65444_Linux_Files_rel20180420.zip dated 04/19/2018), the ISR is called repeatedly during a transfer. In the ISR routine, neither the ch_irq or user_irq register indicates the cause of the interrupt. Each time, the ISR does nothing and returns, only to be immediately called again, still with no indication as to the cause of the interrupt.

 

 

kernel: [ 355.260272] xdma:xdma_isr: schedule_work, 0-C2H0-ST.
kernel: [ 355.260299] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260308] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260316] xdma:xdma_isr: user_irq = 0x00000000
kernel: [ 355.260325] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260331] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260340] xdma:xdma_isr: user_irq = 0x00000000
kernel: [ 355.260349] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260356] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260362] xdma:xdma_isr: user_irq = 0x00000000
kernel: [ 355.260369] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260372] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260379] xdma:xdma_isr: user_irq = 0x00000000
kernel: [ 355.260384] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260387] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260391] xdma:xdma_isr: user_irq = 0x00000000
kernel: [ 355.260396] xdma:xdma_isr: (irq=26, dev 0x00000000a003ae5c) <<<< ISR.
kernel: [ 355.260400] xdma:xdma_isr: ch_irq = 0x00000000
kernel: [ 355.260404] xdma:xdma_isr: user_irq = 0x00000000
[repeat hundreds of times until transfer completes]

 

In my design, I have user_irq_req tied to ground. I have the xdma IP block configured as streaming.

 

I reported a similar issue previously, and I believed that the cause user_irq_req pin being tied to 1 due to a bug in Vivado block automation. When that occurred, user_irq showed 0x00000001 in the ISR. user_irq is showing 0x00000000 in this case, which I believe must be a different issue.

 

Is anyone else seeing this behavior? I had to turn on debug logging to see the kernel logs from the ISR. Thanks.

 

EDIT: This behavior is only seen when interrupt_mode in the xdma module is 0 (MSI-X), which is the default. When it is set to 1, the issue goes away.

 

EDIT 2: I just enabled "Enable MSI-X Capability Structure" (which is disabled by default), and I no longer see the ISR being called repeatedly, even in MSI-X mode.

0 Kudos
5 Replies
743 Views
Registered: ‎08-10-2016

Re: XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

I see this issue with Vivado 2018.2 'DMA subsystem for PCIe' IP working with the XDMA drivers in the 20th April, 2018 release "Xilinx_Answer_65444_Linux_Files_rel20180420.zip". Target is an Artix-7 FPGA on a custom board whose design is similar to AC701, except PCIe is realised with cable.

 

In my design, the IP's "user_irq_req[0:0]" port is tied to '0' with a constant. I am using the module in Polling mode. Turning on debugging as suggested by the AR 65444 PDF as below

#define XDMA_DEBUG 1

gave no useful messages were being logged in dmesg. So I turned on

__LIBXDMA_DEBUG__

by manually defining it in xdma/libxdma.h file.

 

These ISR messages flood the logs and within no time, a "memory-running-out" warning came up on my Linux host.

0 Kudos
Xilinx Employee
Xilinx Employee
675 Views
Registered: ‎08-06-2008

Re: XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

Could you please post your dmesg log?

Also, please post your xci file.

In my test, I don't see repeated xdma_isr being called.

0 Kudos
Adventurer
Adventurer
670 Views
Registered: ‎03-22-2016

Re: XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

Here's my dmesg log and XCI files.

 

This test was done with Vivado 2018.2 in a project for the VCU118. I included an XDMA IP block set as streaming, connecting the H2C port to the C2H port via AXIS Data FIFO.

 

The xdma module is the latest release, the only change is to the Makefile to enable verbose debugging logs. This is necessary to see that the ISR is being called repeatedly.

 

diff --git a/xdma/Makefile b/xdma/Makefile
index d1c07a8..c893821 100644
--- a/xdma/Makefile
+++ b/xdma/Makefile
@@ -14,7 +14,7 @@ topdir := $(shell cd $(src)/.. && pwd)
TARGET_MODULE:=xdma

EXTRA_CFLAGS := -I$(topdir)/include $(XVC_FLAGS)
-#EXTRA_CFLAGS += -D__LIBXDMA_DEBUG__
+EXTRA_CFLAGS += -D__LIBXDMA_DEBUG__
#EXTRA_CFLAGS += -DINTERNAL_TESTING

ifneq ($(KERNELRELEASE),)

Note that this issue only occurs (for me, anyway), after the first read() has been called and when the module is inserted with the default settings. When interrupt_mode is set to 1 (MSI) rather than the default of 0 (MSI-X), it seems to work as expected.

 

I do see this in the logs that might be indicative of an issue:

 

Aug 24 11:16:52 MY-HOSTNAME-HERE kernel: [64532.997002] xdma:enable_msi_msix: MSI/MSI-X not detected - using legacy interrupts

Hope this helps. Let me know if there's anything else I can do.

 

EDIT: All my XCI files got removed when I posted them. I will try to find another way to attach them.

The attachment's design_1_xdma_0_0.xci content type (application/octet-stream) does not match its file extension and has been removed.
The attachment's design_1_xdma_0_0_pcie4_ip.xci content type (application/octet-stream) does not match its file extension and has been removed.
The attachment's design_1_xdma_0_0_pcie4_ip_gt.xci content type (application/octet-stream) does not match its file extension and has been removed.
The attachment's xdma_v4_1_1_blk_mem_64_reg_be.xci content type (application/octet-stream) does not match its file extension and has been removed.
The attachment's xdma_v4_1_1_blk_mem_64_noreg_be.xci content type (application/octet-stream) does not match its file extension and has been removed.

EDIT 2: I enabled the "Enable MSI-X Capability Structure" option in the XDMA block it's disabled by default). With that option enabled, I am no longer seeing the ISR being called repeatedly.

0 Kudos
Xilinx Employee
Xilinx Employee
621 Views
Registered: ‎08-06-2008

Re: XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

Thanks Jeff. We will investigate this and get back as soon as we have an update.

 

 

0 Kudos
Highlighted
Adventurer
Adventurer
605 Views
Registered: ‎03-22-2016

Re: XDMA Linux driver calling ISR repeatedly during transfer without ch_irq or user_irq indicating cause

Now that I've enabled the "Enable MSI-X Capability Structure" option, I'm not seeing the extraneous ISR calls, but I am seeing instability in the driver. Transfers will sometimes work just fine, and sometimes (often) hard freeze the entire machine. I haven't been able to debug much about it, but the issue seems to only occur when interrupt_mode is 0 (MSI-X). When I switch to interrupt_mode=1 (MSI), it seems stable.
0 Kudos