03-20-2019 04:37 PM
I have implemented a simple DMA testing FPGA and am attempting to drive the transfer from Linux using the memory mapped registers of the device. My program follows the following flow:
1) Reset MM2S and S2MM by setting the control registers to 4
2) Halt DMA by setting control registers to 0
3) Write source and destination physical addresses
4) Start channel with masked interrups by setting control to 0xf001
5) Write transfer lengths to both channels
6) Wait for both channels to read IOC_Irq showing completion
The program runs the first two times without issue, successfully transferring all the way through the loop. On the third (and every run after) run it hangs, with the MM2S stream reading "running idle IOC_Irq" in the status register after writing the length to that channel.
I've attached the code I'm running, as well as images showing the runs. What could cause this sort of problem?
03-22-2019 11:04 AM
Does either channel report an error when you are in the stuck condition? (Are Dly_Irq or Err_Irq asserting rather than IOC_Irq?)
03-26-2019 01:19 PM - edited 03-26-2019 01:20 PM
I finally got an error to display in one of my runs today:
MM2S Status: halted DMADecErr IOC_Irq Err_Irq
But when I ran it again (3rd run) after rebooting the board I was back to the status I got in previous runs - running idle IOC_Irq
03-26-2019 03:40 PM
Just tested using separate buffers (instead of repeating the same source and dest) and with all DMA transactions taking place in the same program, still get this behaviour. The third DMA transaction always results in the MM2S channel stalling out with the status bits set for "running idle IOC_Irq".
Worth noting, those are the proper status bits for completion. So that makes me think that the MM2S channel actually behaved properly and succeeded in a write out, but the S2MM channel failed to read back in. Not sure if that guesswork helps at all. The S2MM channel is stuck on "running".
Would the Vivado project be helpful?
03-26-2019 03:52 PM
A picture of the block diagram would help. Do you have M_AXIS_MM2S tied to S_AXIS_S2MM (loop back mode) or is there an IP in the middle?
I would be interested in adding System ILAs to the M_AXI_MM2S, M_AXIS_MM2S, S_AXIS_S2MM and M_AXI_S2MM interfaces to see how far the AXI DMA engine gets in that third transfer.
03-27-2019 08:47 AM
03-27-2019 03:46 PM
I think the next step would be to add ILAs to your design and try to trigger on the third transfer. It's not clear where the transfer is getting stuck.
Another thought: when you issue the soft reset, do you poll to see that the reset has completed before continuing to set up the transfer?
03-28-2019 11:16 AM
So I've now demonstrated that it has something to do with the number of bytes written in/out. Even over separate runs, it looks like once more than 64 bytes have been looped through the system that is when it begins to hang. If you try to write more than 64 bytes even on the first run it fails.
03-28-2019 11:42 AM
Is this the same failure or a different failure?
The upside of hardening the failure is that this makes it easier to capture a trace of the hardware interfaces to see where the data is getting stuck.