UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Explorer
Explorer
762 Views
Registered: ‎12-11-2017

multithreaded xdma AXI4 Lite access fails

A bit of background. My xdma implementation connects to four bidirectional FIFOs using AXI4-MM. The single h2c and c2h is shared by the 4 devices. The driver manages the request queues to service the FIFOs with DMA transfers.


Each end device gets a thread to manage it. To schedule the transfers, each thread polls registers that are on AXI4 Lite (rx and tx FIFO fullness) to see if its device is ready to send or receive data.

 

The platform is 8 cores/16 hyperthreads. So each thread is getting its own CPU - we are seeing overlapping requests. There shouldn't be an issue with this... but if I don't spinlock access to the AXI4 lite space my system will hang. With spinlock, no hang.

 

As a further test I created a simple polling loop to exercise the AXI4 Lite read path. If I run enough threads of these at once, I also get a hang.

 

Based on this I suspect that AXI4 Lite is failing to complete one the requests from the link partner (in this case, the CPU itself with its embedded northbridge) and thus causing it to hang.

Tags (2)
0 Kudos
4 Replies
Xilinx Employee
Xilinx Employee
711 Views
Registered: ‎08-06-2008

Re: multithreaded xdma AXI4 Lite access fails

Hi votex1601,

 

Could you put ILA on AXI4 lite interface and check the status of the interface signals? I would be interested to see if tready signal is asserted or deasserted.

 

In your post, you mentioned the following:

 

As a further test I created a simple polling loop to exercise the AXI4 Lite read path. If I run enough threads of these at once, I also get a hang.

 

When you say enough threads, how many threads do you need to run for the hang to occur?

 

Thanks.

0 Kudos
Explorer
Explorer
705 Views
Registered: ‎12-11-2017

Re: multithreaded xdma AXI4 Lite access fails

Yes, I have an ILA on the bus. The ARREADY and RREADY signals look fine as far as I can tell - AXI handshake appears ok. RREADY is low initially, then is high all the time. ARREADY is normally low, and pulses high 2 cycles after ARVALID.

 

I've determined the number of threads to kill it to be 4, using 10M iterations. However, if I run fewer threads and more iterations it will eventually die too.

 

The behavior is similar if I use resource2 (BAR2/3) direct access, or loop AXI4 Lite resource0 (BAR0/1) back to access XDMA over the AXI4 Lite port. They both die on 4 threads.

 

On the other hand, AXI4 Lite access my own stuff it is much more robust - 8 threads doesn't seem to kill it.

 

This made me think about what is different between my stuff and XDMA slave. A major difference is completion latency. My stuff responds in 1 cycle from ARVALID (total transaction is 2 cycles.) XDMA response is much slower - 13 cycles from ARVALID. My hypothesis is that when multiple threads hammer the XDMA slave it's statistically more likely that it will have a transaction clobbered by another thread because it takes longer to complete.

0 Kudos
Explorer
Explorer
697 Views
Registered: ‎12-11-2017

Re: multithreaded xdma AXI4 Lite access fails

I should clarify a point. To test XDMA on resource0 I add an AXI Interconnect block, which add latency. On my fast-responding slave this will die on 6 threads. XDMA slave will die on 4.

 

If I connect AXI4 Lite slave directly to my stuff (no interconnect) then I can run 8 or more threads on my slave.

0 Kudos
Explorer
Explorer
662 Views
Registered: ‎12-11-2017

Re: multithreaded xdma AXI4 Lite access fails

Another data point. 3 threads hammering on resource2 (XDMA regs) - ok. Add one more thread hammering resource0 (AXI4 Lite) - dies.

0 Kudos