UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Explorer
Explorer
8,782 Views
Registered: ‎03-13-2014

Zynq Linux user mode DMA, cache problem?

Hi,

 

I am debugging a user mode Linux app that uses AXI dma. I made a 128Mbyte DRAM memory hole which Linux does not use and I mmap() it in. I can then use this memory for buffers for dma to my logic. The data is written into the buffers by the processor, then read from the buffers by the AXI DMA (I use the S_AXI_GP0 port on the processor system). Everything runs but the data is not consistent. I wonder if this is a caching problem, the processor has data cache but the AXI_DMA is uncached so maybe it reads stale data. Is there a way to flush the cache in a Linux user space app, or should I use maybe the ACP port for the AXI_DMA?

 

Regards

 

Dave Warren

 

0 Kudos
5 Replies
Xilinx Employee
Xilinx Employee
8,778 Views
Registered: ‎09-10-2008

Re: Zynq Linux user mode DMA, cache problem?

Hi Dave,

 

I personally find that user space DMA without a kernel driver is going to be very challenging as you don't have good cache control and buffer management IMHO. But I could be corrected if someone has data that differs from that.

 

Are you using /dev/mem?, I assume it's mapping that memory in as cached based on your issues.  If you used the UIO framework (even without a kernel driver) then it could treat that memory as I/O (slower) but then it could be coherent without any caching issues I believe. The UIO framework is really easy to use, not hardly any more work that /dev/mem.

 

I'm working on an example similar to this and here's my thoughts, but I'm open to suggestions as I'm not an expert on it.  I am using a small kernel driver that is a char device (for ioctl support),  a UIO device (to allow mmap with user space), and a client of the DMA Engine framework which the AXI DMA driver is part of.  This driver takes care of cache control so that DMA buffers are consistent.

 

The driver allocates a kernel buffer using kmalloc such that it's cached.  The memory is a block that is shared memory with a user space app that is UIO based with mmap.  The user space application read/writes data and control/status to the block and then uses iotcl to start the DMA transfer. 

 

Great conversation, I like these threads as we all learn,

John

0 Kudos
Xilinx Employee
Xilinx Employee
8,774 Views
Registered: ‎09-10-2008

Re: Zynq Linux user mode DMA, cache problem?

Yes ACP could help avoid caching issues with software based on what I understand. If you are DMAing a large amount of memory it could somewhat pollute the cache which may affect the CPU performance.
0 Kudos
Explorer
Explorer
8,773 Views
Registered: ‎03-13-2014

Re: Zynq Linux user mode DMA, cache problem?

Hi John,

 

I guess my 'quick and dirty' approach is because I am a hardware engineer that dabbles in software so I don't want to touch kernel programming.

 

Can you post a link to UIO framework and I will check it out.

 

Meanwhile I am going to try again using the ACP port for the AXI DMA.

 

Regards Dave

0 Kudos
Xilinx Employee
Xilinx Employee
8,766 Views
Registered: ‎09-10-2008

Re: Zynq Linux user mode DMA, cache problem?

I attached a PPT and there are references at the end that may help too.

0 Kudos
Visitor iviktorin
Visitor
8,753 Views
Registered: ‎08-13-2014

Re: Zynq Linux user mode DMA, cache problem?

Hello, developing of DMA transfers is quite challenging and you need to write a kernel driver. Your issue with inconsistency cannot be solved from userspace. The uio is a way... But from my point of view, it is important to understand the kernel space anyway.

You have solved the issue with virtual/physical addresses by reserving the memory. But you still have to flush caches before transfers (dma_map_* kernel interface).

ACP should solve this if you activate the cache coherency features. But it is suitable shorter transfers.

Jan Viktorin
Ph.D. student at Brno University of Technology | System Architect at RehiveTech spin-off company
0 Kudos