UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
9,340 Views
Registered: ‎05-27-2013

Writing and reading data to/from DDR3 with Linux Zynq

Hello,

 

I want to read and write Data from/to a specific address in DDR3 Memory with Linux driver/application. In a driver I tried ioremap to map physical address in DDR3 address space to virtual address, but I get the following warning:

 

WARNING: at arch/arm/mm/ioremap.c:241 __arm_ioremap_pfn_caller+0x108/0x1a4()

Modules linked in: (...)

 

Does anyone know a way how to access DDR3 from driver?

 

Thanks in advance!

0 Kudos
10 Replies
Xilinx Employee
Xilinx Employee
9,321 Views
Registered: ‎03-13-2012

Re: Writing and reading data to/from DDR3 with Linux Zynq

When you try to read specific DDR addresses from a Linux driver, you do something wrong. Use the kernel implementations kmalloc and kfree to allocate and free memory. Ioremap is only of IO memory like peripheral registers etc.

0 Kudos
9,284 Views
Registered: ‎05-27-2013

Re: Writing and reading data to/from DDR3 with Linux Zynq

Hello,

 

thanks a lot for your reply.

 

The reason why I wanted to read and write to specific addresses is that I want that for example an application writes data to ddr3, dma controller reads this data and writes it to PL. I do not want to use the write function to pass data to dma driver, because in later times I want that usb writes data to ddr3, dma controller reads this data and writes it to PL. Maybe in addition an application can access this data to work with it.

 

Now my questions are:

  1. How can I read and write from/to the allocated memory?
  2. How can I write data to ddr3 in application and read the same data from ddr3 in driver?
  3. How can I write data to ddr3 with one driver and read this data with another driver?

Thanks in advance!

0 Kudos
Explorer
Explorer
9,128 Views
Registered: ‎06-23-2013

Re: Writing and reading data to/from DDR3 with Linux Zynq

Same question, because we have implemented DMA engine in PL for AXI_HP.

 

 zc706 address range 0x0010_0000 to 0x3FFF_FFFF is accessible to CPU,ACP, AXI_HP and other BUS masters according to ug585-Zynq-7000-TRM.pdf

 

So on the PS, we want to fill up about 40MB begin at 0x1000_0000 and then we'd write start address to PL register

and length to another PL register.

 

Soren, are you saying there is no way to allocate a buffer at  0x1000_0000, eg. in a small driver, then write a user space program to fill it ?, letting Linux know that we're using that address range.

 

If above not possible then relying on kmalloc, how to guarantee pointer returned in range accessble to AXI_HP ?

dogbytes
0 Kudos
Xilinx Employee
Xilinx Employee
9,125 Views
Registered: ‎03-13-2012

Re: Writing and reading data to/from DDR3 with Linux Zynq


doreenyen wrote:

Soren, are you saying there is no way to allocate a buffer at  0x1000_0000, eg. in a small driver, then write a user space program to fill it ?, letting Linux know that we're using that address range.



The original question was not about DMA. The original problem was something about writing to specific physical addresses w/o giving any additional information. And that is usually wrong since the OS is doing all the memory management for you, so you should never work with physical addresses.

 

However, DMA may be a lot different and honestly I cannot tell you a lot about DMA since I never used it.

Only one thing I kinda know: kmalloc does not work for big buffers there are special allocators for continuous memory as DMA requires it.

0 Kudos
Scholar milosoftware
Scholar
9,114 Views
Registered: ‎10-26-2012

Re: Writing and reading data to/from DDR3 with Linux Zynq

Allocate memory with the "GFP_DMA" flag. That will position it so that the DMA controller can reach it, and it will be physically continuous memory. Usually such a buffer is allocated at boot time, because memory will become fragmented and at some point, the system will be unable to allocate anything larger than a single page.

 

The "Linux Device Drivers" explains it all quite thouroughly:

http://www.makelinux.net/ldd3/

 

An alternative way (though I would consider this a HACK), is to reduce the memory available to Linux, by changing the memory configuration in the devicetree. Then you can use that exceluded memory exclusively in your memory map and PL, without troubling the kernel about it.

0 Kudos
Explorer
Explorer
9,097 Views
Registered: ‎06-23-2013

Re: Writing and reading data to/from DDR3 with Linux Zynq

MiloSoftware suggested using flag __GFP_DMA but perhaps I'm not using it correctly ?

When used in kmalloc it returned addresses that are not in 0010_0000 - 3FFF_FFFF DDR accessible to AXI_HP.

So my question remains, how do I get an allocation in the above range to fill for PL DMA READ ?

 

 

ug585-Zynq-7000-TRM.pdf Address Map has no entries at E600_0000 - F7FF_FFFF

 __GFP_DMA flag allowed kmalloc in that address range.  There is no DDR in that address range !?

 

pbdma::init_module: begin
pbdma::init_module: PL READaddress=ed86f400
pbdma::init_module: status 0
pbdma::cleanup_module:
pbdma::init_module: begin
pbdma::init_module: PL READaddress=eda08400
pbdma::init_module: status 0

 

#define LENGTH 1024
 kbuffer_p = kmalloc((size_t) LENGTH, __GFP_DMA);
   if (kbuffer_p != NULL) {
     printk(KERN_INFO "pbdma::init_module: PL READaddress=%x\n", (int)kbuffer_p);
     status = 0;
     kfree(kbuffer_p);

       }

 

 

Can anybody comment on the maps file shows addresses which should be reserved for PL ?

 

Address Map lists 8000_0000 - BFFF_FFFF as PL M_AXI_GP1

 

Our running Ubuntu from                   git clone git://git.xilinx.com/linux-xlnx.git

 

 

cat /proc/1/maps

b6d96000-b6d9e000 r-xp 00000000 b3:02 654521     /lib/arm-linux-gnueabihf/libnss_files-2.15.so
...

b6f65000-b6f66000 rw-p 00017000 b3:02 654515     /lib/arm-linux-gnueabihf/ld-2.15.so
b6f66000-b6f83000 r-xp 00000000 b3:02 785008     /sbin/init
b6f8a000-b6f8b000 r--p 0001c000 b3:02 785008     /sbin/init
b6f8b000-b6f8c000 rw-p 0001d000 b3:02 785008     /sbin/init
b6f8c000-b6fef000 rw-p 00000000 00:00 0          [heap]
bea2f000-bea50000 rw-p 00000000 00:00 0          [stack]
ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]

My colleague implemented some PL DMA registers at B880_00xx but any B6xx_xxxx is in PL M_AXI_GP1

then wouldn't it have conflicted with above ?

dogbytes
0 Kudos
Explorer
Explorer
9,092 Views
Registered: ‎06-23-2013

Re: Writing and reading data to/from DDR3 with Linux Zynq

Er, kmalloc returns a virtual address.
How do you find out its physical address ?
dogbytes
0 Kudos
Visitor rteune
Visitor
9,069 Views
Registered: ‎11-30-2009

Re: Writing and reading data to/from DDR3 with Linux Zynq

Try virt_to_phys.  The physical address returned can be given to the FPGA to access the memory.  Get a user space address for the same location using mmap with that physical address.

Randy
0 Kudos
Explorer
Explorer
9,061 Views
Registered: ‎06-23-2013

virt_to_phys Re: Writing and reading data to/from DDR3 with Linux Zynq

 

The physical address is the virtual address - 0xc0000000

 

#include <../arch/arm/include/asm/memory.h>  //for virt_to_phys
#define LENGTH 4064

 

 kbuffer_p = kmalloc((size_t) LENGTH, GFP_ATOMIC | __GFP_COLD | __GFP_DMA);
   if (kbuffer_p != NULL) {
     printk(KERN_INFO "pbdma::init_module: %d PL READaddress=%x\n", i,(int)kbuffer_p);
     status += 0;
     vbufptrs[i] = kbuffer_p;
     printk(KERN_INFO "pbdma::init_module: writing to address %x\n", (int) kbuffer_p);
     memset( kbuffer_p, 0x5a, LENGTH);
     kbuffer_phys = virt_to_phys(kbuffer_p);
     pbufptrs[i] = kbuffer_phys;
     printk(KERN_INFO "pbdma::init_module: physical address %x pageoffset %x physoffset %x\n",
            (int) kbuffer_phys, (int) PAGE_OFFSET, (int) PHYS_OFFSET);
       }

 

pbdma::init_module: begin
pbdma::init_module: 0 PL READaddress=ed9d5000
pbdma::init_module: writing to address ed9d5000
pbdma::init_module: physical address 2d9d5000 pageoffset c0000000 physoffset 0
pbdma::init_module: status 0
pbdma::init_module: 1 PL READaddress=ec9b6000
pbdma::init_module: writing to address ec9b6000
pbdma::init_module: physical address 2c9b6000 pageoffset c0000000 physoffset 0
pbdma::init_module: status 0

 

 

dogbytes
0 Kudos
Explorer
Explorer
3,422 Views
Registered: ‎06-23-2013

ioremap Re: Writing and reading data to/from DDR3 with Linux Zynq

I've attempted the HACK mentioned by milosoftware and the same FPGA bit file that DMAed with a driver using kmalloc

does not DMA with the driver using the HACK, ioremap, dma_unmap_single, copy_from_user, dma_map_single, ...

 

The HACK has all the attractions of using insmod, can anybody demonstrate a working example on Zynq ?

 

A FPGA system.bit.bin file works with a driver that used kmalloc,
which returned a virtual address and virt_to_phys
returned the same physical address that was given to kmalloc.
The FPGA does a DMA read from read_phys_addr and then a DMA write to write_phys_addr

But kmalloc is limited to a 4MB allocation, so another driver was written
to use instead, ioremap, dma_unmap_single, dma_map_single, copy_from_user, copy_to_user.

The new driver ioremap return values are not expected offset 0xC00000000 from physical addrs.
Changes for the new driver:

The devicetree.dts was changed from reg = <0x00000000 0x40000000>;
to reserve the upper quarter GB for DMA buffers.

    memory {
        device_type = "memory";
        reg = <0x00000000 0x30000000>;
    };


Driver init_module() {
read_phys_addr = 0x30000000;  write_phys_addr = 0x38000000;
read_virt_addr = ioremap (read_phys_addr, 0x07E00000);  //read_virt_addr is 0xF1000000
write_virt_addr = ioremap(write_phys_addr, 0x00100000);//write_virt_addr is 0xF0200000
mmio = ioremap(0xB8800000, 0x200); // FPGA registers
}

Driver device_write() {
dma_unmap_single( NULL, read_virt_addr, user_buffer, length);

// The user_buffer containing 0xDB is correctly copied to 0x30000000 for length 288.
nbytes_failed = copy_from_user( read_virt_addr, user_buffer, length);

// read_phys_addr returned from dma_map_single is 0x31000000
read_phys_addr = dma_map_single(NULL, read_virt_addr, length, DMA_TO_DEVICE);

read_phys_addr = 0x30000000;  //Force read_phys_addr equal to ioremap read_phys_addr arg

write_phys_addr = dma_map_single( NULL, write_virt_addr, length, DMA_FROM_DEVICE);
// Because write_virt_addr is 0xF0200000, dma_map_single returns 0x30200000

write_phys_addr = 0x38000000;  //Force write_phys_addr equal to ioremap write_phys_addr arg

// FPGA readaddr reg set to read_phys_addr, FPGA writeaddr reg set to write_phys_addr
// Set FPGA the readlen and writelen registers
// DMA read status is busy
// FPGA rd status is busy, the user_buffer data never arrives at write_phys_addr
}
Driver device_read() {
       dma_unmap_single( NULL, write_virt_addr, length, DMA_FROM_DEVICE);
       nbytes_failed = copy_to_user( user_buffer, write_virt_addr, length);
}

dogbytes
0 Kudos