UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Explorer
Explorer
742 Views
Registered: ‎10-19-2017

Mmapping Succeeds with AXI DMA-Proxy Driver but Memory in Kernel not Changed from User Space

I have a really bizarre situation. I have created a DMA-Proxy driver that sets up two DMA-Channels using a platform device and referring to the device tree. It then creates a "proxy_dma" class in sysfs and creates two different  device nodes, registering them with that class. For some reason, I am not able to change the memory of the second device, regardless of whether it is the transmit or receive channel, from the user space. I think it is a problem with how I am registering the character devices but I am not expert enough to know where exactly I am going wrong.

 

Init the module

static int __init dma_proxy_init(void)
{


        int rc;

        rc = platform_driver_register(&rfx_axidmatest_driver);

        return 0;
}

rfx_axidmatest_driver probe function

 

int rfx_axidmatest_driver_probe(struct platform_device *pdev)
{
        printk(KERN_INFO "probe\n");

        create_channel(&channels[1], "_rx", DMA_DEV_TO_MEM, pdev);

        // Memory mapped data to this channel doesn't change in the kernel after     the data is changed in the user space 
        create_channel(&channels[0], "_tx", DMA_MEM_TO_DEV, pdev);

}

Here is how the channels are being initialized and the character device drivers set up

 

 

static int create_channel(struct dma_proxy_channel *pchannel_p,
                        char *name,
                        u32 direction,

                        struct platform_device *pdev)
{
        int rc;

        if(name == "_tx")
        {
                printk(KERN_INFO "Getting transfer channel\n");
                pchannel_p->channel_p = dma_request_slave_channel(&pdev->dev, "axidma0");
                if (!pchannel_p->channel_p) {
                        dev_err(pchannel_p->dma_device_p, "DMA channel request error\n");
                        return ERROR;
                }
        }
        else
        {
                printk(KERN_INFO "Getting receive channel\n");
                pchannel_p->channel_p = dma_request_slave_channel(&pdev->dev, "axidma1");
                if (!pchannel_p->channel_p) {
                        dev_err(pchannel_p->dma_device_p, "DMA channel request error\n");
                        return ERROR;
                }
        }

        /* Request the DMA channel from the DMA engine and then use the device from
         * the channel for the proxy channel also.
         */     
        pchannel_p->dma_device_p = &pchannel_p->channel_p->dev->device;

        /* Initialize the character device for the dma proxy channel
         */
        rc = cdevice_init(pchannel_p, name);
        if (rc) {
                return rc;
        }

        pchannel_p->direction = direction;

        /* Allocate memory for the proxy channel interface for the channel as either
         * cached or non-cache depending on input parameter. Use the managed
         * device memory when possible but right now there's a bug that's not understood
         * when using devm_kzalloc rather than kzalloc, so stay with kzalloc.
         */
        pchannel_p->interface_p = (struct dma_proxy_channel_interface *)
                                kzalloc(sizeof(struct dma_proxy_channel_interface),
                                GFP_KERNEL);
        printk(KERN_INFO "Allocating cached memory at 0x%08X\n",
                                (unsigned int)pchannel_p->interface_p);
        if (!pchannel_p->interface_p) {
                dev_err(pchannel_p->dma_device_p, "DMA allocation error\n");
                return ERROR;
        }
        return 0;
}

Initialization for the character devices called from the above function

 

 

static int cdevice_init(struct dma_proxy_channel *pchannel_p, char *name)
{

        int rc;
        char device_name[32] = "dma_proxy";
        static struct class *local_class_p = NULL;

        /* Allocate a character device from the kernel for this 
         * driver
         */
        rc = alloc_chrdev_region(&pchannel_p->dev_node, 0, 1, "dma_proxy");

        if (rc) {
                dev_err(pchannel_p->dma_device_p, "unable to get a char device number\n");
                return rc;
        }

        /* Initialize the ter device data structure before
         * registering the character device with the kernel
         */
        cdev_init(&pchannel_p->cdev, &dm_fops);
        pchannel_p->cdev.owner = THIS_MODULE;
        rc = cdev_add(&pchannel_p->cdev, pchannel_p->dev_node, 1);

        if (rc) {
                dev_err(pchannel_p->dma_device_p, "unable to add char device\n");
                goto init_error1;
        }

        /* Only one class in sysfs is to be created for multiple channels,
         * create the device in sysfs which will allow the device node 
         * in /dev to be created 
         */
        if (!local_class_p) {
                printk(KERN_INFO "creating a new class"); 
                local_class_p = class_create(THIS_MODULE, DRIVER_NAME);

                if (IS_ERR(pchannel_p->dma_device_p->class)) {
                        dev_err(pchannel_p->dma_device_p, "unable to create class\n");
                        rc = ERROR;
                        goto init_error2;
                }
        }

        pchannel_p->class_p = local_class_p;
        printk(KERN_INFO "local class_p: %x", (void*)(pchannel_p->class_p));

        /* Create the device node in /dev so the device is accessible 
         * as a character device
         */
        strcat(device_name, name);
        pchannel_p->proxy_device_p = device_create(pchannel_p->class_p, NULL,
                                                        pchannel_p->dev_node, NULL, device_name);

        if (IS_ERR(pchannel_p->proxy_device_p)) {
                dev_err(pchannel_p->dma_device_p, "unable to create the device\n");
                goto init_error3;
        }
       return 0;

init_error3:
        class_destroy(pchannel_p->class_p);

init_error2:
        cdev_del(&pchannel_p->cdev);

init_error1:
        unregister_chrdev_region(pchannel_p->dev_node, 1);
        return rc;
}

The userland application

static struct dma_proxy_channel_interface *tx_proxy_interface_p, *rx_proxy_interface_p;;
static int tx_proxy_fd, rx_proxy_fd;

/* The following function is the transmit thread to allow the transmit and the 
 * receive channels to be operating simultaneously. The ioctl calls are blocking
 * such that a thread is needed.
 */
void *tx_thread()
{
        int dummy, i;

        /* Set up the length for the DMA transfer and initialize the transmit
         * buffer to a known pattern.
         */
        tx_proxy_interface_p->length = TEST_SIZE;

        for (i = 0; i < TEST_SIZE; i++)
                tx_proxy_interface_p->buffer[i] = i;

        /* Perform the DMA transfer and the check the status after it completes
         * as the call blocks til the transfer is done.
         */
        printf("tx ioctl user space\n");
        ioctl(tx_proxy_fd, 0, &dummy);

        if (tx_proxy_interface_p->status != PROXY_NO_ERROR)
                printf("Proxy tx transfer error\n");
}

/* The following function uses the dma proxy device driver to perform DMA transfers
 * from user space. This app and the driver are tested with a system containing an
 * AXI DMA without scatter gather and with transmit looped back to receive.
 */
int main(int argc, char *argv[])
{
        int i;
        int dummy;
        pthread_t tid;

        printf("DMA proxy test\n");

        /* Step 1, open the DMA proxy device for the transmit and receive channels with
         * read/write permissions
         */

        tx_proxy_fd = open("/dev/dma_proxy_tx", O_RDWR);

        if (tx_proxy_fd < 1) {
                printf("Unable to open DMA proxy device file\n");
                return -1;
        }

        rx_proxy_fd = open("/dev/dma_proxy_rx", O_RDWR);
        if (tx_proxy_fd < 1) {
                printf("Unable to open DMA proxy device file\n");
                return -1;
        }
        rx_proxy_interface_p = (struct dma_proxy_channel_interface *)mmap(NULL, sizeof(struct dma_proxy_channel_interface),
                                                                        PROT_READ | PROT_WRITE, MAP_SHARED, rx_proxy_fd, 0);



        tx_proxy_interface_p = (struct dma_proxy_channel_interface *)mmap(NULL, sizeof(struct dma_proxy_channel_interface),
                                                                        PROT_READ | PROT_WRITE, MAP_SHARED, tx_proxy_fd, 0);

        printf("tx_proxy_interface_p: %p\n", (void*)tx_proxy_interface_p);
        printf("rx_proxy_interface_p: %p\n", (void*)rx_proxy_interface_p);

        if ((rx_proxy_interface_p == MAP_FAILED) || (tx_proxy_interface_p == MAP_FAILED)) {
                printf("Failed to mmap\n");
                return -1;
        }

        rx_proxy_interface_p->length = TEST_SIZE;
        for (i = 0; i < TEST_SIZE; i++) {
                rx_proxy_interface_p->buffer[i] = 0;
        }


        /* Create the thread for the transmit processing and then wait a second so the printf output is not 
         * intermingled with the receive processing
         */
        pthread_create(&tid, NULL, tx_thread, NULL);
        sleep(2);

        printf("rx ioctl user space\n");
        ioctl(rx_proxy_fd, 0, &dummy);

        if (rx_proxy_interface_p->status != PROXY_NO_ERROR)
                printf("Proxy rx transfer error\n");

        /* Verify the data recieved matchs what was sent (tx is looped back to tx)
         */
        for (i = 0; i < TEST_SIZE; i++) {
                printf("tx: %d\trx: %d\n", tx_proxy_interface_p->buffer[i], rx_proxy_interface_p->buffer[i]);
        }

        /* Unmap the proxy channel interface memory and close the device files before leaving
         */
        munmap(tx_proxy_interface_p, sizeof(struct dma_proxy_channel_interface));
        munmap(rx_proxy_interface_p, sizeof(struct dma_proxy_channel_interface));

        close(tx_proxy_fd);
        close(rx_proxy_fd);
        return 0;
}

Here is the output to prove what I am talking about.

root@dma_no_scatter_gather_unaligned_xfers:~# insmod /lib/modules/4.9.0-xilinx-v2017.3/extra/dma-proxy.ko
dma_proxy: loading out-of-tree module taints kernel.
probe
Getting receive channel
creating a new classlocal class_p: ef3df500
Allocating cached memory at 0xEE832000
Getting transfer channel
local class_p: ef3df500Allocating cached memory at 0xEF192800
root@dma_no_scatter_gather_unaligned_xfers:~# /usr/bin/dma-proxy-test
DMA proxy test
mmap
mmap
tx_proxy_interface_p: 0xb6f80000
ioctl kernel space_p: 0xb6f82000

tx ioctl user space
channel_interface location: ef192800
dma_proxy_channel->length: 0
length = 0 tx direction
xilinx-vdma 40400000.dma: Channel ef101110 has errors 40, cdr 0 tdr 0
no error
^C
root@dma_no_scatter_gather_unaligned_xfers:~# /usr/bin/dma-proxy-test
DMA proxy test
mmap
mmap
tx_proxy_interface_p: 0xb6f7a000
ioctl kernel space_p: 0xb6f7c000

tx ioctl user space
channel_interface location: ef192800
dma_proxy_channel->length: 0
length = 0 tx direction
xilinx-vdma 40400000.dma: Channel ef101110 has errors 20, cdr 0 tdr 0
no error
rx ioctl user space
ioctl kernel space
channel_interface location: ee832000
dma_proxy_channel->length: 1024
length = 1024 rx direction
xilinx-vdma 40400000.dma: Cannot start channel ef101010: 1

If anyone here is enough of a kernel expert to figure out what went wrong when I was

1. Setting up the DMA channels from the device tree with the platform probe

2. Setting up the character devices for the DMA Channels

3. Creating and registering the dma_proxy device class, OR

4. Creating the device nodes

 

Such that mmap doesn't work on only the second character device/device node, then you are a beast because I've been trying to debug this one for days.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

0 Kudos
1 Reply
Visitor roli_s
Visitor
217 Views
Registered: ‎02-03-2016

Re: Mmapping Succeeds with AXI DMA-Proxy Driver but Memory in Kernel not Changed from User Space

Hi,

I am working on similar dma_proxy driver. But in my case I am having trouble adding device node for the proxy driver, the dtc is giving syntax error during petalinux build. I can see the axi_dma_0 device populated in the pl.dtsi. I have edited the system-user.dtsi to add dma_proxy. Attached is the system-user.dtsi. Could you please point out the error. I am using petalinux 2018.3 and following steps listed here -

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842418/Linux+DMA+From+User+Space

Thanks!

0 Kudos