cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Adventurer
Adventurer
2,372 Views
Registered: ‎05-30-2018

Use PL as Userspace IO Device

Hi !

Board: ZCU102 Evaluation Board

Development Tool: Vivado / SDx.

On-Board OS: Ubuntu Desktop 2018.3

My top-level hardware function is simply a function that takes an image as input and returns a number of values inferred from this image. In other words, the input is a 2D array (can be 1D as well), and the output is an array of floats. Input and output are passed to the function as arguments, no value is to be returned.

I would like to use the PL as a device (i.e. Userspace IO) in Ubuntu Desktop, where I can open the device using C++ program, write the image to it, read the results from it and close it.

After building the project using SDx, Vivado gave me the following base address for the device:axi.png

However, this step of system integration on Vivado is done automatically by SDx. In the resulting diagram, the interrupt pin of the PL was not connected:Interrupt.png

 

Nevertheless, Interrupt pins are configured in the PS-PL Interface:PSPLInterface.png

Before compiling the Linux kernel using Petalinux Tool, I added the following device to system-user.dtsi:

/{
    chosen {
        bootargs = "earlycon clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait cma=256M uio_pdrv_genirq.of_id=generic-uio";
    stdout-path = "serial0:115200n8";
    };

    MyPLDevice@a0000000 {
        compatible = "xlnx,generic-uio","uio_pdrv_genirq";
        reg = <0x0 0xa0000000 0x0 0x10000>;
        interrupts = <0x0 0x59 0x1>;
        interrupt-parent = <0x4>;
    };
};

I got the value 0x59 by substracting (32) from IRQ number (121) which gives 89 in decimal or 0x59 in Hexadecimal. I found the IRQ number in PG201

Furthermore, I configured the Userspace I/O Platform Driver ... and Userspace Platform Driver ... as <*> in Petalinux kernel configuration.

After booting, I can find /dev/uio0 but looks like it's used for axi_pmon (i.e. AXI Performance Monitoring).

I tried to search for my device by typing:

find / -iname "*MyPLDevice*"

and I found out that it only exists in the following locations:

/sys/devices/platform/a0000000.MyPLDevice
/sys/firmware/devicetree/base/MyPLDevice@a0000000
/sys/bus/platform/devices/a0000000.MyPLDevice

Now my questions are:

1- Is my module in device tree correctly written?

2- Do I have to connect the interrupt pin to some other pin in Vivado?

3- Do I have to configure the interface of the top-level function using #pragma HLS interface ... ?

4- What is the correct and complete way to have the PL as a device? 

Several tutorials are avaiable in the Internet but are either outdated or incomplete.

5- Is there an easier way to do this using SDx since it generates a FIT image? Ubuntu Desktop provides flexible software programming without cross-compilation hence it is desired.

 

Thank you very much in advance !

0 Kudos
23 Replies
Highlighted
Moderator
Moderator
2,339 Views
Registered: ‎09-12-2007

Re: Use PL as Userspace IO Device

0 Kudos
Highlighted
Adventurer
Adventurer
2,315 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi @stephenm 

Thank you for your reply.

I tried this solution already, but the device tree compilation throws an error (I guess because of &&axi_gpio_0, shouldn't it be & instead of && ? )

Also this is related to GPIO, and not PL as UIO right?

Also the link shows that device /dev/uio0 was detected. Isn't uio0 used for axi performance monitoring as shown in this wiki page?

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842046/APM

I really appreciate any answer to any of my questions in the main post.

Thanks !

0 Kudos
Highlighted
Explorer
Explorer
2,309 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

Hi @rashedkoutayni 

You can check your interrupts by issuing following command. 

cat /proc/interrupts

You should see your device appearing, and the second number is the count of the interrupt trigger. ( dma is not relevant here, I just copy - pasted some random line)

50:          0          0          0          0     GICv2 154 Level     fd4c0000.dma

In user space you can poll the /dev/uio0 virtual file for any interrupt change. ( https://linux.die.net/man/3/poll ) I might be wrong but it was my understanding the generic driver does not distinguish separate interrupts. I think it triggers on every bound IRQ.

You can in fact use the generic driver on every device once you bind it to a device with the "compatible" keyword. I must say in all honesty that I left the generic driver path as it seemed not to work as great as expected. But I was trying to access a DMA not a simple interrupt. 

 

@stephenm  Please stop posing incomplete wiki links as an answer. You can't seem to get a serious answer from Xilinx these days, let alone some assistance.  It looks like you need to wait a day or so to get these generic answers from you guys. ( @rashedkoutayni  Sorry to hijack your thread )   

 

 

0 Kudos
Highlighted
Adventurer
Adventurer
2,298 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi @deville 

The thing is I'm not finding my device in the interrupt list, nor in /dev/

I find Performance Monitor as uio0 which has nothing to do with my dvice.

Does this mean that the kernel module does not exist? or not inserted? what to do in this case ?

in /lib/modules/ I don't find it neither.

I am also not tied to the generic driver path, any method in userspace would in principle work.

0 Kudos
Highlighted
Explorer
Explorer
2,289 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

@rashedkoutayni 

Did you load the driver? The kernel modules are set to a module so you need to load them manually after the kernel boots. Change this to =y to include it directly into the kernel. 

CONFIG_UIO_PDRV_GENIRQ=m
CONFIG_UIO_DMEM_GENIRQ=m

Edit: 

 

OK I misread your previous post. -> The drivers are compiled as a .ko module and should still be placed in your build folders. You have to add them manually into the /lib/modules/"KERNELVERSION" ( example: /lib/modules/4.14.0/ )

sudo mkdir -p /lib/modules/$(uname -r)
sudo mv uio_pdrv_genirq.ko /lib/modules/$(uname -r)
sudo depmod sudo modprobe uio_pdrv_genirq dmesg
0 Kudos
Highlighted
Adventurer
Adventurer
2,284 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

@deville 

As mentioned above, these two options are alreadyset as  <*> means y during petalinux process.

0 Kudos
Highlighted
Explorer
Explorer
2,281 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

One thing i've noticed:

compatible = "xlnx,generic-uio","uio_pdrv_genirq";

should work with only 

compatible = "generic-uio";

You've done the translation of this in the kernel command line. 

 

I would load the module manually so you can seen the differences before and after, and you can do a "lsmod" to be sure is was loaded properly.

 

Deville

 

0 Kudos
Highlighted
Adventurer
Adventurer
2,266 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi again,

I changed compatible to be like:

compatible = "generic-uio";

I also found uio_pdrv_genirq.ko in:

/lib/modules/4.14.0-xilinx-v2018.3/kernel/drivers/uio/uio_pdrv_genirq.ko

The first time I did modprobe I got segmentation fault, but when I try it again it simply hangs until I send Ctrl+C.

 I am not really sure what the mistake is.

0 Kudos
Highlighted
Explorer
Explorer
2,226 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

Hi @rashedkoutayni 

Sounds you've got to take a step back and start all over.  I've experienced some segmentation faults when I tried to double load a certain driver so it might suggest you've already loaded the driver. 

I just tried to load it myself and there was no kernel message popping up. You can add some kernel messages inside the driver samewhere in the init function (uio_pdrv_genirq_probe). By adding some print messages you can get more clues where things are getting wrong. 

example

printk (KERN_INFO "uio_prdv_genirq init() \n");

After recompliling the kernel ( incl the driver) and reloading the driver you can then search for the pattern in the kernel output log

dmesg | grep uio_pdrv_genirq

Good luck

Deville

 

0 Kudos
Highlighted
Adventurer
Adventurer
2,195 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

I think this should work without changing any line in the kernel / driver source code.

I am just following the way recommended by tutorials and wikis (as explained in main post) but it's not working !

Is it because I am using Ubuntu Desktop on the board? 

If you take a look at this nice tutorial:

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18841937/Zynq+UltraScale+MPSoC+Ubuntu+part+2+-+Building+and+Running+the+Ubuntu+Desktop+From+Sources

The instructions tell you to disable PCI Support and to choose SD CARD as root file system type in Image Packaging Configuration.

Does this have any impact on the ability of having PL as UIO device?

Look at this weird post:

https://forums.xilinx.com/t5/Embedded-Processor-System-Design/Petalinux-2014-4-UIO-drivers-do-not-show-up/td-p/707238

and this:

https://forums.xilinx.com/t5/Embedded-Linux/UIO-method-for-custom-slave-and-IPs-for-petalinux-2018-1/td-p/945207

0 Kudos
Highlighted
Explorer
Explorer
2,149 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

Hi @rashedkoutayni 

It would not matter which root file system you choose neither the boot medium. The PL side will be translated inside your device tree where you point out the addresses and configurations to which registers of the bitstream you can use for a certain device. 

The Wiki pages, forum and the example drivers are a bit neglected by Xilinx. I'm also hitting several walls while trying to implement Xilinx products into our own. 

Concerning your driver:  I still think your best chance is to add some printk's in your device so you have some kind of idea what is going on. Debuggers like gdb are kept out of the kernel to keep the code clean, so the prints are your best friend.  For anything else I'm also in the dark as I have no access to your setup. 

This tutorial should get you started  http://derekmolloy.ie/writing-a-linux-kernel-module-part-1-introduction/

 

Deville

Edit: added a link

0 Kudos
Highlighted
Adventurer
Adventurer
2,086 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi again,

Thanks for your replies.

Now I'm pretty sure this issue has to do with Ubuntu Desktop and specifically with initramfs/initrd.

Whenever I disable it from Petalinux kernel config as follows:

[ ] Initial RAM file system and RAM disk (initramfs/initrd) support

I don't see my device in /dev/ nor an interrupt for it.

However, disabling the initramfs/initrd support is needed for Ubuntu booting. When it is enabled, only basic Linux will boot and not Ubuntu.

This is what I observed so far. Any suggestions or clarifications please?

Thanks !

0 Kudos
Highlighted
Adventurer
Adventurer
2,066 Views
Registered: ‎08-23-2018

Re: Use PL as Userspace IO Device

I looked at that Ubuntu Build page (https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18841937/Zynq+UltraScale+MPSoC+Ubuntu+part+2+-+Building+and+Running+the+Ubuntu+Desktop+From+Sources)

The funny thing about it is that it does not package a FPGA bit-file into the BOOT.BIN file, so unless you are using the FPGA Manager or loading FPGA with JTAG, you will not have any active FPGA running in the PL.

ref. petalinux-package --boot --fsbl images/linux/zynqmp_fsbl.elf --u-boot images/linux/u-boot.elf

(Assuming you are not JTAG loading or using FPGA Manager in Linux)

If you have a FPGA bit file ready (exported from Vivado), you may want to try adding it to the BOOT.BIN. This allow the FSBL to pre-load it for you.

For example:

petalinux-package --boot --format BIN --fsbl images/linux/zynqmp_fsbl.elf --u-boot images/linux/u-boot.elf --pmufw images/linux/pmufw.elf --fpga <path-to-your-fpga.bit-file>

See also UG1144 (v2018.3) for a good reference to creating PetaLinux packages, including for SD Cards.

0 Kudos
Highlighted
Adventurer
Adventurer
2,054 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi @rwillis ,

No I'm not using JTAG, but I am already encapsulating the bitstream in BOOT.BIN, and it's not a problem of having the PL in principle. The thing is I am unable to deal with PL as UIO Device.

0 Kudos
Highlighted
Explorer
Explorer
2,048 Views
Registered: ‎09-28-2018

Re: Use PL as Userspace IO Device

hi @rashedkoutayni 

 

I don't understand what you mean by disabling the initramsfs option. ->  http://www.linuxfromscratch.org/blfs/view/cvs/postlfs/initramfs.html

Initial RAM file system and RAM disk (initramfs/initrd) support

Are you saying the standard Petalinux root file system is showing the /dev/uiox entry and Ubuntu can't?  Can you clarify a bit more?

0 Kudos
Highlighted
Adventurer
Adventurer
2,042 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi @deville  !

I followed this tutorial here:

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18841937/Zynq+UltraScale+MPSoC+Ubuntu+part+2+-+Building+and+Running+the+Ubuntu+Desktop+From+Sources

Look at the Configuring the kernel section where you have to disable the support for initramfs/initrd.

Exactly, the standard Petalinux root file system is showing the /dev/uiox entry while Ubuntu isn't !

A bit more clarification:

If you want to boot Ubuntu on ZCU102 you need to configure Petalinux to load the file system that belongs to Ubuntu and exists on the second SD Card partition mmcblk0p2 (my understanding).

If I don't configure it to load fs from mmcblk0p2 (i.e. normal Petalinux boot) I see my /dev/uiox, while if I load fs from mmcblk0p2 (i.e. Ubuntu Desktop boot) I don't see /dev/uiox anymore.

0 Kudos
Highlighted
Adventurer
Adventurer
2,024 Views
Registered: ‎08-23-2018

Re: Use PL as Userspace IO Device

Reason why article is asking to place filesystem into 2nd partitiion:
I imagine the Ubuntu filesystem is quite large as it is a desktop distro, so having the whole thin in RAM is undesirable.

 

"/dev/uiox is in the initrd (RAM-filesystem) but not in the filesystem on partition 2"
Sounds either like the worng filesystem is being written onto the second partition, or that linux is not "pointed" to the correct partition in its boot args.

 

AXI FPGA IP Blocks:
Have you tested memory access to your device? AXI maps it to an address block (if it is not AXI-stream that is, assuming not as I don't think UIO works with AXI-Streams) within the 40-bit address map shared between the PS and the PL.. You can use a test program like devmem to directly read and write into your AXI-based IP block in the FPGA. This confirms that it is accessable from Linux and can be ruled out.

 

0 Kudos
Highlighted
Adventurer
Adventurer
2,014 Views
Registered: ‎08-23-2018

Re: Use PL as Userspace IO Device

Just as an aside, I also just looked at your system-user.dtsi and it does not looks the same as the one that I use. My axi devices work sucessfully.

Here is an example of the format I use:

---

/include/ "system-conf.dtsi"
/ {
};
&amba_pl {
    ddr4_0: ddr4@b0000000 {
        compatible = "generic-uio";
        reg = <0x0 0xb0000000 0x0 0x10000000>;
    };
    my_axi_block: my_axi_block@a0000000 {
        compatible = "generic-uio";
        reg = <0x0 0xa0000000 0x0 0x10000000>;
    };
};

---

As you can see, the new Device tree nodes are added into the "amba_pl" section.

 

This however does not answer why you cannot see the uio devices when you mount the Ubuntu filesystem...

 

0 Kudos
Highlighted
Adventurer
Adventurer
2,011 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

Hi @rwillis 

That's a very important point !

"/dev/uiox is in the initrd (RAM-filesystem) but not in the filesystem on partition 2"

The question is, does a device (like /dev/uiox) belong to the kernel or to the file system?

As you can see in the tutorial of Ubuntu Desktop mentioned before, you don't touch the Ubuntu file system.

Rather, you simply 'burn' (i.e. dd ) the prebuilt file system to the SD card, resulting in two partitions:

  • The BOOT partition, which contains the FIT image : image.ub and the boot wrapper: BOOT.BIN
  • The ROOT partition, which contains the root file system.

Any customizations or Petalinux compilations will have to be replaced with the files in the BOOT partition and not with the ROOT partition. So basically no changes are performed with file system (as far as I understand).

However, I don't think that the correct partition is not pointed to correctly, otherwise Ubuntu wouldn't boot.

I guess we're really too close to the solution !

0 Kudos
Highlighted
Adventurer
Adventurer
2,007 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

@rwillis 

Is this system-user.dtsi what you use for Ubuntu Desktop?

My current system-user.dtsi looks as follows:

/include/ "system-conf.dtsi"
/ {
        chosen {
                bootargs = "earlycon clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait cma=256M uio_pdrv_genirq.of_id=generic-uio";
                stdout-path = "serial0:115200n8";
        };

};

&hand_network_1{
        compatible = "generic-uio";
        interrupts = <0x0 0x59 0x1>;
        interrupt-parent = <0x4>;

};

Since my device (hand_network_1) is already defined in pl.dtsi, I only add the driver and the interrupt things there.

0 Kudos
Highlighted
Adventurer
Adventurer
1,999 Views
Registered: ‎05-30-2018

Re: Use PL as Userspace IO Device

0 Kudos
Highlighted
Adventurer
Adventurer
1,996 Views
Registered: ‎08-23-2018

Re: Use PL as Userspace IO Device

RE: /dev device mount points.
I remember "touching" devices to create a starting point for devices in a file system but that was a long time ago (decades!) in a much older version of Linux. It is quite possible now that nothing has to be done in the filesystem anymore (like sysfs).

If you are not touching the Ubuntu filesystem itself then I guess you are compiling the drivers straight into the kernel, so there should be no problems with the kernel.

That leaves only the drivers themselves failing to boot up. I believe they will create the device file as well, located in /dev/

Regarding my system-user.dtsi, it basically ignores the pl.dtsi and just adds (or over-rides) the whole node in this file. This is the why I did the addition in the past, starting at petalinux 2018.2,  before I knew a lot about how petalinux builds the pl.dtsi from the Vivado BSP.

This is for "native" Petalinux filesystems, not for Ubuntu. I have not tried to run a Ubuntu filesystem.

---

You could try and confirm that you can read/write the FPGA AXI block, using the devmem command. You will likely have to add that into Ubuntu filesystem though... It should be being compiled in the Petalinux filesystem.

 

0 Kudos
Highlighted
Adventurer
Adventurer
1,994 Views
Registered: ‎08-23-2018

Re: Use PL as Userspace IO Device

Also, in your system-user.dtsi, you still need to specify that the node is within the 'amba_pl' block otherwise the dt compiler will not be able to modify the existing node in the pl correctly. See my example above.

 

0 Kudos