UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer koolscooby
Observer
3,492 Views
Registered: ‎10-15-2016

Zynq UltraScale+ PCIe Root Port Lessons Learned

I wanted to add a couple of notes to help out others in case they have bugs/issues bringing up PCIe as I've spent a fair bit of time reaching out into the dark and debugging.

 

Problem: End point enumerates, but linux throws errors and the endpoint driver fails to communicate with the end point device

I was receiving linux kernel PCIe error relating to address space allocation before I made the change:

[ 2.166104] pci 0000:01:00.0: BAR 0: no space for [mem size 0x00100000 64bit pref]
[ 2.173622] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit pref]
[ 2.181493] pci 0000:01:00.0: BAR 2: no space for [mem size 0x00100000 64bit pref]
[ 2.189018] pci 0000:01:00.0: BAR 2: failed to assign [mem size 0x00100000 64bit pref]


In Vivado, the default settings of the Processing Subsystem block were incorrect for Root Port mode and I actually had to make a few changes to set them based on the screenshot on this Wiki. Changing this configuration modifies the FSBL initialization code with respect to PCIe - so be sure to export hardware and rebuild your FSBL.

 

Selection_006.png
Problem: Endpoint enumerates only on ~50% of boots.

My custom hardware has a Si5341 clock generator that is configured in in the FSBL (based on the Xilinx provided template). The clock generator config happens after psu_init is called. Since the PCIe clock is not stable (or presumably there is a race condition), I needed to manually reset the PCIe block.

 

You can try this from linux command line:

echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/remove
devmem 0xFD1A0100
devmem 0xFD1A0100 w 0x28000
devmem 0xFD1A0100 w 0x08000
echo 1 > /sys/bus/pci/rescan

You can make this permanent in the FSBL code after the clocks are stable and running, for example:

// Reset PCIe controller
RegVal = XFsbl_In32(0xFD1A0100);
XFsbl_Out32(0xFD1A0100, RegVal | 0x00020000);
usleep(1000);
// Run PCIe controller
RegVal = XFsbl_In32(0xFD1A0100);
XFsbl_Out32(0xFD1A0100, RegVal & 0xFFFDFFFF);
usleep(1000);


Hope these nuggets help someone else!

Tags (1)
1 Reply
Newbie ch.powe
Newbie
82 Views
Registered: ‎09-25-2018

Re: Zynq UltraScale+ PCIe Root Port Lessons Learned

I spent a fair amount of time debugging the first problem described above. Who knows how much longer I'd be digging without finding this post. Thank you!
0 Kudos