UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
942 Views
Registered: ‎02-07-2008

AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution

Hi,

 

I have a design for the KCU105 using AXI PCIe Gen3 Subsystem IP in root port mode. This design worked well under PetaLinux 2017.4 but since upgrading to PetaLinux 2018.2, the PCIe bridge is no longer able to assign BARs to an end-point. In my case, the end-point is an NVMe PCIe SSD.

 

I have gone through the migration tips in UG1144 and this useful forum post on the same subject.

 

When I run the same Vivado 2018.2 design through the older PetaLinux 2017.4 tools, it works, so I suspect that the problem is with the PetaLinux 2018.2 tools and not the Vivado project or tools.

 

Here is the part of the boot log that relates to the AXI PCIe Gen3 IP (note: PCIe debug is enabled):

 

xilinx-pcie 20000000.axi-pcie: PCIe Link is UP
OF: PCI: host bridge /amba_pl/axi-pcie@20000000 ranges:
OF: PCI:   MEM 0x40000000..0x7fffffff -> 0x40000000
xilinx-pcie 20000000.axi-pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff pref]
pci_bus 0000:00: scanning bus
pci 0000:00:00.0: [10ee:8134] type 01 class 0x060400
pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
pci 0000:00:00.0: calling pcibios_fixup_resources+0x0/0x8
pci_bus 0000:00: fixups for bus
pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 0
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 1
pci_bus 0000:01: scanning bus
pci 0000:01:00.0: [144d:a808] type 00 class 0x010802
pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
pci 0000:01:00.0: calling pcibios_fixup_resources+0x0/0x8
pci_bus 0000:01: fixups for bus
pci_bus 0000:01: bus scan returning with max=01
pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
pci_bus 0000:00: bus scan returning with max=01
pci 0000:00:00.0: BAR 8: no space for [mem size 0x00100000]
pci 0000:00:00.0: BAR 8: failed to assign [mem size 0x00100000]
pci 0000:00:00.0: BAR 6: assigned [mem 0x40000000-0x400007ff pref]
pci 0000:01:00.0: BAR 0: no space for [mem size 0x00004000 64bit]
pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00004000 64bit]
pci 0000:00:00.0: PCI bridge to [bus 01]

As you can see in the log, the Vivado design has 1GB to assign the BARs to, so there is definitely enough space for those BARs.

 

If anyone has experienced this, I would appreciate some help.

 

Thanks.

 

Jeff

0 Kudos
1 Solution

Accepted Solutions
935 Views
Registered: ‎02-07-2008

Re: AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution

Hi,

 

I've managed to find the problem and get this working. Here is what I did for anyone who has the same problem. I compared the PetaLinux project from version 2017.4 with the one from version 2018.2. As there were no changes in the actual Vivado design, I wasn't expecting any big changes in the PetaLinux project files. Instead, I found that the automatically generated device tree contained in pl.dtsi contained a difference in the axi_pcie3 section.

 

The pl.dtsi file is found here:

 

<petalinux-project>/components/plnx_workspace/device-tree/device-tree/pl.dtsi

The axi_pcie3 section from version 2017.4 was like this:

 

axi_pcie3_0: axi-pcie@20000000 {
    ...
    ranges = <0x02000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
    ...
};

The axi_pcie3 section from version 2018.2 is like this:

axi_pcie3_0: axi-pcie@20000000 {
    ...
    ranges = <0x43000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
    ...
};

Notice the first element in the ranges parameter has changed, even though the hardware design did not change at all. When I changed this value back to what it was in version 2017.4, the BARs are all assigned during Linux boot.

 

The way I changed the value was by adding the following code to the system-user.dtsi file:

# project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi

&axi_pcie3_0 {
    ranges = <0x02000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
};

Here is the boot log after this change:

xilinx-pcie 20000000.axi-pcie: PCIe Link is UP
OF: PCI: host bridge /amba_pl/axi-pcie@20000000 ranges:
OF: PCI:   MEM 0x40000000..0x7fffffff -> 0x40000000
xilinx-pcie 20000000.axi-pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
pci 0000:00:00.0: [10ee:8134] type 01 class 0x060400
pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:00.0: [144d:a808] type 00 class 0x010802
pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
pci 0000:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff]
pci 0000:00:00.0: BAR 6: assigned [mem 0x40100000-0x401007ff pref]
pci 0000:01:00.0: BAR 0: assigned [mem 0x40000000-0x40003fff 64bit]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x40000000-0x400fffff]
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
console [ttyS0] disabled
14a00000.serial: ttyS0 at MMIO 0x14a01000 (irq = 4, base_baud = 6250000) is a 16550A
console [ttyS0] enabled
brd: module loaded
nvme nvme0: pci function 0000:01:00.0
pci 0000:00:00.0: enabling device (0000 -> 0002)
nvme 0000:01:00.0: enabling device (0000 -> 0002)

Jeff

 

 

View solution in original post

4 Replies
Xilinx Employee
Xilinx Employee
902 Views
Registered: ‎08-06-2008

Re: AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution

Hi Jeffrey,

 

AXI PCIe Gen3 support as root port with Petalinux has been deprecated. The recommendation is to use XDMA in Bridge mode. 

Please refer to: https://www.xilinx.com/support/answers/71210.html for additional information. 

 

Thanks. 

892 Views
Registered: ‎02-07-2008

Re: AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution
Thanks for that reply. Maybe you have posted the wrong link but that answer record seems only to apply to Zynq US+ and Series 7 devices. I am using Kintex Ultrascale.
0 Kudos
887 Views
Registered: ‎02-07-2008

Re: AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution

@deepeshm

 

After looking into your suggestion, I read that the XDMA in bridge mode only supports Ultrascale+ devices. I also tried it in Vivado and it doesn't allow me to configure the XDMA in Bridge mode on the Kintex Ultrascale - only in DMA mode, end-point. According to PG194 page 6, I seems that I am already using the correct IP for root port mode on the Kintex Ultrascale.

 

So I'm a bit confused. Is Xilinx going to develop support for XDMA in Bridge mode on Ultrascale devices? Or is PetaLinux support going to continue for the AXI Bridge for PCIe Gen3 Subsystem for Ultrascale devices?

 

Thanks in advance.

 

Jeff

 

0 Kudos
936 Views
Registered: ‎02-07-2008

Re: AXI PCIe Gen3 Subsystem in root port mode not assigning BARs in PetaLinux 2018.2

Jump to solution

Hi,

 

I've managed to find the problem and get this working. Here is what I did for anyone who has the same problem. I compared the PetaLinux project from version 2017.4 with the one from version 2018.2. As there were no changes in the actual Vivado design, I wasn't expecting any big changes in the PetaLinux project files. Instead, I found that the automatically generated device tree contained in pl.dtsi contained a difference in the axi_pcie3 section.

 

The pl.dtsi file is found here:

 

<petalinux-project>/components/plnx_workspace/device-tree/device-tree/pl.dtsi

The axi_pcie3 section from version 2017.4 was like this:

 

axi_pcie3_0: axi-pcie@20000000 {
    ...
    ranges = <0x02000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
    ...
};

The axi_pcie3 section from version 2018.2 is like this:

axi_pcie3_0: axi-pcie@20000000 {
    ...
    ranges = <0x43000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
    ...
};

Notice the first element in the ranges parameter has changed, even though the hardware design did not change at all. When I changed this value back to what it was in version 2017.4, the BARs are all assigned during Linux boot.

 

The way I changed the value was by adding the following code to the system-user.dtsi file:

# project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi

&axi_pcie3_0 {
    ranges = <0x02000000 0x00000000 0x0000000040000000 0x0000000040000000 0x00000000 0x40000000>;
};

Here is the boot log after this change:

xilinx-pcie 20000000.axi-pcie: PCIe Link is UP
OF: PCI: host bridge /amba_pl/axi-pcie@20000000 ranges:
OF: PCI:   MEM 0x40000000..0x7fffffff -> 0x40000000
xilinx-pcie 20000000.axi-pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
pci 0000:00:00.0: [10ee:8134] type 01 class 0x060400
pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:00.0: [144d:a808] type 00 class 0x010802
pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
pci 0000:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff]
pci 0000:00:00.0: BAR 6: assigned [mem 0x40100000-0x401007ff pref]
pci 0000:01:00.0: BAR 0: assigned [mem 0x40000000-0x40003fff 64bit]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x40000000-0x400fffff]
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
console [ttyS0] disabled
14a00000.serial: ttyS0 at MMIO 0x14a01000 (irq = 4, base_baud = 6250000) is a 16550A
console [ttyS0] enabled
brd: module loaded
nvme nvme0: pci function 0000:01:00.0
pci 0000:00:00.0: enabling device (0000 -> 0002)
nvme 0000:01:00.0: enabling device (0000 -> 0002)

Jeff

 

 

View solution in original post