cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
emeaton
Observer
Observer
585 Views
Registered: ‎10-21-2018

PCIe switch/enclosure compatibility

I'm trying to use the U250 card with a One Stop Systems PCIe enclosure and Ubuntu 20. The enclosure works with a USB add-in card, but the Alveo U250 has issues. It enumerates in the lspci list, but this is the result of actually trying to do something with it:

$ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan

ERROR: Failed to detect feature ROM.
ERROR: Platform name not found
ERROR: Platform name not found
Shell on FPGA is unknown
Card [0000:09:00.0]
Card type:
Flash type: SPI
Flashable partition running on FPGA:
,[SC=INACTIVE]
Flashable partitions installed in system: (None)

I was able to update the card firmware to the latest version using the new Ubuntu 20, by connecting the U250 to the mother board with a PCIe riser cable. The card validated with the expected warning that the slot is 8 lanes, since no 16-lane slot is available in the machine. However it doesn't work through the enclosure which is supposed to support up to two cards with PCIe 3.0 x16 bandwidth.

I contacted the OSS support and they told me to ask whether the Alveo U250 can work through PCIe switches, the enclosure uses 3 apparently.

Tags (2)
0 Kudos
Reply
6 Replies
emeryw
Xilinx Employee
Xilinx Employee
487 Views
Registered: ‎12-06-2019

Hi @emeaton ,

Thanks for reaching out. Could you please post the outputs of:

sudo lspci -vd 10ee:
sudo xbutil query
sudo xbmgmt flash --scan

 

Is there a particular reason for using the external PCIe enclosure vs. the physical slots on the motherboard? As there is additional hardware inbetween the card and the PCIe slot, I'd be curious of how the device path appears to the OS. I'd also be curious if the additional hardware is compatible with the PCIe capabilities used by the U250. As the card validates when used with the internal PCIe slot, we know the card and the host system are properly configured. For reference, here is the lspci output from the U250 in our lab:

$ sudo lspci -vd 10ee:
af:00.0 Processing accelerators: Xilinx Corporation Device 5004
        Subsystem: Xilinx Corporation Device 000e
        Flags: bus master, fast devsel, latency 0, NUMA node 1
        Memory at 385ff2000000 (64-bit, prefetchable) [size=32M]
        Memory at 385ff4040000 (64-bit, prefetchable) [size=256K]
        Capabilities: [40] Power Management version 3
        Capabilities: [60] MSI-X: Enable- Count=32 Masked-
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [1c0] #19
        Capabilities: [400] Access Control Services
        Capabilities: [410] #15
        Capabilities: [480] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
        Kernel driver in use: xclmgmt
        Kernel modules: xclmgmt

af:00.1 Processing accelerators: Xilinx Corporation Device 5005
        Subsystem: Xilinx Corporation Device 000e
        Flags: bus master, fast devsel, latency 0, IRQ 77, NUMA node 1
        Memory at 385ff0000000 (64-bit, prefetchable) [size=32M]
        Memory at 385ff4000000 (64-bit, prefetchable) [size=256K]
        Memory at 385fe0000000 (64-bit, prefetchable) [size=256M]
        Capabilities: [40] Power Management version 3
        Capabilities: [60] MSI-X: Enable- Count=32 Masked-
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [400] Access Control Services
        Capabilities: [410] #15
        Capabilities: [480] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
        Kernel driver in use: xocl
        Kernel modules: xocl

 

Best,
-Emery
-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------

0 Kudos
Reply
emeaton
Observer
Observer
472 Views
Registered: ‎10-21-2018

$ sudo lspci -vd 10ee:
09:00.0 Processing accelerators: Xilinx Corporation Device 5004
	Subsystem: Xilinx Corporation Device 000e
	Physical Slot: 64-1
	Flags: bus master, fast devsel, latency 0, NUMA node 0
	Memory at f392000000 (64-bit, prefetchable) [size=32M]
	Memory at f394040000 (64-bit, prefetchable) [size=256K]
	Capabilities: [40] Power Management version 3
	Capabilities: [60] MSI-X: Enable- Count=32 Masked-
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [1c0] Secondary PCI Express
	Capabilities: [400] Access Control Services
	Capabilities: [410] Resizable BAR <?>
	Capabilities: [480] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
	Kernel driver in use: xclmgmt
	Kernel modules: xclmgmt

09:00.1 Processing accelerators: Xilinx Corporation Device 5005
	Subsystem: Xilinx Corporation Device 000e
	Physical Slot: 64-1
	Flags: bus master, fast devsel, latency 0, IRQ 37, NUMA node 0
	Memory at f390000000 (64-bit, prefetchable) [size=32M]
	Memory at f394000000 (64-bit, prefetchable) [size=256K]
	Memory at f380000000 (64-bit, prefetchable) [size=256M]
	Capabilities: [40] Power Management version 3
	Capabilities: [60] MSI-X: Enable- Count=32 Masked-
	Capabilities: [70] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [400] Access Control Services
	Capabilities: [410] Resizable BAR <?>
	Capabilities: [480] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
	Kernel driver in use: xocl
	Kernel modules: xocl
$ sudo /opt/xilinx/xrt/bin/xbutil query
INFO: Found total 1 card(s), 0 are usable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
System Configuration
OS name:	Linux
Release:	5.4.0-62-generic
Version:	#70-Ubuntu SMP Tue Jan 12 12:45:47 UTC 2021
Machine:	x86_64
Model:		MS-7C59
CPU cores:	128
Memory:		257692 MB
Glibc:		2.31
Distribution:	Ubuntu 20.04.1 LTS
Now:		Thu Jan 21 19:31:12 2021 GMT
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
XRT Information
Version:	2.8.743
Git Hash:	77d5484b5c4daa691a7f78235053fb036829b1e9
Git Branch:	2020.2
Build Date:	2020-11-16 00:18:20
XOCL:		2.8.743,77d5484b5c4daa691a7f78235053fb036829b1e9
XCLMGMT:	2.8.743,77d5484b5c4daa691a7f78235053fb036829b1e9
ERROR: Card index 0 is out of range
$ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan
ERROR: Failed to detect feature ROM.
ERROR: Platform name not found
ERROR: Platform name not found
Shell on FPGA is unknown
Card [0000:09:00.0]
    Card type:		
    Flash type:		SPI
    Flashable partition running on FPGA:
        ,[SC=INACTIVE]
    Flashable partitions installed in system:	(None)

There is no space in the machine to mount a 2-slot PCIe card. I could rebuild the entire workstation and its custom cooling system in a larger case, but would rather not.

Here is the device path from the AMD Threadripper PCIe root.

$ sudo lspci -tv
# snip
\-[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex
             +-00.2  Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU
             +-01.0  Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
# snip
             +-01.3-[03-0a]--+-00.0-[04-0a]----08.0-[05-0a]----00.0-[06-0a]----00.0-[07-0a]----00.0-[08-0a]--+-00.0-[09]--+-00.0  Xilinx Corporation Device 5004
             |               |                                                                               |            \-00.1  Xilinx Corporation Device 5005
             |               |                                                                               \-01.0-[0a]--
             |               +-00.1  PLX Technology, Inc. Device 87d0
             |               +-00.2  PLX Technology, Inc. Device 87d0
             |               +-00.3  PLX Technology, Inc. Device 87d0
             |               \-00.4  PLX Technology, Inc. Device 87d0

dmesg output from xocl/xclmgmt:

Spoiler
[  +0.006693] xocl: loading out-of-tree module taints kernel.
[  +0.000378] xocl: module verification failed: signature and/or required key missing - tainting kernel
[  +0.012241] xclmgmt init()
[  +0.000756] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xclmgmt_probe: Driver: 2.8.743,77d5484b5c4daa691a7f78235053fb036829b1e9
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xclmgmt_probe: probe(pdev = 0x000000002311ada5, pci_id = 0x000000007f799c89)
[  +0.000611] xclmgmt 0000:09:00.0: enabling device (0000 -> 0002)
[  +0.003097] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xclmgmt_probe: minimum initialization done
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev dma_msix.m multi 0 level 0
[  +0.000713] xclmgmt 0000:09:00.0: dma_msix.m.5242880 ffff99cc58828410 mgmt_msix_probe: legacy platform, identify intr bar by size
[  +0.008053] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev dma_msix inst 5242880 level 0
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev dma_msix.m.5242880 inst 5242880 is active
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev firewall.m multi 0 level 0
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920d0000-0xf3920dffff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920e0000-0xf3920e0fff]
[  +0.000000] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920e1000-0xf3920e1fff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920f0000-0xf3920fffff]
[  +0.000296] xclmgmt 0000:09:00.0: firewall.m.16777216 ffff99cc5882cc10 firewall_probe: probe
[  +0.000204] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev firewall inst 16777216 level 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev firewall.m.16777216 inst 16777216 is active
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev rom.m multi 0 level 0
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920b0000-0xf3920b0fff]
[  +0.000336] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 get_header_from_iomem: Magic number does not match, actual 0xffffffff, expected 0x786e6c78
[  +0.000246] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: ROM magic : 
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: VBNV: 
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: DDR channel count : 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: DDR channel size: 0 GB
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: Major Version: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: Minor Version: 0
[  +0.000000] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: IPBuildID: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: TimeSinceEpoch: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_probe: FeatureBitMap: 0
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev rom inst 0 level 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev rom.m.0 inst 0 is active
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xocl_subdev_reserve: subdev is in-use
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev axigate.m multi 0 level 2
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392030000-0xf39203000b]
[  +0.000347] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev axigate inst 3145728 level 2
[  +0.000000] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev axigate.m.3145728 inst 3145728 is active
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev iores2.m multi 0 level 2
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392032000-0xf392032003]
[  +0.000353] xclmgmt 0000:09:00.0: iores2.m.7340032 ffff99cc58829810 iores_probe: Resource ep_ddr_mem_calib_00[mem 0xf392032000-0xf392032003], id 1, mapped @ffffb9a9c0531000
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev iores2 inst 7340032 level 2
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev iores2.m.7340032 inst 7340032 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev flash.m multi 0 level 0
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392040000-0xf39204007b]
[  +0.113420] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc5882d010 flash_reset_fifo: failed to reset FIFO, status: 0xffffffff
[  +0.000029] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc5882d010 flash_probe: probing failed
[  +0.000116] flash.m: probe of flash.m.8388608 failed with error -22
[  +0.000016] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev flash inst 8388608 level 0
[  +0.315490] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc5882d010 flash_reset_fifo: failed to reset FIFO, status: 0xffffffff
[  +0.000031] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc5882d010 flash_probe: probing failed
[  +0.000116] flash.m: probe of flash.m.8388608 failed with error -22
[  +0.000006] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: failed to probe subdev flash.m.8388608, ret 0
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev mailbox.m multi 0 level 0
[  +0.000005] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392210000-0xf39221002f]
[  +0.000042] xclmgmt 0000:09:00.0: mailbox.m.9437184 ffff99cc5882f810 mailbox_start: Starting Mailbox channels
[  +0.001105] xocl 0000:09:00.1: enabling device (0000 -> 0002)
[  +0.000119] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev rom.u multi 0 level 0
[  +0.000003] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: resource [mem 0xf3900b0000-0xf3900b0fff]
[  +0.000032] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 get_header_from_iomem: Magic number does not match, actual 0xffffffff, expected 0x786e6c78
[  +0.000042] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: ROM magic : 
[  +0.000000] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: VBNV: 
[  +0.000001] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: DDR channel count : 0
[  +0.000001] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: DDR channel size: 0 GB
[  +0.000001] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: Major Version: 0
[  +0.000000] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: Minor Version: 0
[  +0.000001] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: IPBuildID: 0
[  +0.000001] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: TimeSinceEpoch: 0
[  +0.000000] xocl 0000:09:00.1: rom.u.0 ffff99cc47514c10 feature_rom_probe: FeatureBitMap: 0
[  +0.000007] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev rom inst 0 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev rom.u.0 inst 0 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_subdev_reserve: subdev is in-use
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev dma.xdma.u multi 0 level 0
[  +0.000015] xocl:xdma_device_open: xocl device 0000:09:00.1, 0x0000000009c42cc1.
[  +0.000016] xocl:map_single_bar: map bar 0, len 33554432
[  +0.000000] xocl:map_single_bar: BAR #0 is not present - skipping
[  +0.000001] xocl:map_single_bar: map bar 1, len 0
[  +0.000000] xocl:map_single_bar: BAR #1 is not present - skipping
[  +0.000000] xocl:map_single_bar: map bar 2, len 262144
[  +0.000004] xocl:map_single_bar: BAR2 at 0xf394000000 mapped at 0x00000000246d8020, length=262144(/262144)
[  +0.000003] xocl:map_single_bar: map bar 3, len 0
[  +0.000000] xocl:map_single_bar: BAR #3 is not present - skipping
[  +0.000000] xocl:map_single_bar: map bar 4, len 268435456
[  +0.000001] xocl:map_single_bar: BAR #4 is not present - skipping
[  +0.000000] xocl:map_single_bar: map bar 5, len 0
[  +0.000000] xocl:map_single_bar: BAR #5 is not present - skipping
[  +0.000000] xocl:map_bars: Failed to detect XDMA config BAR
[  +0.000145] xclmgmt 0000:09:00.0: dma_msix.m.5242880 ffff99cc58828410 user_intr_register: intr 11 register success, start vec 15
[  +0.000001] xclmgmt 0000:09:00.0: dma_msix.m.5242880 ffff99cc58828410 user_intr_config: configure intr at 0xffffb9a9c7680000
[  +0.000009] xclmgmt 0000:09:00.0: mailbox.m.9437184 ffff99cc5882f810 mailbox_probe: successfully initialized
[  +0.000006] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev mailbox inst 9437184 level 0
[  +0.000050] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev mailbox.m.9437184 inst 9437184 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev xvc_pri.m multi 0 level 0
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3921c0000-0xf3921cffff]
[  +0.000022] xclmgmt 0000:09:00.0: xvc_pri.m.13631488 ffff99cc47515010 xvc_probe: XVC device instance initialized
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev xvc_pri inst 13631488 level 0
[  +0.000035] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev xvc_pri.m.13631488 inst 13631488 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev nifd_pri.m multi 0 level 0
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392028000-0xf39202cfff]
[  +0.000016] xclmgmt 0000:09:00.0: nifd_pri.m.14680064 ffff99cc47517c10 nifd_probe: NIFD: looking from NIFD in FeatureBitMap: 0
[  +0.000001] nifd.m.m: probe of nifd_pri.m.14680064 failed with error -22
[  +0.000007] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev nifd_pri inst 14680064 level 0
[  +0.000006] xclmgmt 0000:09:00.0: nifd_pri.m.14680064 ffff99cc47517c10 nifd_probe: NIFD: looking from NIFD in FeatureBitMap: 0
[  +0.000000] nifd.m.m: probe of nifd_pri.m.14680064 failed with error -22
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: failed to probe subdev nifd_pri.m.14680064, ret 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xocl_subdev_reserve: subdev is in-use
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev xmc.m multi 0 level 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392120000-0xf392121fff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392131000-0xf392131fff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392140000-0xf39215ffff]
[  +0.000000] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392160000-0xf39217ffff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392190000-0xf39219ffff]
[  +0.000013] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: IO start: 0xf392120000, end: 0xf392121fff
[  +0.000002] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_mapio_by_name: resource xmc.m.18874368 not found
[  +0.000005] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: IO start: 0xf392131000, end: 0xf392131fff
[  +0.000001] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_mapio_by_name: resource xmc.m.18874368 not found
[  +0.000002] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: IO start: 0xf392140000, end: 0xf39215ffff
[  +0.000001] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_mapio_by_name: resource xmc.m.18874368 not found
[  +0.000002] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: IO start: 0xf392160000, end: 0xf39217ffff
[  +0.000001] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_mapio_by_name: resource xmc.m.18874368 not found
[  +0.000002] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: IO start: 0xf392190000, end: 0xf39219ffff
[  +0.000001] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_mapio_by_name: resource xmc.m.18874368 not found
[  +0.000005] xclmgmt 0000:09:00.0: xmc.m.18874368 ffff99cc47516810 xmc_probe: Microblaze is not supported.
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev xmc inst 18874368 level 0
[  +0.000018] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev xmc.m.18874368 inst 18874368 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev icap.m multi 0 level 0
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392020000-0xf392020119]
[  +0.000016] xclmgmt 0000:09:00.0: icap.m.22020096 ffff99cc47510c10 icap_probe: icap.m.22020096 mapped in register @ 0x00000000b5a1c8b0
[  +0.000001] xclmgmt 0000:09:00.0: icap.m.22020096 ffff99cc47510c10 icap_refresh_addrs: memcalib @ ffffb9a9c0531000
[  +0.000001] xclmgmt 0000:09:00.0: icap.m.22020096 ffff99cc47510c10 icap_refresh_addrs: icap_reset @ 0
[  +0.000026] xclmgmt 0000:09:00.0: icap.m.22020096 ffff99cc47510c10 icap_probe: successfully initialized FPGA IDCODE 0xffffffff
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev icap inst 22020096 level 0
[  +0.000018] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev icap.m.22020096 inst 22020096 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev fmgr.m multi 0 level 0
[  +0.000026] fpga_manager fpga0: Xilinx PCIe FPGA Manager registered
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev fmgr inst 24117248 level 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev fmgr.m.24117248 inst 24117248 is active
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev clock.m multi 0 level 2
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392050000-0xf392050fff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392051000-0xf392051fff]
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392052000-0xf392052fff]
[  +0.000011] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_prev_refresh_addrs: freq_k1_k2 @ 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_prev_refresh_addrs: freq_hbm @ 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_prev_refresh_addrs: done.
[  +0.000003] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_probe: res[0] ep_aclk_kernel_00 mapped @ ffffb9a9c0591000
[  +0.000003] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_probe: res[1] ep_aclk_kernel_01 mapped @ ffffb9a9c0599000
[  +0.000002] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_probe: res[2] clkfreq_kernel1_kernel2 mapped @ ffffb9a9c05a1000
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_aclk_kernel_00 @ ffffb9a9c0591000
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_aclk_kernel_01 @ ffffb9a9c0599000
[  +0.000000] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_aclk_hbm_00 @ 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: clkfreq_kernel1_kernel2 @ ffffb9a9c05a1000
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_freq_cnt_aclk_kernel_00 @ 0
[  +0.000000] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_freq_cnt_aclk_kernel_01 @ 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_freq_cnt_aclk_hbm_00 @ 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_iores_update_base: ep_ucs_control_status_00 @ 0
[  +0.000000] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_post_refresh_addrs: ret 0
[  +0.000001] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_probe: successfully initialized Clock subdev
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev clock inst 28311552 level 2
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev clock.m.28311552 inst 28311552 is active
[  +0.000046] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xclmgmt_extended_probe: created all sub devices
[  +0.000002] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5004-000e-0000000000000000.xsabin
[  +0.000587] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5004-000e-0000000000000000.xsabin failed with error -2
[  +0.000002] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5005-000e-0000000000000000.xsabin
[  +0.000007] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5005-000e-0000000000000000.xsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5004-000e-0000000000000000.dsabin
[  +0.000007] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5004-000e-0000000000000000.dsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5005-000e-0000000000000000.dsabin
[  +0.000007] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5005-000e-0000000000000000.dsabin failed with error -2
[  +0.000000] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_flash: try loading fw from flash
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_flash: no flash subdev
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware: can't load firmware, give up
[  +0.000035] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5004-000e-0000000000000000.xsabin
[  +0.000006] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5004-000e-0000000000000000.xsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5005-000e-0000000000000000.xsabin
[  +0.000007] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5005-000e-0000000000000000.xsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5004-000e-0000000000000000.dsabin
[  +0.000006] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5004-000e-0000000000000000.dsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_disk: try loading fw: xilinx/10ee-5005-000e-0000000000000000.dsabin
[  +0.000007] xclmgmt 0000:09:00.0: Direct firmware load for xilinx/10ee-5005-000e-0000000000000000.dsabin failed with error -2
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_flash: try loading fw from flash
[  +0.000000] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware_from_flash: no flash subdev
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 load_firmware: can't load firmware, give up
[  +0.000019] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev clock, cdev 0000000000000000
[  +0.000003] xclmgmt 0000:09:00.0: clock.m.28311552 ffff99cc47516010 clock_remove: successfully removed Clock subdev
[  +0.000019] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev fmgr, cdev 0000000000000000
[  +0.000001] fpga_manager fpga0: fpga_mgr_unregister Xilinx PCIe FPGA Manager
[  +0.000037] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev icap, cdev 00000000413d5b46
[  +0.000058] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 raptor_cmc_access: No ep_cmc_mutex_00 resource, skip.
[  +0.000013] xclmgmt 0000:09:00.0: icap.m.22020096 ffff99cc47510c10 icap_remove: cleaned up successfully
[  +0.000024] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev xmc, cdev 000000002a9d3c2d
[  +0.000074] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev firewall, cdev 0000000000000000
[  +0.000028] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev nifd_pri, cdev 0000000000000000
[  +0.000007] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev xvc_pri, cdev 000000004135bcd8
[  +0.000088] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev mailbox, cdev 00000000bb38084b
[  +0.000020] xclmgmt 0000:09:00.0: dma_msix.m.5242880 ffff99cc58828410 user_intr_config: configure intr at 0xffffb9a9c7680000
[  +0.000011] xclmgmt 0000:09:00.0: dma_msix.m.5242880 ffff99cc58828410 user_intr_unreg: intr 11 unreg success, start vec 15
[  +0.001024] xclmgmt 0000:09:00.0: mailbox.m.9437184 ffff99cc5882f810 mailbox_remove: mailbox cleaned up successfully
[  +0.000059] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev flash, cdev 0000000000000000
[  +0.000018] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev iores2, cdev 0000000000000000
[  +0.000019] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev dma_msix, cdev 0000000000000000
[  +0.000352] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev axigate, cdev 0000000000000000
[  +0.000028] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_destroy: Destroy subdev rom, cdev 0000000000000000
[  +0.000002] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc58829010 feature_rom_remove: Remove feature rom
[  +0.000030] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 xclmgmt_extended_probe: failed to fully probe device, err: -19
[  +0.000043] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev rom.m multi 0 level 0
[  +0.000003] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf3920b0000-0xf3920b0fff]
[  +0.000048] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 get_header_from_iomem: Magic number does not match, actual 0xffffffff, expected 0x786e6c78
[  +0.000074] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: ROM magic : 
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: VBNV: 
[  +0.000005] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: DDR channel count : 0
[  +0.000002] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: DDR channel size: 0 GB
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: Major Version: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: Minor Version: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: IPBuildID: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: TimeSinceEpoch: 0
[  +0.000001] xclmgmt 0000:09:00.0: rom.m.0 ffff99cc47517810 feature_rom_probe: FeatureBitMap: 0
[  +0.000006] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev rom inst 0 level 0
[  +0.000001] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: subdev rom.m.0 inst 0 is active
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: creating subdev flash.m multi 0 level 0
[  +0.000002] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_construct: resource [mem 0xf392040000-0xf39204007b]
[  +0.512425] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc47513010 flash_reset_fifo: failed to reset FIFO, status: 0xffffffff
[  +0.000028] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc47513010 flash_probe: probing failed
[  +0.000026] flash.m: probe of flash.m.8388608 failed with error -22
[  +0.000006] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: Created subdev flash inst 8388608 level 0
[  +1.066538] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc47513010 flash_reset_fifo: failed to reset FIFO, status: 0xffffffff
[  +0.000027] xclmgmt 0000:09:00.0: flash.m.8388608 ffff99cc47513010 flash_probe: probing failed
[  +0.000024] xocl 0000:09:00.1: dma.xdma.u.5242880 ffff99cc47517010 xdma_probe: XDMA Device Open failed
[  +0.000024] dma.xdma.u: probe of dma.xdma.u.5242880 failed with error -5
[  +0.000006] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev dma.xdma inst 5242880 level 0
[  +0.000009] xocl:xdma_device_open: xocl device 0000:09:00.1, 0x0000000009c42cc1.
[  +0.000014] xocl:map_single_bar: map bar 0, len 33554432
[  +0.000000] xocl:map_single_bar: BAR #0 is not present - skipping
[  +0.000001] xocl:map_single_bar: map bar 1, len 0
[  +0.000000] xocl:map_single_bar: BAR #1 is not present - skipping
[  +0.000000] xocl:map_single_bar: map bar 2, len 262144
[  +0.000011] xocl:map_single_bar: BAR2 at 0xf394000000 mapped at 0x00000000ef18b125, length=262144(/262144)
[  +0.000003] xocl:map_single_bar: map bar 3, len 0
[  +0.000000] xocl:map_single_bar: BAR #3 is not present - skipping
[  +0.000000] xocl:map_single_bar: map bar 4, len 268435456
[  +0.000000] xocl:map_single_bar: BAR #4 is not present - skipping
[  +0.000001] xocl:map_single_bar: map bar 5, len 0
[  +0.000000] xocl:map_single_bar: BAR #5 is not present - skipping
[  +0.000000] xocl:map_bars: Failed to detect XDMA config BAR
[  +0.000049] flash.m: probe of flash.m.8388608 failed with error -22
[  +0.000004] xclmgmt 0000:09:00.0:  ffff99ccac2d40b0 __xocl_subdev_create: failed to probe subdev flash.m.8388608, ret 0
[  +0.017185] xocl 0000:09:00.1: dma.xdma.u.5242880 ffff99cc47517010 xdma_probe: XDMA Device Open failed
[  +0.000013] dma.xdma.u: probe of dma.xdma.u.5242880 failed with error -5
[  +0.000008] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: failed to probe subdev dma.xdma.u.5242880, ret 0
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev mailbox.u multi 0 level 0
[  +0.000005] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: resource [mem 0xf390200000-0xf39020002f]
[  +0.000131] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 mailbox_start: Starting Mailbox channels
[  +0.000143] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 mailbox_enable_intr_mode: failed to add intr handler
[  +0.000001] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 mailbox_start: failed to enable intr mode
[  +0.000045] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 mailbox_probe: successfully initialized
[  +0.000008] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev mailbox inst 9437184 level 0
[  +0.000134] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev mailbox.u.9437184 inst 9437184 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev mb_scheduler.u multi 0 level 0
[  +0.000003] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: resource [mem 0xf390180000-0xf390180fff]
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: resource [mem 0xf390190000-0xf39019ffff]
[  +0.000125] xocl 0000:09:00.1: mb_scheduler.u.11534336 ffff99cc47512810 exec_create: CQ size is 65536
[  +0.000042] xocl 0000:09:00.1:  ffff99ccac2d60b0 exec_reset: exec_reset(0) cfg(0)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 exec_reset: exec_reset resets
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 exec_reset: exec->xclbin(00000000-0000-0000-0000-000000000000),xclbin(00000000-0000-0000-0000-000000000000)
[  +0.000084] [drm] command scheduler started
[  +0.000005] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev mb_scheduler inst 11534336 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev mb_scheduler.u.11534336 inst 11534336 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev xvc_pub.u multi 0 level 0
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: resource [mem 0xf3900c0000-0xf3900cffff]
[  +0.000092] xocl 0000:09:00.1: xvc_pub.u.12582912 ffff99cc47510c10 xvc_probe: XVC device instance initialized
[  +0.000004] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev xvc_pub inst 12582912 level 0
[  +0.000109] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev xvc_pub.u.12582912 inst 12582912 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev firewall.u multi 0 level 0
[  +0.000109] xocl 0000:09:00.1: firewall.u.16777216 ffff99cc47517c10 firewall_probe: probe
[  +0.000042] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev firewall inst 16777216 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev firewall.u.16777216 inst 16777216 is active
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev xmc.u multi 0 level 0
[  +0.000072] xocl 0000:09:00.1: xmc.u.18874368 ffff99cc47514010 xmc_probe: Microblaze is not supported.
[  +0.000003] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev xmc inst 18874368 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev xmc.u.18874368 inst 18874368 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev icap.u multi 0 level 0
[  +0.000169] xocl 0000:09:00.1: icap.u.22020096 ffff99cc47514410 icap_probe: successfully initialized FPGA IDCODE 0x0
[  +0.000005] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev icap inst 22020096 level 0
[  +0.000000] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev icap.u.22020096 inst 22020096 is active
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev intc.u multi 0 level 0
[  +0.000092] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev intc inst 40894464 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev intc.u.40894464 inst 40894464 is active
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev ert_user.u multi 0 level 0
[  +0.000136] ert_user.u: probe of ert_user.u.48234496 failed with error -12
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev ert_user inst 48234496 level 0
[  +0.000066] ert_user.u: probe of ert_user.u.48234496 failed with error -12
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: failed to probe subdev ert_user.u.48234496, ret 0
[  +0.000046] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_construct: creating subdev p2p.u multi 0 level 0
[  +0.000081] xocl 0000:09:00.1: p2p.u.10485760 ffff99cc47515810 p2p_probe: can not find p2p bar in metadata
[  +0.000012] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: Created subdev p2p inst 10485760 level 0
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 __xocl_subdev_create: subdev p2p.u.10485760 inst 10485760 is active
[  +0.000290] [drm] Initialized xocl 2.8.743 20201116 for 0000:09:00.1 on minor 2
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_init_sysfs: Creating sysfs
[  +0.000144] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_init_persist_sysfs: Creating persist sysfs
[  +0.000021] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_refresh_subdevs: get fdt from peer
[  +0.000106] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 mailbox_request: sending request: 10 via HW
[  +0.187620] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000003] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000002] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.047570] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000002] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000003] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.047284] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000002] [drm] client exits pid(2539)
[  +0.000000] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.047302] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000002] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000003] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.047281] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000001] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.047254] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000002] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.073778] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000003] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000008] [drm] client exits pid(2539)
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.003511] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000003] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.004575] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000003] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000005] [drm] client exits pid(2539)
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.001737] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(2539), ret: 0
[  +0.000003] [drm] client exits pid(2539)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000000] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.023424] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 timeout_msg: found outstanding msg time'd out
[  +0.000001] xocl 0000:09:00.1: mailbox.u.9437184 ffff99cc47513810 timeout_msg: peer becomes dead
[  +5.368224] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000003] [drm] creating scheduler client for pid(3286), ret: 0
[  +0.000008] [drm] client exits pid(3286)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.002484] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000002] [drm] creating scheduler client for pid(3286), ret: 0
[  +0.000005] [drm] client exits pid(3286)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.234044] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000017] [drm] creating scheduler client for pid(4014), ret: 0
[  +0.000026] [drm] client exits pid(4014)
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000003] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.003278] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000002] [drm] creating scheduler client for pid(4014), ret: 0
[  +0.000006] [drm] client exits pid(4014)
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +1.923176] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000003] [drm] creating scheduler client for pid(4680), ret: 0
[  +0.000013] [drm] client exits pid(4680)
[  +0.000002] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
[  +0.002166] xocl 0000:09:00.1:  ffff99ccac2d60b0 _xocl_drvinst_open: OPEN 1
[  +0.000001] [drm] creating scheduler client for pid(4680), ret: 0
[  +0.000004] [drm] client exits pid(4680)
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: CLOSE 2
[  +0.000001] xocl 0000:09:00.1:  ffff99ccac2d60b0 xocl_drvinst_close: NOTIFY 000000003763de02
​
0 Kudos
Reply
emeryw
Xilinx Employee
Xilinx Employee
427 Views
Registered: ‎12-06-2019

Hi @emeaton ,

Thanks for the response. The dmesg shows several errors, upon first glance, configure space access seems okay, but BAR access does not. Could you please post a more verbose lspci?

sudo lspci -vvvd 10ee:

 

That will give us a better idea of what the host system sees. I do see in the lspci posted above the line about the resizable BAR, whereas that is not present in the lspci from our lab U250.

Best,
-Emery
-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------

0 Kudos
Reply
emeaton
Observer
Observer
411 Views
Registered: ‎10-21-2018

Here you go:

Spoiler
09:00.0 Processing accelerators: Xilinx Corporation Device 5004
	Subsystem: Xilinx Corporation Device 000e
	Physical Slot: 64-1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	NUMA node: 0
	Region 0: Memory at f392000000 (64-bit, prefetchable) [size=32M]
	Region 2: Memory at f394040000 (64-bit, prefetchable) [size=256K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI-X: Enable- Count=32 Masked-
		Vector table: BAR=2 offset=00030000
		PBA: BAR=2 offset=00034000
	Capabilities: [70] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 25.000W
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 512 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM not supported
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s (downgraded), Width x8 (downgraded)
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, NROPrPrP-, LTR-
			 10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS-, TPHComp-, ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
			 AtomicOpsCtl: ReqEn-
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [1c0 v1] Secondary PCI Express
		LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
		LaneErrStat: 0
	Capabilities: [400 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl+ DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Capabilities: [410 v1] Resizable BAR <?>
	Capabilities: [480 v1] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
	Kernel driver in use: xclmgmt
	Kernel modules: xclmgmt

09:00.1 Processing accelerators: Xilinx Corporation Device 5005
	Subsystem: Xilinx Corporation Device 000e
	Physical Slot: 64-1
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 37
	NUMA node: 0
	Region 0: Memory at f390000000 (64-bit, prefetchable) [size=32M]
	Region 2: Memory at f394000000 (64-bit, prefetchable) [size=256K]
	Region 4: Memory at f380000000 (64-bit, prefetchable) [size=256M]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI-X: Enable- Count=32 Masked-
		Vector table: BAR=2 offset=00030000
		PBA: BAR=2 offset=00034000
	Capabilities: [70] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 25.000W
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 512 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM not supported
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s (downgraded), Width x8 (downgraded)
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, NROPrPrP-, LTR-
			 10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS-, TPHComp-, ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
			 AtomicOpsCtl: ReqEn-
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [400 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl+ DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Capabilities: [410 v1] Resizable BAR <?>
	Capabilities: [480 v1] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
	Kernel driver in use: xocl
	Kernel modules: xocl

 

0 Kudos
Reply
emeaton
Observer
Observer
209 Views
Registered: ‎10-21-2018

Is there any other information you need?

0 Kudos
Reply
emeryw
Xilinx Employee
Xilinx Employee
30 Views
Registered: ‎12-06-2019

Hi @emeaton ,

I ran the same lspci command on our lab U250 and looked at its results next to the one you posted. I've attached the output from our lab machine.

ne line that jumped out was the SlotPowerLimit 25W reported with the external enclosure, where the lab U250 reports the expected 75W. Per the card datasheet: https://www.xilinx.com/support/documentation/data_sheets/ds962-u200-u250.pdf the card can pull up to 65W from the slot, which it wouldnt be able to do if the slot is limited to 25W. Do you know if that reported slot limit is accurate?

I also see some information regarding speed downgrading: LnkSta: Speed 2.5GT/s (downgraded), Width x8 (downgraded), which isn't ideal, but probably isn't affecting our access to the card.

I'm still suspect of the BAR access as well. You could try the following to see if you are able to access it outside of XRT, as it would tell us if its purely a hardware issue.
1. uninstall XRT and reboot
2. use a tool like devmem2 (http://manpages.ubuntu.com/manpages/bionic/man1/devmem2.1.html) to access PF0 BAR0 offset 0x1f08000 and see if that is successful.

Best,
-Emery

0 Kudos
Reply