cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Participant
Participant
783 Views
Registered: ‎06-27-2018

Alveo U50 xbutil validate failed.

Jump to solution

Hi,

I had set up Alveo U50 development environment(XRT etc,) once, and again, I set up environment from OS clean installation. I followed UG1370 Ubuntu procedure. But, since the Alveo card memory was already flushed, the memory flush procedure "xbmgmt flash --update" was excluded.

When I exec "xbutil validate", I got error as follow:

iwabuchi@PX04101:~$ /opt/xilinx/xrt/bin/xbutil validate
INFO: Found 1 cards

INFO: Validating card[0]:
INFO: == Starting AUX power connector check:
AUX power connector not available. Skipping validation
INFO: == AUX power connector check SKIPPED
INFO: == Starting PCIE link check:
INFO: == PCIE link check PASSED
INFO: == Starting SC firmware version check:
INFO: == SC firmware version check PASSED
INFO: == Starting verify kernel test:
ERROR: Failed to find xclbin in /opt/xilinx/xsa//test/ and /opt/xilinx/dsa//test/ERROR: Failed to find 22_verify.py or verify.xclbin, Shell package not installed properly.
ERROR: == verify kernel test FAILED
INFO: Card[0] failed to validate.

ERROR: Some cards failed to validate.

But /opt/xilinx/xsa and dsa directiries do not exist.

 

iwabuchi@PX04101:~$ ls /opt/xilinx
firmware  platforms  xrt

 

What am I missing?

Other information is as follows:

 

iwabuchi@PX04101:~$ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan
Card [0000:01:00.0]
    Card type:          u50
    Flash type:         SPI
    Flashable partition running on FPGA:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6]
    Flashable partitions installed in system:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0.27]

iwabuchi@PX04101:~$ sudo apt list | grep xrt
xrt/now 2.5.309 amd64 [インストール済み、ローカル]

iwabuchi@PX04101:~$ sudo apt list | grep xilinx
xilinx-cmc-u50/now 1.0.17-2784148 all [インストール済み、ローカル]
xilinx-sc-fw-u50/now 5.0.27-2.e289be9 all [インストール済み、ローカル]
xilinx-u50-gen3x16-xdma-blp/now 1-2784799 all [インストール済み、ローカル]
xilinx-u50-gen3x16-xdma-dev/now 201920.3-2784799 all [インストール済み、ローカル]

iwabuchi@PX04101:~/u50$ ls  # installed files
Xilinx_u50-gen3x16-xdma-201920.3-2784799_16.04_deb.tar.gz
xilinx-cmc-u50-1.0.17-2784148_16.04.deb
xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb
xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb
xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb
xrt_201920.2.5.309_16.04-amd64-xrt.deb

 

 

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Xilinx Employee
Xilinx Employee
573 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @kousei.iwabuchi 

What kernel ($uname -r) and operating system ($cat /etc/*-release) are you using? 

Looks like XRT isn't installing correctly. 

It also looks like XRT is skipping pyopencl install. I think our tools want to use python 2, can you check which python is installed on your system? 

I also see that the satellite controller (SC) is not coming up correctly. You can tell this when running xbutil flash scan. You should follow the directions below until the SC is available in xbutil flash scan, then you should update your shell and SC firmware again. 

The SC is in a bad state, you must remove power completely from the card by either removing power from the server or removing the card from the server. Waiting about 2 minutes, then plugging it back in and trying again. Repeat this process until you are able to see a SC version in xbutil flash scan. When you see that it is an indication XRT has made communication with the SC and is able to program it. If you do not see this, you are not programming the SC. The U50 relies heavily on the SC being operational, so if it is not, you can see unpredictable behavior in the card. 

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------

View solution in original post

0 Kudos
18 Replies
Highlighted
Xilinx Employee
Xilinx Employee
766 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @kousei.iwabuchi 

I think you are either missing python libraries or the shell wasn't installed correctly. 

Can you remove XRT, install it all again, and copy all the install logs to this thread so I can see if the shell was installed correctly? 

If it is installed correctly, check to see if this script completes correctly, if it does install new packages you'll need to install XRT and the shells again. https://github.com/Xilinx/XRT/blob/1d814bce789b0fb0b531669a78aa84bf971088e4/src/runtime_src/tools/scripts/pyopencl_install_check.sh

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Tags (2)
0 Kudos
Highlighted
Participant
Participant
721 Views
Registered: ‎06-27-2018

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi,

Thank you for your reply.

I uninstalled XRT and re-installed.

And checked by pyopencl_install_check.sh, I attached the log file.(xrt_install.log)

It is seemed that I could install correctly....???

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
686 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @kousei.iwabuchi 

Did you install a platform on the system? 

The error message says you are missing a validate kernel as well as python libraries. Since you ran the script telling you that the python libraries needed are installed, then we must check the platform install.

 What shell is installed on the system? 

What card do you have?

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
Highlighted
Participant
Participant
624 Views
Registered: ‎06-27-2018

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi, @mcertosi

Sorry for the late reply.

> What shell is installed on the system?

Is the shell what you say "xilinx_u50_gen3x16_xdma_201920_3"? I couldn't understand what the shell was after reading UG1370(Alveo U50 Data Center Accelerator Card Installation Guide).

When I installed eployment packages, the messages was shown.

iwabuchi@PX04101:~/workspace/u50/tmp$ tar -zxf Xilinx_u50-gen3x16-xdma-201920.3-2784799_16.04_deb.tar.gz
iwabuchi@PX04101:~/workspace/u50/tmp$ ls
Xilinx_u50-gen3x16-xdma-201920.3-2784799_16.04_deb.tar.gz
xilinx-cmc-u50-1.0.17-2784148_16.04.deb
xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb
xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb
iwabuchi@PX04101:~/workspace/u50/tmp$ sudo apt install ./*.deb
    :
xilinx-cmc-u50 (1.0.17-2784148) を設定しています ...
xilinx-sc-fw-u50 (5.0.27-2.e289be9) を設定しています ...
xilinx-u50-gen3x16-xdma-blp (1-2784799) を設定しています ...
Partition package installed successfully.
Please flash card manually by running below command:
sudo /opt/xilinx/xrt/bin/xbmgmt flash --update --shell xilinx_u50_gen3x16_xdma_201920_3

And xbmgmt flash --scan outputs ,

iwabuchi@PX04101:~/workspace/u50$ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan
Card [0000:01:00.0]
    Card type:          u50
    Flash type:         SPI
    Flashable partition running on FPGA:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6]
    Flashable partitions installed in system:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0.27]

After that, I installed "xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb".

iwabuchi@PX04101:~/workspace/u50$ sudo apt install ./xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb
  :
(installation completed successfully) iwabuchi@PX04101:~/workspace/u50$ ls /opt/xilinx/ firmware platforms xrt

/opt/xilinx/xsa and dsa directiries do not exist.

 

> What card do you have?

I have Alveo U50.

 

Best regards.

 

0 Kudos
Highlighted
Participant
Participant
589 Views
Registered: ‎06-27-2018

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi, @mcertosi

The installation logs are as follows:

iwabuchi@PX04101:~/workspace/u50$ ls
Xilinx_u50-gen3x16-xdma-201920.3-2784799_16.04_deb.tar.gz
pyopencl_install_check.sh
tmp
xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb
xrt_201920.2.5.309_16.04-amd64-xrt.deb
iwabuchi@PX04101:~/workspace/u50$ sudo apt install ./xrt_201920.2.5.309_16.04-amd64-xrt.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'xrt' instead of './xrt_201920.2.5.309_16.04-amd64-xrt.deb'
The following NEW packages will be installed:
  xrt
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 0 B/9390 kB of archives.
After this operation, 49.2 MB of additional disk space will be used.
Get:1 /mnt/hdd/u50/xrt_201920.2.5.309_16.04-amd64-xrt.deb xrt amd64 2.5.309 [9390 kB]
Selecting previously unselected package xrt.
(Reading database ... 250199 files and directories currently installed.)
Preparing to unpack .../xrt_201920.2.5.309_16.04-amd64-xrt.deb ...
Unpacking xrt (2.5.309) ...
Setting up xrt (2.5.309) ...
Unloading old XRT Linux kernel modules
rmmod: ERROR: Module xocl is not currently loaded
rmmod: ERROR: Module xclmgmt is not currently loaded
Invoking DKMS common.postinst for xrt
Loading new xrt-2.5.309 DKMS files...
First Installation: checking all kernels...
Building only for 4.4.0-178-generic
Building initial module for 4.4.0-178-generic
Secure Boot not enabled on this system.
Done.

xocl:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.4.0-178-generic/updates/dkms/

xclmgmt.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.4.0-178-generic/updates/dkms/

depmod....

DKMS: install completed.
Finished DKMS common.postinst
Loading new XRT Linux kernel modules
Installing MSD / MPD daemons
Skipping pyopencl installation...
iwabuchi@PX04101:~/workspace/u50$



iwabuchi@PX04101:~/workspace/u50/tmp$ ls
Xilinx_u50-gen3x16-xdma-201920.3-2784799_16.04_deb.tar.gz
xilinx-cmc-u50-1.0.17-2784148_16.04.deb
xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb
xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb
iwabuchi@PX04101:~/workspace/u50/tmp$ sudo apt install ./*.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'xilinx-cmc-u50' instead of './xilinx-cmc-u50-1.0.17-2784148_16.04.deb'
Note, selecting 'xilinx-sc-fw-u50' instead of './xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb'
Note, selecting 'xilinx-u50-gen3x16-xdma-blp' instead of './xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb'
The following NEW packages will be installed:
  xilinx-cmc-u50 xilinx-sc-fw-u50 xilinx-u50-gen3x16-xdma-blp
0 upgraded, 3 newly installed, 0 to remove and 5 not upgraded.
Need to get 0 B/19.3 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /mnt/hdd/u50/tmp/xilinx-cmc-u50-1.0.17-2784148_16.04.deb xilinx-cmc-u50 all 1.0.17-2784148 [29.0 kB]
Get:2 /mnt/hdd/u50/tmp/xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb xilinx-sc-fw-u50 all 5.0.27-2.e289be9 [74.2 kB]
Get:3 /mnt/hdd/u50/tmp/xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb xilinx-u50-gen3x16-xdma-blp all 1-2784799 [19.2 MB]
Selecting previously unselected package xilinx-cmc-u50.
(Reading database ... 250535 files and directories currently installed.)
Preparing to unpack .../xilinx-cmc-u50-1.0.17-2784148_16.04.deb ...
Unpacking xilinx-cmc-u50 (1.0.17-2784148) ...
Selecting previously unselected package xilinx-sc-fw-u50.
Preparing to unpack .../xilinx-sc-fw-u50-5.0.27-2.e289be9_16.04.deb ...
Unpacking xilinx-sc-fw-u50 (5.0.27-2.e289be9) ...
Selecting previously unselected package xilinx-u50-gen3x16-xdma-blp.
Preparing to unpack .../xilinx-u50-gen3x16-xdma-201920.3-2784799_16.04.deb ...
Unpacking xilinx-u50-gen3x16-xdma-blp (1-2784799) ...
Setting up xilinx-cmc-u50 (1.0.17-2784148) ...
Setting up xilinx-sc-fw-u50 (5.0.27-2.e289be9) ...
Setting up xilinx-u50-gen3x16-xdma-blp (1-2784799) ...
Partition package installed successfully.
Please flash card manually by running below command:
sudo /opt/xilinx/xrt/bin/xbmgmt flash --update --shell xilinx_u50_gen3x16_xdma_201920_3
iwabuchi@PX04101:~/workspace/u50/tmp$


iwabuchi@PX04101:~/workspace/u50$ sudo apt install ./xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'xilinx-u50-gen3x16-xdma-dev' instead of './xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb'
The following NEW packages will be installed:
  xilinx-u50-gen3x16-xdma-dev
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 0 B/153 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /mnt/hdd/u50/xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb xilinx-u50-gen3x16-xdma-dev all 201920.3-2784799 [153 MB]
Selecting previously unselected package xilinx-u50-gen3x16-xdma-dev.
(Reading database ... 250573 files and directories currently installed.)
Preparing to unpack .../xilinx-u50-gen3x16-xdma-dev-201920.3-2784799_16.04.deb ...
Unpacking xilinx-u50-gen3x16-xdma-dev (201920.3-2784799) ...
Setting up xilinx-u50-gen3x16-xdma-dev (201920.3-2784799) ...
iwabuchi@PX04101:~/workspace/u50$ ls /opt/xilinx/
firmware  platforms  xrt

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
574 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @kousei.iwabuchi 

What kernel ($uname -r) and operating system ($cat /etc/*-release) are you using? 

Looks like XRT isn't installing correctly. 

It also looks like XRT is skipping pyopencl install. I think our tools want to use python 2, can you check which python is installed on your system? 

I also see that the satellite controller (SC) is not coming up correctly. You can tell this when running xbutil flash scan. You should follow the directions below until the SC is available in xbutil flash scan, then you should update your shell and SC firmware again. 

The SC is in a bad state, you must remove power completely from the card by either removing power from the server or removing the card from the server. Waiting about 2 minutes, then plugging it back in and trying again. Repeat this process until you are able to see a SC version in xbutil flash scan. When you see that it is an indication XRT has made communication with the SC and is able to program it. If you do not see this, you are not programming the SC. The U50 relies heavily on the SC being operational, so if it is not, you can see unpredictable behavior in the card. 

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------

View solution in original post

0 Kudos
Highlighted
Participant
Participant
545 Views
Registered: ‎06-27-2018

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi,

I could solve, thank you for your support!

> What kernel ($uname -r) and operating system ($cat /etc/*-release) are you using? 

iwabuchi@PX04101:~/workspace/u50$ uname -r
4.15.0-101-generic
iwabuchi@PX04101:~/workspace/u50$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

> It also looks like XRT is skipping pyopencl install.

I had installed pyopencl before XRT, so deleted pyopencl and reinstall XRT.

iwabuchi@PX04101:~/workspace/u50$ sudo -E apt install ./xrt_201920.2.5.309_16.04-amd64-xrt.deb
    :
DKMS: install completed.
Finished DKMS common.postinst
Loading new XRT Linux kernel modules
Installing MSD / MPD daemons
Installing pyopencl...
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
/home/iwabuchi/.local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
WARNING: The directory '/home/iwabuchi/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pyopencl
  Downloading pyopencl-2020.1-cp27-cp27mu-manylinux1_x86_64.whl (726 kB)
     |????????????????????????????????| 726 kB 741 kB/s
Requirement already satisfied: decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from pyopencl) (4.4.2)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python2.7/dist-packages (from pyopencl) (1.10.0)
Requirement already satisfied: numpy in /usr/local/lib/python2.7/dist-packages (from pyopencl) (1.16.6)
Requirement already satisfied: appdirs>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from pyopencl) (1.4.4)
Requirement already satisfied: pytools>=2017.6 in /usr/local/lib/python2.7/dist-packages (from pyopencl) (2020.2)
Installing collected packages: pyopencl
Successfully installed pyopencl-2020.1

> I think our tools want to use python 2, can you check which python is installed on your system? 

I had been used python3.5, changed to 2.7.

 

> you must remove power completely 

I tried xbmgmt flash --factory_reset, and power off, wait about 5min. after that executed xbmgmt flash --update. Again power off, and wait 5min, then I can see [SC=5.0.2] at "running on FPGA", too. xbutil validate also succeeded.

iwabuchi@PX04101:~$ sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan
[sudo] password for iwabuchi:
Card [0000:01:00.0]
    Card type:          u50
    Flash type:         SPI
    Flashable partition running on FPGA:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0.27]
    Flashable partitions installed in system:
        xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0.27]
0 Kudos
Highlighted
Visitor
Visitor
320 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

I am having the same trouble getting the u50 to validate.

%> xbmgmt flash --scan

Card [0000:02:00.0]
Card type: u50
Flash type: SPI
Flashable partition running on FPGA:
xilinx_u50_xdma_201920_1,[ID=0x5daa78ef],[SC=5.0.13]
Flashable partitions installed in system:
xilinx_u50_xdma_201920_1,[ID=0x5daa78ef],[SC=5.0.13]

%> xbutil validate -d 0000:02:00.0

INFO: Found 1 cards

INFO: Validating card[0]: xilinx_u50_xdma_201920_1
INFO: == Starting Kernel version check:
INFO: == Kernel version check PASSED
INFO: == Starting AUX power connector check:
AUX power connector not available. Skipping validation
INFO: == AUX power connector check SKIPPED
INFO: == Starting PCIE link check:
LINK ACTIVE, ATTENTION
Ensure Card is plugged in to Gen3x16, instead of Gen2x16
Lower performance may be experienced
WARN: == PCIE link check PASSED with warning
INFO: == Starting SC firmware version check:
INFO: == SC firmware version check PASSED
INFO: == Starting verify kernel test:
ERROR: Failed to find 22_verify.py or verify.xclbin, Shell package not installed properly.
ERROR: == verify kernel test FAILED
INFO: Card[0] failed to validate.

ERROR: Some cards failed to validate.

I make sure to check that the correct python version is installed.

%> ./XRT/src/runtime_src/tools/scripts/pyopencl_install_check.sh

Python version: 2.7
Pip version: 20.1.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
The script wasn't able to determine the issue with pyopencl
installation or import

I am not sure to which issue the above is referring? So, I check the pyopencl installation.

%> sudo pip install pyopencl

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: pyopencl in /usr/lib64/python2.7/site-packages (2020.2)
Requirement already satisfied: decorator>=3.2.0 in /usr/lib/python2.7/site-packages (from pyopencl) (3.4.0)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python2.7/site-packages (from pyopencl) (1.9.0)
Requirement already satisfied: numpy in /usr/lib64/python2.7/site-packages (from pyopencl) (1.7.1)
Requirement already satisfied: appdirs>=1.4.0 in /usr/lib/python2.7/site-packages (from pyopencl) (1.4.4)
Requirement already satisfied: pytools>=2017.6 in /usr/lib/python2.7/site-packages (from pyopencl) (2020.2)

However, from within the /opt/xilinx/dsa/xilinx_u50_xdma_201920_1/test directory I attempt to run xbtest; and get an error related to a missing Xilinx OpenCL library.

%> sudo ./xbtest

./xbtest: error while loading shared libraries: libxilinxopencl.so.2: cannot open shared object file: No such file or directory

I have uninstalled XRT, reinstalled and did not complete a successful validate.

So, what am I missing?

Any insights into what I may have missed?

Maybe it is the wrong XRT and platform installation? Should I be using Xilinx_u50-gen3x16-xdma-201920.3-2784799* rather than xilinx-u50-xdma-201920.1-2699728*?

Any assistance regarding this matter would be greatly appreciated!

I apologize in advance if there is already a much more recent thread that addresses this issue more recently.

Thank you for your time; and have a great day!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
311 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @freeze 

You want to use the following versions

numpy <= 1.16.6
pip <= 20.1.1
pyopencl <= 2019.1.2
pytools <= 2020.1
setuptools <= 0.9.8

You could try uninstalling python and pyopencl and pip and trying again. Otherwise double check that the versions above are what you have in your system.

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
Highlighted
Visitor
Visitor
243 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

Here are steps that I took; and still not the desired result -

1) Removed XRT. This removed all /opt/xilinx/* directory content as expected.

2) Verified that python 2.7.5 was the latest drop.

3) Removed all packages that were members of the list you supplied.

4) Removed pip.

5) Reinstalled pip

6) Added all package versions that you listed.

7) Reverted the U50 image to factory default.

I then went through the entire process of installing the XRT and deployment RPMs; flashed the card and verified that factory default had been replaced by the deployment image; used lspci and xbmgmt to verify the card had been installed correctly.

9) Attempted to validate the DMA and memory with xbutil, but the same issue occurred.

---

INFO: == Starting SC firmware version check:
INFO: == SC firmware version check PASSED
INFO: == Starting verify kernel test:
ERROR: Failed to find 22_verify.py or verify.xclbin, Shell package not installed properly.
ERROR: == verify kernel test FAILED
INFO: Card[0] failed to validate.

---

What might you suggest as a next step?

Here is the CentOS release and kernel version for reference.

%> cat /etc/centos-release

CentOS Linux release 7.7.1908 (Core)

%> uname -r

3.10.0-1062.18.1.el7.x86_64

0 Kudos
Highlighted
Visitor
Visitor
274 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

I made sure that Pyton 2.7.5 was up-to-date and updated the packages to the proper versions.

I then reverted the U50 image to factory default, removed xrt and reinitiated the hardware installation and verification process.

Unfortunately, I was left with the same bad result.

Any suggestion(s) on where I go from here?

0 Kudos
Highlighted
Visitor
Visitor
216 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

A quick update -

After purging my development system and starting the installation from scratch, I was able to successfully run hardware emulation (hw_emu).

Be sure to run the host application from within the build_* directory; otherwise, you will get an error related to not having the XRT_XILINX shell variable set, of which, is should not be the case because I have it is set in ~/.bash_profile. Not sure what is going on here?

I was also able to build the target application for running on the hardware, but have been unable to successfully run the binary (*.xclbin).

0 Kudos
Highlighted
Visitor
Visitor
174 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

Can anyone comment on the output of */src/runtime_src/tools/scripts/pyopencl_install_check.sh?

Python version: 2.7
Pip version: 20.1.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
XRT installation didn't fail because of a pyopencl issue
Please recheck the installation log

So, does this mean pyopencl failed to install for some other reason?

0 Kudos
Highlighted
Visitor
Visitor
172 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

Just to stay true to this thread,

%> xbutil validate -d 0000:02:00.0

still fails as before.

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
160 Views
Registered: ‎10-19-2015

Re: Alveo U50 xbutil validate failed.

Jump to solution

Hi @freeze 

The tool seems to be complaining about not finding the verify xclbin.

 We've updated the shells and xrt, can you update to the latest? 

Regards,

M

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Highlighted
Visitor
Visitor
141 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

These new shells (or platforms) did the trick. As least as far as U50 hardware validation goes.

I assume hardware emulation will work as before, but I still need to test my custom RTL Kernel on the U50.

I will provide a further update(s) regarding emulation and target hardware execution.

Thank you for your time and assistance!

0 Kudos
Highlighted
Visitor
Visitor
99 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

The hardware emulation builds and runs as advertised.

NOTE: As mentioned previously in this thread, you do not need to run the host application from within the ./build_dir.hw_emu.* directory.

[#]$ ./host ./build_dir.hw_emu.xilinx_u50_gen3x16_xdma_201920_3/vadd.xclbin
Found Platform
Platform Name: Xilinx
INFO: Reading ./build_dir.hw_emu.xilinx_u50_gen3x16_xdma_201920_3/vadd.xclbin
Loading: './build_dir.hw_emu.xilinx_u50_gen3x16_xdma_201920_3/vadd.xclbin'
Trying to program device[0]: xilinx_u50_gen3x16_xdma_201920_3
INFO: [HW-EM 01] Hardware emulation runs simulation underneath. Using a large data set will result in long simulation times. It is recommended that a small dataset is used for faster execution. The flow uses approximate models for DDR memory and interconnect and hence the performance data generated is approximate.
Device[0]: program successful!
TEST PASSED
INFO::[ Vitis-EM 22 ] [Time elapsed: 0 minute(s) 14 seconds, Emulation time: 0.0347903 ms]
Data transfer between kernel(s) and global memory(s)
krnl_vadd_rtl_1:m_axi_gmem-HBM[0] RD = 2.000 KB WR = 1.000 KB

INFO: [HW-EM 06-0] Waiting for the simulator process to exit
INFO: [HW-EM 06-1] All the simulator processes exited successfully

The target hardware builds, but does not run successfully.

Before showing the results - here is verification that the hardware is configured/installed correctly.

[root@# krnl]# sudo lspci -vd 10ee:
02:00.0 Processing accelerators: Xilinx Corporation Device 5020
Subsystem: Xilinx Corporation Device 000e
Flags: bus master, fast devsel, latency 0, NUMA node 0
Memory at da000000 (64-bit, prefetchable) [size=32M]
Memory at dffe0000 (64-bit, prefetchable) [size=128K]
Capabilities: [40] Power Management version 3
Capabilities: [60] MSI-X: Enable+ Count=32 Masked-
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [1c0] #19
Capabilities: [e00] Access Control Services
Capabilities: [e10] #15
Capabilities: [e80] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
Kernel driver in use: xclmgmt
Kernel modules: xclmgmt

02:00.1 Processing accelerators: Xilinx Corporation Device 5021
Subsystem: Xilinx Corporation Device 000e
Flags: bus master, fast devsel, latency 0, IRQ 18, NUMA node 0
Memory at dc000000 (64-bit, prefetchable) [size=32M]
Memory at dffc0000 (64-bit, prefetchable) [size=128K]
Memory at c0000000 (64-bit, prefetchable) [size=256M]
Capabilities: [40] Power Management version 3
Capabilities: [60] MSI-X: Enable+ Count=32 Masked-
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [e00] Access Control Services
Capabilities: [e10] #15
Capabilities: [e80] Vendor Specific Information: ID=0020 Rev=0 Len=010 <?>
Kernel driver in use: xocl
Kernel modules: xocl

[root@# krnl]# xbmgmt flash --scan
Card [0000:02:00.0]
Card type: u50
Flash type: SPI
Flashable partition running on FPGA:
xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0]
Flashable partitions installed in system:
xilinx_u50_gen3x16_xdma_201920_3,[ID=0xf465b0a3ae8c64f6],[SC=5.0.27]

[root@# krnl]# xbutil validate -d 0000:02:00.0
INFO: Found 1 cards

INFO: Validating card[0]: xilinx_u50_gen3x16_xdma_201920_3
INFO: == Starting Kernel version check:
INFO: == Kernel version check PASSED
INFO: == Starting AUX power connector check:
AUX power connector not available. Skipping validation
INFO: == AUX power connector check SKIPPED
INFO: == Starting PCIE link check:
LINK ACTIVE, ATTENTION
Ensure Card is plugged in to Gen3x16, instead of Gen2x16
Lower performance may be experienced
WARN: == PCIE link check PASSED with warning
INFO: == Starting SC firmware version check:
SC FIRMWARE MISMATCH, ATTENTION
SC firmware running on board: 5.0. Expected SC firmware from installed Shell: 5.0.27
Please use "xbmgmt flash --scan" to check installed Shell.
WARN: == SC firmware version check PASSED with warning
INFO: == Starting verify kernel test:
INFO: == verify kernel test PASSED
INFO: == Starting DMA test:
Host -> PCIe -> FPGA write bandwidth = 5129.85 MB/s
Host <- PCIe <- FPGA read bandwidth = 5641.5 MB/s
INFO: == DMA test PASSED
INFO: == Starting device memory bandwidth test:
............
Maximum throughput: 52428 MB/s
INFO: == device memory bandwidth test PASSED
INFO: == Starting PCIE peer-to-peer test:
P2P BAR is not enabled. Skipping validation
INFO: == PCIE peer-to-peer test SKIPPED
INFO: == Starting memory-to-memory DMA test:
M2M is not available. Skipping validation
INFO: == memory-to-memory DMA test SKIPPED
INFO: Card[0] validated with warnings.

INFO: All cards validated successfully but with warnings.

After completing hardware emulation and before starting hardware target testing, the XCL_EMULATION_MODE environment was reset.

[root@# krnl]# ./host ./build_dir.hw.xilinx_u50_gen3x16_xdma_201920_3/vadd.xclbin
XRT build version: 2.6.655
Build hash: 2d6bfe4ce91051d4e5b499d38fc493586dd4859a
Build date: 2020-05-22 19:05:52
Git branch: 2020.1
PID: 12193
UID: 0
[Tue Jul 7 19:59:28 2020]
HOST: #.com
EXE: /*/krnl_vadd_rtl3/krnl/host
[XRT] ERROR: Failed to open XDP library '/opt/xilinx/xrt/lib/liboclxdp.so'
libxdp.so.2: cannot open shared object file: No such file or directory
ERROR: [HW-EM 08] Please set XCL_EMULATION_MODE to "hw_emu" to run hardware emulation.
[XRT] ERROR: No devices found
ERROR: [HW-EM 08] Please set XCL_EMULATION_MODE to "hw_emu" to run hardware emulation.
[XRT] ERROR: No devices found
../../Vitis_Accel_Examples//common/includes/xcl2/xcl2.cpp:50 Error calling err = cl::Platform::get(&platforms), error code is: -1001

Not sure why the invocation of the hardware test 'assumes' it is still hardware emulation?

The hardware test build was performed from a clean build directory; therefore, no reference to emulation should have been present.

[root@# krnl]# env | grep XCL
XCL_EMULATION_MODE=

The library should be found because the XILINX_XRT environment variable is set.

[root@# krnl]# env | grep XI
XILINX_VITIS=/mnt/coolsilicon-proj/Vitis/2020.1
XILINX_XRT=/opt/xilinx/xrt
XILINX_VIVADO=/mnt/coolsilicon-proj/Vivado/2020.1

Please advise on how I might go about dealing with this issue. Thanks!

0 Kudos
Highlighted
Visitor
Visitor
98 Views
Registered: ‎07-03-2020

Re: Alveo U50 xbutil validate failed.

Jump to solution

The oclxdp library does exist -

[root@# krnl]# ls /opt/xilinx/xrt/lib/ | grep oclxdp
liboclxdp.so
liboclxdp.so.2
liboclxdp.so.2.6.655

0 Kudos