UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Contributor
Contributor
482 Views
Registered: ‎05-22-2018

PCIe add on card hot insertion & removal

Hello all,

I am designing a PCIe addon card with XCZU21DR. Its a x4 lane board. The power supply to the board will be sourced through a Server SMPS 12V connector. ( not using 12V power pins of PCIe edge connector). Please check the attached schematic for connections. I have following doubts :

1. Switch ON the server power button --> Add on board will be powered up by SMPS instanlty  --> after some boot time Server will be up : Will the card be detected in this case?

2. If I power cycle the board while being inserted & Server running : will the card be detected ?

3. If I restart the server while board being in the slot & powered up  : What will happen ?

How can I make sure that in case 2 board will be detected ? (I envisage case 2 as : Different bitstreams will be frequqently downloaded while in development phase).

Is there any chance that the board might go bad ?

pcie_connector_sch.JPG
0 Kudos
5 Replies
Xilinx Employee
Xilinx Employee
458 Views
Registered: ‎07-26-2012

Re: PCIe add on card hot insertion & removal

As 1 and 3, the card is powered from the server before, the card should be detected. However, In case 2, the configuration information in the card is lost but the host system does not setup agian after power cycleing of the card. As a result, the card got not to work after that.

0 Kudos
Contributor
Contributor
452 Views
Registered: ‎05-22-2018

Re: PCIe add on card hot insertion & removal

Thanks a lot.

Can you suggest me how to make case 2 work ?

It will very frustrating to restart the server everytime after loading bitstream( while in development phase).

0 Kudos
Xilinx Employee
Xilinx Employee
440 Views
Registered: ‎07-26-2012

Re: PCIe add on card hot insertion & removal

It is impossible that PCIe bus system can detect the bit stream re-loaded FPGA because that the PCIe setup is doen by BIOS or OS. The only way is "Tandem with Field Update" which can replace the user design while PCIe link remains active. However, you have to re-design for the methodoloty and it would frustrate you rather than re-starting the server.

388 Views
Registered: ‎01-08-2012

Re: PCIe add on card hot insertion & removal

Of course it's impossible ... if you don't know how to do it.

There's a whole section in the PCIe standard about hot swap.  This needs to read and understood.

I've used it in production designs, with an FPGA (containing a PCIe EP) that wasn't loaded at boot time.  This allowed me to boot, then configure the FPGA at my leisure, then trigger the PCIe rescan from software.

My experience is that:

  • It's nigh on impossible if your PCIe switch doesn't expect to support hot swap, or the motherboard designer didn't connect the right signals, or the PCIe switch has bugs (I'm looking at you, Texas Instruments).
  • You need to be very careful with OS memory map allocation.  Most operating systems will expect the memory map to be set up by the BIOS.  That clearly isn't going to happen here.
  • You need to be vary careful regarding OS bugs.  Most operating systems' hot swap support isn't tested that much.  Occasionally someone will rewrite or refactor part of the kernel and break it (sometimes simply for aesthetic reasons!), not understanding what they've done.  (This happens far too often with Linux, BTW.)
  • You need to unload the driver(s) before "unplugging" (i.e. reconfiguring the FPGA).  Otherwise you'll find that some process will try to access the FPGA's memory space and not get a PCIe completion packet, which typically crashes the computer.
Contributor
Contributor
380 Views
Registered: ‎05-22-2018

Re: PCIe add on card hot insertion & removal

Thanks a lot for this perspective. 

This seems to be a doable but tough task. 

I will update this thread in future after experimenting with it

0 Kudos