UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
357 Views
Registered: ‎08-12-2010

PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Hello,

Until recently, all the datasheets and product tables showed support for PCIe Gen3x16 and Gen4x8. A recent update shows that Gen4x8 is now only supported for the Ultrascale+ devices with HBM2 technology, for the other devices gen4 has magically disappeared.

In vivado the ip core has a hidden option to enable gen4 support which I am not allowed tho show here, but you can pm me to get the tcl line.

What is the reason for silently removing gen4 support? Was there a problem with the gen4 compliance? Did it not work in some cases?

I would like to know whether we can use gen4 or that this is a road with a dead end, we have already designed our hardware taking gen4 into account and we don't want to throw it all in the bin.

0 Kudos
1 Solution

Accepted Solutions
Xilinx Employee
Xilinx Employee
141 Views
Registered: ‎09-02-2009

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Hi Frans - sorry for the confusion.  I'm the marketing manager for PCIe.  Let me try to clarify.

When our UltraScale+ device first taped out, the PCISIG 4.0 spec was only at v0.5.  That was the spec that our first PCIe blocks used.  There were two main changes going from v0.5 to v0.7 (and subsequent versions all the way to 1.0).  First, there was an updated Electrical Idle Ordered Set and second there was a requirement for a new CTRL SKP Ordered Set.  Because of Xilinx's deep involvement in PCISIG, we knew that the Electrical Idle OS was coming and added this as option in silicon.  However, the CTRL SKP Ordered Set spec work did not even start until after we taped out.

At the time our devices started sampling, there were some devices (including Power 9 from IBM) that had an option for v0.5 (with the updated EIEOS).  Because of the 'stamp and repeat' nature of the way we build our devices, all UltraScale+ devices were built with this original PCIe block (called PCIE4 in hardware).

As time moved on, all devices that we know of moved away from having a v0.5 option.  Also, our testing had shown that the new CTRL SKP caused our UltraScale+ devices not to link train.  When we introduced the new UltraScale+ HBM devices, that included a spin to the PCIe block to add CCIX.  When we added CCIX, we took the opportunity to update the PCIe block to have support for the new CTRL SKP OS as well.  We call this block PCIE4C to differentiate it from the original PCIE4 blocks in UltraScale.  We have found that this new PCIE4C block, while not fully compliant, works at Gen4 with every Gen4 devices we have tested with so far (which is quite a few devices from different vendors).

What we have found though is that while our PCIE4 devices only work with other v0.5 devices, many people miss that note.  And because we don't see any v0.5 devices in the market anymore, we decided that it was causing more confusion and thus decided to remove the Gen4 designation from the datasheet.  It's also worth noting that the option you put above will be removed from Vivado and no longer work.

Just to recap:

1. Xilinx UltraScale+ devices PCIe block supported 4.0 v0.5.  There are no other v0.5 devices anymore.  Causes confusion and being removed.

2. Xilinx UltraScale+ HBM devices have a new PCIE4C block that is compatible to Gen4 4.0 and interoperates with other 4.0 devices even though not fully compliant.  A list of limitations can be found in PG213.  (Also note that some HBM devices have both PCIE4 and PCIE4C blocks).

3. Xilinx Versal devices (7nm) will be fully compliant to the 4.0 PCIe spec.

Sorry for the long winded explanation but hopefully this provides you some context and helps you choose the right device for your application.

Jason  

 

 

6 Replies
331 Views
Registered: ‎08-12-2010

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

I do realize that the Ultrascale+ devices were released before the PCIe Gen4 standard became official. I can also understand that because of this the PCIe hard block has limitations regardsing PCIe Gen4.

However I don't think it is fair to just remove all the gen4 from the datasheets as if it had never been there and just be silent about it. Please make a document with the limitations and conditions under which the Gen4 hard block in the lower range Virtex Ultrascale+ devices can be used if any.

0 Kudos
Moderator
Moderator
303 Views
Registered: ‎02-11-2014

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Hello @fransschreuder,

The PCIE4 hardblocks were designed for PCIe gen4 spec 0.5+ (which as the time was the currently used market standard.) The current market now is gen4 spec 0.7 - 1.0. Quite a bit changed between the 0.5 spec and the 1.0 spec. Some features that were originally optional in the 0.5 spec, became required in the 1.0 spec. The hidden gen4 switch was never publicly documented for the PCIE4 hard blocks for the above reasoning.

As you have stated the PCIE4C hardblocks (that you can find in HBM devices) DO support some gen4 spec 1.0 features (that the PCIE4 hard blocks do not support) which can be seen in the following Table from PG213. Take special consideration of the IMPORTANT note at the bottom. These features allow our PCIE4C hard blocks to interoperate with other gen4 devices, but we are not gen4 compliant.

image.png

We have also updated DS890 to reflect gen4 support which can be seen here:

image.png

Thanks,
Cory

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
289 Views
Registered: ‎08-12-2010

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Dear @coryb,

Thank you for the answer. I would like some more clarification though:

You state that the PCIE4 blocks were Gen4 0.5+ compliant, but PG213 states that the PCIE4C block is compliant to Gen4 0.5.

I would like to see a list of limitations regarding the PCIE4 blocks with respect to PCIe Gen4 1.0, rather than just removing every reference from the datasheet.

Additionally we will test our Gen4 design with the VU9P FPGA in an IBM power9 server to actually experiment with Gen4. If we don't get information from Xilinx we have to find it out for ourselves.

Regards,

Frans

0 Kudos
177 Views
Registered: ‎08-12-2010

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

@coryba comment would be highly appreciated.

0 Kudos
Xilinx Employee
Xilinx Employee
142 Views
Registered: ‎09-02-2009

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Hi Frans - sorry for the confusion.  I'm the marketing manager for PCIe.  Let me try to clarify.

When our UltraScale+ device first taped out, the PCISIG 4.0 spec was only at v0.5.  That was the spec that our first PCIe blocks used.  There were two main changes going from v0.5 to v0.7 (and subsequent versions all the way to 1.0).  First, there was an updated Electrical Idle Ordered Set and second there was a requirement for a new CTRL SKP Ordered Set.  Because of Xilinx's deep involvement in PCISIG, we knew that the Electrical Idle OS was coming and added this as option in silicon.  However, the CTRL SKP Ordered Set spec work did not even start until after we taped out.

At the time our devices started sampling, there were some devices (including Power 9 from IBM) that had an option for v0.5 (with the updated EIEOS).  Because of the 'stamp and repeat' nature of the way we build our devices, all UltraScale+ devices were built with this original PCIe block (called PCIE4 in hardware).

As time moved on, all devices that we know of moved away from having a v0.5 option.  Also, our testing had shown that the new CTRL SKP caused our UltraScale+ devices not to link train.  When we introduced the new UltraScale+ HBM devices, that included a spin to the PCIe block to add CCIX.  When we added CCIX, we took the opportunity to update the PCIe block to have support for the new CTRL SKP OS as well.  We call this block PCIE4C to differentiate it from the original PCIE4 blocks in UltraScale.  We have found that this new PCIE4C block, while not fully compliant, works at Gen4 with every Gen4 devices we have tested with so far (which is quite a few devices from different vendors).

What we have found though is that while our PCIE4 devices only work with other v0.5 devices, many people miss that note.  And because we don't see any v0.5 devices in the market anymore, we decided that it was causing more confusion and thus decided to remove the Gen4 designation from the datasheet.  It's also worth noting that the option you put above will be removed from Vivado and no longer work.

Just to recap:

1. Xilinx UltraScale+ devices PCIe block supported 4.0 v0.5.  There are no other v0.5 devices anymore.  Causes confusion and being removed.

2. Xilinx UltraScale+ HBM devices have a new PCIE4C block that is compatible to Gen4 4.0 and interoperates with other 4.0 devices even though not fully compliant.  A list of limitations can be found in PG213.  (Also note that some HBM devices have both PCIE4 and PCIE4C blocks).

3. Xilinx Versal devices (7nm) will be fully compliant to the 4.0 PCIe spec.

Sorry for the long winded explanation but hopefully this provides you some context and helps you choose the right device for your application.

Jason  

 

 

114 Views
Registered: ‎08-12-2010

Re: PCIe Gen4 x8 support for Virtex UltraScale+ devices

Jump to solution

Dear @jlawley,

Thanks for the detailed explanation. This was exactly what I was looking for and now we have at least a proper foundation on which we can base our further design decisions. You have to understand that if you specify Gen4, people already start designing with it, then if it gets removed without notice or explanation, people get pissed off.

I think that a note similar to your answer belongs in the PG213 document, and it is a pity that you are going to remove the hidden enable option from Vivado. Luckily we can choose to keep using the current version of Vivado if we decide to go on with this development and the IBM power9 device or something similar with Gen 4.0 v0.5 support.

0 Kudos