What are your requirements?
A single gen1 lane can transport 2 GBit/s, each direction, gross. Cut by 10–20% for real-life max_payload_size values.
• gen1x8 is 16 GBit/s. In an FPGA, use 128 bit/125 MHz or 64 bit/250 MHz on the PCIe TLP-level IP interface (AXI4, etc.).
• gen2x8 is 32 GBit/s, use 128 bit/250 MHz.
• gen2x16 is 64 GBit/s, use 256 bit/250 MHz. x16 not supported by FPGA hard IPs to my knowledge. 256 bit interfaces typically allow multiple TLPs to be handled on the interface to better utilize the interface, this drives complexity high.
• gen3x8 is 64 GBit/s, use 256 bit/250 MHz. Requires high-end FPGAs (Virtex-7 XT/HT).
• gen3x16 is 128 GBit/s. Currently not possible in FPGAs due to the high bandwidth interface. 512 bit/250 MHs is an option, but handling of up to 4 TLPs at a time is no fun anymore, on both sides of the AXI interface.
Additionally, you need that bandwidth on the main system memory and some decent support for handling this high load on the CPU as well. A zero-copy OS/driver architecture is a must, CPU multi-core optimization for interrupt and queue handling as well, MSI-X typically required. How about the data storage path? or transport on other network interfaces?
IMHO, gen3x8 is already very ambitious and requires bleeding edge FPGA silicon, so it would be clever to start with gen2x8 for the time being. I think the dual-gen2x8 as available on the board you mentioned looks quite reasonable to me. Upgrade to gen3x8 or dual-gen3x8 when available and necessary or go for ASIC if gen3x16 is really required.
– Matthias
