04-09-2021 01:32 AM - edited 04-09-2021 04:48 AM
I am working on a project where the FPGA needs to be ready for enumeration within 100-120ms.
My Artix-7 FPGA design is AXI based and is made up of several AXI masters/slaves. For the PCIe connection, it therefore made the most sense to use the AXI Memory Mapped To PCI Express bridge from Xilinx because it conveniently offers a master AXI interface, it performs address translation between PCIe and the AXI and I do not need to worry about interpreting/generating TLP packets.
I also want to use Multiboot in order to allow safe FPGA updates in the field, however in the worst case scenario where the update design is faulty, the FPGA will fall back to golden and boot the golden design in SPIx1 always (regardless of the fact that I have a QSPI interface to the flash). Because of this, the FPGA needs a very long time to configure. In my case, the uncompressed bitstream size is 77,845,216 bits and my configuration clock is 80 MHz. Therefore, to boot the golden design using SPIx1, the FPGA needs at least 973 ms (almost a second), which is too slow to meet the PCIe enumeration requirements. I would like to be able to force the FPGA to also boot the golden design in SPIx4, in order to shorten my time by a factor of 4. Do you know if this is possible? Even then I would bring it down to 243ms, so it would still not be enough.
Other options for me right now are Partial Reconfiguration or Tandem Configuration, which would allow the PCIe part of the design to be alive fast enough for PCIe enumeration to occur, and then the rest of the design can be booted in the meantime. I would rather first try Tandem Configuration. The AXI Memory Mapped To PCI Express v2.9 however does not support Tandem Configuration (PG055 page 7), even tough the basic 7 Series Integrated Block for PCI Express v3.3 does support Tandem configuration. I know that at deep down, the AXI Memory Mapped To PCI Express uses the same 7 Series Integrated Block for PCI Express and just adds the AXIS-AXIMM translation on top, so is there any reason why Tandem is not supported with the AXI Memory Mapped to PCI Express? Is there any option for me to still get Tandem Configuration to work and not have to implement the AXIS-AXIMM translation myself? Is there any easy way to use the 7 Series Integrated Block for PCI Express and get the same AXIS-AXIMM functionality around it using Xilinx IPs?
04-20-2021 12:20 PM
Hi @anr7 ,
I understand your problem as I struggled with something similar. Here are a few things that I would suggest trying in order to get the FPGA to configure in time.
If your board is set up for QSPI, make sure that you're selecting that here: First, open implemented design. Then go to Project Manager > Settings > bitstream > configure additional bitstream settings > configuration modes > select Master SPI x4.
It may also help to have the following constraints in your .XDC file, if your flash is compatible with these changes:
set_property BITSTREAM.CONFIG.SPI_BUSWIDTH 4 [current_design]
set_property BITSTREAM.CONFIG.CONFIGRATE 66 [current_design]
set_property CONFIG_MODE SPIx4 [current_design]
set_property BITSTREAM.CONFIG.SPI_32BIT_ADDR YES [current_design]
set_property BITSTREAM.CONFIG.SPI_FALL_EDGE YES [current_design]
Hopefully this is helpful and not information that you already knew. Feel free to leave a kudo or accepted solution if it solved the problem.
04-20-2021 09:01 PM - edited 04-20-2021 09:13 PM
Thanks a lot for getting back. I am already aware of all these constraints and am already using them.
Tandem Configuration is needed for me on top of all these constraints in order to meet the PCIe enumeration timing (in my specific case, the constraints you mentioned above are not sufficient on their own given my bitstream size for Artix-7 200T + my external configuration clock of 80MHz). Sadly, Tandem does not seem to be set up in a very straightforward manner in Vivado, at least not for Artix-7 devices.
04-20-2021 09:20 PM
I don't have time to dig into this too much, but "ouch" on the tandem configuration option not being supported. Have you tried bitstream compression?
set_property BITSTREAM.GENERAL.COMPRESS TRUE [current_design]
It's a longshot and doubtful you'll get compression to less than 10%. Short of somebody else coming in with a workaround, looks like you may need to resort to PR/DFX. Hopefully somebody from Xilinx will stop in.
04-21-2021 01:30 AM
Unfortunately Tandem was never considered for the AXI Memory Mapped for PCIe IP core for 7-series. I can confirm that advice is to use the 7-Series Integrated block PCIe IP if you have a hard 100ms boot time requirement that is no achieved with another load methodology.
04-21-2021 02:18 AM
Dear @garethc ,
Thanks a lot for getting back to me.
Based on what I have seen in Vivado and user guides, I assumed that this would be the case (Tandem not being supported for the AXI Memory Mapped to PCIe core). Unfortunate.
Because of this, I have been experimenting with getting Tandem to work with the 7-Series Integrated Block for PCIe. I have followed the guidelines in its specification and also the flow in the application note for the KC705 board (xapp1179). I generated the example design for the Integrated PCIe core and adjusted the physical constraints to match the pin locations on my board. I took note of how the PBlocks are defined in both the constraints of the example design and the build_stage1.tcl script. When implementing the example design however, the Bitstream that I am generating does not work (pcie link never goes up), even though with a separate example design which is the same, except that it has Tandem disabled, it does. I can only assume I am missing a step somewhere. I also do not see the "Enabled Tandem bitstream" info message during Bitstream generation in the run.log of my implementation run.
Can you please point me to a detailed documentation of the Tandem flow for a series7 device? Or can you describe it? Thank you very much.
04-21-2021 02:45 AM
Can you create another forum post for this issue as it has moved away from the initial post question and we should not dilute forum posts with new issues.
04-21-2021 10:08 AM