05-13-2020 04:53 PM
Here is a strange problem. About 10% of the boards I have built using the xcku035-fbva676-1-c are giving me problems when I try to access the EEPROM (Cypress S25FL128) after configuration. They configure the FPGA without any problem but after configuration, when I try to read from or write to the EEPROM through the STARTUPE3, they fail to pull down the chip select pin (FCS_B) or toggle the SDO pin (D). The signals go into the startup instance but never come out. It is almost like the tri-state signal is asserted but as you can see in the instantiation I used below, it isn't. Also, this only happens on some boards. Most of the boards we built work just fine. A few boards fail initially but then start working after a short while (warm-up time?). One or two boards never work. I have tried this in Vivado 17.3 and 19.2. Same result.
STARTUPE3_inst : STARTUPE3
generic map (
PROG_USR => "FALSE", -- Activate program event security feature. Requires encrypted bitstreams.
SIM_CCLK_FREQ => 0.0 -- Set the Configuration Clock Frequency (ns) for simulation
port map (
CFGCLK => open , -- 1-bit output: Configuration main clock output
CFGMCLK => open , -- 1-bit output: Configuration internal oscillator clock output
DI => sdi_from_pin , -- 4-bit output: Allow receiving on the D input pin
EOS => open , -- 1-bit output: Active-High output signal indicating the End Of Startup
PREQ => open , -- 1-bit output: PROGRAM request to fabric output
DO => sdo_to_pin , -- 4-bit input: Allows control of the D pin output
DTS => "0010", -- 4-bit input: Allows tristate of the D pin
FCSBO => eeprom_cs, -- 1-bit input: Controls the FCS_B pin for flash access
FCSBTS => '0', -- 1-bit input: Tristate the FCS_B pin
GSR => '0', -- 1-bit input: Global Set/Reset input (GSR cannot be used for the port)
GTS => '0', -- 1-bit input: Global 3-state input (GTS cannot be used for the port name)
KEYCLEARB => '1', -- 1-bit input: Clear AES Decrypter Key input from Battery-Backed RAM (BBRAM)
PACK => '0', -- 1-bit input: PROGRAM acknowledge input
USRCCLKO => eeprom_cclk, -- 1-bit input: User CCLK input
USRCCLKTS => '0', -- 1-bit input: User CCLK 3-state enable input
USRDONEO => '1', -- 1-bit input: User DONE pin output control
USRDONETS => '0' -- 1-bit input: User DONE 3-state enable output
Any idea's? I think I'll try a heat gun and the cold spray...
05-13-2020 05:45 PM
No change of behavior when heated or cooled.
I should also point out that when we drive the STARTUPE3 DO pins, the ILA shows that they toggle as expected. The ILA also shows the DI pins doesn't toggle at all. Once again, it looks very much like the tristate signal is asserted...but it isn't !!!!
05-14-2020 12:34 PM
I have ILA outputs from a working board and a failing board. I'm reading 8 bytes from address 0xff08d0. The EEPROM contains decimal 1 through 8. This is the ILA output from a working board:
For the working board, a scope on the eeprom pins matched what the ILA is displaying perfectly.
This is the ILA output from a failing board:
In this case the only activity visible on the scope is the toggling of the CCLK pin. The CS and DO pins are high.
It looks to me like there is something broken in the STARTUPE3 block. I'm afraid my only option at this point is to scrap the board or replace the FPGA's. Very disappointing especially when it is affecting such a high percentage if the boards.
05-16-2020 10:11 AM
One more interesting data point. I thought I may be able to make these boards somewhat functional by changing the design to fail more gracefully. I'd detect the failing condition by observing when D0(0) and DI(0) did not match. The failing boards would then no longer be in-system programmable and I would have to program them manually using the jtag access to the configuration memory using the Vivado hardware manager.
It turns out that I cannot re-program the eeprom on these boards in this way. The JTAG access is also failing. Interestingly, the CS works but the DO(0) line doesn't. A board without this issue can be reprogrammed fine in this way.
Clearly the issue with this board has gotten worse. The EEPROM was programmed fine at some point. There must have been a voltage, temperature, static discharge ... that has damaged something in the STARTUPE3 block. It is quite strange that the board still configures fine and all the other functionality other than the EEPROM access works just fine. Nevertheless, this was my last option other than changing the FPGA or scrapping the board.
05-16-2020 06:06 PM - edited 05-16-2020 06:32 PM
I appreciate people like you who take the time to document all the things they have done to solve a problem - even if the solution is not found. So, thanks!
1) this kinda seems like a wrong voltage somewhere, sometime
2) check your set_property constraints for CONFIG_VOLTAGE and CFGBVS
3) check physical connection of FPGA CFGBVS pin (ref UG570) - maybe a loose jumper on CFGBVS?
4) use oscope to see how voltage supply on EEPROM is powering up - and simultaneously watch how VCCO for FPGA bank-0 is powering up
5) use oscope to check voltage on IO pins of EEPROM during power-up. Are any above absolute max spec for Vin on bank-0 (ref Table 1, DS892)?
05-26-2020 03:59 PM
Thanks for the suggestions. I finally got a chance to check the voltages on the scope. Everything looks OK with no significant overshoot. Vint, Vaux and Vio are nicely sequenced. The EEPROM and bank 0 are powered from the same 3.3V rail so there are no differences in the timing of the supplies. The 3.3 rail is actually 3.34V so within the 5% tolerance.
I do see a roughly 20% over/under-shoot on the clock signal to the eeprom. The other signals have more like 10% overshoot. I don't think that would be enough to damage the IO ports ... I certainly hope not.
I'm planning to replace the FPGA. It will be interesting to see if that fixes it.
07-15-2020 06:11 PM
Did you fix this ? we have a similar problem with KU040 and external Micron NOR flash.
The STARTUPE3 blocks signals randomly.
The EOS sometimes transitions back to low !
we see EOS signal not remaining HIGH ?, which indicated END OF STARTUP
So this prevents the AXI QUAD from programming, as the pins go tri-state again.
Something is affecting EOS, which affects AXI_QUAD, which stops SPI bus
We are re-building IP for HWICAP, and AXI_QUAD and STARTUPE3 to ensure the EOS signal is routed correctly, to avoid any conflict
07-15-2020 08:59 PM
I'm afraid that I haven't gotten around to changing the FPGA.
It sure sound like you have the same problem. Does it affect all or just some of your boards? If you do find any answers please post it here!
07-17-2020 11:02 AM
We removed the HWICAP module, and the problem with AXI-QUAD-SPI and access to STARTUPE3 pins (SPI Pins) was fixed, i.e the EOS signal behaves correctly
Hence we have a conflict when trying to instantiate all 3 modules, waiting Xilinx to educate us...
07-17-2020 11:13 AM
Interesting, removing the ICAPE may solve the problem but then we would have to power cycle the FPGA to get a newly programmed configuration to take effect. Not a great option but perhaps better than changing FPGAs.
I sure hope Xilinx will provide a solution.
07-31-2020 04:01 AM
We have similar problem.
We have design where STARTUPE3 used SPI pins (directly) and on the same board (with xcku060) with some different details in design (which hasn't to affect on my module with STARTUPE3) I saw that MISO (DI) signal is always HIGH.
But if I load my original design, STARTUPE3 work fine. Constraints settings is equal (in my design and my coworkers).