We have used the Spartan 2 series for many years without issues. We have a new design using the Spartan 3E 500 devices. We use 4 of these per board and each uses its own boot Prom. We experience very erratic boot performance where some of the FPGAs load while others do not. It is not always the same FPGA and we also observe differences between boards.
Schematics are attached. Any input is appreciated.
The only thing I see right off is that you have the DONE pin pulled up to 3.3V, while the configuration user guide says it should be pulled up to VccAux (2.5V for Spartan3E). Also the pull-up value in the configuration guide is recommended at 330 ohms to 3.3K ohms. I've had issues with Virtex-5 parts where the DONE pullup was not strong enough to transition high within one CCLK period, and occasionally the startup logic would get stuck (configuration was complete but the FPGA did not start up). I would suggest using Impact to read the device configuration status when it doesn't load correctly to see if you can get any more information about just where in the configuration process the failure occurred. Also make sure that the DONE signal at 2.5V gives a valid high level for the XCF04S. If you don't want to change the DONE pullup resistor, you could re-build the bitstream with the setting "DriveDone Yes." Note that this only works if you don't tie the DONE pins of the FPGAs together.