10-11-2012 11:39 PM
I'm using the GTX reference clock in as an input to an MMCM (allowed in Virtex-6 designs). The MMCM has problems generating the proper output clock frequencies when locking to this clock signal. I noticed a difference when configuring via download cable versus loading from a flash.
As I narrowed it down, it appears that when loading from download cable, the internal termination is already enabled at LOCK (incoming clock signal looks much better). However, when loading from flash, the internal termination is not enabled (clock signal quality is not nearly as good). I trigger on the LOCK signal of the MMCM and I can see the difference in the external clock signal. It doesn't look terminated when loading directly from flash.
I have eliminated (as much as possible) any possiblity of this being an external clock quality issue or power sequencing issue. I can cause the problem to appear or disappear, at will, by just pulsing PROGRAM (load from flash) or downloading via cable with Chipscope or Impact.
I have also tried changing the bitgen startup options such as Match_cycle and LCK_cycle but so far, no change.
Any help or advice is greatly appreciated.
10-15-2012 12:44 PM
The GTX reference clock inputs have dedicated 100 ohm termination and these would not be affected by the configuration bitstream. The likely problem is that you are not keeping the MMCM in reset until the device has come out of reset.
10-16-2012 04:48 AM
Ok, that is good to know. I appreciate the reply and the information.
However, isn't the MMCM reset an optional input? The clocking wizard doesn't provide it unless you specifically tell it to. Am I required to do anything with it? I only added it to debug this problem specifically. Without adding the optional reset and then 'manually' resetting it at a later point, the MMCM will lock but puts out an incorrect frequency. This doesn't happen when using configuring with the cable.
In the meantime, I did find a change that appears to fix the problem. I had 0.1uF external AC coupling caps on my 156.25MHz GTX ref clock. I changed these to 10nF caps and it appears to fix this problem. The input clock looks much better at locking time, and it looks clean like it does when configuring via download cable (also at lock time). Note, the input clock also looked just as 'clean' before, with 0.1uF caps, and downloading via cable. So this isn't a static problem.
Could the calibration input circuit on the MMCM (new in Virtex 6 I believe) somehow be causing this due to the extra capacitance? I had the same basic design with Virtex 5/DCMs and no problems. It also only seems to happen when using the GTX ref clock as a 'free running' source to an MMCM. This wasn't possible in Virtex 5.
10-18-2012 07:54 AM
I'm having a very similar issue where the frequency from an MMCM is incorrect only if the FPGA (LX75T) is configured from Flash (in this case SPI) i.e. OK every time if configured from JTAG.
The 100nF / 10nF cap comment is very interesting as I'm AC coupling the MGTRefClk with 100nF capacitors as well. However why does this make a difference? I don't understand the mechanism of what's going on here.
I'm using version 13.4 of the ISE tools if that's relevant.
Any help, gratefully received.
10-18-2012 01:28 PM - edited 10-18-2012 01:29 PM
My speculation in both cases is that the AC coupling capacitors have not fully charged by the time that the MMCM starts to lock after configuration. In the case of flash/SPI the configuration time will be much faster than with JTAG cable download and the 10nF capacitor will be fully charged quicker than the 100nF.
I'm not fully on board with this answer without fully understanding the system clock implementationthat is driving the MGT reference clock inputs. If the startup of this is delayed or controlled by the FPGA then I think my answer is likely, but if this starts up with the board and then it should be ready to go before the FPGA could finish configuring in both cases. The only way to know is to put a scope probe on the clock after the AC coupling caps.
10-18-2012 01:45 PM
Thanks for the reply. It's clearly a strange one which presumably hasn't been experienced before. My design simply has a 125MHz LVDS oscillator driving the MGTRefClk inputs through the AC capacitors. I hold the INIT_B line low for 200ms after completion of the power sequencing so everything should be steady before configuration starts.
10-18-2012 05:22 PM
Note that any at-configuration-end changes to the internal RX termination voltage (RCV_TERM_VTTRX,RCV_TERM_GND), or other RX termination settings, could also cause the same sort of cap-charging-induced transients as Ed mentioned earlier for oscillator startup.
A quick test would be to slow down the config rate of the Flash, or speed up the JTAG, to see if it has any effect.
Better would be to monitor the clock ( before and after the caps ) with a good FET probe at startup to look for any odd common-mode transients on the clock lines at the end of configuration.
I have also seen strange loading effects in some LVDS driver families. ( ie, in the absence of any DC termination on the LVDS driver side of the caps, a cap-charging startup event could be mucking with the LVDS driver's common mode control loop circuitry )
10-18-2012 06:20 PM
A couple more notes:
Datasheet says 100 nf:
DS152 (v3.4) Table 18
" CEXT Required external AC coupling capacitor –100 nF
But UG366 (v2.6), Fig 5-10, shows 10 nf clock caps
Also, UG366 Fig 5-9 shows the clock termination voltage as fixed at 4/5 MGTAVCC
( it's the data lines that have the switchable RCV_TERM settings )
10-18-2012 11:02 PM
Well I'm glad to see that I'm not the only one having this problem. This has driven me crazy for quite a while. When the clock frequency is wrong, it is a difficult problem to debug, since things may basically work, but not consistently.
In my case, the FPGA takes over 2 seconds to configure from Flash. This is an HX250T with quite a large bit file, using the default bitgen options, which make CCLK quite slow. So I don't think the difference in configuration time has anything to do with it. It also doesn't matter if I configure from Flash after power on, or force a Flash config by pulsing the PROGRAM pin low.
I have scope traces of the incoming clock at lock time. You can clearly see the difference in the incoming clock quality, at MMCM lock time, between configuring via Flash, and download cable. I no longer see this difference, after changing the capacitors.
Note, the locking and generating an incorrect frequency is an intermittent problem. Well, at least of 10 boards, it happened all the time on 3, but almost never on the other 7 (but I have seen it at least once on one of those 7). However, the incoming clock quality at LOCK time is NOT intermittent at all. It is ALWAYS better when configuring via download cable, versus configuring via Flash. Except of course, if I change the caps, then it is the same in both cases. I think it just happens to work in most cases.
I have two clock generators with four outputs each, and I've tried this using all as inputs to the MMCM. All have better clock quality at LOCK time, with configuration via cable versus configuration via Flash. So it's pretty clear to me, that the FPGA is doing something different, when configuring from Flash, versus download cable, that affects the incoming clock.
10-18-2012 11:14 PM - edited 10-18-2012 11:18 PM
Here are the scope traces. I'm not sure how to do more than one attachment per post, so this is the "failing" case first. The top signal is the incoming reference clock, just after the AC caps. This is with a high quality differential probe.
The middle signal is the same as the top, however, coming back out of the FPGA. The bottom signal is LOCK from the MMCM.
10-18-2012 11:16 PM - edited 10-18-2012 11:17 PM
2nd scope trace, the "passing" case. This is with the capacitor change.
Note, I see the same clock quality, in the "failing" case, but later on. I don't see good quality at LOCK however, I do later. So the FPGA has finished or done something, that stopped affecting the clock quality.
10-19-2012 05:08 AM
> The top signal is the incoming reference clock, just after the AC caps.
That does look like the termination is being switched out or changed somehow.
I would trigger mid-record on DONE doing high, and look at different timescales to see when the waveform changes.( you could also re-configure, and see if it goes good->bad at the start of configuration)
In past families, probing the DCI reference resistor gives a clue as to when termination calibrations are in effect, not sure whether that applies to the GTX cailbrator. ( Answer Records 12573 and 13012 used to have pictures of the V2 DCI adjustment waveforms, but they seem to have vanished )
>This is with a high quality differential probe.
Probing each side of the cap single-ended might also be informative ( as the differential probe will subtract out any common mode shifts. )
10-25-2012 10:14 PM
Just an update on this. I haven't had time to try other experiments, since at this point I have it working with the capacitor changes.
My conclusions so far:
- It doesn't seem to have anything to do with power up or timing of anything external to the FPGA, since the problem happens when configuring from the Flash by pulsing program low as well (incoming clocks and power fully stable).
- I don't see how the charging time of the external capacitors could be a problem, unless something inside the FPGA is changing later (after DONE and LOCK) that affects the charging time.
- It doesn't appear to be the MMCM that is somehow influencing the external signal since keeping it in powerdown or reset doesn't affect the clock quality and change.
- My best speculation is that it is something in the FPGA, that is changing, long after DONE and MMCM LOCK. Unfortunately, it's not easy to trigger on the change, since I have to trigger on "when the clock quality improves" and it's far enough away from DONE or LOCK such that you can't "see" it on the scope at such a long timescale. This also affects the ability of the scope to trigger on the change.
- This doesn't only apply to the GTX reference clock (input on IBUFDS_GTXE1), it also happens when using other clocks inputs on non-GTX ref clocks with an IBUFGDS.