UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Observer manninosi
Observer
5,016 Views
Registered: ‎06-01-2017

Timing Errors with 8-channel ADC

I am a graduate student working with FPGAs for digital pulse processing with radiation detection. 

 

My research group has designed an analog to converter board that utilizes an Analog Devices ADC (AD9681) that is 8 channel (two 4-channel blocks) that runs out serial data. The ADC out puts LVDS data clock DCO_1 and DCO_1 and LVDS frame clocks FCO_1 and FCO_2. This board is connected to an Opal Kelly board xem7350, that uses a Kinetex-7 FPGA, with a FMC. Analog devices provided a de-serializer for stream of data. 

 

The clocks are generated as follows: 

 

DCO +/-  =>  IBUFDS  => DCM (MMCM) => ISER (Provided by Analog Devices) 

 

The FCO clock goes straight to the ISER where it is buffed as the following: 

 

FCO +/- => IBUFDS => BUFG => Sys_Clk

 

The FCO from the 2nd channel bank is the clock that is used to clock the rest of the data processing for the design. 

 

I've attached the timing error summary as well as my constraints file. This summary report is from synthesis. I have noticed when I run implementation that the data from the ADC and DCO clock are not connected in the implementation schematic. I'm assuming this was done since these were some of the signals that were not meeting the timing. 

 

I've tried using one of Avrumw's responses in this link to constrain my design. 

 

https://forums.xilinx.com/t5/Timing-Analysis/Source-synchronous-ADC-timing/m-p/572674#M7688

 

I believe the overall design is accurately since some of the channels operate as expected. (i.e. I test it with a simple square wave pulse and the oscilloscope output from the FPGA is the same pulse). 

 

Please let me know if I can provide any additional information to clarify the problem. 

 

Thank you for any help!

 

0 Kudos
16 Replies
Highlighted
Historian
Historian
4,964 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

The problem is not your constraints (well, it may also be your constraints), but your clocking structure. What you have done here is illegal.

 

The ISERDES is doing deserialization, and hence it needs a high speed clock and a low speed clock. There is no question that the high speed (CLK) clock is related to DCO, but we need to think about the low speed clock (CLKDIV).

 

If we look at the documentation for the ISERDES, the CLK and CLKDIV must be "in phase". Rather than define what "in phase" means in terms of static timing, they specifically tell you what the legal clocking structures of the ISERDES are. These are shown in the 7 Series FPGAs SelectIO Resources User Guide (UG471), in the section ISERDESE2 Clocking Methods (p. 152 in version 1.6). What you are doing is not one of them.

 

So your clocking is illegal - you need to choose one of the two legal ones

  - DCO -> IBUFG -> BUFIO       -> CLK

                            -> BUFR (/N) -> CLKDIV

 

or

 

  - DCO -> MMCM: CLKOUT0        -> BUFG -> CLK

                             : CLKOUT1 (/N) -> BUFG -> CLKDIV

 

This gives you a legal clock structure. In neither of these is the FCO used as a clock.

 

For capturing the framing, you treat the FCO as a data input using another ISERDES. By definition, the framing of the FCO capture and the framing of the D capture are aligned. So you need to look at the output of the FCO ISERDES. If the framing is correct, the sampled FCO will be 00001111 (or 11110000 depending on which bit comes first). If it is in the wrong alignment, then both it and the captured data are incorrectly framed, and need to be shifted. This is done by asserting BITSHIFT of all the ISERDES (together) until the sampled FCO has the correct alignment.

 

Once you have the clocking (and BITSHIFT) implementation correct, we can worry about constraints.

 

However, you need to be certain that you can capture this interface. You don't tell us what frequency this is running at, but the capture of very high speed ADC interfaces can be very challenging. Depending on clocking structure and speedgrade, FPGAs need more than 1ns and sometimes as long as 2.5ns of stable data to be able to statically capture an interface. If your bit rate on the data is high, you may not have this much stable data window available...

 

Avrum

Observer manninosi
Observer
4,951 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Hey Avrumw, 

 

Thank you for the speedy reply, it is greatly appreciated. 

 

The device I am using is Kintex-7 the full project part number is xc7k160tffg676 with speed of "-1". The speed of the DCO is 500 MHz and the frame clock is 125 MHz. 


I am actually a little confused on the "illegal" timing. Would this current structure still allow for implementation and generating a bit-stream? The analog devices de-serializer that we are using in the project does not use a 7-series primitive and just provided us with a verilog code. The current code does work with some of the channels. 

 

I'll start trying what you have stated in your solution and report back later. Thank you so much for the help. 

0 Kudos
Historian
Historian
4,931 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

Sorry - I assumed when you said ISER, you meant the Xilinx ISERDES, but it appears this is some RTL code from Analog Devices (why is it called ISER when it is actually an input deserializer???)

 

So, I can't comment on it - I have no idea what the module is doing. It needs to take care of both framing and clock domain transferring between the DCO domain and the FCO domain. The mechanisms it uses will determine if the clocking is valid for the RTL.

 

But regardless of the framing and clock domain crossing, they both will share the same initial problem - how do you guarantee accurate sampling (on the DCO domain) of the high speed data. You say the DCO is 500MHz - is the data SDR or DDR. At SDR rates that is 2ns per bit period - assuming that the DCO/DATA uncertainty from the ADC is good (say +/-100ps) then there are clocking mechanisms that will accurately sample that (although the global clocking with MMCM that you are using may be too slow). At DDR rates, the data is 1ns per bit period - this is too fast for any static clocking mechanism.

 

If the interface is too fast for static clocking, then you will need some form of dynamic calibration...

 

I am actually a little confused on the "illegal" timing. Would this current structure still allow for implementation and generating a bit-stream?

 

Yes. Failing timing does not prevent bitstream generation. Furthermore, a design with failing timing may still work at particular combinations of process, voltage and temperature (PVT). But if it fails timing, then there are (legal) PVT combinations where it will fail.

 

As for "illegal" clocking to the ISERDES, I don't know if this fails a DRC check or not. If it doesn't fail a DRC (and hence allows a bitstream to be generated), then the same argument as above holds true - it may work at certain PVT, but will not at others.

 

Avrum

Tags (1)
Observer manninosi
Observer
4,919 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

I am also unsure of what the Analog Devices module is doing and am willing to switch to the ISERDESE2 for de-serialization. This should allow me more control over the entire process. I'm reading the UG471 and UG953 manuals describing the primitive. 

 

To provide more information on the ADC, it is running DDR at 500 MHz, with the FCO running at 125 MHz. It is also two lanes for each channel, each lane is 8 bits. The output is a 16 bit data stream, but the two MSB's are padded 0's. Before I start working on doing dynamic calibration, I am trying to discern how the ISERDESE2 is operating when I instantiate. 

 

From my current understanding is that I will need to instantiate an ISERDESE2 for each lane and do a 1:8 conversion. So as an example, this is what I'm thinking Channel 1 would look like using lane A and lane B. The adc_data streams into the FPGA as a 16 bit bus (LVDS), with bits 0 and 1 represent lanes A and B. 

 

 adc_p[0], adc_n[0] => IBUFDS_DIFF_OUT  : output => Lane_A

 adc_p[1], adc_n[1] => IBUFDS_DIFF_OUT  : output => Lane_B

 

Now for the first instantiation of the ISERDESE2 for Lane_A:

Clk => DCO_clk (after doing the buffers you described in your previous reply)

ClkDiv => FCO_clk

Data_With => "1"

D => Lane_A

Q1 => Channel_1_Data[0]

Q2 => Channel_1_Data[1]

Q3 => Channel_1_Data[2]

Q4 => Channel_1_Data[3]

Q5 => Channel_1_Data[4]

Q6 => Channel_1_Data[5]

Q7 => Channel_1_Data[6]

Q8 => Channel_1_Data[7]

 

I would repeat this then for Lane B but for Channel_1_Data[8:15]. Is this the right approach, because it seems the data sheet indicates that using "shiftout" and "shiftin" methods work for only a single lane? Or am a misunderstanding the manual? 

 

Thank you, 

 

-Mitch

0 Kudos
Historian
Historian
4,902 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

You still haven't answered the most important question - is the data 500MHz SDR or DDR. Before we discuss anything else, we need to know if you are wasting your time with static capture (and hence timing constraints).

 

As for the ISERDES, re-read my earlier post regarding CLK and CLKDIV - it is not legal to drive CLKDIV from something related to FCO; CLKDIV must be driven by an in-phase divided version of DCO. FCO will be used as a data signal for framing only (not a clock).

 

The shiftout/shiftin is only for situations where a single bit is doing more than 8:1 deserialization (i.e. 10, or 14. which are the other legal values). For 8:1, you don't need the shiftout/shiftin (so if you think you do, then you are misunderstanding the manual).

 

Avrum

Observer manninosi
Observer
4,876 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Sorry for all the jumbled mess. The ADC is running DDR at 500 MHz, so it looks like static capturing is out of the question. Though looking at these application notes in the posted link (pg. 3), the Virtex-7 has an I/O Clock Network frequency of 710 MHz, or this not related to my current design?

 

https://www.xilinx.com/support/documentation/application_notes/xapp585-lvds-source-synch-serdes-clock-multiplication.pdf

 

Also, only 1-bit is providing 8 bits of data, so I won't have to use the shift-in/shift-out. 

 

 

Thanks, 

 

-Mitch

 

0 Kudos
Historian
Historian
4,863 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

The 710MHz is the maximum frequency of the clock network - it just won't operate above that. This is different than the maximum frequency at which you can do static capture - that is specific for each interface based on static timing analysis. Basically a given clock structure and capture mechanism will require a certain amount of setup and hold time. At any PVT corner the width of the setup/hold window is pretty small, but over the combination of all legal PVTs the window gets pretty large. If the window is larger than the data valid window provided by the ADC, then you can't capture it statically.

 

In your case, the bit time is 1ns, and even with an almost perfect driver, you will lose some of that to skew. Even the best clocking and capture architecture requires more setup/hold window than 1ns.

 

So, you need to give up on finding a single configuration that works at all PVT corners, and instead use a dynamic mechanism that adjusts to the current PVT.

 

Avrum

Tags (1)
Observer manninosi
Observer
4,824 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

I am still having trouble trying to implement the ISERDESE2 into my adc interface. I created the DCO Clk as you presented in your previous reply: 

 

  - DCO -> IBUFG -> BUFIO       -> CLK

                             -> BUFR (/N) -> CLKDIV

 

I'm buffering the lvds adc data as following

 

lvds_adc_data_a1 -> IBUFDS_DIFF_OUT -> lane_a1_data

lvds_adc_data_b1 -> IBUFDS_DIFF_OUT -> lane_b1_data

 

I'm connecting the first two lanes to two different ISERDESE2, just to test channel 1. I figure once I can get this established I can this also connect the FCO clk to check the framing capture as you also mention. 

 

When I synthesize the design the TOP I/O ports are connected to the ADC interface (i.e. Adc_data, dco_clk) while checking the schematic. The check timing summary indicates there are thousands of "unconstrained Pins for maximum delay due to constant clock". I've tried to remedy this by establishing a set_max_delay constraint and it did not seem to have an affect: 

 

set_max_delay 5 -from [all_inputs] -to [all_outputs]

 

When I implement the design and check the schematic the top I/O ports are not connected to the ADC interface and appear to be open. Are these two issues correlated? 

0 Kudos
Historian
Historian
4,817 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

When I implement the design and check the schematic the top I/O ports are not connected to the ADC interface and appear to be open. Are these two issues correlated? 

 

Probably...

 

The only thing I see is that you shouldn't be using an IBUFDS_DIFF_OUT - you should be using a regular IBUFDS (or IBUFGDS, which is the same thing).

 

The set_max_delay command you have won't (normally) do anything - [all_inputs] are input ports of the FPGA and [all_outputs] are output ports of the FPGA, so this will only affect paths that go combinatorially through the FPGA (i.e. from IBUF -> combinatorial logic -> OBUF).

 

So you have to figure out why the tools aren't leaving your input ports connected. It could be a problem with the clock -  "Constant clock" seems to indicate that you have a clock pin of some clocked element connected to a constant (perhaps left unconnected - the tools will tie a floating internal net to ground). It could also be a reset - if the cells are held in continual reset then the inputs are redundant. Another option is that your design doesn't "do" anything and the tools are removing all the (what it sees as) redundant logic...

 

Avrum

Tags (1)
Observer manninosi
Observer
4,055 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Hi Avrum, 

 

Thanks a lot for your previous reply. That got the ball rolling for me and I was able to connect the top I/O pins and start running some simulations on the ISERDESE2. I believe I understand what is going on with the ISERDESE2 and I am now trying to feed the FCO_clk into the ISER as data to see what kind of pattern I'm getting from the ADC. 

 

I'm getting a weird clocking error when I attempt to feed the FCO_clk into the data input. This is how I have the FCO_clk buffers set-up: 

 

FCO_clk_Test : IBUFDS
generic map ( DIFF_TERM => FALSE, IBUF_LOW_PWR => FALSE, IOSTANDARD => "LVDS")
port map (
O => fco_clk1_ibufgds, -- Buffer output
I => lvds_fco1_p, -- Diff_p buffer input (connect directly to top-level port)
IB => lvds_fco1_n-- Diff_n buffer input (connect directly to top-level port)
);

 

 

 

And when I take the output (fco_clk1_ibufgds) and feed it into the ISERDESE2 I get the following error during implementation: 

 

[Place 30-806] Clock placer fails to converge to a solution. Please try to LOC the following instances, which may allow clock placer to converge and find a legal solution:
Driver inst: ad9681_intrfc0/dco_DIVCLK1_BUFR
Load inst: ad9681_intrfc0/Lane_A1/CH1_LaneA

Driver inst: ad9681_intrfc0/dco_clk1_BUFIO
Load inst: ad9681_intrfc0/Lane_A1/CH1_LaneA
Load inst: ad9681_intrfc0/Lane_A1/CH1_LaneA

 

 

I do not see this error when I am using the adc_data as the signal into the ISERDESE.  I've played around with different buffers like IBUFGDS and IBUFDS_DIFF_OUT to see if the outputs of these buffers would work, but still get the same error. I've also commented out any clock designation on the FCO_clk ports in the xdc file.  Any thoughts?

 

Thank you, 

 

-Mitch

0 Kudos
Historian
Historian
4,045 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

I don't recognize the error, but it might be a clock region error...

 

Both the BUFIO and BUFR can only reach resources in the same clock region. If the clock capable pin (and hence BUFIO and BUFR) are in one clock region and even one data pin (including the FCO pin) are in a different clock region, then the connection cannot be made.

 

So check if that is the case - are all your data pins (and FCO pin)  in the same clock region (and hence same I/O bank) as the clock capable pin?

 

If the pins are not in the same bank, then there are some solutions (but they start to get ugly...)

 

Avrum

Tags (1)
0 Kudos
Observer manninosi
Observer
4,040 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Great suggestion! I will check that soon and get back to you.
0 Kudos
Observer manninosi
Observer
4,018 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Hi Avrum, 

 

This is the way I've checked to ensure all the pins are in the same clock region. I opened up the "Device" which displayed all the clock regions i.e. "X0Y0, X0Y1...etc." I clicked on the nets for the adc_interface to see if any of the pins were placed in different clock regions and they were not. 

 

I believe I might have figured out the issue but am not sure how to resolve it. I have frame_clock_1 set-up as the following: 

 

adc_fco -> IBUFGDS ->  Frame_clk_1 -> clocks for all other modules

                                        Frame_clk_1 -> Data into the ISERDESE2  

 

I discovered if I put instead of using Frame_clk_1 as clocks for all the other modules in my design, I just use Frame_clk_2 and then feed only Frame_clk_1 to the ISERDESE2 it will implement correctly. 

 

Since the frame_clk is used to "Frame" the data what is the best way to have it fed into the ISERDESE2 and then also used as the clock for the other modules? My initial thought is to perhaps use a DCM and split the frame_clk into two synchronous signals and feed one into the ISERDESE and the other to modules. 

 

Thanks, 

 

-Mitch

0 Kudos
Historian
Historian
4,002 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

what is the best way to have it fed into the ISERDESE2 and then also used as the clock for the other modules

 

You don't. The only clocks you use in the system are the outputs of the BUFIO and BUFR. The BUFR and the FCO have the same frequency, so you only use the FCO as a framing signal, never as a clock. You use the output of the FCO ISERDES with the BITSLIP operation of all the ISERDES to determine framing. 

 

If you need a global clock (related to FCO / BUFR clocks) then drive the output of the BUFR to the input of a BUFG - but be aware that the outputs of the BUFR and BUFG are mesochronous - you need a (shallow) clock crossing FIFO to move data between these domains. 

 

Avrum

Tags (1)
Observer manninosi
Observer
3,934 Views
Registered: ‎06-01-2017

Re: Timing Errors with 8-channel ADC

Hi Avrum, 

 

I was able to send the BUFR to a BUFG and distribute it to the rest of the modules in my design and it appears to work well. 

 

I'm now having issues attempting to monitor the deserialized data from the ADC data. I've initially looked at the FCO_data and sent it to a BRAM. I used a state machine to write data to the BRAM, read the BRAM, then reset the address to start the process over. Basically mimicking an oscilloscope.  The FCO data is showing some combination "11110000" continuously. Though, it is not always in that pattern since I'm waiting to monitor the adc data before I start messing with BITSLIP, but it is at least acting in a way that I am suspecting

 

When I now de-serialized the adc-data and feed that into a BRAM oscilloscope state machine, I read a constant number on the PC. The number will be either one of three constant numbers when I re-start the FPGA. When I convert them into binary, there appears to be no pattern between these constant numbers. The other weird thing is the constant numbers will stay the same whether I apply a square pulse or not to the data inputs. 

 

Do you have any suggestions on how to go about trouble shooting? 

 

Thanks, 

 

-Mitch

0 Kudos
Historian
Historian
3,916 Views
Registered: ‎01-23-2009

Re: Timing Errors with 8-channel ADC

No, not really.

 

Based on the fact that you state that this doesn't change when you change the ADC's input, this doesn't sound like a capture problem, but either something outside the FPGA or inside the logic of the FPGA (getting the data to the RAM).

 

Avrum

0 Kudos