UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Adventurer
Adventurer
4,640 Views
Registered: ‎12-30-2015

Routable but not routed. D pin of ISERDESE2.

Jump to solution

Hi, 

 

I wrote a VHDL code where I instantiate 4 ISERDESE2.

2 of them are for 1 wire of an ADC, the other 2 for another wire.

 

Each pair, cascaded, ISERDESE2 Master and Slave, to do a 14x deserialization, DDR.

 

The sampling clock of the ADC is very slow. The chip is fed with 28.533MHz, and provides data in 2 wires, synchronous to a FRAME CLK that is 14.2665MHz. The data-centered clock is DCLK (bit clock), being 99.8655MHz. (7 times FRAME CLK). (yes, Im deserializing 2 samples). 

 

What I wanted to try, step by step, was to implement my own design, using ISERDESE2, IDELAY, and a bit align machine.

But, for this frequency, my first guess is that it might work straight away without introducing any delay control. So the only thing I did was a state machine to control bitslip. As soon as I get this working, I will carry on controlling the delays, previous to increase the frequency. Step by step. (still learning!).

 

The simulation works perfect: ADC is configured through SPI (this was tested in the real hardware and works), and when the configuration is done, a flag is sent to the Deserialization block, where:

 

-ADC DATA from both wires, differential pairs, go into an IBUFDS.

-FCLK goes into IBUFDS 

-DCLK goes into IBUFDS -> BUFIO + BUFR(/7). The CLR pin of the BUFR is set to high. 

-Process receives "configuration done" externally. CLR of BUFR is released. A counter waits the divided clock to be stable.

-Another process awaits. After some clock cycles, the RESETs for the ISERDESE2s are released. 

-State machine starts working. This state machine:

        *Puts bitslip high for the MASTER SERDES of one of the wires, during 1 clock cycle of clk_div.

        *Waits for the outputs to be trained. 

        *Checks if the outputs are correct. No? Back to set the bitslip. Yes? Then set bitslip for the second wire, deasserting the first.

        *When both cascaded SERDES are trained, the state machine puts bitslip to 0, and remains forever in STEADY STATE (a flag         called "training_done" is asserted) .

 

Now, what is the data going into the MASTER SERDES blocks? (pin D). 

D_A0M <= ADC_DA0 when training_done = '1' else
                  FCLK_n;

D_A1M <= ADC_DA1 when training_done = '1' else
                  FCLK_n;

 

This means I use FRAME CLK as the pattern to train the SERDES. When the training is done, I multiplex the ADC data.

This works perfectly in the simulation, as the data I generate (ADC data) in my testbench seems to be correct after the SERDES are trained. Now, when I try to build this in the real thing, I don't understand why D_A0M and D_A1M are not routed.

 

I initially thought, well, some processes are running with the output of BUFR, so I limited the BUFR and BUFIO to the SERDES blocks (just in case there are not enough resources in that clock domain for everything). But no, I limited them to drive just the SERDES, that's all they are driving. The rest of the logic is being driven by IBUFDSs, or BUFGs, I tried everything. 

I even put a BUFMR before the BUFR and BUFIO just in case. Nothing works. This 2 signals can't be routed.

 

Any help?

 

Timoteo

0 Kudos
1 Solution

Accepted Solutions
Historian
Historian
7,712 Views
Registered: ‎01-23-2009

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

The reason this isn't routed is that it is structurally illegal.

 

The only valid connection to the D input of the ISERDES is the output of the IBUF in the same IOB, either directly or via the IDELAY. You are trying to drive it from a MUX, which means from a fabric element - it just can't be done.

 

For bitslip operation, you don't have to calibrate each pin, you need to calibrate the interface. If I understand it, you have a DCLK (which drives the BUFIO and BUFR), two data pins and the frame clock (FCLK).

 

The way to deal with this interface is to have three sets of ISERDES pairs (master/slave). One for each of the data pins and one for the FCLK pin. Since they are all clocked by the same BUFR, they start with the same framing. Therefore, if you apply bitslip to all three of them, they will all slip one bit, but still remain in alignment with each other.

 

So, for the framing, your state machine only needs to look at the output of the ISERDES that is used for FCLK. If it isn't properly aligned, then apply bitslip to all three ISERDES and re-check the FCLK alignment. When FCLK is properly aligned, the two data bits will also be aligned.

 

Avrum

0 Kudos
10 Replies
Historian
Historian
7,713 Views
Registered: ‎01-23-2009

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

The reason this isn't routed is that it is structurally illegal.

 

The only valid connection to the D input of the ISERDES is the output of the IBUF in the same IOB, either directly or via the IDELAY. You are trying to drive it from a MUX, which means from a fabric element - it just can't be done.

 

For bitslip operation, you don't have to calibrate each pin, you need to calibrate the interface. If I understand it, you have a DCLK (which drives the BUFIO and BUFR), two data pins and the frame clock (FCLK).

 

The way to deal with this interface is to have three sets of ISERDES pairs (master/slave). One for each of the data pins and one for the FCLK pin. Since they are all clocked by the same BUFR, they start with the same framing. Therefore, if you apply bitslip to all three of them, they will all slip one bit, but still remain in alignment with each other.

 

So, for the framing, your state machine only needs to look at the output of the ISERDES that is used for FCLK. If it isn't properly aligned, then apply bitslip to all three ISERDES and re-check the FCLK alignment. When FCLK is properly aligned, the two data bits will also be aligned.

 

Avrum

0 Kudos
Adventurer
Adventurer
4,542 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

This answer was amazing, thanks. 

I'll try this, and give feedback. 

 

 

Regards,

Timoteo

0 Kudos
Adventurer
Adventurer
4,526 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution
Definitely that was the problem! Adding another pair of SERDES for FRAME solved it. I had some problems, as I was using Frame for some processes too, and vivado automatically instantiates a BUFG when it thinks it's necessary, driving the SERDES with BUFG and making it unroutable again. But as long as the user controls where the BUFG is used, that's not an issue anymore. Training working in simulation, and the real thing works, though some patterns are glitchy, which probably means I need the delay control as well.

Regards,
Timoteo
0 Kudos
Highlighted
Historian
Historian
4,516 Views
Registered: ‎01-23-2009

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

I had some problems, as I was using Frame for some processes too,

 

You shouldn't need to do this.

 

When implemented properly, the BUFR output is the same frequency as the FRAME "clock". You should never need to use the FRAME signal as a clock - use the BUFR instead.

 

If you need the clock to be on a global domain, then drive the output of the BUFR to the input of a BUFG. However, you must be aware that the BUFR clock and the BUFG clock are not phase aligned with eachother; unless your frequency is very low (which it may be in your case), you should treat these two clocks as mesochronous, and hence use a clock crossing FIFO to bring data between these two domains.

 

Avrum

0 Kudos
Adventurer
Adventurer
4,501 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

Hi,

 

I see you answered me with this information, which I actually wondered here:

https://forums.xilinx.com/t5/Implementation/BUFG-inferred-in-between-IBUFDS-and-ISERDES/td-p/781206

 

The reason I use FRAME and not the output of CLK_DIV, although they have the same frequency, is because FRAME is aligned with the data, while CLK_DIV is slightly phased, aligned with DCLK.

 

As I'm deserializing 2 samples, I'm struggling with separating both samples. For that, I need to update the deserialized data, every FRAME/CLK_DIV clock cycle, and multiplex each sample, depending on FRAME/CLK_DIV being '1' or '0'. 

 

I say FRAME/CLK_DIV, because as you say, CLK_DIV should be fine for this. But with any of them, Vivado infers a BUFG in the path in between the IBUFDS and SERDES. How can I control that?

 

Regards,

Timoteo

0 Kudos
Historian
Historian
4,487 Views
Registered: ‎01-23-2009

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

The tools will automatically insert a BUFG on any net coming from a pin that is used as a clock that doesn't already have a clock buffer on it. In your case, you want to have the output of the IBUF go directly to the ISERDES (with no BUFG), but to the other loads (which are being used as a clock) through a BUFG. I suspect that if you manually instantiate a BUFG with this configuration, the tools will keep the structure you put in (and keep the IBUF -> ISERDES connection not going through the BUFG).

 

But again, I don't see why this is necessary - you should be able to use the BUFR clock...

 

As for separating the two samples, it should be easy. Your bitslip state machine is asserting bitslip until you get the deserialized FRAME signal into the correct position. When you have done so, the data bits that correspond to the ones in your deserailized FRAME signal are one sample, the ones that are 0 are the other...

 

Avrum

Tags (2)
0 Kudos
Adventurer
Adventurer
4,473 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

Hi, 

 

 

First, thanks for all your feedback Avrum, it's being really helpful.

The board I'm using, has an adc (ADC3244), and a dac (AD9747). 

The firmware configures the PLLs on board through I2C, ADC and DAC through SPI, and when the configuration is done, a signal flags that to a VHDL module where I have all what we were talking about. 


The reason why I need to use FRAME apart from training: (maybe you come up with a better solution!)

 

adc3244.png

This image is from the ADC3244 datasheet, and I added the CLK_DIV I generate. 

Because the deserialization is DDR x14, (2 samples), I need to really generate CLK_DIV syncrhonous to DCLK, but after a rising_edge of FRAME. What I do for this, is to release the CLR pin of BUFR that receives DCLK as input, when FRAME has a rising edge. After that, I wait some clock cycles so that CLK_DIV is stable, to release the RST pin of the SERDES blocks.

 

I need FRAME for releasing the BUFR basically. Otherwise, how do I know the samples won't be mixed up?

 

When I separate the samples after the deserialization is done, I just check as you say, when FRAME is '1' or '0'. Or even CLK_DIV. I tried both. 

 

The problem is, that a small percentage of times I program the FPGA (let's say 10%), the data appears shifted (I generate an 8-point sine wave from the ADC, one of its patterns, and when the waveform is generated wrong, 4 of those points are fine, 4 are just inverted, or with more offset, being the waveform everything except a sine wave!).

 

I have an asynchronous reset on board, with which I can reset the whole thing, and it's being synchronized in the FPGA with the corresponding clocks in every block. So every time I reset, the ADC/DAC/PLLs are re-configured, the clocks re-generated, the bitslip training happens, etc. It's a full reset. So as I say, just resetting, I can see that the waveform appears wrong a small percentage of times. Now, why is this happening?

I will start a bit align machine, and instantiate an IDELAY. If after controlling the phase of DCLK with the data, this is still happening, it must be something related to the way I multiplex the samples after the deserialization. (I know the problem is there, as I placed an ILA core to scope the bitslip training, and the SERDES are trained well every time. It's the separation of the samples where the problem is). If somehow I'm multiplexing the samples just checking if FRAME/CLK_DIV is '1' or '0', but there are timing delays between data and the clocks (I assumed not, being the frequencies so low...) then the IDELAY will solve it.

 

I hope you understand the need of FRAME out of the SERDES now, and I'm open to suggestions if there is another way to do this using SERDES primitives.

 

Regards,

Timoteo 

0 Kudos
Historian
Historian
4,455 Views
Registered: ‎01-23-2009

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

It seems like you aren't following how framing is supposed to be done on this interface...

 

When you transfer data serially, framing is lost, and has to be recovered by some mechanism. The FRAME signal from the ADC is there to help you restore framing.

 

So the question is how to use it in the FPGA. I thought we had agreed on how this was to be done - you don't need (and shouldn't use or rely on) any mechanism of "releasing" the BUFR or resetting the ISERDES based on any given phase of the FRAME signal - the BUFR divider will be running to fast for any of that...

 

When an BUFR comes up, the divider should be treated as if it starts randomly. Thus when you deserialize data with it, you will get data with random framing. However, all ISERDES clocked by the same BUFR will have the same framing. This is the key to recovering framing.

 

So we have 3 ISERDES, one for Dx0, one for Dx1, and one for FCLK. Each of these are driven by the same BUFR/BUFIO combination.

 

When the board comes out of reset, all three of these will have a random framing, but they will all be framed together. So what we do is "fix" the framing of all 3 channels using BITSLIP. Whenever we assert BITSLIP, we assert it to all three channels simultaneously. Our control logic that determines when to assert BITSLIP, though, is only looking at the framing of the ISERDES that deserialises FCLK.

 

(for the sake of argument, I am going to draw the first arriving bit in the most significant bit of my bit pattern)

 

Lets say when we start, the deserialized 14 bit value we get from the FCLK ISERDES is 00011111110000. This is framed incorrectly. However, we actually now know enough - the FCLK pattern is shifted by 3 bits to the right; thus we already know our data on the other channels is

D12, D11, D10, D26, D25, D24, D23, D21, D20, D36, D35, D34, D33

 

(where Dxy means the y'th bit of the x'th sample)

 

Now, all we need to do is use BITSLIP to realign the framing. If you apply BITSLIP, then the framing will change - lets say it now changes to 00001111111000. This too is incorrect, and we need to keep going. If we apply it again, it will change again, this time we may get 0011111100000 - again incorrect (the actual pattern of FRAMING with bitslip is not "linear" when DDR inputs are used, they seem to go one forward, 3 back, one forward, 3 back). While we don't know exactly how the BITSLIP will affect framing, we know that in 14 assertions of BITSLIP we will have ended up with each one of the 14 possible alignments. One of these is correct - namely the one that returns with the FCLK pattern - 11111110000000.

 

Once we get the correct pattern on the FCLK ISERDES, and assuming we have been asserting BITSLIP on the two data channels every time we asserted it on the FCLK channel, now your interface is framed. The data interfaces will now be returning

 

D26, D25, D24, D23, D21, D20, D36, D35, D34, D33, D32, D31, D30

 

Your interface is framed, and you know that most significant 7 bits are the data that was returned when FCLK=1 (hence SAMPLE N) and least significant 7 bits are the data returned when FCLK=0 (hence SAMPLE N+1).

 

Avrum

0 Kudos
Adventurer
Adventurer
4,443 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

Thanks very much for your answers Avrum. Very helpful indeed.


Kind regards,

Timoteo

0 Kudos
Adventurer
Adventurer
1,994 Views
Registered: ‎12-30-2015

Re: Routable but not routed. D pin of ISERDESE2.

Jump to solution

I want to update this thread to say that I successfully made it work.

For those who will struggle as I did, and are in the learning curve as I am:

 

reply.png

The bit align machine basically finds the corresponding edges, applying different tap values for the IDELAYE2 block. Later, it applies the bitslip training, checking frame serdes' outputs. Please be careful with this, as the taps are up to 2.496ns for a reference clock of 200MHz on IDELAYCTRL. For the input clock I was applying to the ADC (28.533 MHz), every bit coming from the ADC was changing every aprox. 5ns. The maximum value of taps wasn't enough, so the bit align machine was getting stuck trying to find edges. For higher frequencies, it's tested it works fine. For low frequencies I just apply 16 taps and jump to bitslip operation.

 

I hope this information is helpful for other users.

Regards,

Timoteo