12-20-2018 04:31 PM
I am using the ISERDESE3 along with IDELAYE3 to capture incoming LVDS data to internal fabric over a custom interface. There are many of these bits in the FPGA (11 IO banks, or 264 LVDS channels).
I am using the FIFO_EMPTY signal to align the incoming data between ISERDES to avoid any skew to simplify the alignment of channels in my fabric logic. Basically, I hold FIFO_RD_EN low, and my logic detects the FIFO_EMPTY of up to 8 LVDS channels together every 8 FIFO_RD_CLKs, and will pulse FIFO_RD_EN by a single clock until the 8 FIFO_EMPTY signals are all aligned.
This has been working, but recently i had to move the fabric clock (and thus FIFO_RD_CLK) to a different output of the MMCM from the CLK/CLK_B/CLKDIV clocks in the ISERDES (so FIFO_RD_CLK and CLKDIV are same frequency, but phase independent). Now i am getting channels where i never detect any transition of FIFO_EMPTY. It is always 0. This makes me suspect that FIFO_EMTPY is on a different clock domain then FIFO_RD_CLK and my logic is not detecting the signal. It is a very sporadic, and build dependent issue.
Below is my instantiation of ISERDESE3, but basically i want to get more details then the SelectIO Resources User Guide provides on why I would not see these FIFO_EMPTY signals transition every 8 clocks when holding FIFO_RD_EN low.
ISERDESE3 # (
) u_iserdese3_master (
.Q (i_rx_din), // spyglass disable W120, HangingInstOutput-ML
01-08-2019 06:01 AM
That is an interesting one I've not seen before.
The MMCM CLK/CLK_B/CLKDIV are coming from two BUFG_DIVs? Are you controlling the CE and CLR to ensure they are aligned to each other?
How are you controlling the FIFO_RD_EN?
When I've seen issues with the FIFO so far it was related to the Reset sequence. Have you captured the signals (FIFO_EMPTY, FIFO_RD_EN, MMCM Locked and the different resets) with an ILA?
I'm assuming that this would be very difficult to see in a simulation, I've tried to build up a testcase but I dont see it in simulation. I'll put it on HW but if you are seeing it be build dependant I may not see it.
The ISERDES generates the INTERNAL_DIVCLK and uses that for writing to the FIFO from the CLK (clk_x2). I would expect that the FIFO_EMPTY could be generated in that clock domain. Is the clk_x2/FIFO_WR_CLK free running? Do you see the INTERNAL_DIVCLK toggling?
01-08-2019 10:52 AM
CLK/CLK_B are coming from one BUFGCE, and CLKDIV is on same MMCM output, but BUFGCE with divide by 2. CLK/CLK_B are 2x frequency of CLKDIV so I'm unclear what phase alignment you mean (i.e. there is no divide by for CLK/CLK_B).
FIFO_RD_CLK is coming from a separate MMCM output with separate BUFGCE. This is same frequency of CLKDIV, but has NO phase relationship to CLKDIV.
FIFO_RD_EN is based on FIFO_EMPTY. However, I am trying to sync up multiple channels to be bit aligned (or at least close to bit aligned as I can get before aligning them properly in my downstream logic). The ISERDES FIFOs in a bank don't have their write pointers synchronized to each other and I can see FIFO_EMPTYs between FIFOs off by several clocks at startup, so i use FIFO_RD_EN to align all of the FIFO_EMPTYs in my channel. Once they are aligned, I drive FIFO_RD_EN=1 all the time. This circuit has worked quite well and I only started seeing an issue when going to a different clock for FIFO_RD_CLK.
CLK (and thus FIFO_WR_CLK) is free running. I am convinced that FIFO_EMPTY is not synchronous to FIFO_RD_CLK, and that is the issue, but I can't find anything about it in the documentation. The BITSLICE does mention that FIFO_EMPTY is synchronous to FIFO_RD_CLK in it's implementation, but my guess is that they are doing some synchronization of the ISERDES FIFO_EMPTY before providing this signal to the user.
01-09-2019 10:01 AM
Can you share you .ila results showing the behaviour? I've tried a few different iterations on HW and I can't reproduce it.
If you would be willing to send the design I can private message you an EZMOVE link to share the design with me.
Reading more closely the FIFO_EMPTY is when the Read Pointer is equal Write Pointer, the Write Pointer is obviously going to Synchronous to the WRCLK which needs to synchronised to the RDCLK. But the FIFO_EMPTY should assert synchronous to the RDCLK.
01-09-2019 05:27 PM
I currently don't have a build with ILA output. The ILA to capture this was very simple though, but doesn't show much.
I take the FIFO_EMPTY signal directly from ISERDES and capture it in ILA using the FIFO_RD_CLK clock.
Then I hold FIFO_RDEN_EN=0, which causes the FIFO_EMPTY to pulse 0-1->0 every 8 FIFO_RD_CLKs. When doing this there are certain FIFO_EMPTIES that never show a 1 on the ILA (just flat 0).
Please note that I am using 528 ISERDES in my design, and usually it is less than 10 that have this problem. Also, the problem ISERDES change from build to build. If I change design so that FIFO_RD_CLK=CLKDIV then I never see this issue. It only occurs (and varies build-build) when FIFO_RD_CLK != CLKDIV (also, there are no timing constraints between FIFO_RD_CLK and CLKDIV since they are separate MMCM outputs and the phase of the 2 clocks is not guarenteed).
This makes it pretty certain to me that FIFO_EMPTY on ISERDES is not synchronized to FIFO_RD_CLK.