UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Visitor AhmedShaaban
Visitor
4,517 Views
Registered: ‎01-11-2016

OC48 issue

Hi everyone,

I have to interface two zc706 boards using oc48 protocol through gtx. I split this task into two scenarios:

  • Using one Zc706 board and test it with a EXFO (OC48 handheld testing device).
  • Using the two Zc706 board and test them with a EXFO (OC48 handheld testing device).

The first scenario worked ok, but the second is not working. A detailed steps is described below.

 

First, I have the following tools/modules:

  • Two Zc706 boards.
  • Two FM-S14 Quad SFP/SFP+ transceiver FMC.
  • SFP loopback module.
  • Two fiber cables (LC-LC: 9/125).
  • Four SFP modules 2.4 gbps .
  • EXFO device (is a fully integrated SONET handheld tester).
  • Vivado 2016.1.

 

Second, I have generated the 7 series transceiver ip core for OC48 protocol and with the following features :

  • Enable bit slip logic.
  • Using two Gtx (X0Y0,X0Y3).

 I have modified  the generated example design with my own bitslip logic and the frame_gen logic has been removed.

 

scenario #1,

 

Scen1.png

 

  • EXFO will generate OC48 data (on the recovered clock) then send it to Zc706 through fiber cable and FM-s14 (SFP0).
  • Received data (gt0_rxdata) will be saved in FIFO1 on gt0_rxuserclk2 clock.
  • Data will be read (from FIFO1) on gt1_txuserclk2 clock and transmitted out of FPGA through SFP3.
  • Using the SFP loopback module, received data (gt1_rxdata) will be saved in FIFO2 on gt1_rxuserclk2 clock.
  • Data will be read (from FIFO2) on gt0_txuserclk2 clock and transmitted back to EXFO device through SFP0.

The above scenario has been worked fine without any errors on either EXFO or chipscope.

 

scenario #2,

 

Scen2.png

 

It seems nothing is wrong with data on chipscope but EXFO test is not passed and there is a high bit error rate.

So, what I need to do to get this setup works?

Thanks in advance.

0 Kudos
16 Replies
Scholar austin
Scholar
4,505 Views
Registered: ‎02-27-2008

Re: OC48 issue

a, It is likely jitter accumulation in this setup is leading to errors. I would pass the recovered clock through a PLL to remove the accumulated jitter.
Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Visitor AhmedShaaban
Visitor
4,499 Views
Registered: ‎01-11-2016

Re: OC48 issue

Hi Austin,

The recoverd clock which I mean is the clock the exfo is used to transmit data to fpga.

The exfo has two mode of data transfer either transmit data using it's internal clock or using the recoverd clock. Here I have two recoverd clock (gt0_rxusrclk2 , gt1_rxusrclk2) which I used to save(write) data to fifo1 and fifo2 .

So which recoverd clock should I add a pll to?. And why the jitter problem doesn't appear in the first scenario?

Thanks.

0 Kudos
Scholar austin
Scholar
4,360 Views
Registered: ‎02-27-2008

Re: OC48 issue

a, Typically, a PLL is used for any looped or forwarded clock (always after recovered clock before used to transmit). In your case that is before each GT Tx.
Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Visitor AhmedShaaban
Visitor
4,287 Views
Registered: ‎01-11-2016

Re: OC48 issue

Hi Austin,

sorry for being late. I have tried to add a PLL to the recovered clock from gt0 (gt0_rxoutclk) and a PLL to gt1 (gt1_rxoutclk) then these clocks (gt0_rxusrclk_PLL,gt1_rxusrclk_PLL) are used to read data from FIFOs and transmit data to gt0 and gt1.

 

The following modifications have been done in "oc48_gt_usrclk_source.vhd".

gt0_txusrclk   <= gt1_rxusrclk_PLL;

gt0_txusrclk2 <= gt1_rxusrclk_PLL;

gt0_rxusrclk   <= gt0_rxusrclk;

gt0_rxusrclk2 <= gt0_rxusrclk;

 

 

gt1_txusrclk   <= gt0_rxusrclk_PLL;

gt1_txusrclk2 <= gt0_rxusrclk_PLL;

gt1_rxusrclk   <= gt1_rxusrclk;

gt1_rxusrclk2 <= gt1_rxusrclk;

 

Also, scenario #1 is still working perfect, but scenario #2 doesn't work.

 

I have attached the project to tell me (if you don't mind) if I have a something wrong.

I really appreciate your help.

Thanks.

0 Kudos
Scholar austin
Scholar
4,257 Views
Registered: ‎02-27-2008

Re: OC48 issue

a,

 

If it is still not working, I would measure the tx jitter at each port to see if this is the issue.  I only suspect it is, and I also know that to meet the specified jitter mask, one needs to probably use an external PLL (often a crystal controlled VCO PLL).

 

The SONET standard is pretty tough regarding jitter (I know as I was on the standards committee that wrote the jitter and wander parts).

 

Another issue is the data itself.  What is your test pattern?  Some stress test patterns would never occur in use.  Does it pas a much simpler pattern (like 2^7-1)?

Austin Lesea
Principal Engineer
Xilinx San Jose
Visitor AhmedShaaban
Visitor
4,255 Views
Registered: ‎01-11-2016

Re: OC48 issue

Hi Austin,

 

After some digging, I think I should use an external jitter attenuator (As you mentioned) and I'll proceed to use it. The data pattern generated from EXFO is PRBS23.

 

So, I'll use the Si5324 (located on ZC706) to minimize the jitter from the recovered clock, and then apply this clock to GTX ref clock. Also I'll try to measure the tx jitter.

 

Thanks for your help & guidance.

 

I'll let you know the result as soon as I apply this scenario.

 

Thanks.

 

0 Kudos
Visitor AhmedShaaban
Visitor
2,845 Views
Registered: ‎01-11-2016

Re: OC48 issue

Hi Austin,

 

I'm sorry for being too late.

 

Now, I have used Jitter attenuator clock (Si5324)  IC and also the setup didn't work. Here what I have done so far:

 

1- I have configured the Si5324 in free running mode to output 155.52 MHz ( SI5324_OUT) which will drive the GTX reference clock.

2- The recovered clock form GTX (RXOUTCLK) is connected to  CLK1in (REC_CLOCK) of Si5324  through ODDR (the CE of ODDR is connected to CPLLLOCK) followed by OBUFDS.

3- Input clock selection of Si5324 is adjusted to be "Automatic Retrieve".

 

scenario#1 (refer to first post):  is working perfect (as it was without using the Jitter attenuator) .

 

scenario#2 (refer to first post):  is not working although the EXFO display that the RX PPM = 8.3.

 

What can I do so it works?

 

Thanks in advance.

 

0 Kudos
Scholar austin
Scholar
2,835 Views
Registered: ‎02-27-2008

Re: OC48 issue

In scenario #2,

 

Who is the master for timing (better be the upstream source)?

 

Who are loop timed (recover timing and use for transit back, and any forwarded link transmit)?

 

The last in line is, of course loop timed?

 

Have you verified each link by itself is error free? (both near and far end loopbacks?)

 

Have you measured the transmit jitter of each transmit?  Is it within spec?

 

Have you verified the receive input jitter tolerance (how much jitter can you add before you get errors)?

 

Having helped write the SONET/SDS standard jitter section many years ago, I fully understand what may cause bit errors.  It could also be a logic design error as well, so do not rule anything out.

Austin Lesea
Principal Engineer
Xilinx San Jose
Xilinx Employee
Xilinx Employee
2,834 Views
Registered: ‎11-29-2007

Re: OC48 issue

hello,

please can you add to your drawing

  • all the clocks sources used to write and read from the FIFOs
  • each transceiver REFCLK sources

Do you get FIFO errors in the second board?

thanks

GG

2,449 Views
Registered: ‎02-14-2018

Re: OC48 issue

Dear gguasti,

I am Mahmoud, Ahmed Shaaban's colleague. I hope my comment will be an answer for your question, in the following image you may find all the clocks sources used to write and read from the FIFOs, and also the reference clock for both transceivers:

FIFOs.PNG

 

Also find here the whole project uploaded, hoping that you can help us identify where the problem is.

 

 

0 Kudos
Scholar austin
Scholar
2,444 Views
Registered: ‎02-27-2008

Re: OC48 issue

M,

 

I have a few questions:

 

On the si5324, are you using a crystal as the timing source? (I understand you lock the DSPLL to your receive reference)

 

What is the jitter attenuation pole frequency set at in the si5324?

 

Have you measured each clock to see if it meets the jitter level output, and input jitter tolerance levels (not chained)?

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
2,428 Views
Registered: ‎02-14-2018

Re: OC48 issue

Dear Austin,

 

"On the si5324, are you using a crystal as the timing source?"

 

  • The Si5324 is configured to have two clock sources upon which it will multiplex; CLKIN1 is the recovered clock, and CLKIN2 is the one coming from the 114.285 oscillator (as the free running mode is enabled). We also gave clock 1 higher priority, thus -according to our understanding- the Si5324 shall switch between clocks whenever it senses the generation of CLKIN1. We have connected the output of the Si5324 provides the reference clock for the two GTx transceivers.

"What is the jitter attenuation pole frequency set at in the si5324?"

 

  • Would you please illustrate more what do you mean by the jitter attenuation pole frequency and how to set it.

 

"Have you measured each clock to see if it meets the jitter level output, and input jitter tolerance levels (not chained)?"

 

  • No we didn't, I believe we don't have the required hardware that enables us to do so.

 

 

 

 

 

 

0 Kudos
Scholar austin
Scholar
2,413 Views
Registered: ‎02-27-2008

Re: OC48 issue

OK,

 

If the unit is to use recovered clock (downstream slave to an upstream master), it has to at least meed the requirements of stratum 4 when holding over (lost sync source to upstream node), and be able to pass yellow alarm state (link may have high error rate due to slips).  The crystal mode does that, I agree.  While locked to the upstream in the si5324 data sheet it mentions that one can set the pole frequency for jitter attenuation.  You may need to set that to the lowest value to attenuate jitter enough to use it as a timing source for any links downstream (I believe 4 Hz is the lowest setting, 525 Hz is the highest).

 

Without testing, you are blind, and there is no way to verify if timing is correct.  Rent an OC48 test set that is able to measure jitter, and test for jitter tolerance.  Not that your problem is timing, but you need to get timing right first, before further debugging.

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
2,334 Views
Registered: ‎02-14-2018

Re: OC48 issue

Dear Austin, 

 

I hope you are well.

 

Regrading to set pole frequency for jitter attenuation to the lowest value, I have already set it to 4 Hz.

 

Currently I'm trying to borrow a Jitter measuring device but It isn't available now. So in the mean while, I have observed the PPM offset and alarms through the EXFO and here what I have found so far.

 

For Scenario#1 : The PPM offset = 10, and No alarms are detected.

For Scenario#2 : The PPM offset = 10, and a LOF-S (Loss of Frame section) alarm is detected.

 

Attached Images for both scenarios.

 

Thanks in advance.

 

 

 

Scenario#1.png
Scenario#2.png
0 Kudos
Scholar austin
Scholar
2,300 Views
Registered: ‎02-27-2008

Re: OC48 issue

If this is loop timed,

 

There should be 0 offset.  It must be loop timed.

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Visitor AhmedShaaban
Visitor
1,772 Views
Registered: ‎01-11-2016

Re: OC48 issue

Hello Everyone,

 

First of all, thank you so much Austinfor your great help.

 

Secondly, Now The link is up and running with 0 PPM offset as you mentioned and I had no errors.

 

Here what I had done to get it work:

1- Generate 155.52 Mhz clock from System clock (200 Mhz).

2- connect this clock to BUFGMUX (input 1) and connect gt0_rx_usrclk2 (gt0 Recovered clock ) to BUFGMUX (input 2)  and connect the selection pin to " gt0_rx_reset_fsm_done".

3- Connect BUFGMUX output to SI5324 input clock ( REC_CLOCK_P and  REC_CLOCK_N).

4- I configured the SI5324 in manual mode and select clock 1 as a source .

 

So Scenario Two currently is working perfect.But now I have another issue when I use Two EXFO's as illustrated below:

original.png

I had FIFO_2_EMPTY in one FPGA and FIFO_2_FULL in the other (No problems found for FIFO_1 in both FPGA's). when I use the gt1_rx_usrclk2  to be the input for BUFGMUX instead of gt0_rx_usrclk2, FIFO_2 has no problems in both FPGA's and I had FIFO_1_EMPTY in both FPGA's .

 

So does It mean, I need to use jitter attenuator for each recovered clock so that the tx and rx rate for each transceiver  is identical and so no errors are found?!

 

Thanks in advance.

0 Kudos