Solved: Re: Critical Warning on clock interaction due to r...

namabo · ‎01-09-2020

Hi everybody,

I have a problem with a project using 100G Xilinx IP.

I have 2 clocks rxoutclk_out[0] and txoutclk_out[0] reported as "Timed (unsafe)" in the Clock Interaction Report.

On report Methodology I get

"TIMING #1 Critical Warning The clocks rxoutclk_out[0] and txoutclk_out[0] are related (timed together) but they have no common node. The design could fail in hardware. To find a timing path between these clocks, run the following command: report_timing -from [get_clocks rxoutclk_out[0]] -to [get_clocks txoutclk_out[0]] "

I started debugging this Critical Warning, following my analysis.

First of all I run report_timing -from [get_clocks {rxoutclk_out[0]}] -to [get_clocks {txoutclk_out[0]}] command and I got:

Timing Report

Slack (MET) :             1.099ns  (required time - arrival time)
  Source:                 design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_top/obsibdaaf4ker2wujpra0sjb/RX_CLK
                            (rising edge-triggered cell CMACE4 clocked by rxoutclk_out[0]  {rise@0.000ns fall@1.551ns period=3.103ns})
  Destination:            design_1_i/cmac_usplus_0/inst/master_watchdog_reg[0]/S
                            (rising edge-triggered cell FDSE clocked by txoutclk_out[0]  {rise@0.000ns fall@1.551ns period=3.103ns})
  Path Group:             txoutclk_out[0]
  Path Type:              Setup (Max at Slow Process Corner)
  Requirement:            3.103ns  (txoutclk_out[0] rise@3.103ns - rxoutclk_out[0] rise@0.000ns)
  Data Path Delay:        1.516ns  (logic 0.773ns (50.989%)  route 0.743ns (49.011%))
  Logic Levels:           1  (LUT4=1)
  Clock Path Skew:        -0.200ns (DCD - SCD + CPR)
    Destination Clock Delay (DCD):    1.199ns = ( 4.302 - 3.103 ) 
    Source Clock Delay      (SCD):    1.399ns
    Clock Pessimism Removal (CPR):    0.000ns
  Clock Uncertainty:      0.214ns  ((TSJ^2 + DJ^2)^1/2) / 2 + PE
    Total System Jitter     (TSJ):    0.424ns
      User System Jitter         :    0.300ns
    Discrete Jitter          (DJ):    0.060ns
    Phase Error              (PE):    0.000ns
  Clock Net Delay (Source):      1.184ns (routing 0.350ns, distribution 0.834ns)
  Clock Net Delay (Destination): 1.007ns (routing 0.298ns, distribution 0.709ns)

    Location             Delay type                Incr(ns)  Path(ns)    Netlist Resource(s)
  -------------------------------------------------------------------    -------------------
                         (clock rxoutclk_out[0] rise edge)
                                                      0.000     0.000 r  
    GTYE4_CHANNEL_X0Y8   GTYE4_CHANNEL                0.000     0.000 r  design_1_i/cmac_usplus_0/inst/design_1_cmac_usplus_0_0_gt_i/inst/gen_gtwizard_gtye4_top.design_1_cmac_usplus_0_0_gt_gtwizard_gtye4_inst/gen_gtwizard_gtye4.gen_channel_container[2].gen_enabled_channel.gtye4_channel_wrapper_inst/channel_inst/gtye4_channel_gen.gen_gtye4_channel_inst[0].GTYE4_CHANNEL_PRIM_INST/RXOUTCLK
                         net (fo=2, routed)           0.085     0.085    design_1_i/cmac_usplus_0/inst/cmac_gtwiz_userclk_rx_inst/rxoutclk_out[0]
    BUFG_GT_X0Y52        BUFG_GT (Prop_BUFG_GT_I_O)
                                                      0.130     0.215 r  design_1_i/cmac_usplus_0/inst/cmac_gtwiz_userclk_rx_inst/gen_gtwiz_userclk_rx_main.bufg_gt_usrclk_inst/O
    X3Y2 (CLOCK_ROOT)    net (fo=676, routed)         1.184     1.399    design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_top/rx_clk
    CMACE4_X0Y0          CMACE4                                       r  design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_top/obsibdaaf4ker2wujpra0sjb/RX_CLK
  -------------------------------------------------------------------    -------------------
    CMACE4_X0Y0          CMACE4 (Prop_CMACE4_RX_CLK_STAT_RX_ALIGNED)
                                                      0.683     2.082 r  design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_top/obsibdaaf4ker2wujpra0sjb/STAT_RX_ALIGNED
                         net (fo=3, routed)           0.326     2.408    design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_cmac_cdc_sync_gt_txresetdone_int/stat_rx_aligned
    SLICE_X101Y174       LUT4 (Prop_H6LUT_SLICEM_I0_O)
                                                      0.090     2.498 r  design_1_i/cmac_usplus_0/inst/i_design_1_cmac_usplus_0_0_cmac_cdc_sync_gt_txresetdone_int/master_watchdog[0]_i_1/O
                         net (fo=29, routed)          0.417     2.915    design_1_i/cmac_usplus_0/inst/master_watchdog0
    SLICE_X101Y169       FDSE                                         r  design_1_i/cmac_usplus_0/inst/master_watchdog_reg[0]/S
  -------------------------------------------------------------------    -------------------

                         (clock txoutclk_out[0] rise edge)
                                                      3.103     3.103 r  
    GTYE4_CHANNEL_X0Y8   GTYE4_CHANNEL                0.000     3.103 r  design_1_i/cmac_usplus_0/inst/design_1_cmac_usplus_0_0_gt_i/inst/gen_gtwizard_gtye4_top.design_1_cmac_usplus_0_0_gt_gtwizard_gtye4_inst/gen_gtwizard_gtye4.gen_channel_container[2].gen_enabled_channel.gtye4_channel_wrapper_inst/channel_inst/gtye4_channel_gen.gen_gtye4_channel_inst[0].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK
                         net (fo=2, routed)           0.078     3.181    design_1_i/cmac_usplus_0/inst/cmac_gtwiz_userclk_tx_inst/txoutclk_out[0]
    BUFG_GT_X0Y58        BUFG_GT (Prop_BUFG_GT_I_O)
                                                      0.114     3.295 r  design_1_i/cmac_usplus_0/inst/cmac_gtwiz_userclk_tx_inst/gen_gtwiz_userclk_tx_main.bufg_gt_usrclk_inst/O
    X3Y2 (CLOCK_ROOT)    net (fo=1817, routed)        1.007     4.302    design_1_i/cmac_usplus_0/inst/gt_txusrclk2
    SLICE_X101Y169       FDSE                                         r  design_1_i/cmac_usplus_0/inst/master_watchdog_reg[0]/C
                         clock pessimism              0.000     4.302    
                         clock uncertainty           -0.214     4.088    
    SLICE_X101Y169       FDSE (Setup_AFF_SLICEM_C_S)
                                                     -0.074     4.014    design_1_i/cmac_usplus_0/inst/master_watchdog_reg[0]
  -------------------------------------------------------------------
                         required time                          4.014    
                         arrival time                          -2.915    
  -------------------------------------------------------------------
                         slack                                  1.099

From this report I watched the schematic involved:

This block cdc_sync is a simply synchronizer, in particular it synchronize the signal gtwiz_reset_tx_done_out signal wrt the tx_usr clock (path in blue).

The red line is the stat_rx_align singnal that unfurtunately has been synthetized without any synchrionizer.

Following the chunck of code that generates the issue:

The synchonizer itself

(* DowngradeIPIdentifiedWarnings="yes" *)
  module design_1_cmac_usplus_0_0_cdc_sync (
   input clk,
   input signal_in,
   output wire signal_out
  );

                               wire sig_in_cdc_from ;
      (* ASYNC_REG = "TRUE" *) reg  s_out_d2_cdc_to;
      (* ASYNC_REG = "TRUE" *) reg  s_out_d3;
      (* max_fanout = 500 *)   reg  s_out_d4;
      
// synthesis translate_off
      
      initial s_out_d2_cdc_to = 1'b0;
      initial s_out_d3        = 1'b0;
      initial s_out_d4        = 1'b0;
      
// synthesis translate_on   
   
      assign sig_in_cdc_from = signal_in;
      assign signal_out      = s_out_d4;

      always @(posedge clk) 
      begin
        s_out_d2_cdc_to  <= sig_in_cdc_from;
        s_out_d3         <= s_out_d2_cdc_to;
        s_out_d4         <= s_out_d3;
      end
  
  endmodule


design_1_cmac_usplus_0_0_cdc_sync i_design_1_cmac_usplus_0_0_cmac_cdc_sync_gt_txresetdone_int
(
.clk (gt_txusrclk2),
.signal_in (gt_txresetdone_int), 
.signal_out (gt_txresetdone_int_sync)
);

and the piece of code that generates the master LUT showed in the schematic

  localparam [28:0] MASTER_WATCHDOG_TIMER_RESET = 29'b01110011010000001001011001011;
  reg  [28:0] master_watchdog = MASTER_WATCHDOG_TIMER_RESET;
  reg  master_watchdog_barking;
  wire master_watchdog_barking_sync;

  always @(posedge gt_txusrclk2)
  begin
    if (tx_reset_done == 1'b0 || stat_rx_aligned == 1'b1)
      master_watchdog <= MASTER_WATCHDOG_TIMER_RESET;
    else
      master_watchdog <= master_watchdog - 1;
  end

Now this piece of code are inside the same verilog file.

I'm not expert of timing issue but in this situation I think I neeed to add a synchronizer for the stat_rx_align signal to synchronize it with gt_txusr_clk. Please Can someone confirm my hypothesis?

This verilog file is provided by Xilinx as part of the wrapper of CMAC IP. The file is reported in Vivado a read-only.

How can I add a syncronizer in this scenario?

Than after inserting the syncro if I get the same error Can I quietly insert an async group between this 2 clocks?

For mw this is not resolvable by a false path Am I right?

Really thank you in advance.

Regards

avrumw · ‎01-09-2020

I agree with your assessment - this appears to be an illegal clock crossing.

My only question is if there is a possibility that you have misconnected a clock at the upper level. This path is clearly clocked by two different domains. It's also clear that both clocks come from GTs - however, are the GTs (and hence the connections of the clocks to the startpoint and endpoint of this path) inside the IP block, or are they outside, and at some higher level you are responsible for connecting the clocks to the IP? If you connected the clocks to the IP, is there any possibility that you have connected an incorrect clock - a port that should be connected to txoutclk is actually connected to rxoutclk (I don't think this is the case, but you need to rule it out).

If the connection is purely inside the IP block, then it is a "bug" in the core, and, as you suspected, there is nothing you can do to fix it (at least within the normal flow of how you are supposed to use IP) - this is a bug in a Xilinx block and Xilinx needs to fix it.

That being said, this is a watchdog timer - it is probably just holding off real activity until "long enough" after the latter of the two events (the deassertion of reset or the RX being aligned). While the length of this timer is oddly specific, one wonders how critical it really is - the effect of metastability will result in the count being incorrect around the metastable event. Furthermore, since the reset amount ends in '2b11, on the first clock after reset, only the LSbit will change - the risk with metastability in a counter is that more than one bit may be changing, so down counting from 1000 could end up with any value, but down counting from 1011 can only end up being interpreted as 1011 or 1010 - which isn't really a problem.

So while the code is incorrect - it is an illegal clock crossing, the impact of the metastable event is probably pretty negligible. So it should be fixed (i.e. you should file a service request with Xilinx), but it probably won't make your system unreliable.

Given that it is at least "mostly harmless" you can waive the error with a timing exception. However, I would not use a set_clock_groups command - that has the potential to affect all the clock domain crossings between these to clock domains. Instead, I would put something like

set_max_delay -datapath_only 3.103 -from <startpoint> -to [get_cells design_1_i/cmac_usplus_0/inst/master_watchdog_reg*]

Since the <startpoint> is in obfuscated code, I don't know if you can use the full cell name, so you may need to at least wildcard some portion of it (although if the name is consistent from run to run you can use it even if it is just a sequence of characters). The -to is in plain text, so you should be able to use it. Limiting the exception as much as possible reduces the risk of it affecting something else...

Avrum

View solution in original post

avrumw · ‎01-09-2020