UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Adventurer
Adventurer
733 Views
Registered: ‎03-31-2017

Direct data crossing between synchronous clocks

Jump to solution

Can we have a definitive yes/no answer on whether the tools will handle this?

As a concrete example: I have inherited a chip which has a single 156.25MHz differential clock input. This drives two different 10GE PCS/PMA CoreGen modules on two different tiles. Each one generates a 156.25 clock for use in different parts of the chip, but each clock has the same waveform and phase (I'm assuming they can't de-phase in some way). Currently, all data crossings between the two domains are explicitly synchronised. Is this necessary?

In the recent mesochronous thread  @yashp suggests, for sync clocks, "user needs to make sure there is proper CDC to manage data transfer", but @avrumw thinks ("as far as I know") that direct crossings will be timed correctly.

I haven't found anything in the docs, but UG903 does say that, for set_multicycle_path, the source and destination clocks only have to have the same waveform with no phase offset, so direct crossing seems to be Ok in this case.

0 Kudos
1 Solution

Accepted Solutions
Historian
Historian
510 Views
Registered: ‎01-23-2009

Re: Direct data crossing between synchronous clocks

Jump to solution

I have a couple of comments...

First, we have to separate the idea of analyzing clock path skew/insertion delay/propagation delay from the idea of "automatically generated clocks" - they are interrelated, but are separate.

For the first one (analyzing clock path skew/insertion delay/propagation delay) the tools are 100% correct - they understand the propagation delays completely, including the variation in paths between different components - including the variation in path delays that you would get between multiple MMCMs/PLLs (the equivalent of the CLKIN->CLKOUT phase error of MMCMs).

Now the second part. In "pure" SDC, there are no automatically generated clocks - all internally generated clocks need to be explicitly defined by the user (usually using create_generated_clock). Thus if your ASIC library has built in clock dividers or PLLs, you, the user, are required to manually write "create_generated_clock" commands for each output of these cells.

However, since MMCMs (and some other clock modifying cells - PLLs, BUFRs, BUFGCE_DIV [in UltraScale]) are part of the Xilinx library, and are used in almost all designs, Xilinx decided to add the "automatically generated clocks" mechanism. However, you have to understand that it has limitiations.

The main limitation is that it is designed around a single MMCM/PLL/BUFR/BUFGCE_DIV kind of system (or for BUFR/BUFGCE_DIV systems, only ones that explicitly synchronize the dividers). So, we know that it always works for different outputs of one MMCM. We also know it completely understands the relationship between a BUFIO clock (which is undivided) and a BUFR clock (which is divided). But that's pretty much it. Anything outside this, comes back to us, the user.

So, if you have (for example) two BUFRs (or BUFGCE_DIV) that don't use the "BUFR Alignment" mechanism described in UG472, Appendix A, then the relationship between the two automatically generated clocks on the outputs of the two BUFRs is incorrect - you, the user, need to intervene in order to describe this more complex relationship to the tools (probably with a set_max_delay).

Similarly if you use two MMCMs, then you need to figure out if the automatically generated clocks generated by the tool are sufficient or not. This post talks about the conditions where the MMCMs are guaranteed to be in phase or potentially out of phase. If you fall into the category where the MMCMs are guaranteed to be in phase, then the automatically generated clocks are sufficient. If you are using the MMCMs where they can be out of phase, then they are not, and you need to override the constraints (again, most likely a set_max_delay).

Finally if you start talking about GT's (GTP, GTH, GTX, GTY, etc...) then the relationship between the REFCLK and the TXOUTCLK/RXOUTCLK is very complex. In general, two GTs that share the same REFCLK should probably consider the RXOUTCLK/TXOUTCLK as asynchronous clocks. (There are ways around this when using multiple "bonded" channels, but the IP has to be built to take advantage of this).

So, with all this, if we come back to your original question - without knowing exactly what is being done to the clocks in these two cores, it is impossible to know if the relationship between the automatically generated clocks within these two cores is sufficient or not - we need to inspect the clocking inside the cores to see how they fit into the above cases in order to determine if the "default" analysis performed by the tools is correct or not.

Avrum

Tags (2)
13 Replies
Scholar drjohnsmith
Scholar
710 Views
Registered: ‎07-09-2009

Re: Direct data crossing between synchronous clocks

Jump to solution

without seeing your code and the constraints,

 

its a definate may be.

 

 

<== If this was helpful, please feel free to give Kudos, and close if it answers your question ==>
0 Kudos
Adventurer
Adventurer
688 Views
Registered: ‎03-31-2017

Re: Direct data crossing between synchronous clocks

Jump to solution

I'm not sure that specific source code or constraints would help here - the clocks are generated identically, and have the same waveform and phase, but come from different quads. I constrain the device-input differential clock the obvious way, but after that the same CoreGen module is instantiated twice, once in each quad. The module creates three xdcs which again just set the output clock to 6.4ns.

So, in this case, can data cross between the two domains without a synchroniser? STA has enough information, but the docs are ambiguous.

UG901 effectively states that all that matters is the clock definition. If two different clocks have the same waveform and phase, then they can be used as the src and dst clocks in a multi-cycle path. In other words, CDC is safe.

UG949 doesn't say this. The box on p118 says, for instance:

IMPORTANT: If the CDC paths are between clocks that originate from different MMCM/PLLs, the clock insertion delays across the MMCMs/PLLs are more difficult to control. In this case, Xilinx recommends that you treat these clock domain crossings as asynchronous and make design changes accordingly.

In other words, CDC is not safe.

 

0 Kudos
678 Views
Registered: ‎01-22-2015

Re: Direct data crossing between synchronous clocks

Jump to solution

@eml 

I don’t know the answer to your question, but here’s how I’d think about it.

In your situation, it’s all about the phase relationship between the two clocks.  If we know that the FPGA (and these cores) will power up the same way every time and have a constant and well-defined phase relationship between the clocks and if Vivado understands this relationship – then – yes – the two clocks are synchronous.  However, that’s some really big “ifs”. 

You could simply try a synchronous crossing and see what Vivado says about it.  If Vivado says the crossing is not possible (ie. it recognizes the clocks as async/mesochronous, which it probably won’t) then the answer is “no” – you cannot treat the clocks as synchronous.  However, as you saw from the “mesochronous thread”, if Vivado says “yes” that the crossing is possible then the true answer is “maybe” – because Vivado will not always properly identify synchronous clocks.

Unless the core documentation specifically describes an alignment procedure for the clock outputs from two separate cores, then I think you must assume the two clock outputs are asynchronous – and that crossings between the two clock domains needs a synchronizer.

I am starting to think that synchronous clocks are a rare and endangered species – and that imposters are frequent and hard to identify.

Moderator
Moderator
654 Views
Registered: ‎01-16-2013

Re: Direct data crossing between synchronous clocks

Jump to solution

Hi @eml ,

Thanks for tagging me in this post. 

I have read the complete thread and I would like to start with the reference you have given in your one of post regarding UG/documents.

-------

UG901 effectively states that all that matters is the clock definition. If two different clocks have the same waveform and phase, then they can be used as the src and dst clocks in a multi-cycle path. In other words, CDC is safe.

UG949 doesn't say this. The box on p118 says, for instance:

IMPORTANT: If the CDC paths are between clocks that originate from different MMCM/PLLs, the clock insertion delays across the MMCMs/PLLs are more difficult to control. In this case, Xilinx recommends that you treat these clock domain crossings as asynchronous and make design changes accordingly.

In other words, CDC is not safe.

------

UG901 statement states that if waveform and phase is consistent and same then we can use just multi-cycle constraints (if required) and CDC is safe (i.e. only if the clocks are generated from same source i.e. MMCM/PLL for example). Because when we have same MMCM/PLL under usage we can say there will be no changes in phase/waveform across all the output clocks.

UG949 is more precise while putting the statements when user needs to consider the synchronous looking clocks as asynchronous and handle it with proper CDC. In your earlier post (mesochronous thread) while comenting I used the same reference (UG949) and for your current use case I will still go with UG949. As the two clocks are generated from different source better to have CDC circuit.

Theoratically might be your use case looks like the clocks are always in phase with same waveform but there might be variation on board. To nullify those adverse effect UG949 warns the users to handle it using CDC.

Thanks,
Yash

 

0 Kudos
Adventurer
Adventurer
630 Views
Registered: ‎03-31-2017

Re: Direct data crossing between synchronous clocks

Jump to solution

Hi @yashp - so, basically, you're of the view that the multicycle section of UG903 is incorrect, and that, by implication, a create_clock definition is not sufficient to allow the tools to determine if a CDC is safe or not. Note that your statement "i.e. only if the clocks are generated from same source" is not, I think, taken from UG903.

There are only two options here:

  1. create_clock is sufficient to allow the STA tool to carry out its job under any circumstances (ignoring the cases of random start up of dividers/etc), which includes determining if two clocks are 'synchronous' and CDCs are safe (which is clearly what the author of UG903 thought), or
  2. there are Unknown Unknowns which prevent the STA from calculating cross-domain paths (which is clearly what the author of UG949 thought).

Since we don't know what the Unknown Unknowns are, we've got nothing more than the list of Known Unknowns from markg@prosensing.com, and the forum posts referenced may not be entirely reliable.

I can see which way this is going, though. It looks like the official answer might end up as "only clocks sourced from the same MMCM can be analysed to determine if a direct CDC is safe", and "only clocks sourced from the same MMCM can be considered to be 'synchronous'". However, this isn't a particularly useful case, except possibly in the context of power reduction - who's going to generate 'synchronous' clocks from different MMCMs if they could have done it in the same MMCM?

0 Kudos
Moderator
Moderator
620 Views
Registered: ‎01-16-2013

Re: Direct data crossing between synchronous clocks

Jump to solution

Hi @eml ,

UG903 multi-cycle constraints define how one can relax the requirement for some circuits. This constraints is valid for inter and intra clocks both depending upon the use case.

In first example there is source and destination driven by same clock.

In second example the clock1 is used for source and clock2 is used for destination (one of the clock has phase shift which maybe intentionally done by user  OR have know phase shift {0.3 is used in example}) in such use cases how one can use multicycle path is mentioned.

In 3rd and 4th example SLOW-FAST and FAST-SLOW clock relationship has been discussed.

In above all examples no where it is mentioned that the clocks are generated from different clock source. This section shows how one can relax the requirements when waveform and phase shift is known so tool will get beniefit for using effort in really critical paths. As complete tool is timing driven.

----

In UG949 the actual use case of two different clock source has been mentioned because theoratically the clocks may look similar and in phase but may have difference. That's the reasone document have warned the users to consider it as asynchronous clocks and manage accordingly.

Thanks,
Yash

 

0 Kudos
Scholar drjohnsmith
Scholar
609 Views
Registered: ‎07-09-2009

Re: Direct data crossing between synchronous clocks

Jump to solution

Hi @eml

 

you seem very frustrated by this, please rember we are all giving our time Free here, doing the best we can for others.

 

Timming is a week long cource, so you can see it has many many bits to consider.  

 

End of day, th eapps are explaining their situatoins, which as you have seen are different.

 

you need to assure yourself  as the desinger, that you have covered CDC,

    there is no magic, for instance one I had a while back, user had two clock circuits in the FPGA.

        both driven by the same external clock, 

But there design failed some times.

     turns out they had not matched the two tracks driving the external pins, or taken account of the pin delay differences.   So although the internal clocks were the same and phase matched, the absolute phase relationship between them was not garunteed betwee the boards. so they had to retro add CDC.

As a hint, Ok, Im old, but I find it useflu to draw the clock trees, to clarify for who ever picks up later where the clocks are comming from and what assumptons weve made.

 

 

 

<== If this was helpful, please feel free to give Kudos, and close if it answers your question ==>
0 Kudos
Adventurer
Adventurer
606 Views
Registered: ‎03-31-2017

Re: Direct data crossing between synchronous clocks

Jump to solution

@yashp wrote:

In above all examples no where it is mentioned that the clocks are generated from different clock source. This section shows how one can relax the requirements when waveform and phase shift is known so tool will get beniefit for using effort in really critical paths. As complete tool is timing driven.

 


Hi @yashp - search for the phrase 'same waveform' in the multi-cycle section; there are 5 occurrences. The one on p110, for example, says "A Multicycle constraint defined within the same clock domain or between two clocks with the same waveform (no phase-shift) work the same way".

This second part of this statement clearly implies two different domains. The whole of the multi-cycle section, in my reading, is independent of clock domain, and relies solely on clock waveform.

0 Kudos
Adventurer
Adventurer
601 Views
Registered: ‎03-31-2017

Re: Direct data crossing between synchronous clocks

Jump to solution

Hi @drjohnsmith - yes, we're all giving our time for free, including myself. I'm not frustrated; I'm seeking clarification. Clarification may or may not come; that's how these things work. If you don't point out the issues, the documents will remain inconsistent.

And, as it happens, I've been working with Xilinx constraints since the XC4000, so I may be even older than you are.

0 Kudos
Scholar drjohnsmith
Scholar
583 Views
Registered: ‎07-09-2009

Re: Direct data crossing between synchronous clocks

Jump to solution

Thanks @eml

<== If this was helpful, please feel free to give Kudos, and close if it answers your question ==>
0 Kudos
Explorer
Explorer
545 Views
Registered: ‎06-25-2014

Re: Direct data crossing between synchronous clocks

Jump to solution

This is an interesting subject which may help me better understand STA (this is definitely a weak area of mine!) so thought I would join in with my thoughts! :)

For me, set_multicycle_path is only ever used on a synchronous path between either a common clock or 2 clocks that are frequency and phase locked. (Where phase locked can also include any integer relationship) Anything that cannot meet this must be considered asynchronous and CDC be considered.

So, I would consider UG903’s set_multicycle_path test cases all as synchronous examples and not covering cases such is dual MMCMs or anything else where you cannot guarantee a consistent phase alignment. Although, I do find the dual MMCM example you guys discuss a bit confusing in that if you are:

  • Running both from same clock input 
  • Meet whatever setup/hold requirements are necessary with a common reset signal to both MMCMs
  • The MMCM startup is 100% deterministic

Surely you can then guarantee 2 MMCMs will give a consistent phase alignment? (e.g. Each one producing a ½ clock output that will always have same phase relationship)

** Other than any extra jitter introduced by having 2 MMCMs.

Or are you guys saying that Xilinx lacks the timing model to correctly perform STA on the 2 MMCM example (or the 10GE example) and because of its inability to determine timing was met we must add CDC logic?

0 Kudos
Highlighted
Adventurer
Adventurer
533 Views
Registered: ‎03-31-2017

Re: Direct data crossing between synchronous clocks

Jump to solution

@andrewlan wrote:

 

Or are you guys saying that Xilinx lacks the timing model to correctly perform STA on the 2 MMCM example (or the 10GE example) and because of its inability to determine timing was met we must add CDC logic?


I don't know what we're saying... :)

However, this is my take-away so far:

  1. UG903 implies that the STA has the required knowledge
  2. UG949 implies that the STA does not have the required knowledge
  3. Various forum posts by various authors come down with either (1) or (2) above (but primarily with (2))

In principle, the STA tool should be able to handle 'direct' CDC with unrelated but nominally synchronous clocks. In the two-MMCM case, the complication is that there's an additional jitter to take into account, but that's just another margin. If the numbers don't add up, it should just tell you that it's not possible.

What we actually need is someone who works on the STA code to write something more definitive, but that's a long shot.

 

0 Kudos
Historian
Historian
511 Views
Registered: ‎01-23-2009

Re: Direct data crossing between synchronous clocks

Jump to solution

I have a couple of comments...

First, we have to separate the idea of analyzing clock path skew/insertion delay/propagation delay from the idea of "automatically generated clocks" - they are interrelated, but are separate.

For the first one (analyzing clock path skew/insertion delay/propagation delay) the tools are 100% correct - they understand the propagation delays completely, including the variation in paths between different components - including the variation in path delays that you would get between multiple MMCMs/PLLs (the equivalent of the CLKIN->CLKOUT phase error of MMCMs).

Now the second part. In "pure" SDC, there are no automatically generated clocks - all internally generated clocks need to be explicitly defined by the user (usually using create_generated_clock). Thus if your ASIC library has built in clock dividers or PLLs, you, the user, are required to manually write "create_generated_clock" commands for each output of these cells.

However, since MMCMs (and some other clock modifying cells - PLLs, BUFRs, BUFGCE_DIV [in UltraScale]) are part of the Xilinx library, and are used in almost all designs, Xilinx decided to add the "automatically generated clocks" mechanism. However, you have to understand that it has limitiations.

The main limitation is that it is designed around a single MMCM/PLL/BUFR/BUFGCE_DIV kind of system (or for BUFR/BUFGCE_DIV systems, only ones that explicitly synchronize the dividers). So, we know that it always works for different outputs of one MMCM. We also know it completely understands the relationship between a BUFIO clock (which is undivided) and a BUFR clock (which is divided). But that's pretty much it. Anything outside this, comes back to us, the user.

So, if you have (for example) two BUFRs (or BUFGCE_DIV) that don't use the "BUFR Alignment" mechanism described in UG472, Appendix A, then the relationship between the two automatically generated clocks on the outputs of the two BUFRs is incorrect - you, the user, need to intervene in order to describe this more complex relationship to the tools (probably with a set_max_delay).

Similarly if you use two MMCMs, then you need to figure out if the automatically generated clocks generated by the tool are sufficient or not. This post talks about the conditions where the MMCMs are guaranteed to be in phase or potentially out of phase. If you fall into the category where the MMCMs are guaranteed to be in phase, then the automatically generated clocks are sufficient. If you are using the MMCMs where they can be out of phase, then they are not, and you need to override the constraints (again, most likely a set_max_delay).

Finally if you start talking about GT's (GTP, GTH, GTX, GTY, etc...) then the relationship between the REFCLK and the TXOUTCLK/RXOUTCLK is very complex. In general, two GTs that share the same REFCLK should probably consider the RXOUTCLK/TXOUTCLK as asynchronous clocks. (There are ways around this when using multiple "bonded" channels, but the IP has to be built to take advantage of this).

So, with all this, if we come back to your original question - without knowing exactly what is being done to the clocks in these two cores, it is impossible to know if the relationship between the automatically generated clocks within these two cores is sufficient or not - we need to inspect the clocking inside the cores to see how they fit into the above cases in order to determine if the "default" analysis performed by the tools is correct or not.

Avrum

Tags (2)