cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
1,293 Views
Registered: ‎06-05-2019

1G/2.5G Ethernet PCS/PMA or SGMII v16.1 - resetdone timing for shared logic in core versus in example design

Jump to solution

I have a Kintex UltraScale design in Vivado 2018.1 that uses two “1G/2.5G Ethernet PCS/PMA or SGMII v16.1” IP blocks. These are being configured for 1G, SGMII, Device Specific Transceiver, SGMII PHY Mode, with unique Transceiver Location for each. One is configured to include shared logic in the core and the other to include shared logic in the example design. This was done to allow one block to generate "*_out" signals and supply them to the second block.

Block Design (portion)Block Design (portion)

I use the "resetdone" output from both IP blocks in a conditional statement in the design’s top-level VHDL (along with some other signal values) to determine when to pull the external CPU out of reset. PG047 indicates the resetdone signal is an “Indication that reset sequence of the transceiver is complete.”

When testing on hardware, we have an LED that indicates when the external CPU has come out of reset (basically). When both blocks' resetdone outputs are part of the condition to release that reset, we experience a long delay before that LED lights up (at least 40 seconds). When I comment out only the resetdone associated with the IP block configured to include shared logic in the example design from that condition, the LED lights in a fraction of that time (say around 4 seconds).

Is this kind of delay expected when the two IP blocks have been configured to handle shared logic differently? It certainly doesn't seem like it would be. Some explanation for the difference, if unavoidable, would be very helpful. If avoidable, some guidance for revising the design to reduce that lengthy resetdone assertion would be great.

0 Kudos
1 Solution

Accepted Solutions
nanz
Moderator
Moderator
986 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

I was asking if you could check the TX/RXRESETDONE from the GT and see if there are any delays. I just run the example design simulation:

nanz_0-1616084330917.png

Have you experimented with generating both IPs without shared logic and connect them up to clocking/reset? 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------

View solution in original post

17 Replies
nanz
Moderator
Moderator
1,221 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

How about when commenting out the resetdone from the IP that includes the shared logic? Does this happen after 40secs? The delay seems a bit big to me. 

In the implemented example designs for both IPs' configuration, the resetdone logic is connected the same way output from the transceiver. The only difference is the clock_reset is instantiated in the top-level exmpale design:

nanz_1-1615382929766.png

Once you confirm the resetdone from the IP which has shared logic in the example design has a big delay. We can add some debugging signals to check why this is happening. 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
1,216 Views
Registered: ‎06-05-2019

When the CPU-reset-release condition includes both resetdone_0 and resetdone_1, the delay is 40secs or so. When resetdone_1 is removed from that condition, the delay is only 4secs or so. resetdone_1 is from the IP configured to include shared logic in the example design.

Perhaps the delay is due, maybe only in part, to the IP with shared logic in the core taking some time to establish the "*_out" signals that get routed to the IP with shared logic in the example design? IOW, maybe the amount of time needed for both transceivers to come out of reset isn't all that different but one of those transceivers cannot begin its "come out of reset sequence" until much later making it look like one takes significantly longer?

0 Kudos
nanz
Moderator
Moderator
1,200 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

Yes, this makes sense. What is your system's requirement? Can you tolerate such delay? 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
1,197 Views
Registered: ‎06-05-2019

We don't have a requirement, per se. The customer is concerned about the delay difference between the two Ethernet options. If this delay is unavoidable, I would like to be able to supply a justification that isn't just based on my conjectures. When you say, "this makes sense", are you reacting to my suspicions or are you stating that based on some knowledge of the IP that hasn't yet been shared in this forum entry?

Thanks for your help! 

0 Kudos
nanz
Moderator
Moderator
1,163 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

Sorry, I meant the logic from 1st core to the 2nd might cause it. But the 40s seems quite long to me.

I wonder if there is some logic in your design that holds the reset of the 2nd core? How does the pma_reset_out insert from the 1st core? You can look into that signal and see. Even 4s for the first IP to insert resetdone is a bit much. How is this measure? Is it from the complete power up + configuration etc? 

The resetdone is a combination of the tx reset and rx reset done. There isn't a timeout logic to hold rx reset to cause such issues. 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
1,137 Views
Registered: ‎06-05-2019

Here are the results of some VIO/ILA debug with the hardware.

  1. I decided to temporarily swap the Eth0-related signals with the Eth1-related signals at the top-level where the block design is instantiated. In other words, the processor still thinks its communicating over Eth0 but those signals are being routed through the Eth1 portion of the block design (that has shared logic included in the example design). This was done just to ensure that portion of the block design is functioning properly since it hadn't been tested before. All good.
  2. With the swapped build, there was no change to the duration between assertion of FPGA done (complete power-up + config) and LED lighting up. That is, still over 40 seconds. This is what I expected would be the case.
  3. With the swapped build, I got a better capture of the timing when the Eth1 IP's resetdone is not part of the condition. I measured 7secs instead of 4secs, so a little bit larger than the earlier rough estimate, but certainly acceptable to our customer. Again, this was an expected duration.
  4. The time between FPGA done asserting and seeing the rising edge of the pma_reset_out for the Eth1 IP being used was only about 1sec. The same is true of the timing between FPGA done and the falling edge of the pma_reset_out. So there is definitely a larger portion of time between pma_reset_out deasserting and resetdone asserting.

To answer your question, there is nothing between the pma_reset_out of the Eth0 IP to the pma_reset input of the Eth1 IP, as seen in the original block diagram screen above. (The only new change being that I also route pma_reset_out to an ILA for debug purposes.)

I am working remotely so cannot put my hands on the hardware and so am working with an intermediary, so please note that there may be delays between your replies and my responses. Thanks!

0 Kudos
nanz
Moderator
Moderator
1,067 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

Would you please send me an ILA capture of the pma_reset_out deasserting and the resetdone asserting and rgmii_reset signal of the 2nd IP which has the shared logic outside of the core? 

It will also be good if you can capture the internet TX reset done and RX reset done so we could take a closer look. If on the RX path, the clock is not stable it may also delay it from asserting the resetdone. 

 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
1,004 Views
Registered: ‎06-05-2019

First up is a screenshot showing triggering on rising edge of resetdone_0 (the quick one) in the bottom window and rising edge of resetdone_1 (the slow one) in the top window. The reason for the two windows is so that the timestamps in the lower-left corners can be used to calculate a rough time difference. This corroborates the approx. 43sec delay before seeing the transceiver for the IP with shared logic in example design indicate reset is done.

Timestamp Diff for Resetdone AssertsTimestamp Diff for Resetdone Asserts

The next screenshot shows the time from rgmii_reset (the reset input to both 1G/2.5G Ethernet PCS/PMA or SGMII IP blocks) deassertion to when pma_reset_out deasserts from the IP block with shared logic in the core.

rgmii_reset Deassert to pma_reset_out Deassertrgmii_reset Deassert to pma_reset_out Deassert

The final screenshot shows the time between rgmii_reset deasserting and resetdone_0 asserting for the IP with shared logic in the core. A similar screenshot for the "slow" IP with shared logic in the example design was not possible to capture given the extreme delay. The ILAs' clock is 50MHz, so one sample every 20ns. The screenshot shows a duration of 31,984 samples, or approx. 640us. Contrast that with the ~40sec time for resetdone_1.

rgmii_reset Deassert to resetdone_0 Assertrgmii_reset Deassert to resetdone_0 Assert

You'd asked me to capture "internet TX reset done and RX reset done" but I'm not sure where those signals are; they don't seem to be available to me from the IP, as configured. See the original screenshot at the top of this forum post for the signals available for the two 1G/2.5G Ethernet PCS/PMA or SGMII IP blocks. Can you please clarify what you meant? Maybe you meant for me to add debug options for the RXRESETDONE and TXRESETDONE outputs of the GTHE3_CHANNEL_PRIM_INSTs?

Finally, I'm not seeing any way around this issue other than to change both IP to include shared logic in the example design and deal with the needed clocking logic externally but I'd like to avoid having to do that if. Are there alternatives to this?

0 Kudos
nanz
Moderator
Moderator
987 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

I was asking if you could check the TX/RXRESETDONE from the GT and see if there are any delays. I just run the example design simulation:

nanz_0-1616084330917.png

Have you experimented with generating both IPs without shared logic and connect them up to clocking/reset? 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------

View solution in original post

956 Views
Registered: ‎06-05-2019

An earlier iteration of the design actually did have both IPs set with shared logic in the example design and the clocking/reset logic supplied from the top-level of the design. I had changed that to the current design for a variety of reasons that aren't relevant here, none of which require the design to stay this way. So, I reverted the treatment to the prior style (as you suggested in your prior response).

The delay before the LED lights up is now the shorter 7sec time, as desired. (In actuality, both transceivers assert resetdone much earlier than 7sec, but there are other conditions on that LED lighting up.)

So, in some sense, my issue is resolved. In another, it is not. While I can move forward with this design revision, I still don't have an answer for the initial question of why when one IP provides clocking/reset signals to the other IP, do we see a significant delay on one transceiver's resetdone assertion? Said differently, if that is indeed a valid way to connect two of these 1G/2.5G Ethernet PCS/PMA or SGMII IP blocks, as Xilinx documentation suggests is true, then what about this configuration causes such a drastic duration difference in reset completion between transceivers?

Thanks for your help!

0 Kudos
nanz
Moderator
Moderator
909 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

I did do a search internally and see if we had similar issues reported with 2018.1 tool, and I find one CR - the transceiver reset FSM got stuck & resetdone signal not asserted. This issue has been resolved in 2018.3, and the reason is - 

It is observed that the free-run DRP clock (i.e. independent_clock_bufg input) to GT is not driven with the correct value as per the GUI setting. Due to which the CPLLLOCK was not getting asserted from the bit_sync module output.

Once the clock is updated with the correct input, it worked fine. 

This does not look exactly the same issue as you reported. But it will be great if you could test with 2018.3 and later version of the IP and tools and see if that works OK. 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
882 Views
Registered: ‎06-05-2019

Thank you for looking into that further. At this point, I am unable to try things out in 2018.3, but I will make note of this in the case that I am able to test further at a later time.

592 Views
Registered: ‎06-05-2019

UPDATE: Since this forum thread wound down in March, I have subsequently had a chance to try looking at similar design choices in Vivado 2018.3, as nanz had requested checking. I created two versions of the new project, which makes use of six Xilinx AXI Ethernet Subsystem IP blocks instead of the two described above:

  1. A "core" version, where one of those IP blocks has Shared Logic in the Core and feeds the generated clock/reset signals to the other five IP blocks with Shared Logic in the Example Design.
  2. A "no-core" version, where all six IP blocks are configured with Shared Logic in the Example Design; thus, the needed clock/reset signals are generated in the top-level VHDL and fed to the six IP blocks in the same fashion.

With this now being done in Vivado 2018.3, that means this IP is at rev 5.0 rather than rev 3.0. I also found the following which may be related to the transceiver release-from-reset delay issue originally described in this post for Vivado 2018.1: https://www.xilinx.com/support/answers/70875.html

The "core" version does not exhibit the ~40sec delay we had experienced in 2018.1 (nor does the "no-core", but that is expected). It therefore appears the update to rev 4.0 of the IP has resolved this issue (in Vivado 2018.2 and later). We would prefer to knowingly continue using the "core" design but would also prefer doing so with concrete knowledge that it was the IP update that resolved the original issue.

Can a Xilinx representative officially confirm that?

If not, is there a way to downgrade the "core" version of this 2018.3 project to 2018.1 to see if it then starts showing the delay?

0 Kudos
nanz
Moderator
Moderator
448 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

It's very likely the updated IP version resolved the issue you have seen in 2018.1. We do not recommend back porting the core to an older version of the tools to test. This flow has not been tested. So we are not sure if this is working or not. Users can try this though by creating a user repository and added the new version of the IP there for testing.

The AR that you pointed out applies to 1000BASE-X or SGMII over LVDS configuration but you are not using LVDS if I remember correctly. 

I am not exactly sure what your concern is right now. IAre you not able to work on 2018.3 and need to stick with 2018.1? 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
409 Views
Registered: ‎06-05-2019

Ah yes, I see what you mean about the LVDS aspect of AR 70875. You are correct, we are not using LVDS. I also looked at the change logs of the sub-blocks, but nothing listed there jumped out at me as being related to the transceiver delay we'd been experiencing in 2018.1.

We are able to stick with 2018.3 for this new project, so that is not a concern. What is a potential concern is still not knowing what specifically resolved the issue. It would be nice to be able to document what exactly resolved that delay when moving from 2018.1 to 2018.3, so that there is no mystery that perhaps we might not be seeing the issue at the moment and can rest assured it won't somehow "pop up" down the line after the design has been finalized, loosely speaking.

0 Kudos
nanz
Moderator
Moderator
401 Views
Registered: ‎08-25-2009

Hi james.laser@ge.com ,

Great to know that you are able to stick with 2018.3. 

The problem is to root cause the issue in 2018.1, we need to be able to reproduce the issue on our end either in simulation or on HW to debug it. But we have not gone down that root. As the 2018.3 seems working fine, I wonder if it's worth the effort. 

I also checked the changelog on my end and I agree that your issue is not documented, and the only internal CR that I could find is the one I mentioned earlier in the thread.

But overall, I think you should be confident to go ahead with 2018.3. 


-------------------------------------------------------------------------------------------

Don’t forget to reply, kudo, and accept as solution.

If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs and our Versal Ethernet Sticky Note.

-------------------------------------------------------------------------------------------
0 Kudos
398 Views
Registered: ‎06-05-2019

Thank you for the additional checks and thoughts. I think we've taken this as far as we can through this forum thread now. Much appreciated.