06-14-2019 08:19 AM
Hi there~ Anybody who can see the following messages on running ptp4l master. It may arise from 10g/25g AXIE network driver. Further, will be greatly appreciated with any of how to attack this crash issue.
ptp4l[61451.0709]: port 1: master sync timeout
ptp4l[61452.0291]: port 1: master tx announce timeout
Block lock has been lost : 0x00000000 : 0x00000000 : 0
ptp4l[61452.0709]: port 1: delay timeout [61452.739001]
xilinx_axienet a0020000.ethernet: Did't get FIFO rx interrupt 164626432
[61452.739982] Synchronous External Abort: synchronous external abort (0x96000210) at 0xffffff8009540024
[61452.741137] Internal error: : 96000210 [#1] PREEMPT SMP
[61452.741790] Modules linked in: xlnx_ptp_timer macb xilinx_emac zcu102(O) [61452.742642] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.14.0-xilinx-ultrascale-kmw #36
[61452.743784] Hardware name: ZynqMP ZCU102 Rev1.0 XROE (DT)
[61452.744460] task: ffffff80091c1f80 task.stack: ffffff80091b0000
[61452.745214] PC is at axienet_tx_hwtstamp+0x68/0x1d0 [xilinx_emac]
[61452.745983] LR is at axienet_tx_hwtstamp+0x48/0x1d0 [xilinx_emac]
[61452.746743] pc :  lr :  pstate: 600001c5 [61452.747669] sp : ffffff8008003d70 [61452.748083] x29: ffffff8008003d70 x28: ffffffc87a17b000 [61452.748749] x27: 0000000080000060 x26: ffffffc87a7fbc00 [61452.749414] x25: ffffff80085c7110 x24: 000000000000000c [61452.750079] x23: 000037e44e9502b8 x22: ffffffc87a17b700 [61452.750744] x21: ffffffc87bbd4690 x20: ffffffc87bbd4680 [61452.751409] x19: ffffff8009485e00 x18: 0000000000000010 [61452.752074] x17: 0000007f93c571c8 x16: ffffff80081abad0 [61452.752739] x15: ffffffffffffffff x14: 3631207470757272 [61452.753404] x13: 65746e6920787220 x12: 4f46494620746567 [61452.754069] x11: 207427646944203a x10: 74656e7265687465 [61452.754734] x9 : 2e30303030323030 x8 : 612074656e656978 [61452.755399] x7 : 615f786e696c6978 x6 : 00000596cd504ad3 [61452.756064] x5 : 00ffffffffffffff x4 : 000631c796720566 [61452.756729] x3 : 0000000000000018 x2 : 000000000137afb8 [61452.757394] x1 : 000000000a00417d x0 : ffffff8009540024 [61452.758060] Process swapper/0 (pid: 0, stack limit = 0xffffff80091b0000) [61452.758898] Call trace: [61452.759205] Exception stack(0xffffff8008003c30 to 0xffffff8008003d70) [61452.760012] 3c20: ffffff8009540024 000000000a00417d [61452.760993] 3c40: 000000000137afb8 0000000000000018 000631c796720566 00ffffffffffffff [61452.761975] 3c60: 00000596cd504ad3 615f786e696c6978 612074656e656978 2e30303030323030 [61452.762956] 3c80: 74656e7265687465 207427646944203a 4f46494620746567 65746e6920787220 [61452.763937] 3ca0: 3631207470757272 ffffffffffffffff ffffff80081abad0 0000007f93c571c8 [61452.764918] 3cc0: 0000000000000010 ffffff8009485e00 ffffffc87bbd4680 ffffffc87bbd4690 [61452.765899] 3ce0: ffffffc87a17b700 000037e44e9502b8 000000000000000c ffffff80085c7110 [61452.766880] 3d00: ffffffc87a7fbc00 0000000080000060 ffffffc87a17b000 ffffff8008003d70 [61452.767861] 3d20: ffffff80005cc5a8 ffffff8008003d70 ffffff80005cc5c8 00000000600001c5 [61452.768842] 3d40: ffffff8008003d70 ffffff80005cc5a8 0000008000000000 ffffffc87bbd4680 [61452.769822] 3d60: ffffff8008003d70 ffffff80005cc5c8 [61452.770441]  axienet_tx_hwtstamp+0x68/0x1d0 [xilinx_emac] [61452.771341]  axienet_start_xmit_done+0x118/0x1c0 [xilinx_emac] [61452.772295]  axienet_tx_irq+0x148/0x1b0 [xilinx_emac] [61452.773148]  __handle_irq_event_percpu+0x5c/0x148 [61452.773954]  handle_irq_event_percpu+0x1c/0x58 [61452.774727]  handle_irq_event+0x48/0x78 [61452.775426]  handle_fasteoi_irq+0xb8/0x1a8
06-17-2019 11:58 PM
Can you please confirm on the setup and design which you are running. Is it the demo of RoE?
I beleive there is no link established for Core when you got this error.
06-18-2019 08:58 AM - edited 06-18-2019 09:02 AM
Thanks for your comment and I am unfortunately not remembering where I got the design, but I posted the screenshot of design shown through Vivado 2019.1.
It might be questionable how it can make you catch up what version(design) I am now using, but this design could be downloaded from Xilinx site by free without license agreement. It could be a sort of demonstration matter, but it was full enough to catch up how well IEEE1588 PTP 1 PPS tracing is achievable through Xilinx solution.
Linux drivers and DTS I tested is all originated from "xroe_framer_demo_2018_3.bsp" file obtainable from the Xilinx site as well. Yes, the error came from the case when we turned off PTP slave ZCU102 board. We used 2 ZCU102 board for either master or slave, and when we turned off the slave. The error happens in the master ZCU102 board. Do you guarantee the reliability of 10g/25g AXIe Ethernet driver even with such a crash?
When we use the both ZCU102 boards two masters (not SynE evironment), this error would not happen. When I switched to "two masters mode" only PTP protocol contributes to 1 PPS clock synchronization with its own built-in algorithm. Two ZCU102 boards might use their own TPU clocks.
11-19-2019 07:25 AM - edited 11-19-2019 07:30 AM
I can reproduce this issue in 2019.2 build. It only errors out with master command.
If you try with ptp4l -m -A -i eth0 -f /usr/bin/xroe-ptp4lsyncE.cfg -s, it should work fine.
We have reported this issue, and ethernet driver team is still investigating this.
2018.3 build shouldn't have problem.
11-19-2019 07:46 AM - edited 11-19-2019 07:46 AM
Thanks Xud for your comment.
Wasn't 2019.1 recommended for RoE job more than 2018.3 by Xilinx? Which one is more useful when we work with RFSoC not MPSoC ?
11-19-2019 07:51 AM
11-25-2019 08:02 AM - edited 11-25-2019 08:06 AM
I have tested 2019.1 build on ZCU102 board, it works fine.
My 2019.2 issue was related to following fix in driver. When I revert the fix, it doesn't crash anymore.
Can you PM your email address please? I will work with you offline, so you can transfer your petalinux project to me.
I will have a close look at your design.