Booting into Petalinux causes 10G Eth to break

[dima2882]

Hi all,

This one has me stumped.

I've got a Vivado 2018.2 ZCU102 system working great in u-boot. There's a 10G/25G eth subsystem in the PL that is completely unrelated to the PS,doesn't use it's clocks or resets or anything else. The eth subsystem is working great. My boot cycle is load bitfile via jtag, then the various .elf files, fsbl, then u-boot, then load linux image via TFTP. 10G eth subsystem doesn't use AXI regs, but it's doing everything right when I'm in u-boot, doesn't need any PS software support at all. Ethernet comes through using SFP+ modules in the ZCU102 card cage.

The eth subsystem loses sync as soon as Petalinux boots. The rx_status signal goes low, indicating that block lock and K character register signals are low, and core is stuck in the INIT state. Basically, as soon as I get the "Starting udev", or thereabout, message in the console, eth subsystem stops working. FPGA needs a full power cycle to get the eth subsystem to work again in u-boot.

It's as if Petalinux turns off my SFPs. What could be happening in Petalinux to make this occur?! I thought the kernel booting is exclusively a PS software operation... My dmesg file is attached for reference.

Please help!

[jadhavs · Accepted solution]

Hi @dima2882,

May be the Device tree or clock might be wrong as if you use a ZCU102 board preset values (zcu102-rev1.0.dtsi) then the clock I2C changes to 148.5Mhz rather than 156.25Mhz. Check your final DTB/DTS and this will give you some clue. Please see my "system-user.dtsi" file for cross check. If you need full project please let me know and I can send you via Xilinx FTP server. (FYI - my project is based on Xapp1305 and has 1G + 10G both enabled in it.)  

[dima2882]

Big thanks for the hint! This got me really close, but still the Si570 is at 148.5MHz, now that you told me about this I can see it in dmesg.

I can verify that my final dts has the right value of clock-frequency = <0x9502f90>, or I can delete the node entierly and verify that it's gone. I verified all this by taking my image.ub in /tftpboot, and running dumpimage -T flat_dt -p 1 -i /tftpboot/image.ub recovered_dtb.dtb on it, then looked at the .dtb with dtc -I dtb -O dts recovered_dtb.dtb -o device_tree.dts and seeing the right stuff for the node at I2C address 74.

However, somehow, after loading that image, I'm still registering the Si570 at 148.5MHz. I am certain that the right image.ub is specified - is there any other file u-boot could be using instead that doesn't change?!

Alternatively, I can see the wrong value in /proc/device-tree/amba/i2c@ff030000/i2c-mux@74/i2c@3/clock-generator3@5d/clock-frequency - it's the hex for 148.5MHz. Is there a way to echo the right value into it? Even with 777 permissions, clock-frequency claims no permission to change its value...

[shabbirk]

Hi

Try programming the si570 to required clock frequency using SCUI and see if that helps

Best Regards

Shabbir

[dima2882]

@jadhavs, @shabbirk thank you both for helping me with this.

The solution that @jadhavs suggested is ultimately the right one to use. It didn't work right away, so I experimented with the SCUI and did get the right clock rate through that... Went back and noticed I was reading from the wrong TFTP server in the lab and was not getting the updated image. Wit hthat fixed the right clock rate is being programmed.

Also, this thread - https://forums.xilinx.com/t5/Embedded-Linux/Petalinux-2016-4-on-ZCU102-resets-Si570-MGT-to-148-5-MHz-during/td-p/769996 - had a similar experience.