02-06-2017 04:35 PM - edited 02-06-2017 04:43 PM
Hi,
I have created the attached design, starting from the design provided in xilinx-zcu102-v2016.4-final1.bsp
I have added three PCS/PMA cores, and connected them to the SFP+ connectors.
After exporting the hardware from Vivado, and importing them into Petalinux, I get the attached Petalinux project.
The compiled Linux image boots correctly with:
petalinux-boot --jtag --fpga --bitstream images/linux/design_1_wrapper.bit --kernel
But unfortunately the PCS/PMA based PHYs are not recognized (below is the related part of the boot log):
[ 3.322719] libphy: Fixed MDIO Bus: probed [ 3.327260] tun: Universal TUN/TAP device driver, 1.6 [ 3.332284] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> [ 3.338601] CAN device driver interface [ 3.343172] macb ff0b0000.ethernet: invalid hw address, using random [ 3.349527] gpiod_set_value: invalid GPIO [ 3.353917] libphy: MACB_mii_bus: probed [ 3.360544] macb ff0b0000.ethernet eth0: no PHY found [ 3.404512] macb ff0c0000.ethernet: invalid hw address, using random [ 3.410856] gpiod_set_value: invalid GPIO [ 3.415220] libphy: MACB_mii_bus: probed [ 3.421849] macb ff0c0000.ethernet eth0: no PHY found [ 3.460497] macb ff0d0000.ethernet: invalid hw address, using random [ 3.466847] gpiod_set_value: invalid GPIO [ 3.471209] libphy: MACB_mii_bus: probed [ 3.474629] ata2: SATA link down (SStatus 0 SControl 330) [ 3.474659] ata1: SATA link down (SStatus 0 SControl 330) [ 3.488703] macb ff0d0000.ethernet eth0: no PHY found [ 3.528494] gpiod_set_value: invalid GPIO [ 3.532800] libphy: MACB_mii_bus: probed [ 3.540111] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 33 (00:0a:35:00:02:90) [ 3.549975] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0e0000.etherne:0c, irq=-1) [ 3.561766] usbcore: registered new interface driver asix
In the final dts file all Ethernet controllers are described as follows:
ethernet@ff0b0000 { compatible = "cdns,zynqmp-gem"; status = "okay"; interrupt-parent = <0x2>; interrupts = <0x0 0x39 0x4 0x0 0x39 0x4>; reg = <0x0 0xff0b0000 0x0 0x1000>; clock-names = "pclk", "hclk", "tx_clk"; #address-cells = <0x1>; #size-cells = <0x0>; #stream-id-cells = <0x1>; iommus = <0x6 0x874>; power-domains = <0xc>; clocks = <0xd 0xd 0xd>; phy-mode = "gmii"; xlnx,ptp-enet-clock = <0x0>; linux,phandle = <0x1f>; phandle = <0x1f>; }; ethernet@ff0c0000 { compatible = "cdns,zynqmp-gem"; status = "okay"; interrupt-parent = <0x2>; interrupts = <0x0 0x3b 0x4 0x0 0x3b 0x4>; reg = <0x0 0xff0c0000 0x0 0x1000>; clock-names = "pclk", "hclk", "tx_clk"; #address-cells = <0x1>; #size-cells = <0x0>; #stream-id-cells = <0x1>; iommus = <0x6 0x875>; power-domains = <0xe>; clocks = <0xd 0xd 0xd>; phy-mode = "gmii"; xlnx,ptp-enet-clock = <0x0>; linux,phandle = <0x20>; phandle = <0x20>; }; ethernet@ff0d0000 { compatible = "cdns,zynqmp-gem"; status = "okay"; interrupt-parent = <0x2>; interrupts = <0x0 0x3d 0x4 0x0 0x3d 0x4>; reg = <0x0 0xff0d0000 0x0 0x1000>; clock-names = "pclk", "hclk", "tx_clk"; #address-cells = <0x1>; #size-cells = <0x0>; #stream-id-cells = <0x1>; iommus = <0x6 0x876>; power-domains = <0xf>; clocks = <0xd 0xd 0xd>; phy-mode = "gmii"; xlnx,ptp-enet-clock = <0x0>; linux,phandle = <0x21>; phandle = <0x21>; }; ethernet@ff0e0000 { compatible = "cdns,zynqmp-gem"; status = "okay"; interrupt-parent = <0x2>; interrupts = <0x0 0x3f 0x4 0x0 0x3f 0x4>; reg = <0x0 0xff0e0000 0x0 0x1000>; clock-names = "pclk", "hclk", "tx_clk"; #address-cells = <0x1>; #size-cells = <0x0>; #stream-id-cells = <0x1>; iommus = <0x6 0x877>; power-domains = <0x10>; clocks = <0xd 0xd 0xd>; phy-mode = "rgmii-id"; xlnx,ptp-enet-clock = <0x0>; local-mac-address = [00 0a 35 00 02 90]; phy-handle = <0x11>; linux,phandle = <0x22>; phandle = <0x22>; phy@c { reg = <0xc>; ti,rx-internal-delay = <0x8>; ti,tx-internal-delay = <0xa>; ti,fifo-depth = <0x1>; linux,phandle = <0x11>; phandle = <0x11>; }; };
Only the third one, that uses the external PHY is operational.
Should I add the phy definitions for the PCS/PMA connected controllers myself?
TIA & Regards,
Wojtek
PS. The attached files should be renamed to des_plnx.tar.xz and xilinx-zcu102-2016.4_xpr_zip, respectively.
I was not able to post them with original names (they were rejected with complaint, that extension does not match the contents).
05-19-2017 11:36 AM
I have prepared an improved version of my sources. The design has been organized in a hierarchy.
You can select the block with Ethernet interfaces and copy it to your own design, as described here.
Moving the Ethernet interfaces with PCS/PMA blocks has exposed a problem with Vivado 2016.4. Therefore, the MDIO lines are connected separately. That solves the problem of not recognized PHY.
The sources have been stripped to the minimum - so the compressed archive allowing recreating of the design occupy only 400KB.
Current setup is prepared for SFP transceivers connected via the FMC0 connector. If you want to change it, you may simply change the locations of pins in the io_ports.xdc. Theoretically you should also adjust the location of the PCS/PMA blocks, but it seems, that Vivado adjusts their location automatically.
Please note, that you may also need to modify the constant defining the state of the sfp_tx_disable lines. In on-board SFP cage on ZCU102 there is an inverting MOS-FET transistor driving the SFP pins. In the FMC board there was a direct connection.
Regards,
Wojtek
02-07-2017 12:40 PM - edited 02-07-2017 12:41 PM
Using examples posted by other users, I have added the dtsi file defining the phys:
&gem0 { local-mac-address = [00 0a 35 00 00 00]; phy-handle = <&phy10>; phy10: phy@6 { compatible = "Xilinx PCS/PMA PHY"; device_type = "ethernet-phy"; reg = <0x6>; }; }; &gem1 { local-mac-address = [00 0a 35 00 00 01]; phy-handle = <&phy11>; phy11: phy@6 { compatible = "Xilinx PCS/PMA PHY"; device_type = "ethernet-phy"; reg = <0x6>; }; }; &gem2 { local-mac-address = [00 0a 35 00 00 02]; phy-handle = <&phy12>; phy12: phy@6 { compatible = "Xilinx PCS/PMA PHY"; device_type = "ethernet-phy"; reg = <0x6>; }; };
Now, when booting the system, I get the following:
[ 3.288509] libphy: Fixed MDIO Bus: probed [ 3.293046] tun: Universal TUN/TAP device driver, 1.6 [ 3.298077] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> [ 3.304392] CAN device driver interface [ 3.308955] gpiod_set_value: invalid GPIO [ 3.313304] libphy: MACB_mii_bus: probed [ 3.317202] mdio_bus ff0b0000.etherne:06: mdio_device_register [ 3.325876] macb ff0b0000.ethernet eth0: no PHY found [ 3.374538] gpiod_set_value: invalid GPIO [ 3.378855] libphy: MACB_mii_bus: probed [ 3.382757] mdio_bus ff0c0000.etherne:06: mdio_device_register [ 3.391409] macb ff0c0000.ethernet eth0: no PHY found [ 3.430522] gpiod_set_value: invalid GPIO [ 3.434837] libphy: MACB_mii_bus: probed [ 3.438737] mdio_bus ff0d0000.etherne:06: mdio_device_register [ 3.440654] ata1: SATA link down (SStatus 0 SControl 330) [ 3.440690] ata2: SATA link down (SStatus 0 SControl 330) [ 3.458256] macb ff0d0000.ethernet eth0: no PHY found [ 3.498521] gpiod_set_value: invalid GPIO [ 3.502831] libphy: MACB_mii_bus: probed [ 3.510143] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0) [ 3.520012] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867])
It seems, that the essential problem is that the "macb" is not informed properly about the EMIO lines used to connect the PCS/PMA internal PHY (the "gpiod_set_value: invalid GPIO" error message).
How can I pass that info to the macb driver?
Should I somehow enable the EMIO GPIO support in the kernel?
TIA & Regards,
Wojtek
02-07-2017 12:47 PM
It's a funny thing, that the similar problem is displayed in the "Expected output" on one of official Xilinx sites:
http://www.wiki.xilinx.com/Linux+I2C+Driver
Creating 4 MTD partitions on "spi0.0": 0x000000000000-0x000000500000 : "boot" 0x000000500000-0x000000520000 : "bootenv" 0x000000520000-0x000000fa0000 : "kernel" 0x000000fa0000-0x000001000000 : "spare" CAN device driver interface gpiod_set_value: invalid GPIO libphy: MACB_mii_bus: probed mdio_bus e000b000.etherne:07: mdio_device_register macb e000b000.ethernet eth0: no PHY found
Regards,
Wojtek
02-08-2017 07:55 AM
Is it possible, that my problem is related to the old https://www.xilinx.com/support/answers/67930.html problem?
Is it still valid in Vivado 2016.4?
Regards,
Wojtek
02-09-2017 03:12 PM - edited 02-09-2017 04:08 PM
The problem with not detected PHY was caused by the fact, that I haven't configured the GMT clock correctly
(using the controller accessible via /dev/ttyUSB3).
After correct configuration of the clock, the PHY is detected, the link goes up but with incorrect speed (10/Full):
PMUFW: PmMmioWrite: (NODE_APU) addr=0xFF180318, mask=0x3F0000, value=0x50000 PMUFW: PmMmioWrite: (NODE_APU) addr=0xFF180358, mask=0x40000, value=0x0 [ 4.760476] mmc0: new high speed SDHC card at address 0007 [ 4.770840] mmcblk0: mmc0:0007 SD04G 3.71 GiB [ 4.777156] mmcblk0: p1 [ 5.380098] EXT4-fs (mmcblk0p1): recovery complete [ 5.384881] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null) [ 5.650588] random: dd urandom read with 7 bits of entropy available [ 5.747168] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 7.352940] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz [ 7.361058] macb ff0b0000.ethernet eth0: link up (10/Full) [ 7.366575] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0 plnx_aarch64 login: root Password: root@plnx_aarch64:~# ifconfig eth1 up [ 31.579411] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready root@plnx_aarch64:~# ifconfig eth1 [ 33.456940] macb ff0c0000.ethernet eth1: unable to generate target frequency: 2500000 Hz [ 33.465056] macb ff0c0000.ethernet eth1: link up (10/Full) [ 33.470575] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready ifconfig eth2 up [ 38.623307] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready root@plnx_aarch64:~# [ 40.560937] macb ff0d0000.ethernet eth2: unable to generate target frequency: 2500000 Hz [ 40.569053] macb ff0d0000.ethernet eth2: link up (10/Full) [ 40.574573] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
However I can't receive anything:
root@plnx_aarch64:~# ifconfig eth0 Link encap:Ethernet HWaddr 72:06:FC:B4:4B:30 inet6 addr: fe80::7006:fcff:feb4:4b30%4890480/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:46 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:13644 (13.3 KiB) Interrupt:30 eth1 Link encap:Ethernet HWaddr 6A:39:5B:1D:1E:52 inet6 addr: fe80::6839:5bff:fe1d:1e52%4890480/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B) Interrupt:31 eth2 Link encap:Ethernet HWaddr 26:59:A8:57:FC:F6 inet6 addr: fe80::2459:a8ff:fe57:fcf6%4890480/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:648 (648.0 B) Interrupt:32
I have played with different settings regarding the 1000BaseX or SGMII mode, turning on and off the "SGMII PHY mode", but nothing helps.
I attach the current version of the hardware project with VEXTPROJ scripts, that allow to rebuild the project from a minimal set of files.
Regards,
Wojtek
02-11-2017 06:35 AM - edited 02-11-2017 07:26 AM
Xilinx published the new XAPP1305 note mentioned here: http://www.wiki.xilinx.com/PS+and+PL+based+Ethernet+in+Zynq+MPSoC
Unfortunately, the design files are not available yet: https://forums.xilinx.com/t5/UltraScale-Architecture/Where-are-design-files-for-XAPP-1305/td-p/745318
I'm trying to anayze the Zynq Ultrascale+ TRM in hope that I'll find the solution myself.
It is still unclear how to connect the GMII_ENETx ports in PS with GMII_GEM_PCS_PMA ports, as they contain different pins (in PS there are additional rx_clk, tx_clk, crs,col and speed_mode lines). Splitting the ports and connecting lines by hand make design illegible and difficult to maintain. Below is just a part of the design in it's last form (I also attach the VEXTPROJ version of the design):
Now I have added the GPIO cores to monitor the PCS/PMA status and to reset PCS/PMA.
Unfortunately I still get the PCS/PMA status 0x806 after booting the system.
When I reset the PCS/PMA, the status starts to oscillate between: 0x818, 0x810, 0x850, and 0x870.
I have checked different settings of PCS/PMA - SGMII, 1000BaseX, with and without autonegotiation and with and without "SGMII PHY mode".
I use GLC-T compatible transceivers:
Regards,
Wojtek
02-11-2017 10:19 AM
02-12-2017 08:00 AM - edited 02-12-2017 11:46 AM
I have prepared a special FPGA firmware equipped with virtual GPIOs for diagnostic and control functions.
They can be controlled with devmem utility.
To work with that firmware, it is necessary, after power up of the ZUC102 to configure the Si570 MGT clock to 125MHz (using the controller accessible via the 3rd UART channel - /dev/ttyUSB3 in my development system).
If I compile the Petalinux with macb and xilinx-phy compiled into the kernel, I get the following results:
[ 5.722125] random: dd urandom read with 8 bits of entropy available [ 5.818468] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 7.416931] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz [ 7.425053] macb ff0b0000.ethernet eth0: link up (10/Full) [ 7.430570] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0 plnx_aarch64 login: root Password: root@plnx_aarch64:~# devmem 0x80000000 0x00000806 root@plnx_aarch64:~# devmem 0x80010000 0x00000806 root@plnx_aarch64:~# devmem 0x80020000 0x00000806 root@plnx_aarch64:~# devmem 0x80040000 32 0xc9a01 root@plnx_aarch64:~# devmem 0x80050000 32 0x11 root@plnx_aarch64:~# devmem 0x80030000 32 0x47 root@plnx_aarch64:~# devmem 0x80030000 32 0x40 root@plnx_aarch64:~# devmem 0x80030000 32 0x47[ 400.428880] macb ff0b0000.ethernet eth0: link down root@plnx_aarch64:~# devmem 0x80030000 32 0x47 root@plnx_aarch64:~# [ 426.432933] macb ff0b0000.ethernet eth0: unable to generate target frequency 2500000 Hz [ 426.441057] macb ff0b0000.ethernet eth0: link up (10/Full) devmem 0x80030000 32 0x40 root@plnx_aarch64:~# [ 440.444881] macb ff0b0000.ethernet eth0: link down root@plnx_aarch64:~# devmem 0x80030000 32 0x58 root@plnx_aarch64:~# devmem 0x80030000 32 0x70 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80000000 0x00000870 root@plnx_aarch64:~# devmem 0x80010000 0x00000850 root@plnx_aarch64:~# devmem 0x8000[ 513.982585] random: nonblocking pool is initialized 2000 0x00000810 root@plnx_aarch64:~# devmem 0x80020000 0x00000810 root@plnx_aarch64:~# devmem 0x80020000 0x00000850 root@plnx_aarch64:~# root@plnx_aarch64:~# devmem 0x80030000 32 0x7 root@plnx_aarch64:~# devmem 0x80030000 32 0x7[ 818.448937] macb ff0b0000.ethernet eth0: unable to gz [ 818.457051] macb ff0b0000.ethernet eth0: link up (10/Full) 00000^C root@plnx_aarch64:~# devmem 0x80030000 32 0x0 root@plnx_aarch64:~# devmem 0x80030000 32 0x7[ 835.460883] macb ff0b0000.ethernet eth0: link down 10000^C root@plnx_aarch64:~# devmem 0x80030000 32 0x18 root@plnx_aarch64:~# devmem 0x80030000 32 0x20 root@plnx_aarch64:~# devmem 0x80030000 32 0x0 root@plnx_aarch64:~# devmem 0x80000000 0x00000830 root@plnx_aarch64:~# devmem 0x80000000 0x00000870 root@plnx_aarch64:~# devmem 0x80000000 0x00000870 root@plnx_aarch64:~# devmem 0x80020000 0x00000810 root@plnx_aarch64:~# devmem 0x80010000 0x00000870
The interesting thing is, that the link "goes up", when PCS/PMA is being reset, and goes down after I release the reset. However the speed of the link is detected as 10Mb/s.
After the reset, the statuses of PCS/PMAs starts to fluctuate between 0x810, 0x818, 0x830, 0x850 and 0x870.
When I compile the Petalinux with macb and xilinx-phy as modules, I get other results:
[ 5.378323] random: dd urandom read with 8 bits of entropy available PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0 plnx_aarch64 login: root Password: root@plnx_aarch64:~# devmem 0x80000000 0x0000D80B root@plnx_aarch64:~# devmem 0x80010000 0x0000D80B root@plnx_aarch64:~# devmem 0x80020000 0x0000D80B root@plnx_aarch64:~# modprobe xilinx-phy root@plnx_aarch64:~# devmem 0x80000000 0x0000D80B root@plnx_aarch64:~# devmem 0x80010000 0x0000D80B root@plnx_aarch64:~# devmem 0x80020000 0x0000D80B root@plnx_aarch64:~# modprobe macb [ 64.063976] macb ff0b0000.ethernet: invalid hw address, using random [ 64.070361] gpiod_set_value: invalid GPIO [ 64.074936] libphy: MACB_mii_bus: probed [ 64.081581] macb ff0b0000.ethernet eth0: no PHY found [ 64.121180] macb ff0c0000.ethernet: invalid hw address, using random [ 64.127552] gpiod_set_value: invalid GPIO [ 64.132105] libphy: MACB_mii_bus: probed [ 64.138773] macb ff0c0000.ethernet eth0: no PHY found [ 64.181171] macb ff0d0000.ethernet: invalid hw address, using random [ 64.187565] gpiod_set_value: invalid GPIO [ 64.192129] libphy: MACB_mii_bus: probed [ 64.198802] macb ff0d0000.ethernet eth0: no PHY found [ 64.249176] gpiod_set_value: invalid GPIO [ 64.253762] libphy: MACB_mii_bus: probed [ 64.261022] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 33 (00:0a:3) [ 64.270899] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff) root@plnx_aarch64:~# devmem 0x80000000 0x0000D80B root@plnx_aarch64:~# devmem 0x80010000 0x0000D80B root@plnx_aarch64:~# devmem 0x80020000 0x0000D80B root@plnx_aarch64:~#
So now, the status of the links is correct, but when I load the driver, it is not able to find the PHY.
Even in that case, the reset of {PCS/PMA results in unstable operation of the link (why?)
root@plnx_aarch64:~# devmem 0x80000000 0x0000D80B root@plnx_aarch64:~# devmem 0x80010000 0x0000D80B root@plnx_aarch64:~# devmem 0x80020000 0x0000D80B root@plnx_aarch64:~# devmem 0x80030000 32 7 root@plnx_aarch64:~# devmem 0x80000000 0x0000D80B root@plnx_aarch64:~# devmem 0x80030000 32 0 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80010000 0x00000870 root@plnx_aarch64:~# devmem 0x80020000 0x00000810 root@plnx_aarch64:~# devmem 0x80040000 32 0xc9a1 root@plnx_aarch64:~# devmem 0x80050000 32 0x10 root@plnx_aarch64:~# devmem 0x80030000 32 0x18 root@plnx_aarch64:~# devmem 0x80030000 32 0x20 root@plnx_aarch64:~# devmem 0x800q0000 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80000000 0x00000850 root@plnx_aarch64:~# devmem 0x80010000 0x00000850 root@plnx_aarch64:~# devmem 0x80020000 0x00000810 root@plnx_aarch64:~# devmem 0x80030000 32 0x07 root@plnx_aarch64:~# devmem 0x80030000 32 0x0 root@plnx_aarch64:~# devmem 0x80030000 32 0x20 root@plnx_aarch64:~# devmem 0x80030000 32 0x0 root@plnx_aarch64:~# devmem 0x80000000 0x00000830 root@plnx_aarch64:~# devmem 0x80010000 0x00000870 root@plnx_aarch64:~#
Why the PCS/PMA behaves so strange, and how can I combine its correct operation with working macb+xilinx-phy drivers?
Regards,
Wojtek
PS. I attach the sources of the diagnostic firmware in VEXTPROJ format to keep its size as small as possible.
02-13-2017 05:17 PM - edited 02-13-2017 05:19 PM
It appeared, that during the debugging I have done incorrect adjustment of the design.
I have moved the MGT reference clock from B10 (ZCU102 Si5328 MGT Clock) to C8 (ZCU102 Si570 MGT Clock).
It appears, that the clock at C8 gets reconfigured to 156 MHz when Linux is booting.
So I have changed back the corresponding line in io_ports.xdc:
set_property PACKAGE_PIN B10 [get_ports diff_clock_rtl_clk_p]
Of course after that change, you have to set the ZCU102 Si5328 MGT Clock Frequency to 125MHz after the ZCU102 power-up.
After that, the PCS/PMA status is correct even after booting with macb and xilinx-phy drivers compiled into the kernel, but the driver still thinks, that the link works at 10Mb/s (even though the PCS/PMA status shows 1Gb/s!)
The PCS/PMA also responds correctly to the reset:
5.754872] random: dd urandom read with 7 bits of entropy available [ 5.851401] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 9.498048] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz [ 9.506170] macb ff0b0000.ethernet eth0: link up (10/Half) [ 9.511687] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80030000 32 7
root@plnx_aarch64:~# devmem 0x80000000
0x00000800
root@plnx_aarch64:~# devmem 0x80010000
0x00000800
root@plnx_aarch64:~# devmem 0x80020000
0x00000800
root@plnx_aarch64:~# devmem 0x80030000 32 0
root@plnx_aarch64:~# [ 728.557998] macb ff0b0000.ethernet eth0: link down
[ 730.562038] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[ 730.570144] macb ff0b0000.ethernet eth0: link up (10/Half)
devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000
0x0000D80B
So now the only problem still remaining is the communication with the kernel driver...
02-14-2017 02:16 PM - edited 02-14-2017 03:39 PM
I have got my links working at 1000Mb/s both in BASEX and SGMII mode, when I have switched off the autonegotiation, and manually set the parameters with ethtool:
[ 5.723371] random: dd urandom read with 7 bits of entropy available [ 5.819537] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0 plnx_aarch64 login: root Password: root@plnx_aarch64:~# ethtool eth0 speed 1000 duplex full autoneg off ethtool: bad command line argument(s) For more information run ethtool -h root@plnx_aarch64:~# ethtool -s eth0 speed 1000 duplex full autoneg off root@plnx_aarch64:~# [ 57.417137] macb ff0b0000.ethernet eth0: link up (1000/Full) [ 57.422813] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready root@plnx_aarch64:~# udhcpc udhcpc (v1.24.1) started Sending discover... Sending select for 192.168.95.190... Lease of 192.168.95.190 obtained, lease time 39600 /etc/udhcpc.d/50default: Adding DNS 192.168.95.253 /etc/udhcpc.d/50default: Adding DNS 194.29.161.2 /etc/udhcpc.d/50default: Adding DNS 194.29.160.10
However I still can't get working the link with 100Mb/s device:
root@plnx_aarch64:~# ethtool -s eth2 speed 100 duplex full autoneg off root@plnx_aarch64:~# [ 130.825144] macb ff0d0000.ethernet eth2: unable to generate target frequency: 25000000 Hz [ 130.833355] macb ff0d0000.ethernet eth2: link up (100/Full) root@plnx_aarch64:~# ifconfig eth2 192.168.0.1 promisc up [ 160.939951] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready [ 160.945867] device eth2 entered promiscuous mode root@plnx_aarch64:~# ethtool -s eth2 speed 100 duplex full autoneg off root@plnx_aarch64:~# [ 172.837155] macb ff0d0000.ethernet eth2: unable to generate target frequency: 25000000 Hz [ 172.845360] macb ff0d0000.ethernet eth2: link up (100/Full) [ 172.850969] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready [ 232.999321] random: nonblocking pool is initialized
Update
Looking at [1] and [2], I can state that the problem with "unable to generate target frequency: 25000000 Hz" is related to clocks available for the Linux kernel. So either those clocks are not configured correctly, or the kernel is not informed correctly about the available clocks...
02-14-2017 04:41 PM
Trying to understand, why the macb driver has problems with generation of 25000000 Hz clock, I have discovered that it is unclear why it is needed. Both clocks (gmii_rx_clk and gmii_tx_clk) are delivered from PCS/PMA to GMII_ENET port.
So, why the macb must to generate independent clocks?
I have also found the following message. Maybe it explains, why I had problems with autonegotiation:
>> To go to 100Mbps mode: >> >> ethtool -s eth0 advertise 0x0f >> >> Back to gigabit: >> >> ethtool -s eth0 advertise 0x3f >> >> >> On the mpsoc this is broken too, but I guess that's just work in progress... >> >> root at zcu102-zynqmp:~# ethtool -s eth0 advertise 0x0f >> macb ff0e0000.ethernet eth0: link down >> macb ff0e0000.ethernet eth0: unable to generate target frequency: 25000000 Hz If you are using the in kernel device trees for zcu102,.they currently have fixed 125MHz clocks for the gems.
02-15-2017 12:29 AM
I have found yet another AR that may explain the problems reported in that thread:
In PetaLinux 2015.4, 2016.1 and 2016.2, dynamic speed switching is not possible. Only the default speed of 1G is supported. This is NOT a silicon or board limitation but a Linux infrastructure (to enable dynamic speed switching) being under development and not ready yet. Work-around: Configure the host machine to "Auto-negotiation" or "1G/Full". This is (tentatively) planned to be fixed for the 2016.3 release.
Does it mean that the problem is not resolved yet in 2016.4?
Regards,
Wojtek
03-07-2017 06:44 PM
Wzab,
Thanks for the work documenting your troubleshooting of this issue. Would it be possible for you to share your latest Vivado project scripts and steps to set up your petalinux 2016.4 project?
Thanks,
Adam
05-10-2017 04:07 PM
05-19-2017 11:36 AM
I have prepared an improved version of my sources. The design has been organized in a hierarchy.
You can select the block with Ethernet interfaces and copy it to your own design, as described here.
Moving the Ethernet interfaces with PCS/PMA blocks has exposed a problem with Vivado 2016.4. Therefore, the MDIO lines are connected separately. That solves the problem of not recognized PHY.
The sources have been stripped to the minimum - so the compressed archive allowing recreating of the design occupy only 400KB.
Current setup is prepared for SFP transceivers connected via the FMC0 connector. If you want to change it, you may simply change the locations of pins in the io_ports.xdc. Theoretically you should also adjust the location of the PCS/PMA blocks, but it seems, that Vivado adjusts their location automatically.
Please note, that you may also need to modify the constant defining the state of the sfp_tx_disable lines. In on-board SFP cage on ZCU102 there is an inverting MOS-FET transistor driving the SFP pins. In the FMC board there was a direct connection.
Regards,
Wojtek
06-06-2017 08:48 AM
Hi Wojtek,
I am considering to utilize the ZCU102 to generate the Ethernet streams to other device through FMC connector / flat cable, I do not need the PHY layer , xGMII signaling only
Do you have any benchmark for multi-ports Ethernet using the ARM cores ( with Linux IP stack) inside the MPSOC ,
Thanks , Oleg
06-06-2017 02:45 PM
Dear Oleg,
It seems that your question is not related to that thread (which was about problems with recognizing of PCS/PMA PHY in ZCU102). I suppose, that you'll get better feedback if you ask a new question in the Embedded Linux forum.
The first thing that comes to my mind is just iperf, which is available in Petalinux.
Regards,
Wojtek
06-07-2017 06:02 AM
Thanks for the files!
Adam
02-17-2020 11:02 AM
Helo @wzab
I am facing silimar issues to those you describe in your post using vivado and petalinux 2019.2
I am not able to open your files because or version differences. So I wanted to know what you did to resolve the issue with:
I modified my clock source for the SFP ports as you mentioned to be pin B10 and B9.
Thanks in Advance.
root@red_board:~# unabl[ 4320.721693] macb ff0b0000.ethernet eth0: link down [ 4324.817730] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz [ 4324.825818] macb ff0b0000.ethernet eth0: link up (10/Half) ifconfig -sh: unablifconfig: command not found root@red_board:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0A:35:00:22:01 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:474 errors:474 dropped:0 overruns:0 carrier:474 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:158548 (154.8 KiB) Interrupt:30 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)