cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Scholar
Scholar
10,296 Views
Registered: ‎08-24-2011

ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Hi,

 

I have created the attached design, starting from the design provided in xilinx-zcu102-v2016.4-final1.bsp

I have added three PCS/PMA cores, and connected them to the SFP+ connectors.

des.png

After exporting the hardware from Vivado, and importing them into Petalinux, I get the attached Petalinux project.

The compiled Linux image boots correctly with:

petalinux-boot --jtag --fpga --bitstream images/linux/design_1_wrapper.bit --kernel

But unfortunately the PCS/PMA based PHYs are not recognized (below is the related part of the boot log):

[    3.322719] libphy: Fixed MDIO Bus: probed
[    3.327260] tun: Universal TUN/TAP device driver, 1.6
[    3.332284] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.338601] CAN device driver interface
[    3.343172] macb ff0b0000.ethernet: invalid hw address, using random
[    3.349527] gpiod_set_value: invalid GPIO
[    3.353917] libphy: MACB_mii_bus: probed
[    3.360544] macb ff0b0000.ethernet eth0: no PHY found
[    3.404512] macb ff0c0000.ethernet: invalid hw address, using random
[    3.410856] gpiod_set_value: invalid GPIO
[    3.415220] libphy: MACB_mii_bus: probed
[    3.421849] macb ff0c0000.ethernet eth0: no PHY found
[    3.460497] macb ff0d0000.ethernet: invalid hw address, using random
[    3.466847] gpiod_set_value: invalid GPIO
[    3.471209] libphy: MACB_mii_bus: probed
[    3.474629] ata2: SATA link down (SStatus 0 SControl 330)
[    3.474659] ata1: SATA link down (SStatus 0 SControl 330)
[    3.488703] macb ff0d0000.ethernet eth0: no PHY found
[    3.528494] gpiod_set_value: invalid GPIO
[    3.532800] libphy: MACB_mii_bus: probed
[    3.540111] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 33 (00:0a:35:00:02:90)
[    3.549975] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0e0000.etherne:0c, irq=-1)
[    3.561766] usbcore: registered new interface driver asix

In the final dts file all Ethernet controllers are described as follows:

ethernet@ff0b0000 {
			compatible = "cdns,zynqmp-gem";
			status = "okay";
			interrupt-parent = <0x2>;
			interrupts = <0x0 0x39 0x4 0x0 0x39 0x4>;
			reg = <0x0 0xff0b0000 0x0 0x1000>;
			clock-names = "pclk", "hclk", "tx_clk";
			#address-cells = <0x1>;
			#size-cells = <0x0>;
			#stream-id-cells = <0x1>;
			iommus = <0x6 0x874>;
			power-domains = <0xc>;
			clocks = <0xd 0xd 0xd>;
			phy-mode = "gmii";
			xlnx,ptp-enet-clock = <0x0>;
			linux,phandle = <0x1f>;
			phandle = <0x1f>;
		};

		ethernet@ff0c0000 {
			compatible = "cdns,zynqmp-gem";
			status = "okay";
			interrupt-parent = <0x2>;
			interrupts = <0x0 0x3b 0x4 0x0 0x3b 0x4>;
			reg = <0x0 0xff0c0000 0x0 0x1000>;
			clock-names = "pclk", "hclk", "tx_clk";
			#address-cells = <0x1>;
			#size-cells = <0x0>;
			#stream-id-cells = <0x1>;
			iommus = <0x6 0x875>;
			power-domains = <0xe>;
			clocks = <0xd 0xd 0xd>;
			phy-mode = "gmii";
			xlnx,ptp-enet-clock = <0x0>;
			linux,phandle = <0x20>;
			phandle = <0x20>;
		};

		ethernet@ff0d0000 {
			compatible = "cdns,zynqmp-gem";
			status = "okay";
			interrupt-parent = <0x2>;
			interrupts = <0x0 0x3d 0x4 0x0 0x3d 0x4>;
			reg = <0x0 0xff0d0000 0x0 0x1000>;
			clock-names = "pclk", "hclk", "tx_clk";
			#address-cells = <0x1>;
			#size-cells = <0x0>;
			#stream-id-cells = <0x1>;
			iommus = <0x6 0x876>;
			power-domains = <0xf>;
			clocks = <0xd 0xd 0xd>;
			phy-mode = "gmii";
			xlnx,ptp-enet-clock = <0x0>;
			linux,phandle = <0x21>;
			phandle = <0x21>;
		};

		ethernet@ff0e0000 {
			compatible = "cdns,zynqmp-gem";
			status = "okay";
			interrupt-parent = <0x2>;
			interrupts = <0x0 0x3f 0x4 0x0 0x3f 0x4>;
			reg = <0x0 0xff0e0000 0x0 0x1000>;
			clock-names = "pclk", "hclk", "tx_clk";
			#address-cells = <0x1>;
			#size-cells = <0x0>;
			#stream-id-cells = <0x1>;
			iommus = <0x6 0x877>;
			power-domains = <0x10>;
			clocks = <0xd 0xd 0xd>;
			phy-mode = "rgmii-id";
			xlnx,ptp-enet-clock = <0x0>;
			local-mac-address = [00 0a 35 00 02 90];
			phy-handle = <0x11>;
			linux,phandle = <0x22>;
			phandle = <0x22>;

			phy@c {
				reg = <0xc>;
				ti,rx-internal-delay = <0x8>;
				ti,tx-internal-delay = <0xa>;
				ti,fifo-depth = <0x1>;
				linux,phandle = <0x11>;
				phandle = <0x11>;
			};
		};

Only the third one, that uses the external PHY is operational.

Should I add the phy definitions for the PCS/PMA connected controllers myself?

 

TIA & Regards,

Wojtek

PS. The attached files should be renamed to des_plnx.tar.xz and xilinx-zcu102-2016.4_xpr_zip, respectively.

I was not able to post them with original names (they were rejected with complaint, that extension does not match the contents).

Tags (1)
0 Kudos
1 Solution

Accepted Solutions
Highlighted
Scholar
Scholar
11,476 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

I have prepared an improved version of my sources. The design has been organized in a hierarchy.

You can select the block with Ethernet interfaces and copy it to your own design, as described here.

Moving the Ethernet interfaces with PCS/PMA blocks has exposed a problem with Vivado 2016.4. Therefore, the MDIO lines are connected separately. That solves the problem of not recognized PHY.

The sources have been stripped to the minimum - so the compressed archive allowing recreating of the design occupy only 400KB.

Current setup is prepared for SFP transceivers connected via the FMC0 connector. If you want to change it, you may simply change the locations of pins in the io_ports.xdc. Theoretically you should also adjust the location of the PCS/PMA blocks, but it seems, that Vivado adjusts their location automatically.

Please note, that you may also need to modify the constant defining the state of the sfp_tx_disable lines. In on-board SFP cage on ZCU102 there is an inverting MOS-FET transistor driving the SFP pins. In the FMC board there was a direct connection.

 

Regards,

Wojtek

 

View solution in original post

18 Replies
Highlighted
Scholar
Scholar
10,250 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Using examples posted by other users, I have added the dtsi file defining the phys:

&gem0 {
        local-mac-address = [00 0a 35 00 00 00];
	phy-handle = <&phy10>;
	phy10: phy@6 {
		compatible = "Xilinx PCS/PMA PHY";
		device_type = "ethernet-phy";
		reg = <0x6>;
	};
};
&gem1 {
        local-mac-address = [00 0a 35 00 00 01];
	phy-handle = <&phy11>;
	phy11: phy@6 {
		compatible = "Xilinx PCS/PMA PHY";
		device_type = "ethernet-phy";
		reg = <0x6>;
	};
};
&gem2 {
        local-mac-address = [00 0a 35 00 00 02];
	phy-handle = <&phy12>;
	phy12: phy@6 {
		compatible = "Xilinx PCS/PMA PHY";
		device_type = "ethernet-phy";
		reg = <0x6>;
	};
};

Now, when booting the system, I get the following:

 

[    3.288509] libphy: Fixed MDIO Bus: probed                                   
[    3.293046] tun: Universal TUN/TAP device driver, 1.6                        
[    3.298077] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>           
[    3.304392] CAN device driver interface                                      
[    3.308955] gpiod_set_value: invalid GPIO                                    
[    3.313304] libphy: MACB_mii_bus: probed                                     
[    3.317202] mdio_bus ff0b0000.etherne:06: mdio_device_register               
[    3.325876] macb ff0b0000.ethernet eth0: no PHY found                        
[    3.374538] gpiod_set_value: invalid GPIO                                    
[    3.378855] libphy: MACB_mii_bus: probed                                     
[    3.382757] mdio_bus ff0c0000.etherne:06: mdio_device_register               
[    3.391409] macb ff0c0000.ethernet eth0: no PHY found                        
[    3.430522] gpiod_set_value: invalid GPIO                                    
[    3.434837] libphy: MACB_mii_bus: probed                                     
[    3.438737] mdio_bus ff0d0000.etherne:06: mdio_device_register               
[    3.440654] ata1: SATA link down (SStatus 0 SControl 330)                    
[    3.440690] ata2: SATA link down (SStatus 0 SControl 330)                    
[    3.458256] macb ff0d0000.ethernet eth0: no PHY found                        
[    3.498521] gpiod_set_value: invalid GPIO                                    
[    3.502831] libphy: MACB_mii_bus: probed                                     
[    3.510143] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0)
[    3.520012] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867])

It seems, that the essential problem is that the "macb" is not informed properly about the EMIO lines used to connect the PCS/PMA internal PHY (the "gpiod_set_value: invalid GPIO" error message).

How can I pass that info to the macb driver?

Should I somehow enable the EMIO GPIO support in the kernel?

 

TIA & Regards,

Wojtek

0 Kudos
Highlighted
Scholar
Scholar
10,246 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

It's a funny thing, that the similar problem is displayed in the "Expected output" on one of official Xilinx sites:

http://www.wiki.xilinx.com/Linux+I2C+Driver

 

Creating 4 MTD partitions on "spi0.0":
0x000000000000-0x000000500000 : "boot"
0x000000500000-0x000000520000 : "bootenv"
0x000000520000-0x000000fa0000 : "kernel"
0x000000fa0000-0x000001000000 : "spare"
CAN device driver interface
gpiod_set_value: invalid GPIO
libphy: MACB_mii_bus: probed
mdio_bus e000b000.etherne:07: mdio_device_register
macb e000b000.ethernet eth0: no PHY found
 

Regards,

Wojtek

 

 

0 Kudos
Highlighted
Scholar
Scholar
10,226 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Is it possible, that my problem is related to the old https://www.xilinx.com/support/answers/67930.html problem?

Is it still valid in Vivado 2016.4?

 

Regards,

Wojtek

0 Kudos
Highlighted
Scholar
Scholar
10,178 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

The problem with not detected PHY was caused by the fact, that I haven't configured the GMT clock correctly

(using the controller accessible via /dev/ttyUSB3).

After correct configuration of the clock, the PHY is detected, the link goes up but with incorrect speed (10/Full):

PMUFW: PmMmioWrite: (NODE_APU) addr=0xFF180318, mask=0x3F0000, value=0x50000
PMUFW: PmMmioWrite: (NODE_APU) addr=0xFF180358, mask=0x40000, value=0x0
[    4.760476] mmc0: new high speed SDHC card at address 0007
[    4.770840] mmcblk0: mmc0:0007 SD04G 3.71 GiB 
[    4.777156]  mmcblk0: p1
[    5.380098] EXT4-fs (mmcblk0p1): recovery complete
[    5.384881] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[    5.650588] random: dd urandom read with 7 bits of entropy available
[    5.747168] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    7.352940] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[    7.361058] macb ff0b0000.ethernet eth0: link up (10/Full)
[    7.366575] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0

plnx_aarch64 login: root
Password: 
root@plnx_aarch64:~# ifconfig eth1 up
[   31.579411] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
root@plnx_aarch64:~# ifconfig eth1 [   33.456940] macb ff0c0000.ethernet eth1: unable to generate target frequency: 2500000 Hz
[   33.465056] macb ff0c0000.ethernet eth1: link up (10/Full)
[   33.470575] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
ifconfig eth2 up                                                                                                                            
[   38.623307] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
root@plnx_aarch64:~# [   40.560937] macb ff0d0000.ethernet eth2: unable to generate target frequency: 2500000 Hz
[   40.569053] macb ff0d0000.ethernet eth2: link up (10/Full)
[   40.574573] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready

However I can't receive anything:

root@plnx_aarch64:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 72:06:FC:B4:4B:30  
          inet6 addr: fe80::7006:fcff:feb4:4b30%4890480/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:46 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:13644 (13.3 KiB)
          Interrupt:30 

eth1      Link encap:Ethernet  HWaddr 6A:39:5B:1D:1E:52  
          inet6 addr: fe80::6839:5bff:fe1d:1e52%4890480/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)
          Interrupt:31 

eth2      Link encap:Ethernet  HWaddr 26:59:A8:57:FC:F6  
          inet6 addr: fe80::2459:a8ff:fe57:fcf6%4890480/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:648 (648.0 B)
          Interrupt:32 

I have played with different settings regarding the 1000BaseX or SGMII mode, turning on and off the "SGMII PHY mode", but nothing helps.

 

I attach the current version of the hardware project with VEXTPROJ scripts, that allow to rebuild the project from a minimal set of files.

 

Regards,

Wojtek

 

0 Kudos
Highlighted
Scholar
Scholar
10,122 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Xilinx published the new XAPP1305 note mentioned here: http://www.wiki.xilinx.com/PS+and+PL+based+Ethernet+in+Zynq+MPSoC

Unfortunately, the design files are not available yet: https://forums.xilinx.com/t5/UltraScale-Architecture/Where-are-design-files-for-XAPP-1305/td-p/745318

 

I'm trying to anayze the Zynq Ultrascale+ TRM in hope that I'll find the solution myself.

It is still unclear how to connect the GMII_ENETx ports in PS with GMII_GEM_PCS_PMA ports, as they contain different pins (in PS there are additional rx_clk, tx_clk, crs,col  and speed_mode lines). Splitting the ports and connecting lines by hand make design illegible and difficult to maintain. Below is just a part of the design in it's last form (I also attach the VEXTPROJ version of the design):

ttt.png

 

Now I have added the GPIO cores to monitor the PCS/PMA status and to reset PCS/PMA.

Unfortunately I still get the PCS/PMA status 0x806 after booting the system.

When I reset the PCS/PMA, the status starts to oscillate between: 0x818, 0x810, 0x850, and 0x870.

I have checked different settings of PCS/PMA - SGMII, 1000BaseX, with and without autonegotiation and with and without "SGMII PHY mode".

I use GLC-T compatible transceivers:

IMG_20170209_111842.jpg

 

 

Regards,

Wojtek

0 Kudos
Highlighted
Scholar
Scholar
10,104 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

I have attempted to create a version based on the firmware MACs, but unfortunately my license does not allow to test it in hardware. I attach this version for those, who have access to the full TEMAC license.

 

Regards,

Wojtek

0 Kudos
Highlighted
Scholar
Scholar
10,076 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

I have prepared a special FPGA firmware equipped with virtual GPIOs for diagnostic and control functions.

They can be controlled with devmem utility.

  • 0x80000000 (RO) - status of the PCS/PMA connected to the ENET0
  • 0x80010000 (RO) - status of the PCS/PMA connected to the ENET1
  • 0x80020000 (RO) - status of the PCS/PMA connected to the ENET2
  • 0x80030000 (WO) - control register
    • bit 0 (0x1) - reset of PCS/PMA connected to the ENET0
    • bit 1 (0x2) - reset of PCS/PMA connected to the ENET1
    • bit 2 (0x4) - reset of PCS/PMA connected to the ENET2
    • bit 3 (0x8) - configuration_valid connected to all PCS/PMAs
    • bit 4 (0x10) - an_adv_config_val connected to all PCS/PMAs
    • bit 5 (0x20) - an_restart_config connected to all PCS/PMAs
    • bit 6 (0x60) - basex_or_sgmii connected to all PCS/PMAs
  • 0x80040000 (WO) - an_adv_config_vector connected to all PCS/PMAs
  • 0x80050000 (WO) - configuration_vector connected to all PCS/PMAs

To work with that firmware, it is necessary, after power up of the ZUC102 to configure the Si570 MGT clock to 125MHz (using the controller accessible via the 3rd UART channel - /dev/ttyUSB3 in my development system).

 

If I compile the Petalinux with macb and xilinx-phy compiled into the kernel, I get the following results:

[    5.722125] random: dd urandom read with 8 bits of entropy available
[    5.818468] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    7.416931] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[    7.425053] macb ff0b0000.ethernet eth0: link up (10/Full)
[    7.430570] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0

plnx_aarch64 login: root
Password: 
root@plnx_aarch64:~# devmem 0x80000000
0x00000806
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000806
root@plnx_aarch64:~# devmem 0x80020000                                                               
0x00000806
root@plnx_aarch64:~# devmem 0x80040000 32 0xc9a01                                                    
root@plnx_aarch64:~# devmem 0x80050000 32 0x11                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x47                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x40
root@plnx_aarch64:~# devmem 0x80030000 32 0x47[  400.428880] macb ff0b0000.ethernet eth0: link down
root@plnx_aarch64:~# devmem 0x80030000 32 0x47
root@plnx_aarch64:~# [  426.432933] macb ff0b0000.ethernet eth0: unable to generate target frequency 2500000 Hz
[  426.441057] macb ff0b0000.ethernet eth0: link up (10/Full)
devmem 0x80030000 32 0x40
root@plnx_aarch64:~# [  440.444881] macb ff0b0000.ethernet eth0: link down

root@plnx_aarch64:~# devmem 0x80030000 32 0x58
root@plnx_aarch64:~# devmem 0x80030000 32 0x70
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80000000
0x00000870
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000850
root@plnx_aarch64:~# devmem 0x8000[  513.982585] random: nonblocking pool is initialized
2000                                                                                                 
0x00000810
root@plnx_aarch64:~# devmem 0x80020000
0x00000810
root@plnx_aarch64:~# devmem 0x80020000
0x00000850
root@plnx_aarch64:~# 
root@plnx_aarch64:~# devmem 0x80030000 32 0x7                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x7[  818.448937] macb ff0b0000.ethernet eth0: unable to gz
[  818.457051] macb ff0b0000.ethernet eth0: link up (10/Full)
00000^C
root@plnx_aarch64:~# devmem 0x80030000 32 0x0                                                        
root@plnx_aarch64:~# devmem 0x80030000 32 0x7[  835.460883] macb ff0b0000.ethernet eth0: link down
10000^C
root@plnx_aarch64:~# devmem 0x80030000 32 0x18                                                      
root@plnx_aarch64:~# devmem 0x80030000 32 0x20
root@plnx_aarch64:~# devmem 0x80030000 32 0x0
root@plnx_aarch64:~# devmem 0x80000000
0x00000830
root@plnx_aarch64:~# devmem 0x80000000
0x00000870
root@plnx_aarch64:~# devmem 0x80000000
0x00000870
root@plnx_aarch64:~# devmem 0x80020000                                                               
0x00000810
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000870

The interesting thing is, that the link "goes up", when PCS/PMA is being reset, and goes down after I release the reset. However the speed of the link is detected as 10Mb/s.

After the reset, the statuses of PCS/PMAs starts to fluctuate between 0x810, 0x818, 0x830, 0x850 and 0x870.

 

When I compile the Petalinux with macb and xilinx-phy as modules, I get other results:

[    5.378323] random: dd urandom read with 8 bits of entropy available

PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0

plnx_aarch64 login: root
Password: 
root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000                                                               
0x0000D80B
root@plnx_aarch64:~# modprobe xilinx-phy
root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000
0x0000D80B
root@plnx_aarch64:~# modprobe macb
[   64.063976] macb ff0b0000.ethernet: invalid hw address, using random
[   64.070361] gpiod_set_value: invalid GPIO
[   64.074936] libphy: MACB_mii_bus: probed
[   64.081581] macb ff0b0000.ethernet eth0: no PHY found
[   64.121180] macb ff0c0000.ethernet: invalid hw address, using random
[   64.127552] gpiod_set_value: invalid GPIO
[   64.132105] libphy: MACB_mii_bus: probed
[   64.138773] macb ff0c0000.ethernet eth0: no PHY found
[   64.181171] macb ff0d0000.ethernet: invalid hw address, using random
[   64.187565] gpiod_set_value: invalid GPIO
[   64.192129] libphy: MACB_mii_bus: probed
[   64.198802] macb ff0d0000.ethernet eth0: no PHY found
[   64.249176] gpiod_set_value: invalid GPIO
[   64.253762] libphy: MACB_mii_bus: probed
[   64.261022] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 33 (00:0a:3)
[   64.270899] TI DP83867 ff0e0000.etherne:0c: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff)
root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000
0x0000D80B
root@plnx_aarch64:~# 

So now, the status of the links is correct, but when I load the driver, it is not able to find the PHY.

Even in that case, the reset of {PCS/PMA results in unstable operation of the link (why?)

root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80030000 32 7
root@plnx_aarch64:~# devmem 0x80000000     
0x0000D80B
root@plnx_aarch64:~# devmem 0x80030000 32 0                                                          
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000870
root@plnx_aarch64:~# devmem 0x80020000                                                               
0x00000810
root@plnx_aarch64:~# devmem 0x80040000 32 0xc9a1                                                     
root@plnx_aarch64:~# devmem 0x80050000 32 0x10                                                   
root@plnx_aarch64:~# devmem 0x80030000 32 0x18                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x20
root@plnx_aarch64:~# devmem 0x800q0000
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80000000
0x00000850
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000850
root@plnx_aarch64:~# devmem 0x80020000                                                               
0x00000810
root@plnx_aarch64:~# devmem 0x80030000 32 0x07                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x0 
root@plnx_aarch64:~# devmem 0x80030000 32 0x20                                                       
root@plnx_aarch64:~# devmem 0x80030000 32 0x0
root@plnx_aarch64:~# devmem 0x80000000
0x00000830
root@plnx_aarch64:~# devmem 0x80010000                                                               
0x00000870
root@plnx_aarch64:~# 

Why the PCS/PMA behaves so strange, and how can I combine its correct operation with working macb+xilinx-phy drivers?

 

Regards,

Wojtek

 

PS. I attach the sources of the diagnostic firmware in VEXTPROJ format to keep its size as small as possible.

0 Kudos
Scholar
Scholar
10,023 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

It appeared, that during the debugging I have done incorrect adjustment of the design.

I have moved the MGT reference clock from B10 (ZCU102 Si5328 MGT Clock) to C8 (ZCU102 Si570 MGT Clock).

It appears, that the clock at C8 gets reconfigured to 156 MHz when Linux is booting.

So I have changed back the corresponding line in io_ports.xdc:

 

set_property PACKAGE_PIN B10 [get_ports diff_clock_rtl_clk_p]

Of course after that change, you have to set the ZCU102 Si5328 MGT  Clock Frequency to 125MHz after the ZCU102 power-up.

After that, the PCS/PMA status is correct even after booting with macb and xilinx-phy drivers compiled into the kernel, but the driver still thinks, that the link works at 10Mb/s (even though the PCS/PMA status shows 1Gb/s!)

The PCS/PMA also responds correctly to the reset:

 

 

    5.754872] random: dd urandom read with 7 bits of entropy available
[    5.851401] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    9.498048] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[    9.506170] macb ff0b0000.ethernet eth0: link up (10/Half)
[    9.511687] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
root@plnx_aarch64:~# devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000                                                                                   
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000                                                                                   
0x0000D80B
root@plnx_aarch64:~# devmem 0x80030000 32 7                                                                              
root@plnx_aarch64:~# devmem 0x80000000
0x00000800
root@plnx_aarch64:~# devmem 0x80010000
0x00000800
root@plnx_aarch64:~# devmem 0x80020000
0x00000800
root@plnx_aarch64:~# devmem 0x80030000 32 0
root@plnx_aarch64:~# [  728.557998] macb ff0b0000.ethernet eth0: link down
[  730.562038] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[  730.570144] macb ff0b0000.ethernet eth0: link up (10/Half)
devmem 0x80000000
0x0000D80B
root@plnx_aarch64:~# devmem 0x80010000                                                                                   
0x0000D80B
root@plnx_aarch64:~# devmem 0x80020000                                                                                   
0x0000D80B

So now the only problem still remaining is the communication with the kernel driver...

 

0 Kudos
Highlighted
Scholar
Scholar
9,992 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

I have got my links working at 1000Mb/s both in BASEX and SGMII mode, when I have switched off the autonegotiation, and manually set the parameters with ethtool:

[    5.723371] random: dd urandom read with 7 bits of entropy available
[    5.819537] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready

PetaLinux 2016.4 plnx_aarch64 /dev/ttyPS0

plnx_aarch64 login: root
Password: 
root@plnx_aarch64:~# ethtool eth0 speed 1000 duplex full autoneg off
ethtool: bad command line argument(s)
For more information run ethtool -h
root@plnx_aarch64:~# ethtool -s eth0 speed 1000 duplex full autoneg off                                                                
root@plnx_aarch64:~# [   57.417137] macb ff0b0000.ethernet eth0: link up (1000/Full)
[   57.422813] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

root@plnx_aarch64:~# udhcpc
udhcpc (v1.24.1) started
Sending discover...
Sending select for 192.168.95.190...
Lease of 192.168.95.190 obtained, lease time 39600
/etc/udhcpc.d/50default: Adding DNS 192.168.95.253
/etc/udhcpc.d/50default: Adding DNS 194.29.161.2
/etc/udhcpc.d/50default: Adding DNS 194.29.160.10

 However I still can't get working the link with 100Mb/s device:

root@plnx_aarch64:~# ethtool -s eth2 speed 100 duplex full autoneg off                                                                 
root@plnx_aarch64:~# [  130.825144] macb ff0d0000.ethernet eth2: unable to generate target frequency: 25000000 Hz
[  130.833355] macb ff0d0000.ethernet eth2: link up (100/Full)

root@plnx_aarch64:~# ifconfig eth2 192.168.0.1 promisc up
[  160.939951] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
[  160.945867] device eth2 entered promiscuous mode
root@plnx_aarch64:~# ethtool -s eth2 speed 100 duplex full autoneg off
root@plnx_aarch64:~# [  172.837155] macb ff0d0000.ethernet eth2: unable to generate target frequency: 25000000 Hz
[  172.845360] macb ff0d0000.ethernet eth2: link up (100/Full)
[  172.850969] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
[  232.999321] random: nonblocking pool is initialized

Update

 Looking at [1] and [2], I can state that the problem with "unable to generate target frequency: 25000000 Hz" is related to clocks available for the Linux kernel. So either those clocks are not configured correctly, or the kernel is not informed correctly about the available clocks...

0 Kudos
Highlighted
Scholar
Scholar
7,782 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Trying to understand, why the macb driver has problems with generation of 25000000 Hz clock, I have discovered that it is unclear why it is needed. Both clocks (gmii_rx_clk and gmii_tx_clk) are delivered from PCS/PMA to GMII_ENET port.

So, why the macb must to generate independent clocks?

I have also found the following message. Maybe it explains, why I had problems with autonegotiation:

>> To go to 100Mbps mode:
>>
>> ethtool -s eth0 advertise 0x0f
>>
>> Back to gigabit:
>>
>> ethtool -s eth0 advertise 0x3f
>>
>>
>> On the mpsoc this is broken too, but I guess that's just work in progress...
>>
>> root at zcu102-zynqmp:~# ethtool -s eth0 advertise 0x0f
>> macb ff0e0000.ethernet eth0: link down
>> macb ff0e0000.ethernet eth0: unable to generate target frequency: 25000000 Hz

If you are using the in kernel device trees for zcu102,.they currently
have fixed 125MHz clocks for the gems.
0 Kudos
Highlighted
Scholar
Scholar
7,775 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

I have found yet another AR that may explain the problems reported in that thread:

In PetaLinux 2015.4, 2016.1 and 2016.2, dynamic speed switching is not possible. Only the default speed of 1G is supported.

This is NOT a silicon or board limitation but a Linux infrastructure (to enable dynamic speed switching) being under development and not ready yet.

Work-around:

Configure the host machine to "Auto-negotiation" or "1G/Full".

This is (tentatively) planned to be fixed for the 2016.3 release.

Does it mean that the problem is not resolved yet in 2016.4?

Regards,

Wojtek

0 Kudos
Highlighted
Adventurer
Adventurer
7,514 Views
Registered: ‎01-26-2017

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Wzab,

 

Thanks for the work documenting your troubleshooting of this issue. Would it be possible for you to share your latest Vivado project scripts and steps to set up your petalinux 2016.4 project?

 

Thanks,

Adam

0 Kudos
Highlighted
Scholar
Scholar
6,840 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Ethernets handled via PCS/PMA and SFP - PHY is not recognized

Jump to solution

Attached is the last version of my demo projects.

To build it, you should customize paths to Vivado and Petalinux in the petalinux/multi_eth1/buduj.sh file.

Then you should simply run this script.

buduj2.sh just skips compilation of FPGA.

 

Regards,

Wojtek

0 Kudos
Highlighted
Scholar
Scholar
11,477 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

I have prepared an improved version of my sources. The design has been organized in a hierarchy.

You can select the block with Ethernet interfaces and copy it to your own design, as described here.

Moving the Ethernet interfaces with PCS/PMA blocks has exposed a problem with Vivado 2016.4. Therefore, the MDIO lines are connected separately. That solves the problem of not recognized PHY.

The sources have been stripped to the minimum - so the compressed archive allowing recreating of the design occupy only 400KB.

Current setup is prepared for SFP transceivers connected via the FMC0 connector. If you want to change it, you may simply change the locations of pins in the io_ports.xdc. Theoretically you should also adjust the location of the PCS/PMA blocks, but it seems, that Vivado adjusts their location automatically.

Please note, that you may also need to modify the constant defining the state of the sfp_tx_disable lines. In on-board SFP cage on ZCU102 there is an inverting MOS-FET transistor driving the SFP pins. In the FMC board there was a direct connection.

 

Regards,

Wojtek

 

View solution in original post

Highlighted
Visitor
Visitor
6,467 Views
Registered: ‎06-05-2017

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

Hi Wojtek, 

I am considering to utilize the ZCU102 to generate the Ethernet streams to other device through FMC connector / flat cable, I do not need the PHY layer , xGMII signaling only 

Do you have any benchmark for multi-ports Ethernet using the ARM cores ( with Linux  IP stack)  inside the MPSOC , 

Thanks , Oleg 

Tags (1)
0 Kudos
Highlighted
Scholar
Scholar
6,454 Views
Registered: ‎08-24-2011

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

Dear Oleg,

It seems that your question is not related to that thread (which was about problems with recognizing of PCS/PMA PHY in ZCU102). I suppose, that you'll get better feedback if you ask a new question in the Embedded Linux forum.

The first thing that comes to my mind is just iperf, which is available in Petalinux.

 

Regards,

Wojtek

0 Kudos
Highlighted
Adventurer
Adventurer
6,435 Views
Registered: ‎01-26-2017

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

Thanks for the files!

 

Adam

0 Kudos
Highlighted
Contributor
Contributor
762 Views
Registered: ‎12-04-2019

Re: ZCU102 Petalinux 2016.4 - Multiple Ethernets. Problem with not recognized PHY solved!

Jump to solution

Helo @wzab 

I am facing silimar issues to those you describe in your post using vivado and petalinux 2019.2 

I am not able to open your files because or version differences. So I wanted to know what you did to resolve the issue with:

I modified my clock source for the SFP ports as you mentioned to be pin B10 and B9. 

Thanks in Advance. 

root@red_board:~# unabl[ 4320.721693] macb ff0b0000.ethernet eth0: link down
[ 4324.817730] macb ff0b0000.ethernet eth0: unable to generate target frequency: 2500000 Hz
[ 4324.825818] macb ff0b0000.ethernet eth0: link up (10/Half)
ifconfig
-sh: unablifconfig: command not found
root@red_board:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0A:35:00:22:01
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:474 errors:474 dropped:0 overruns:0 carrier:474
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:158548 (154.8 KiB)
          Interrupt:30

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
0 Kudos