cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
badFITimage
Explorer
Explorer
937 Views
Registered: ‎09-06-2019

No SGMII link when booted in Linux

Hi,

We're experiencing issues through the SGMII interface of our ZUS+ based system. The system uses all 4 GEMs comprised of 2 SGMII (GEM0 and GEM1) along with 2 RGMII (GEM2 and GEM3). Additionally, the system utilizes a shared MDIO architecture controlled by GEM2. We've proven in u-boot that the physical design of the SGMII is valid. I generated a new FSBL where GEM0 (SGMII) controlled the MDIO and was able to ping successfully (though when booting to the kernel I was not able to get the SGMII up correctly).

Do to the nature of the MDIO scheme we only enable GEM2 in the device tree since GEM0 and GEM1 probe prior to GEM2 if enabled causing issues with PHY registration. At this point we do not believe it is a hardware issue. The PHYs are being correctly ID'd in U-boot as well as by Linux.

From the Macb Driver Confluence page:

 

Common MDIO DT
To use multiple GEM→PHY connections using a common MDIO bus, please use the following devicetree convention:

gem0 {
......
    phy-handle = <&phya>;
 
    phya {
        reg = <0xa>;
    };
 
    phyb {
        reg = <0xb>;
    };
};
 
gem1 {
.....
    phy-handle = <&phyb>;
};
Where:

→ gem0 is the instance whose phy management is being used (and whose MDC and MDI lines are connected to both PHYs)

→ gem0 is communicating via phya and gem1 is communicating via phyb

Note that gem0 needs to come up before gem1 and stay up (because the MDIO interface is expected to be up first; otherwise, the dependent MAC-PHY link (gem1-phyb) will come up on next ifconfig up/down).

As a result of this gem0's runtime PM will not be effective if gem1 is still active in this configuration.

 

Device Tree:

 

&gem3 {
	status = "disabled";
	phy-handle = <&phy1>;
	phy-mode = "rgmii";
	pinctrl-names = "default";
	local-mac-address = [00 0a 35 00 00 03];
	ti,dp83867-rxctrl-strap-quirk;
};

&gem2 {
	status = "okay";
	phy-handle = <&phy0>;
	phy-mode = "rgmii";
	pinctrl-names = "default";
	local-mac-address = [00 0a 35 00 00 02];
	ti,dp83867-rxctrl-strap-quirk;
	phy0: phy@0 {
		reg = <0x0>;
		ti,rx-internal-delay = <0x8>;
		ti,tx-internal-delay = <0xa>;
		ti,fifo-depth = <0x1>;
		ti,rxctrl-strap-worka;
		reset-gpios = <&gpio 6 0>;
	};
	phy1: phy@1 {
		reg = <0x1>;
		ti,rx-internal-delay = <0x8>;
		ti,tx-internal-delay = <0xa>;
		ti,fifo-depth = <0x1>;
		ti,rxctrl-strap-worka;
	};
	phy2: phy@2 {
		reg = <0x2>;
		ti,rx-internal-delay = <0x8>;
		ti,tx-internal-delay = <0xa>;
		ti,fifo-depth = <0x1>;
		ti,dp83867-rxctrl-strap-quirk;
        phy-mode = "sgmii";
        xlnx,phy-type = <0x4>;
        reset-gpios = <&gpio 45 0>;
	};
	phy3: phy@3 {
		reg = <0x3>;	
		ti,rx-internal-delay = <0x8>;
		ti,tx-internal-delay = <0xa>;
		ti,fifo-depth = <0x1>;
		ti,dp83867-rxctrl-strap-quirk;
        phy-mode = "sgmii";
        xlnx,phy-type = <0x4>;
        reset-gpios = <&gpio 13 0>;
	};		
};

&gem1 {
	status = "disabled";
	phy-handle = <&phy3>;
	phy-mode = "sgmii";
	pinctrl-names = "default";
	local-mac-address = [00 0a 35 00 00 01];
	ti,rxctrl-strap-worka;  
	ti,dp83867-rxctrl-strap-quirk;
	//is-internal-pcspma;  
	//reset-gpios = <&gpio 45 0>;
};

&gem0 {
	status = "disabled";
	phy-handle = <&phy2>;
	phy-mode = "sgmii";	
	pinctrl-names = "default";
	local-mac-address = [00 0a 35 00 00 00];
	ti,rxctrl-strap-worka;  
	ti,dp83867-rxctrl-strap-quirk;
	//is-internal-pcspma;  
	//reset-gpios = <&gpio 13 0>;
};

 

U-Boot:

ZynqMP> mii info
PHY 0x00: OUI = 0x80028, Model = 0x23, Rev = 0x01, 1000baseT, FDX
PHY 0x01: OUI = 0x80028, Model = 0x23, Rev = 0x01, 1000baseT, FDX
PHY 0x02: OUI = 0x80028, Model = 0x23, Rev = 0x01,  10baseT, FDX
PHY 0x03: OUI = 0x80028, Model = 0x23, Rev = 0x01, 1000baseT, FDX

Linux dmesg (GEM2 only due to the MDIO configuration and probe order requirements as mentioned before):

 

[    5.019660] macb ff0d0000.ethernet: Not enabling partial store and forward
[    5.027052] libphy: MACB_mii_bus: probed
[    5.030970] mdio_bus ff0d0000.ethernet-ffffffff: Loop over the child nodes and register a phy_device for each phy
[    5.041224] mdio_bus ff0d0000.ethernet-ffffffff: address found. scan phy phy at address 0
[    5.049401] mdio_bus ff0d0000.ethernet-ffffffff: This child node is a phy node of mdio
[    5.060493] mdio_bus ff0d0000.ethernet-ffffffff: address found. scan phy phy at address 1
[    5.068677] mdio_bus ff0d0000.ethernet-ffffffff: This child node is a phy node of mdio
[    5.084472] mdio_bus ff0d0000.ethernet-ffffffff: address found. scan phy phy at address 2
[    5.092643] mdio_bus ff0d0000.ethernet-ffffffff: This child node is a phy node of mdio
[    5.103497] mdio_bus ff0d0000.ethernet-ffffffff: address found. scan phy phy at address 3
[    5.111675] mdio_bus ff0d0000.ethernet-ffffffff: This child node is a phy node of mdio
[    5.128066] TI DP83867 ff0d0000.ethernet-ffffffff:00: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0d0000.ethernet-ffffffff:00, irq=POLL)
[    5.141106] macb ff0d0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0d0000 irq 32 (00:0a:35:00:00:02)

 

Appling DTS Overlay:

> mkdir /configfs
> mount -t configfs configfs /configfs
> mkdir /configfs/device-tree/overlays/full
> echo -n "gem-enable-overlay.dtbo" > /configfs/device-tree/overlays/full/path

[  106.623414] macb ff0d0000.ethernet eth0: link up (1000/Full)
[  106.623434] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  165.977571] macb ff0e0000.ethernet: Not enabling partial store and forward
[  165.978100] libphy: MACB_mii_bus: probed
[  165.978105] mdio_bus ff0e0000.ethernet-ffffffff: Loop over the child nodes and register a phy_device for each phy
[  166.072241] TI DP83867 ff0d0000.ethernet-ffffffff:01: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0d0000.ethernet-ffffffff:01, irq=POLL)
[  166.072254] macb ff0e0000.ethernet eth1: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 72 (00:0a:35:00:00:03)
[  166.072832] macb ff0b0000.ethernet: Not enabling partial store and forward
[  166.073385] libphy: MACB_mii_bus: probed
[  166.073390] mdio_bus ff0b0000.ethernet-ffffffff: Loop over the child nodes and register a phy_device for each phy
[  166.073943] [drm] load() is defered & will be called again
[  166.168164] TI DP83867 ff0d0000.ethernet-ffffffff:02: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0d0000.ethernet-ffffffff:02, irq=POLL)
[  166.168176] macb ff0b0000.ethernet eth2: Cadence GEM rev 0x50070106 at 0xff0b0000 irq 73 (00:0a:35:00:00:02)
[  166.168752] macb ff0c0000.ethernet: Not enabling partial store and forward
[  166.169286] libphy: MACB_mii_bus: probed
[  166.169291] mdio_bus ff0c0000.ethernet-ffffffff: Loop over the child nodes and register a phy_device for each phy
[  166.169842] [drm] load() is defered & will be called again
[  166.264389] TI DP83867 ff0d0000.ethernet-ffffffff:03: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0d0000.ethernet-ffffffff:03, irq=POLL)
[  166.264401] macb ff0c0000.ethernet eth3: Cadence GEM rev 0x50070106 at 0xff0c0000 irq 74 (00:0a:35:00:00:01)
[  166.265054] [drm] load() is defered & will be called again

ifconfig:

> ifconfig eth0 192.168.4.200 up
> ifconfig eth1 192.168.4.201 up
> ifconfig eth2 192.168.4.202 up
> ifconfig eth3 192.168.4.203 up
> ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0A:35:00:00:02
          inet addr:192.168.4.200  Bcast:192.168.4.255  Mask:255.255.255.0
          inet6 addr: fe80::20a:35ff:fe00:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:546 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2587 (2.5 KiB)  TX bytes:42524 (41.5 KiB)
          Interrupt:32

eth1      Link encap:Ethernet  HWaddr 00:0A:35:00:00:03
          inet addr:192.168.4.201  Bcast:192.168.4.255  Mask:255.255.255.0
          inet6 addr: fe80::20a:35ff:fe00:3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:446 (446.0 B)
          Interrupt:72

eth2      Link encap:Ethernet  HWaddr 00:0A:35:00:00:02
          inet addr:192.168.4.202  Bcast:192.168.4.255  Mask:255.255.255.0
          inet6 addr: fe80::20a:35ff:fe00:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:540 (540.0 B)
          Interrupt:73

eth3      Link encap:Ethernet  HWaddr 00:0A:35:00:00:01
          inet addr:192.168.4.203  Bcast:192.168.4.255  Mask:255.255.255.0
          inet6 addr: fe80::20a:35ff:fe00:1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:540 (540.0 B)
          Interrupt:74

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2968 (2.8 KiB)  TX bytes:2968 (2.8 KiB)

Both rgmii interfaces can correctly ping the PC but both sgmii fail. We've searched through forums on TI as well as Xilinx in an attempt to resolve this issue with no avail. Additionally, we've patched the macb driver (2019.1) for fixed link sgmii with no success as well. Can anyone help us resolve this issue?

Thanks!

EDIT:

Some additional info from the dmesg looks like the target frequencies are not being correctly generated here is the dmesg following an ifconfig down/up cycle. Has anyone ran into this issue? Our design and PS are configured to provide a reference clock of 125 MHz to both of those GEMs.

[ 1179.551147] pps pps0: new PPS source ptp0
[ 1179.551158] macb ff0d0000.ethernet: gem-ptp-timer ptp clock registered.
[ 1179.551285] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 1179.556100] pps pps1: new PPS source ptp1
[ 1179.556111] macb ff0e0000.ethernet: gem-ptp-timer ptp clock registered.
[ 1179.556222] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[ 1179.560961] pps pps2: new PPS source ptp2
[ 1179.560972] macb ff0b0000.ethernet: gem-ptp-timer ptp clock registered.
[ 1179.561081] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
[ 1180.187337] pps pps3: new PPS source ptp3
[ 1180.187349] macb ff0c0000.ethernet: gem-ptp-timer ptp clock registered.
[ 1180.187463] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
[ 1183.647707] macb ff0e0000.ethernet eth1: link up (1000/Full)
[ 1183.647843] macb ff0d0000.ethernet eth0: link up (1000/Full)
[ 1183.647923] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[ 1183.648031] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 1186.335368] macb ff0c0000.ethernet eth3: unable to generate target frequency: 125000000 Hz
[ 1186.335377] macb ff0c0000.ethernet eth3: link up (1000/Full)
[ 1186.335393] IPv6: ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
[ 1200.031759] macb ff0b0000.ethernet eth2: unable to generate target frequency: 25000000 Hz
[ 1200.031769] macb ff0b0000.ethernet eth2: link up (100/Full)
[ 1200.031785] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready

EDIT 2:

I've switched the shared MDIO in the PS configuration (as well as the DT) to be controlled by GEM0 and enable all 4 GEMs in the device tree. After attempting an ifconfig up I get the following:

[   44.307045] TI DP83867 ff0b0000.ethernet-ffffffff:02: Master/Slave resolution failed, maybe conflicting manual settings?
[   44.563099] TI DP83867 ff0b0000.ethernet-ffffffff:03: Master/Slave resolution failed, maybe conflicting manual settings?

The two sgmii happen to be the two shown above with the Master/Slave resolution failure. Can anyone provide some guidance to resolve this? I'm sifting through the source code where this is thrown (phy_device.c) 

0 Kudos
4 Replies
danstallman
Visitor
Visitor
759 Views
Registered: ‎03-14-2019

I am working on a Avent UlltraZed-EG SOM, implementing 3 GEM to SGMII PHY's.  I think I am seeing the same issue:

"unable to generate target frequency: 125000000 Hz, and the SGMII paths don't work"

I have implemented a separate MDIO interface for each of the 3 paths, and they work.

I have tried various Device Tree arrangements to no avail. I also have eliminated all of the "quirks" and/or "straps" for the SGMII nodes with no change.

Can you post a .pdf of your Block Design or indicate the IP you have in it?

 

0 Kudos
badFITimage
Explorer
Explorer
747 Views
Registered: ‎09-06-2019

Unfortunately I cannot share our block design but our SGMII and RGMII interfaces were implemented completely in the PS so the block design would not be of much help anyway. 

We were able to get it up by changing the device tree to have the mdio on GEM0 so when booting to the kernel GEM0 (and therefore the shared mdio) are probed first allowing for the other GEMs to come up correctly. 

0 Kudos
jaychoi
Visitor
Visitor
558 Views
Registered: ‎08-26-2019

Dear danstallman,

What does exact meaning about separate MDIO for each path?

Can you share the resolution how to fix it?

 

Thanks,

Jay.

0 Kudos
danstallman
Visitor
Visitor
534 Views
Registered: ‎03-14-2019

My design has 3 DP83867ISRGZ's hanging off the UltraZED, each with its own MDIO Interface.

0 Kudos