GMII2RGMII Ethernet gem1 with Kernel 4.19 not working with Zynq7000

[csporer]

Hello,

since I have updatet the kernel from 4.14 to 4.19 the gem1 with gmii2rgmii converter can not find the mdio device.

Bootlog with kernel 4.14:

[ 1.203759] libphy: MACB_mii_bus: probed
[ 1.302900] macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 28 (00:90:b8:20:b6:80)
[ 1.312762] Micrel KSZ9031 Gigabit PHY e000b000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=e000b000.ethernet-ffffffff:01, irq=POLL)
[ 1.329654] libphy: MACB_mii_bus: probed
[ 1.432955] macb e000c000.ethernet eth1: Cadence GEM rev 0x00020118 at 0xe000c000 irq 29 (00:90:b8:20:b6:81)
[ 1.442811] Micrel KSZ9031 Gigabit PHY e000c000.ethernet-ffffffff:02: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=e000c000.ethernet-ffffffff:02, irq=POLL)

Bootlog with 4.19:

libphy: Fixed MDIO Bu
Micrel KSZ9031 Gigabit PHY e000b000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=e000b000.ethernet-ffffffff:01, irq=POLL)
macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 28 (00:90:b8:20:b6:80)
libphy: MACB_mii_bus: probed
mdio_bus e000c000.ethernet-ffffffff: MDIO device at address 2 is missing.
xgmiitorgmii e000c000.ethernet-ffffffff:1f: Couldn't find phydev

The devicetree (without zynq-7000.dtsi):

&gem1 {
status = "okay";
phy-handle = <&phyremote>;

mdio {
#address-cells = <1>;
#size-cells = <0>;

phyremote: ethernet-phy@2 {
device_type = "ethernet-phy";
reg = <2>;
/*first emio gpio as reset*/
reset-gpios = <&gpio0 54 0>;
phy-mode = "rgmii";
};

gmiitorgmii: gmiitorgmii@31 {
compatible = "xlnx,gmii-to-rgmii-1.0";
reg = <31>;
phy-handle = <&phyremote>;
};
};
};

I have nothing changed at the devicetree because I have not seen any change in the documentation.

Kernel config should be ok because the gem0 is working and the converter is found by the kernel.

By the way the documentation for the macb driver on the xilinx page is not up to date. 

The CONFIG_NET_CADENCE has to be CONFIG_NET_VENDOR_CADENCE with the new kernel.

Thanks for your help!

[shabbirk]

Hi @csporer 

Can you try modifying device tree inline with below example (You dont need device_type = "ethernet-phy"; in phy node, as you are doing gpio reset, keep that parameter)

&gem1 {

    mdio:mdio {

        #address-cells = <1>;

        #size-cells = <0>;

        phy0: phy@0 {

            reg = <0x0>;

        };

        gmiitorgmii: gmiitorgmii@8 {

            compatible = "xlnx,gmii-to-rgmii-1.0";

            reg = <8>;

            phy-handle= <&phy0>;

        };

    };

};

[csporer]

Hello @shabbirk ,

thanks for your reply!

I tried it, but it is not working. New log with your changes:

libphy: MACB_mii_bus: probed
mdio_bus e000c000.ethernet-ffffffff: MDIO device at address 2 is missing.
xgmiitorgmii e000c000.ethernet-ffffffff:1f: Couldn't find phydev
macb e000c000.ethernet eth1: no PHY found

This is the devicetree I tested:

&gem1 {
status = "okay";

mdio:mdio {
#address-cells = <1>;
#size-cells = <0>;

phyremote: ethernet-phy@2 {
reg = <2>;
};

gmiitorgmii: gmiitorgmii@31 {
compatible = "xlnx,gmii-to-rgmii-1.0";
reg = <31>;
phy-handle = <&phyremote>;
};
};
};

[shabbirk]

Hi @csporer 

Can you recheck in your kernel configuration whether PHY support has been enabled?

Best Regards

Shabbir

[csporer]

CONFIG_PHYLIB=y
CONFIG_FIXED_PHY=y
CONFIG_MICREL_PHY=y

This are the phy configs i enabled in this context. The configs which are mentioned from the xilinx wiki are all enabled.

[shabbirk]

I hope you have cloned the correct kernel from xilinx github with tag xlnx_rebase_v4.19_2019.1

[csporer]

I use the meta-xilinx yocto layer from the branch 2019.1. 

I checked the commit id and it matches to the xlnx_rebase_v4.19_2019.1 tag from the kernel.

[shabbirk]

Did the MDIO is shared across two PHY's?

[csporer]

No it is only one phy connected with the gmii2rgmii ip core. The other ethernet phy is connected with the mio pins and working.

[csporer]

Actually I changed the kernel from the recipe to the version 2018.3 (4.14) and the second ethernet is working!

I have no idea what's the problem with the new kernel...

[shabbirk]

Hi @csporer 

Could you please share your complete device tree, as we cant see phy-mode and phy-handle properties in GEM?

It seems that micrel PHY itself is not getting detected and is nothing to do with whether gmii2rgmii is up or not

[csporer]

Hi @shabbirk,

Here is the actually used devicetree with kernel version 4.14.

The changes I tryed from your replys are not included.

Thanks again for your help!

[stephenm]

can the complete bootlog be posted here please?

[csporer]

Now the kernel log and the used devicetree for the 4.19 kernel :)

[shabbirk]

Hi @csporer 

Can you try removing compatible string in gmii2rgmii device tree node and recheck?

[csporer]

Hi @shabbirk,

new bootlog:

[ 1.093551] libphy: Fixed MDIO Bus: probed
[ 1.099176] libphy: MACB_mii_bus: probed
[ 1.200508] Micrel KSZ9031 Gigabit PHY e000b000.ethernet-ffffffff:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=e000b000.ethernet-ffffffff:01, irq=POLL)
[ 1.216350] macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 28 (00:90:b8:20:b6:80)
[ 1.227092] libphy: MACB_mii_bus: probed
[ 1.231117] mdio_bus e000c000.ethernet-ffffffff: MDIO device at address 2 is missing.
[ 1.239302] Generic PHY e000c000.ethernet-ffffffff:1f: attached PHY driver [Generic PHY] (mii_bus:phy_addr=e000c000.ethernet-ffffffff:1f, irq=POLL)
[ 1.252543] macb e000c000.ethernet eth1: Cadence GEM rev 0x00020118 at 0xe000c000 irq 29 (00:90:b8:20:b6:81)

[shimko]

Hi, shabbirk

Is it really necessary to modify device tree with your example?

When we open file <plnx-proj-root>/components/plnx_workspace/device-tree/device-tree/system-top.dts we can see the code:

/* CAUTION: This file is automatically generated by Xilinx. */

#include "zynq-7000.dtsi"
#include "zc702.dtsi"
#include "pcw.dtsi"
...
#include "system-user.dtsi"

When we open file <plnx-proj-root>/components/plnx_workspace/device-tree/device-tree/pcw.dtsi we can see the code:

/* CAUTION: This file is automatically generated by Xilinx. */

&gem0 {
	enet-reset = <&gpio0 7 0>;
	phy-handle = <&phy1>;
	phy-mode = "gmii";
	status = "okay";
	xlnx,ptp-enet-clock = <0x69f6bcb>;
	ps7_ethernet_0_mdio: mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		gmii_to_rgmii_0: gmii_to_rgmii_0@8 {
			compatible = "xlnx,gmii-to-rgmii-1.0";
			phy-handle = <&phy1>;
			reg = <8>;
		};
	};
};
&gem1 {
	phy-handle = <&phy1>;
	phy-mode = "gmii";
	status = "okay";
	xlnx,ptp-enet-clock = <0x69f6bcb>;
	ps7_ethernet_1_mdio: mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		gmii_to_rgmii_1: gmii_to_rgmii_1@8 {
			compatible = "xlnx,gmii-to-rgmii-1.0";
			phy-handle = <&phy1>;
			reg = <8>;
		};
	};
};

So it looks like that Petalinux SDK already inserted all necessary nodes into device tree..

Nevertheless on my board both emacs doesn't see Micrel KSZ9031 Gigabit PHYs connected over GMII2RGMII.

So, Q:

Is it possible to use the same "phy-handle = <&phy1>" for both gem0 and gem1 in device tree?

[stephenm]

There is a great reference design using the gmii2rgmii at ethernetfmc.com. There is example devicetree:

https://github.com/fpgadeveloper/ethernet-fmc-zynq-gem/blob/master/PetaLinux/src/ports-012-/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi

You are missing the PHY node, also the phy-handle shouldnt be the same. You will need to add this in the system-user.dtsi file.

So, something like this:

&gem0 {
 phy-handle = <&phy1>;
 ps7_ethernet_0_mdio: mdio {
                #address-cells = <1>;
                #size-cells = <0>;
                phy1: phy@0 {
                        /* compatible = "marvell,88e1510"; */
                        device_type = "ethernet-phy";
                        reg = <0>;
                };
  gmii_to_rgmii_0: gmii_to_rgmii_0@8 {
   compatible = "xlnx,gmii-to-rgmii-1.0";
   phy-handle = <&phy1>;
   reg = <8>;
  };
 };
};
&gem1 {
 phy-handle = <&phy2>;
 ps7_ethernet_1_mdio: mdio {
                #address-cells = <1>;
                #size-cells = <0>;
                phy2: phy@0 {
                        /* compatible = "marvell,88e1510"; */
                        device_type = "ethernet-phy";
                        reg = <0>;
                };
  gmii_to_rgmii_1: gmii_to_rgmii_1@8 {
   compatible = "xlnx,gmii-to-rgmii-1.0";
   phy-handle = <&phy1>;
   reg = <8>;
  };
 };
};

If you add your HDF, and the version you want to use then I can test this for you.

[shimko]

Hi, stephenm

Thank you for reply!

There are my project files:

[shimko]

PS. Version: Kernel 4.19

[stephenm]

Are you using Petalinux or Yocto to create the Linux Image. Or are you using OSL flow?

I have patched the DTG (Devicetree generator) to fix the issue where the phy-handle is the same.

I have this setup to increment now when a phy-handle is created. I will pass this to the developers to get added to later releases.

However, for now, you can always manually override the nodes. For example the dtsi that I sent last time will override any node settings.

I have taken your dtsi that you sent to me (which look fine), and I added this dtsi include to the system.top.dts:

To pre-compile, we can use the GCC

Note: I copied this system-user.dtsi to the same location as the other dt files otherwise the path would need to be added in the -I in the gcc command line:

gcc -I my_dts -E -nostdinc -undef -D__DTS__ -x assembler-with-cpp -o my_dts/system-top.dts.tmp my_dts/system-top.dts

dtc -I dts -O dtb -o my_dts/system-top.dtb my_dts/system-top.dts.tmp

For reference, here is a wiki that shows this:

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/136904764/Creating+Devicetree+from+Devicetree+Generator+for+Zynq+Ultrascale+and+Zynq+7000

I converted this DTB to a DTS to verify the contents:

dtc -I dtb -O dts -o my_dts/dump.dts my_dts/system-top.dtb

If you are using Petalinux, then this system-user.dtsi would be in the project-spec\meta-user\recipes-bsp\device-tree\files folder.

So, in summary. It looks like the system-user.dtsi file you have is correct. Can you try this in linux, and let me know how you get on?

Note: you can see the driver here for debugging:

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/net/phy/xilinx_gmii2rgmii.c

[shimko]

Hi, stephenm!

Thanks for your reply!

I'm using Petalinux. I have tested system-user.dtsi file before I sent it to you. It does not work!..

An interesting comment about driver xilinx_gmii2rgmii.c: phy has a priv field

[stephenm]

Can you send the complete log including the uboot?

Can you do an mii dump? I want to see if the issue is with the driver or your HW.

[shimko]

Hi, stephenm!

Sorry, but I'm newbie in Linux programming. So I can't do an mii dump without some help.

But I found some strange solution of the problem..

I have changed the MACHINE_NAME in DTS Settings from zc702 to zedboard and after that GEM found out both PHYs..

But now I have another trouble:

"macb e000b000.ethernet eth0: unable to generate target frequency: 125000000 Hz"

This is my complete boot log:

[shimko]

PS. The Block Design schematic:

[stephenm]

I have changed the MACHINE_NAME in DTS Settings from zc702 to zedboard and after that GEM found out both PHYs..

This will load the DTSI  file for the zedboard:

https://github.com/Xilinx/device-tree-xlnx/tree/master/device_tree/data/kernel_dtsi/2019.1/BOARD

Are you able to ping in u-boot now?

The 1G needs the tx clk to be 125Mhz:

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/net/ethernet/cadence/macb_main.c

There is a mismatch between your tx clk, and the expected 125Mhz. What is your tx clk set to?

[shimko]

Hi, stephenm!

I added tx-skew-ps/rx-skew-ps values into system-user.dtsi file according to an advise from scwoods751.

As a result I was able to ping: PC > phy0 > gmii2rgmii_0 > gem0,

but I wasn't able to ping: PC > phy1 > gmii2rgmii_1 > gem1.

I tried to swap pins in GbEX.xdc file for swapping phy0 <=> phy1.

As a result I was able to ping: PC > phy1 > gmii2rgmii_0 > gem0,

but again I wasn't able to ping: PC > phy0 > gmii2rgmii_1 > gem1.

So, I'm sure that HW and skews for phy1 are OK.

Test ping log looks like this:

root@GbEX:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:0A:35:00:1E:53
          inet6 addr: fe80::20a:35ff:fe00:1e53/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17 errors:2 dropped:0 overruns:0 frame:2
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3012 (2.9 KiB)  TX bytes:1034 (1.0 KiB)
          Interrupt:27 Base address:0xb000

eth1      Link encap:Ethernet  HWaddr 00:0A:35:00:1E:54
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:28 Base address:0xc000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

sit0      Link encap:IPv6-in-IPv4
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@GbEX:~# ifconfig eth0 inet 192.168.1.10 netmask 255.255.255.0
root@GbEX:~# ping 192.168.1.100
PING 192.168.1.100 (192.168.1.100): 56 data bytes
64 bytes from 192.168.1.100: seq=0 ttl=128 time=0.694 ms
64 bytes from 192.168.1.100: seq=1 ttl=128 time=0.273 ms
64 bytes from 192.168.1.100: seq=2 ttl=128 time=0.245 ms
64 bytes from 192.168.1.100: seq=3 ttl=128 time=0.262 ms
64 bytes from 192.168.1.100: seq=4 ttl=128 time=0.246 ms
64 bytes from 192.168.1.100: seq=5 ttl=128 time=0.240 ms
64 bytes from 192.168.1.100: seq=6 ttl=128 time=0.252 ms
64 bytes from 192.168.1.100: seq=7 ttl=128 time=0.253 ms
64 bytes from 192.168.1.100: seq=8 ttl=128 time=0.244 ms
64 bytes from 192.168.1.100: seq=9 ttl=128 time=0.270 ms
^C
--- 192.168.1.100 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max = 0.240/0.297/0.694 ms
root@GbEX:~# macb e000b000.ethernet eth0: link down

root@GbEX:~# ifconfig eth1 inet 192.168.1.11 netmask 255.255.255.0
IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
root@GbEX:~# macb e000c000.ethernet eth1: wait for end of transfer timed out

root@GbEX:~# ping 192.168.1.100
PING 192.168.1.100 (192.168.1.100): 56 data bytes
^C
--- 192.168.1.100 ping statistics ---
12 packets transmitted, 0 packets received, 100% packet loss
root@GbEX:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:0A:35:00:1E:53
          inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20a:35ff:fe00:1e53/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:30 errors:2 dropped:0 overruns:0 frame:2
          TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4180 (4.0 KiB)  TX bytes:6588 (6.4 KiB)
          Interrupt:27 Base address:0xb000

eth1      Link encap:Ethernet  HWaddr 00:0A:35:00:1E:54
          inet addr:192.168.1.11  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:28 Base address:0xc000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:336 (336.0 B)  TX bytes:336 (336.0 B)

sit0      Link encap:IPv6-in-IPv4
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@GbEX:~#

[kwy4262]

Hi Stephen:

I have the same problem (by comparing my boot.log) with the user. 

-- Couldn't find phydev

-- unable to generate target frequency of 125000000 Hz

However, I am using ZCU102 and DP83869 PHY chip. Is there some patch that I need for my board to be able to ping via the gmii-to-rgmii?

Thanks,
CK

[csporer]

Hello,

I tried again with the version 2019.2 but there is the same problem. Has anybody an idea?

I searched at github for changes but I can not find something what there was changed that the devicetree

is not longer working...

It would be great to find a answer...

I dont think that the FPGA has to rebuild? All other stuff is working fine.

[shabbirk]

Hi @csporer 

Have you tried reverting the patches that went into 2019.1/2 for gmii2rgmii driver?

You will find the patch details here (Refer to Change Log):

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842246/Xilinx+GMII2RGMII+convertor

Best Regards

Shabbir