cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Observer
Observer
2,120 Views
Registered: ‎10-08-2019

Building PL Ethernet (10G) using XAPP1305 for Vivado 2019.1 and Petalinux 2019.1

This post is a guide to build the XAPP1305 for PL Ethernet 10G using Vivado 2019.1 and Petalinux 2019.1 as the available guide would not work for the aforementioned versions of the software. This post is a collection of information from several forum posts.

  1. Follow the guide available for XAPP1305 from Xilinx Wiki. When the Vivado bitstream generation and hardware SDK export is completed, continue on to Petalinux Build Procedures for 2019.1. Before starting this procedure, continue to Note 01, where directions for creating a project using the HDF file is specified.
  2. Do not follow the step In petalinux-config DTG Settings ---> (template) MACHINE_NAME, change the template to zcu102-rev1.0. Keep the name as it is.
  3. In the next note Changes to system-user.dtsi, rather than the given changes, copy and replace the content on system-user.dtsi with the system-user.txt attached to this post.
  4.  After system-user.dtsi is updated, continue the Petalinux Build Procedures from Configure the Kernel onwards.

This will build a working image!

NOTE: For the curious mind, the given system-user.dtsi is made from the default Device Tree from Petalinux 2018.3 as the device tree on Petalinux 2019.1 is broken. Further, changes to system-user.dtsi given in the XAPP1305 Wiki is incorporated while adding the entry for i2c@3, which is deleted in the Wiki. This would set up the receiver side clock to the proper frequency for a 10G Connection. (10 000 000 000 bits/sec / 64 bits wide axi stream = 156.25 Mhz) [Note that this is an assumption of what it really does. The real scenario is unknown to me]

Few of the errors that will be solved by this solution:

ERROR: device-tree-xilinx+gitAUTOINC+73e546e312-r0 do_compile: Function failed: devicetree_do_compile
eth1: XXV MAC block lock not complete! Cross-check the MAC ref clock configuration

Known Issues:

Following errors/warnings will still appear at boot, even though the system is working properly.

root@ref_10g_xilinx:~# dmesg | grep enet
[    2.767945] xilinx_axienet 80010000.ethernet: missing/invalid xlnx,addrwidth property, using default
[    2.768001] xilinx_axienet 80010000.ethernet: Ethernet clock init failed -517
[    5.596995] xilinx_axienet 80010000.ethernet: missing/invalid xlnx,addrwidth property, using default
[    5.606175] xilinx_axienet 80010000.ethernet: couldn't find phy i/f

References and Kudos to following forum posts to tackling different aspects of the problem:

  1. So, what now? (Petalinux build error) by @satguy. Especially the replies by @stephenm and @eliezer.
  2. Xapp1305 Block lock issue by @kieucua1503. Especially the reply from @simon.beaudoin.
16 Replies
Highlighted
Adventurer
Adventurer
1,887 Views
Registered: ‎12-04-2019

Hello

This post is very helpful. 

Can you update the system.dtsi file to use with vivado and petalinux 2019.2?

I am trying to recreate the PS EMIO 1G 1000Base-X ethernet from XAPP1305 and I am its not working. I believe it might be related to device tree updates needed as you mentioned the device tree for 2019.1 is broken. 

 

0 Kudos
Highlighted
Observer
Observer
1,876 Views
Registered: ‎10-08-2019

Hey mansuramin, 

I believe the attached file will do the trick. (I only undid the changes mentioned here at Changes to system-user.dtsi under 10G implementation while incorporating the change given under "i) For the PS EMIO 1000BASE-X, and PL 1G make the following modifications to system-user.dtsi". But I adjusted the factory-fout and clock-frequency of i2c@2 to <15625000> as that is the frequency for 1G implementation.)

You should also apply the patches given under Apply Patches too.

In case the above device tree still fails, try again with factory-fout and clock-frequency of i2c@2 set to <300000000>. 

Do note that I have NOT implemented PS EMIO 1000BASE-X design myself. I am only commenting based on a similar experience with 10G implementation.

0 Kudos
Highlighted
Adventurer
Adventurer
1,804 Views
Registered: ‎12-04-2019

Hello @chettige 

 

I added the recommended dtsi updates to my petalinux example project and when I do a device tree dump at uboot i dont see those updates being applied. 

In the device tree recommendations, you set the clock frequency to 15625000 or 3000000000.

When I print out the device tree at uboot using fdt print at the respective address I dont see that frequency being set. 

 

I am doing a compare between my rebuilt example design with the dtsi updates, and the example images provided by the reference design. See attached. 

I have the device tree from the example images as well as the console bootup dump. Along with my device tree and console dump. The device tree dtsi settings I am using is also attached. 

0 Kudos
Highlighted
Observer
Observer
1,780 Views
Registered: ‎10-08-2019

Hey @mansuramin,

My colleague made some new discoveries today and I incorporated them in the original post. Just to set the background, weirdly, my 10G MAC was operating a few minutes after bootup, not right away. Apparently, the receiver side of the transaction is not clocked properly and this was fixed by adding the i2c@3 entry to the device tree with proper frequencies and keeping the i2c@2 frequencies at 300MHz (As it was in the guide). Can you try the following device tree with your design and it might work? 

TIP: If you check the boot log for entries relating to "SI570", in the correct design, these clocks should come live.

0 Kudos
Highlighted
Adventurer
Adventurer
1,760 Views
Registered: ‎05-08-2018

Hi @chettige ,

Concerning what you said about your MAC operating only after a few minutes, I wanted to add that we have a similar behavior too. My collegue found out that if you ping a computer connected to the link, the link will sudently come up. So we created a recipe that would ping a remote at startup in order to bring the link up right away. Very strange. Honnestly, strange things are going on with this IP. The driver OOps the kernel once in a while when you 'down' the link too :s. 

0 Kudos
Highlighted
Observer
Observer
1,698 Views
Registered: ‎10-08-2019

Hey @simon.beaudoin,

I updated the original post with a new device tree as my colleague @brasilino found out that the i2c@3 should be there for the MAC to really work. The changes were as follows:

&i2c1 {
pinctrl-names = "default", "gpio";
pinctrl-0 = <&pinctrl_i2c1_default>;
pinctrl-1 = <&pinctrl_i2c1_gpio>;
scl-gpios = <&gpio 16 0>;
sda-gpios = <&gpio 17 0>;

/* FIXME PL i2c via PCA9306 - u45 */
/* FIXME MSP430 - u41 - not detected */
i2c-mux@74 { /* u34 */
compatible = "nxp,pca9548";
#address-cells = <1>;
#size-cells = <0>;
reg = <0x74>;
i2c@0 { /* i2c mw 74 0 1 */
#address-cells = <1>;
#size-cells = <0>;
reg = <0>;
/*
* IIC_EEPROM 1kB memory which uses 256B blocks
* where every block has different address.
* 0 - 256B address 0x54
* 256B - 512B address 0x55
* 512B - 768B address 0x56
* 768B - 1024B address 0x57
*/
eeprom: eeprom@54 { /* u23 */
compatible = "at,24c08";
reg = <0x54>;
};
};
i2c@1 { /* i2c mw 74 0 2 */
#address-cells = <1>;
#size-cells = <0>;
reg = <1>;
si5341: clock-generator1@36 { /* SI5341 - u69 */
compatible = "si5341";
reg = <0x36>;
};
};
i2c@2 { /* i2c mw 74 0 4 */
#address-cells = <1>;
#size-cells = <0>;
reg = <2>;
si570_1: clock-generator2@5d { /* USER SI570 - u42 */
#clock-cells = <0>;
compatible = "silabs,si570";
reg = <0x5d>;
temperature-stability = <50>;
factory-fout = <300000000>;
clock-frequency = <300000000>;
};
};
i2c@3 { /* i2c mw 74 0 8 */
#address-cells = <1>;
#size-cells = <0>;
reg = <3>;
si570_2: clock-generator3@5d {
#clock-cells = <0>;
compatible = "silabs,si570";
reg = <0x5d>;
temperature-stability = <50>;
factory-fout = <156250000>;
clock-frequency = <156250000>;
};
};
i2c@4 { /* i2c mw 74 0 10 */
#address-cells = <1>;
#size-cells = <0>;
reg = <4>;
si5328: clock-generator4@69 {/* SI5328 - u20 */
compatible = "silabs,si5328";
reg = <0x69>;
/*
* Chip has interrupt present connected to PL
* interrupt-parent = <&>;
* interrupts = <>;
*/
};
};
/* 5 - 7 unconnected */
};
};
According to what we found out, i2c@2 (si570_1) containes a user clock and the frequency doesn't really make a difference here. But apparently i2c@3 (si570_2) clocks the receiving side of the MAC. The only reason for assuming this was, on the previous bootup, messages (ping) do go out but the responses don't come back. But this fixes it. Which indeed is a bit strange. 
 
Do try out this device tree update and you should have a working MAC from bootup, w/o any ping a remote or similar shenanigan! 😅
0 Kudos
Highlighted
Adventurer
Adventurer
1,690 Views
Registered: ‎05-08-2018

We are not using the ZCU102 board anymore. I'm talking about our custom pcb... We have a fixed frequency 156.25MHz oscillator hocked up on refclk0, so fudging with the device-tree isn't required anymore to set the frequency!

0 Kudos
Highlighted
Observer
Observer
1,681 Views
Registered: ‎10-08-2019

Oh! Then I have no idea of a solution for that!

0 Kudos
Highlighted
Adventurer
Adventurer
1,633 Views
Registered: ‎12-04-2019

Hello @chettige 

I tried your updated dtsi file and no luck.

My question is why the clock frequency needs to be set to 300000000 or 15625000 when the reference clock for in the IP is set to 125000000? 

Shouldn't the factor-fout be set to 125000000 to match what the example design has in the 1G IP? 

See My rebuild device tree and prebuild xilinx example device tree. Note the differences and please let me know if you spot anything of question in my device tree. 

 

0 Kudos
Highlighted
Adventurer
Adventurer
1,588 Views
Registered: ‎12-04-2019

Hello @chettige 

I was finally able to get the 1000Basex design to work however it requires human intervention. 

Here is what i have done so far: 

1) Apply patches AR72806 and AR71295.

2) Modify kernel as follows: 

Device Drivers > Network device support > PHY Device support and infrastructure > < * > Drivers for xilinx PHYs

Device Drivers> DMA Engine Support> < > Xilinx AXI DMAS Engine

3) Use your updated device tree. 

 

Only caveat is when the board boots up, the link is not established correctly. I have to use the SCUI tool to read the frequency from the SI570 chip and then the link becomes established and a ping request response works. Any idea why I have to read the frequency from the chip vs it just working on power up? 

0 Kudos
Highlighted
Observer
Observer
1,561 Views
Registered: ‎10-08-2019

Hey @mansuramin,

I am also getting variable results from the build and I believe the issues with the 1000Basex design are a completely different set of issues from what I am experiencing with the 10G design, given the block designs are quite different from one another. Sorry, I am not quite sure how to assist.

Best to start a new thread in this regard as Xilinx Moderators do not monitor already existing treads. 

0 Kudos
Highlighted
Adventurer
Adventurer
1,547 Views
Registered: ‎05-08-2018

Check out with a scope the frequency from the oscillator before and after you to that step, see if reading the chip actually triggers it?

0 Kudos
Highlighted
Adventurer
Adventurer
1,471 Views
Registered: ‎12-04-2019

Hello @simon.beaudoin 

 

I placed a diff probe on the following locations and ran through a series of power downs to see if the clock shows any problematic signs as the board boots up and as you read the clock frequency. I was not able to visibly see any differences before the read and after the read. 

u56.pin4 to u56.pin5

Across R248

u51.pin13 to u51.pin14

u51.pin11 to u51.pin12

image.png

 

It seems as the board boots up and is programmed vis the i2c bus, the programming is not done properly and by riggering a read operation using hte scui too, it makes things work as they should have previously. Any ideas on what to look at for debug? 

0 Kudos
Highlighted
Adventurer
Adventurer
1,460 Views
Registered: ‎05-08-2018

@mansuramin yea... I will say like chettige; all the experience I have is with the xxv ethernet core unfortunately. The vivado block design and the linux driver are quite different and I wouldn't know how to guide you properly from there... it would be best to start a new thread :S beft of luck my friend

0 Kudos
Highlighted
Visitor
Visitor
764 Views
Registered: ‎10-18-2018

Hello,

I tried your method to create a 10g ethernet system using only .hdf (instead of .bsp) and succeeded!

However, when I want to further build a dual-channel 10g ethernet system, only one channel can appear in Linux. When I type "ifconfig eth1 up", the first 10g Ethernet interface starts. When I use "ifconfig eth2 up", it shows "SIOCGIFFLAGS: No such device".

My block design is as shown in the figure below. The xxv_ethernet_0 is set to 2 cores, and the DMA and FIFOs for channel 2 are copied and connected exactly according to the channel 1.

无标题.png

At the petalinux level, I didn't make any changes at all. I just used the .hdf generated by vivado to build the project and copied the system.dtsi you provided.

I wonder how to modify system.dtsi to make linux recognize 2 channels of 10g ethernet.
Do you have ideas?

Looking forward to your reply.

0 Kudos
Highlighted
Adventurer
Adventurer
713 Views
Registered: ‎05-08-2018

Hi @wgg!

Indeed, the first step was to duplicate the components in the block design. Not, I would begin by confirming that everything is okay in the pl.dtsi autogenerated. I want to know if the device tree generator properly parsed the info from the .hdf file. Could you provide the pl.dtsi file?

Second of all, from my souvenirs, you have to provide a mac address for the (each) interface in the system-user.dtsi. Please provide the content of that file. 

Third, I would be interested in looking at your boot log. Please provide it too. I want to check if when booting some kind of message appears regarding that second interface (like not having a mac address provided in its device-tree node)

Regards

0 Kudos