cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
rajatrao
Explorer
Explorer
1,219 Views
Registered: ‎08-04-2016

10G ethernet between ZCU102, ZCU111

Hello,

We are using the 10G ethernet subsystem IP and our design is based on XAPP1305. First we tested the design on ZCU102 in loopback mode and between two ZCU102 boards. Then, we ported the design onto ZCU111 with appropriate changes to the constraints file. Here's our situation now -

  1. 10G on ZCU102 in loopback works fine.
  2. 10G between two ZCU102 boards works fine.
  3. 10G on ZCU111 in loopback works fine.
  4. 10G between two ZCU111 boards works fine.
  5. 10G between ZCU102 and ZCU111 does not work!

Any idea why this might be happening? We are using exactly the same design on both boards and it works (first four cases) but fails between the two boards (case 5). How do we debug this?

8 Replies
rajatrao
Explorer
Explorer
1,172 Views
Registered: ‎08-04-2016

Update. Things we tried -

  1. Program 3 boards. (Link between 1 and 2 is known to work and between 1 and 3 does not.)
  2. Check block_lock status on board 1 when no ethernet cable is connected. Block_lock is low.
  3. Connect a cable between boards 1 and 2. Block_lock on board 1 goes high. Transmitting data from board 2 is received on board 1.
  4. Disconnect the cable from board 2 and connect the cable between boards 1 and 3. Block_lock goes low.
  5. Reconnect cable between boards 1 and 2. Block_lock becomes high again and the ethernet link works.

Why does block_lock go low? How do we solve the problem?

0 Kudos
joe306
Scholar
Scholar
610 Views
Registered: ‎12-07-2018

Hello, did you get your program working? If so, would you be willing to show me your Block Design? I am trying to get 10GE connected to the PS properly.

Thank you

Joe

0 Kudos
rajatrao
Explorer
Explorer
557 Views
Registered: ‎08-04-2016

This was a long time ago, but if I remember correctly it was a clocking + device tree issue.

  • The ZCU111 clock needs to be routed from a neighbouring GT bank for 10G.
  • The ZCU102 Si570 (or similar device...I can't remember) output frequency should match whatever was configured in the 10G ethernet IP. The clock chip has a device tree entry to program the freq.
  • The 10G device tree entries are key. Don't trust anything that comes out of HSI (except the irq numbers); use something from a reference design.

Also note, 10G to PS under Linux or baremetal achieves only about 6-7Gbps with multi-channel DMA, even lower with single channel DMA.

joe306
Scholar
Scholar
532 Views
Registered: ‎12-07-2018

Thank you very much. Would you mind commenting on what I have so far?

ZCU102_10GIGE_BLOCK.jpg

I did an Aurora 64B/66B design an I used a FIFO:

AURORA_BLOCK.jpg

Do I need to use a AXI Data FIFO?

Thank you

Joe

 

0 Kudos
rajatrao
Explorer
Explorer
482 Views
Registered: ‎08-04-2016

Looks alright to me. You'd need the FIFO only to meet timing since the DMA handles the clock crossing and has enough buffering capability by itself.
Make sure you monitor the status of the 10G IP for debugging. And the S_AXI port is only needed if you're using Linux. For baremetal, you don't really need to worry about it.

joe306
Scholar
Scholar
464 Views
Registered: ‎12-07-2018

Hello, thank you for looking over my block diagrams. I have found an example for the ZCU102 board. Here is the block diagram:

ZCU106_10G_25G_PL_Side.jpg

Here is the link I used to get the example.

https://github.com/Xilinx-Wiki-Projects/ZCU102-Ethernet/tree/main/2019.2/pl_eth_10g

I have a lot to learn how things are connected.

I see they use an AXI Stream Data FIFO.

I will model my design from this example.

Do you think I would be able to do a Baremetal UDP application in the beginning?

Thank you

Joe

0 Kudos
rajatrao
Explorer
Explorer
414 Views
Registered: ‎08-04-2016

Yeah, your best bet is to start with the reference design.

I don't know about baremetal UDP...I never tried it. At that time, there were no SDK drivers or example programs to use with the 10G IP. Not sure if things have changed since then.

I did 10G under Linux (drivers are available) and once that worked, 10G controlled via custom PL logic.

0 Kudos
joe306
Scholar
Scholar
394 Views
Registered: ‎12-07-2018

Hello, and thank you for responding to my message.

Do I need a license to the 10G IP to generate a bitstream?

 

Thank you

0 Kudos