cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Voyager
Voyager
650 Views
Registered: ‎05-25-2016

AXI Ethernet driver brings up interface but no packets

Hello,

Could someone check out my device tree and PL block diagram to see if anything is wrong that I've missed?  I can boot linux and use "ifconfnig eth1 up" to bring up the ethernet link and see no errors.  I see the following:

86.jpg

I'm using the zedboard and the opsero FMC 4 port ethernet card.  I'm also using the axi ethernet subsystem IP in PL.

 

 

 

0 Kudos
16 Replies
Highlighted
Voyager
Voyager
642 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Here are some of my pertinent files and a shot of my interrupts.  It looks like the ethernet interrupts (eth1) are firing.

87.jpg

0 Kudos
Highlighted
Voyager
Voyager
575 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi Xilinx,

I have figured out that my ethernet link actually isn't working and there is a problem on the receive side.  I do (after waiting a while) get an error from the  driver saying that my rx buffer length is zero.  As you can see I have enabled "Use Rxlength in Status Stream" which I think is correct.

88.jpg

My error is the following:

89.jpg

The 0x15019 shows the status of the DMA status register in SG mode and the bit that represents my error (I think) is bit 4 which conveys the following information:

90.jpg

Here is a screenshot from vivado of the SG PL link that shows the SG read channel

91.jpg

does anyone know what would cause this?  I can't find a device tree setting that would cause this.

 

 

0 Kudos
Highlighted
Voyager
Voyager
542 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

I've finally gotten an ILA connected up to the SG link, the CTRL link and STS link of the DMA.  It seems there is a lot of traffic, but the CTRL stream doesn't match what I would expect.  It seems fairly asynchronous and might be due to the error I'm seeing.  Does the DMAIntError bit mean that the buffer length in the APP section of the buffer descriptor is 0?  If I'm not mistaken the APP words should be transmitted over the CTRL/STS links and not the SG link for this configuration.

How do you troubleshoot this from here?  What part of the driver is supposed to be filling out this length value (or what part of the driver is supposed to be filling out the desciriptors) so I can try to get to the bottom of why my buffer descriptors are wrong?  Below are some ILA screenshots of the traffic I'm seeing.

93.jpg94.jpg

 

 

0 Kudos
Highlighted
Moderator
Moderator
502 Views
Registered: ‎12-04-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @m3atwad 

From the error log, it appears that the Rx interrupt handling is failed as the status bit is not set. The below parts of the code actually handles the BD after calling receive function

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/net/ethernet/xilinx/xilinx_axienet_dma.c#L342

https://github.com/Xilinx/linux-xlnx/blob/master/drivers/net/ethernet/xilinx/xilinx_axienet_main.c#L1243

 

Best Regards

Shabbir

0 Kudos
Highlighted
Voyager
Voyager
487 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi Shabbirk,

It looks like I'm failing out before anything really happens.  Does this mean there is a hardware/configuration problem with the DMA?  It looks like the first thing the driver rx isr does is check these flags.

 

Why does this

irqreturn_t __maybe_unused axienet_rx_irq(int irq, void *_ndev)

say "maybe unused"?

 

Also, what status bit are you referring to in which register?

I didn't realize this was an RX error so I think the internal dma error bit I should be looking at is this

97.jpg

0 Kudos
Highlighted
Voyager
Voyager
449 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

I think I may have finally located the rx data / buffer mismatch with the ILA.  I can see the DMA reading the buffer descriptors with the read bus shown below.  Slot_0 is connected to the DMA SG link.  It looks like the buffer size here in my control byte of the SG read (S2MM) side is 1522 bytes.  This seems reasonable since the MTU for ethernet is 1500.  

98.jpg

The questionable part is the STS link from the ethernet MAC to the DMA.  This is showing much different values in the APP fields. Particularly the APP4 fields which shows 0x11120076 which makes no sense to me.  This is supposed to represent received bytes, but why would there be so many rx'd bytes?  The state of the ethernet link is actually pretty inactive.  For this test I just did "ifconfig eth1 up" which brought the eth1 up, but there is no IP address assigned to eth1 yet (no ipv4 ip address).  

99.jpg

 

I also see no SOF/EOF bits set in any of the SG descriptor transactions.

Are APP0-3 fields not used - the data in them meaningless?  The DMA users guide doesn't seem to be specific about what exactly they are supposed to contain and whether they are even useful.  APP4 seems to just be the total number of bytes rx'd is that correct?

Highlighted
Moderator
Moderator
418 Views
Registered: ‎12-04-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @m3atwad 

Do you see any Tx and RX count other than 0 with ifconfig output?

I will check further to see if there are any issues with your DMA configurations.

I hope you took the reference as XAPP1082 for zynq-7000, which also make use of PL(AXI 1g/2.5g) ethernet with AXI DMA

 

Best Regards
Shabbir

0 Kudos
Highlighted
Voyager
Voyager
401 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @shabbirk I did try to use xapp1082, but unfortunately there are too many missing details regarding how to implement this with the driver.  I will check today, but I am pretty sure there is no TX and when I ping nothing happens.  I also don't see any packets with Wireshark.  Same with RX - I believe that is always zero with 100% packet loss as well.  

I will verify today.

0 Kudos
Highlighted
Voyager
Voyager
395 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Well very much to my surprise I do see a RX/TX packet count.  This is very interesting because I can't ping anything (in or out of the zynq) but do see packet counts increase in ifconfig.

100.jpg101.jpg

 

I've also noticed that I can't connect an ILA to the receive side of the axi ethernet MAC IP.  I was planning on looking at the received data stream along with the status/control links, but when I connect the axi stream rs data from the mac to dma the entire eth1 disappears.  It seems that vivado or the axi ethernet mac really doesn't like when you connect an ILA to the RX or TX data stream in and out of the axi ethernet MAC IP block.

0 Kudos
Highlighted
Voyager
Voyager
389 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Some other questions that may be helpful for me troubleshooting this:

1. How do I print the buffer descriptors for RX and TX to console.  Since these are all preset in software - does the driver allow me to dump the BD contents so I can verify the information is correct?

2. Does the status stream only move/communication the rx byte count?  The users guide for the DMA doesn't really say what values APP0-3 should contain.

3. The ethernet link I'm struggling with comes up with IPv6 information as you can see in the screenshots, but no IPv4 info (ip address, netmask etc...).  Would it be beneficial for me to disable IPv6 all together and try to get the eth1 link to automatically come up and pull an IP address?

0 Kudos
Highlighted
Moderator
Moderator
298 Views
Registered: ‎12-04-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @m3atwad 

The issue in your case seems to be likely with the design. Debugging it from the software/driver point is challenging to catch the issues with DMA configurations, and I would rather suggest to take the reference design as a baseline and develop the custom design. We also have AXI ethernet dma based design targeting zynq-7000 here:

https://github.com/fpgadeveloper/ethernet-fmc-axi-eth

 

Best Regards

Shabbir

0 Kudos
Highlighted
Voyager
Voyager
269 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @shabbirk thanks for the reply!  I have overlooked some things in that example and am still debugging.  I have a question that is probably fairly obvious to you, but I'm having a hard time understanding it.

In this file:

https://github.com/fpgadeveloper/ethernet-fmc-axi-eth/blob/master/PetaLinux/src/common/project-spec/meta-user/recipes-kernel/linux/linux-xlnx/user.cfg

The definition:

CONFIG_XILINX_GMII2RGMII=y

is added.  This seems to imply a conversion between GMII to RGMII has to happen.  Why is this?  Is there no native support for RGMII?  Also, what is this being applied to - the driver?

0 Kudos
Highlighted
Voyager
Voyager
242 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @shabbirk ,

I've made some substantial progress.  There were a few errors in my project when I went back and looked at the example you linked.  I'll note them for others:

1. RX/TX checksum hardware offloading not checked

2. Clock sources were connected to some nodes incorrectly - followed BD in example

3. Added additional source files from link petalinux repo.  You need the common, port-0123 and zedboard directories.

4. Apply xilinx axi ethernet driver patch found here

https://github.com/fpgadeveloper/ethernet-fmc-axi-eth/tree/master/PetaLinux

5. Added constraints file as well to constrain RGMII path (found in linked git repo - vivado project) and set the "late" property for vivado compilation

 

With that said, I've managed to get the link up and running and can ping in/out.  I am unfortunately still getting the same error, but it looks to happen less frequently.  Is there a way I can figure out what is going on by catching this with an ILA in the PL?  

 

Image of what I'm seeing in console attached

104.jpg

0 Kudos
Highlighted
Voyager
Voyager
215 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

I've also finally found that the example project from opsero (eth fmc manufacturer) has the same dma rx problem.  So I think my project is working as good as the example project in github you linked me to works.

0 Kudos
Highlighted
Moderator
Moderator
130 Views
Registered: ‎12-04-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @m3atwad 

yes, this change is applied to the driver. Basically EMIO based Ethernet connection (that has gmii interface) uses the Gmii-to-Rgmii IP between the PS and the Ethernet PHY, as most of the PHY's are rgmii compatible

Do you have any update on the progress of this issue?

Best Regards

Shabbir

0 Kudos
Highlighted
Voyager
Voyager
100 Views
Registered: ‎05-25-2016

Re: AXI Ethernet driver brings up interface but no packets

Hi @shabbirk ,

Thanks for the reply that makes sense.  So the PS/EMIO side expects only GMII?  This confused me because the PL MAC IP core allows you to configure the MAC interface for any of these.  

 

I don't particularly have a final solution, I've rebuilt the project from the company that makes the card and see the same issue.  I got a prebuilt image from them that does not seem to have this issue.  I'm further investigating the build process and some constraints, but so far I'm still unsure.  There are some HDL constraints I don't understand so maybe there is a timing problem I just haven't been able to figure out yet.  The design meets timing, but I noticed there are a couple lines in a constraint file that are not present in my custom project.

0 Kudos