cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Contributor
Contributor
9,069 Views
Registered: ‎02-19-2009

Xilinx LL TEMAC and Xiilnx LLTEMAC

Can anyone tell me what is the difference between Xilinx LL TEMAC and Xilinx LLTEMAC (the one without space between LL and TEMAC). I see two types of kernel configuration option and drivers in linux kernel
0 Kudos
24 Replies
Highlighted
Xilinx Employee
Xilinx Employee
9,067 Views
Registered: ‎09-10-2008

Hi,

 

The LLTEMAC is our traditional local link driver that we've had in the Xilinx tree for years and is not in the mainline kernel. It's what we test with but it's not designed to be put into the mainline as it was designed a long time ago before we realized the mainline requirements.

 

The other one, LL TEMAC, is a new driver that is in the mainline kernel.  It was written by a number of people, but Grant Likely got it into mainline. It's only running on the powerpc 440 as I have a patch that's in the next net branch for making it run on powerpc405 and microblaze.  It's really the way we want to go in the future, but still needs more work to replace our current one (checksum offload).

 

Hope that helps.

0 Kudos
Highlighted
Contributor
Contributor
9,058 Views
Registered: ‎02-19-2009

Thanks for the reply John. That answers my question. However, The kernel menuconfiguration provides options for both type of drivers (LLTEMAC and LL TEMAC).

 

# CONFIG_XILINX_LL_TEMAC is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
# CONFIG_XILINX_TEMAC is not set
# CONFIG_ATL1E is not set
CONFIG_XILINX_LLTEMAC=y
# CONFIG_XILINX_LLTEMAC_MARVELL_88E1111_RGMII is not set
# CONFIG_XILINX_LLTEMAC_MARVELL_88E1111_GMII is not set
CONFIG_XILINX_LLTEMAC_MARVELL_88E1111_MII=y
# CONFIG_XILINX_LLTEMAC_NATIONAL_DP83865_GMII is not set

 

I'm able to bring the ethernet interface by just selecting the LLTEMAC(the old one)  but not by selecting LL TEMAC (the new one). Is there any other manual configuration that has to be done to use the new driver LL TEMAC. Also in "software platform settings" in xps only LLTEMAC and generic is listed as available driver. Shouldn't it have LL TEMAC also.

 

-Karthick

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
9,055 Views
Registered: ‎09-10-2008

>I'm able to bring the ethernet interface by just selecting the LLTEMAC(the old one)  but not by selecting LL TEMAC (the new one). Is there any other >manual configuration that has to be done to use the new driver LL TEMAC.

 

What processor are you one, it should work on PowerPC 440 right now but not others yet as that's what my patch that's headed into the mainline does. 

 

>Also in "software platform settings" in xps only LLTEMAC and generic is listed as available driver. Shouldn't it have LL TEMAC also.

 

Not necessarily as we can't support everything in the EDK that's in the mainline.  The LLTEMAC is our driver for now, if you want to use the LL TEMAC that's up to you.

 

Hope that is clear.

 

 

0 Kudos
Highlighted
Contributor
Contributor
9,053 Views
Registered: ‎02-19-2009

I'm using PPC 440 which is a hard block in Virtex 5 FPGA.
0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
9,049 Views
Registered: ‎09-10-2008

I guess I'm not following you on the problem.

 

After configuring the kernel for the 440, (make ARCH=powerpc 44x/virtex5_defconfig)  I can see both drivers in the kernel configuration when running menuconfig.  I can select either one.

 

Maybe I'm just misunderstanding it.

 

0 Kudos
Highlighted
Contributor
Contributor
9,045 Views
Registered: ‎02-19-2009

Yes, you are right. I can see both drivers in the kernel configuration after running 44x/virtex5_defconfig and i can select either one. When i select  Xilinx LL TEMAC(new driver)  only and build the image I cannot bring the ethernet interface (eth0) up. However, when i build the image by selecting LLTEMAC (old driver) only the ethernet interface seems to come up just fine and i can configure it using ifconfig. Hope this is clear.
0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
9,043 Views
Registered: ‎09-10-2008

Ok good, you need a phy in your device tree for the new driver to work.

 

Add the following to the end of the temac in the device tree. 

 

    phy-handle = < &phy7 >;
    phy7: phy@7 {
     reg = <7>;
    };

 

That should help, just realize this new driver has less testing and features in some ways, but it is the right direction as it uses the kernel phy layer.

 

Thanks.

0 Kudos
Highlighted
Adventurer
Adventurer
9,011 Views
Registered: ‎10-28-2007

Do you want some beta testers for the new LL TEMAC driver on Microblaze?  I'm keen and my project involves creating a network performance testbench, so we might be able to provide you with some useful information.

 

Is it possible for me to get your expected patch from git.xilinx.com?

 

Thanks in advance.

 

Joshua

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
9,009 Views
Registered: ‎09-10-2008

That's a great idea, here's the complication.

 

The patch is against the mainline and a newer kernel version. 

 

What's your timeframe for this work?

 

Thanks.

0 Kudos
Highlighted
Adventurer
Adventurer
8,438 Views
Registered: ‎10-28-2007

I'll be working on this until November or December this year.  I've got some pressure to get some Ethernet performance numbers out fairly soon though.  Right now, I'm using:

 

 

Linux q6-hpc 2.6.33-12341-g1743154 #147 PREEMPT Wed Apr 28 14:59:11 EDT 2010 microblaze GNU/Linux

 

I might be a little confused about which specific kernel versions you are talking about. Do you mean:

 

 

I'm keen to put my teeth into whatever new code I can get.  If there is anything I can do to help, such as helping merging/testing 2.6.33 code up to 2.6.34, let me know.

 

Joshua

 

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,431 Views
Registered: ‎09-10-2008

So a little more heads up to set expectations. 

 

I don't think you're going to see good ethernet performance with the new driver on MicroBlaze (from what I've heard) as it's not using checksum offload yet.  The checksum offload doesn't appear to help much on powerpc, but I'm told it helps a lot on MicroBlaze since it's a lower performance processor.

 

Yes the patch is against 2.6.34-rcX.  I attached the patch that should apply to the mainline kernel.  The mainline kernel is fine for MicroBlaze without our tree if you don't need some of our stuff that's not in mainline yet.

 

Hope that's clearer.

 

 

0 Kudos
Highlighted
Adventurer
Adventurer
8,412 Views
Registered: ‎10-28-2007

Here's some results, but first my process:

 

1) I downloaded 2.6.34-rc5 and patched it with your changes.
2) I then took my working microblaze .dts file, .config, and initramfs.conf from the linux-2.6-xlnx git trunk and placed them in the appropriate places. (arch/microblaze/boot/dts/q6-hpc.dts, arch/microblaze/configs/q6-hpc_defconfig respectively).
3) I added the following to my .dts file

phy-handle = < &phy0 >;
phy0: phy@0 {
    reg = <0>;
};

 4) Ran 'make q6-hpc_defconfig'

 5) Ran 'make menuconfig' and removed all sorts of stuff that I had been test building (e.g. I2C, SPI, USB etc.) and selected the LL TEMAC, and also in the phy menu added support for MARVELL phy.

 6) Ram 'make -j 4 simpleImage.q6-hpc' (gotta love a quad core). And saw the following warning during the .dtb build.

 

  DTC     arch/microblaze/boot/q6-hpc.dtb
DTC: dts->dtb  on file "/home/jpl/dev/exnet/Q6/xilinx/linux-2.6.34-rc5/arch/microblaze/boot/dts/q6-hpc.dts"
Warning (reg_format): "reg" property in /plb@0/xps-ll-temac@23000000/ethernet@23000000/phy@0 has invalid length (4 bytes) (#address-cells == 2, #size-cells == 1)
Warning (avoid_default_addr_size): Relying on default #address-cells value for /plb@0/xps-ll-temac@23000000/ethernet@23000000/phy@0
Warning (avoid_default_addr_size): Relying on default #size-cells value for /plb@0/xps-ll-temac@23000000/ethernet@23000000/phy@0

 

 

7) Used xmd to download and run the ELF image.

8) I saw some bug output about  temac_indirect_busywait on the console.

 

The output is rather large, so I'm attaching the complete dump.

 

Does this look familiar?

 

Joshua

 

 

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,405 Views
Registered: ‎09-10-2008

Hi Joshua,

 

No I didn't see that problem, not sure what the difference it. Will look at the output more.

 

Thanks.

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,404 Views
Registered: ‎09-10-2008

Are you confident that's the right phy address in the dts file, although it shouldn't cause that output you saw even if it is wrong?

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,398 Views
Registered: ‎09-10-2008

Hi Joshua,

 

I'm working on retesting with my current tree and pulling a new update from the mainline.

 

The other thought I had was it might have been worthwhile to test with the emaclite 1st to make sure that something didn't get busted in the mainline.  I say emaclite as the other LL TEMAC is not in the mainline and is only in our tree.

 

Not sure it's worth going there, but another data point.  I'll let you know what I see.

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,397 Views
Registered: ‎09-10-2008

Hi Joshua,

 

Here's the bad news, it works fine in my system on both my rc3 and my current pull today of the kernel, rc6.

 

I use NFS root in my automated test and i just reran that on both kernels without problem. My experience is that's a reasonable test.

 

Not sure what could be going wrong with yours.  I'll attach my kernel config and device tree to see if helps you in any way.

 

Maybe your uncovering some problem I've not seen in testing (and needs to be found).  If you have the to push forward it's good for everyone probably, but I'm not sure how to help exactly.

 

Sorry.

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,393 Views
Registered: ‎09-10-2008

Couldn't attach more than one file?  so this has my kernel config.

0 Kudos
Highlighted
Adventurer
Adventurer
8,388 Views
Registered: ‎10-28-2007

Well, there are a couple of small differences in the .dts file.

 

1) I've enabled RX and TX checksum offloading in the logic, and this is in the .dts.

2) My fifos are 0x4000

3) I'm using the soft TEMAC (temac-type = <2>)

 

I'll disable the checksum offloading in the .dts and rerun the test.

 

Joshua

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
8,385 Views
Registered: ‎09-10-2008

Just another point, I'm testing on a strictly mainline kernel here, nothing from Xilinx merged into it.

 

Not sure that matters, but wanted to make sure we're in sync.

0 Kudos
Highlighted
Adventurer
Adventurer
5,887 Views
Registered: ‎10-28-2007

Okay, I have it booting now, after changing the timeout in 'temac_indirect_busywait' of ll_temac_main.c (line 72).

 

Original:

 

 

int temac_indirect_busywait(struct temac_local *lp)
{
        long end = jiffies + 2;

 

 

My version

 

 

int temac_indirect_busywait(struct temac_local *lp)
{
        /* jpl adding 2 to the timeout */
        long end = jiffies + 4;

 And the output of the kernel boot is:

 

 

[    0.000000] setup_cpuinfo: initialising
[    0.000000] setup_cpuinfo: No PVR support. Using static CPU info from FDT
[    0.000000] cache: wt_msr_noirq
[    0.000000] setup_memory: max_mapnr: 0x8000
[    0.000000] setup_memory: min_low_pfn: 0x80000
[    0.000000] setup_memory: max_low_pfn: 0x88000
[    0.000000] On node 0 totalpages: 32768
[    0.000000] free_area_init_node: node 0, pgdat c02e25c4, node_mem_map c0446000
[    0.000000]   Normal zone: 256 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 32512 pages, LIFO batch:7
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
[    0.000000] Kernel command line: console=ttyS0,115200 root=/dev/q5sda2 init=/init rdinit=/init
[    0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[    0.000000] Memory: 125436k/131072k available
[    0.000000] NR_IRQS:32
[    0.000000] xlnx,xps-intc-1.00.a #0 at 0xc8000000, num_irq=9, edge=0x24
[    0.000000] xlnx,xps-timer-1.00.a #0 at 0xc8004000, irq=1
[    0.000000] microblaze_timer_set_mode: shutdown
[    0.000000] microblaze_timer_set_mode: periodic
[    0.000000] Calibrating delay loop... 49.56 BogoMIPS (lpj=247808)
[    0.190000] Mount-cache hash table entries: 512
[    0.260000] NET: Registered protocol family 16
[    0.270000] bio: create slab <bio-0> at 0
[    0.280000] Switching to clocksource microblaze_clocksource
[    0.290000] microblaze_timer_set_mode: oneshot
[    0.290000] NET: Registered protocol family 2
[    0.290000] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.290000] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[    0.300000] TCP bind hash table entries: 4096 (order: 4, 81920 bytes)
[    0.300000] TCP: Hash tables configured (established 4096 bind 4096)
[    0.300000] TCP reno registered
[    0.310000] NET: Registered protocol family 1
[    2.430000] msgmni has been set to 244
[    2.430000] io scheduler noop registered
[    2.430000] io scheduler deadline registered
[    2.430000] io scheduler cfq registered (default)
[    3.100000] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    3.120000] 21000000.serial: ttyS0 at MMIO 0x21001003 (irq = 6) is a 16550
[    3.340000] console [ttyS0] enabled
[    3.350000] 22000000.serial: ttyS1 at MMIO 0x22001003 (irq = 7) is a 16550
[    3.430000] brd: module loaded
[    3.440000] Xilinx TEMAC MDIO: probed
[    3.480000] eth0: Dropping NETIF_F_SG since no checksum feature.
[    3.490000] TCP cubic registered
[    3.490000] NET: Registered protocol family 17
[    3.510000] Freeing unused kernel memory: 1249k freed
Starting rcS... PID(14)
++ Creating device points
++ Mounting filesystem
++ Starting telnet daemon
++ Enabling ethernet
[    4.360000] net eth0: Promiscuous mode disabled.
udhcpc (v1.16.1) started
sh: can't execute '/usr/sbin/ethtool': No such file or directory
Sending discover...
[    6.700000] PHY: 23000000:00 - Link is Down
[    7.800000] PHY: 23000000:00 - Link is Up - 100/Full
Sending discover...
Sending select for 192.168.200.215...
Lease of 192.168.200.215 obtained, lease time 86400
sh: can't execute '/usr/sbin/ethtool': No such file or directory
deleting routers
route: SIOCDELRT: No such process
route: resolving gw
adding dns 192.168.200.168

rcS Complete.


BusyBox v1.16.1 (2010-04-22 17:12:31 EDT) hush - the humble shell

/ # 

 

 

 

I'm going to do some stupid performance tests now.

 

Joshua

0 Kudos
Highlighted
Adventurer
Adventurer
5,885 Views
Registered: ‎10-28-2007

As it happens, I'd just run some tests on my 100baseT router connected to this hardware last night, using the code from git (2.6.33).

 

My sample code (attached) is pretty dumb:

 

  1. Open file.
  2. Load it into buffer cache [ readahead()]
  3. Open UDP socket and connect it, so I can have a file descriptor I can pass to write.
  4. Break the file up into 1450 byte chunks by calling sendfile() repeatedly.

At the other end I have 'nc' and 'tcpdump' running on the server.  I then get the timestamp for the first and last packets and use it to get an average throughput.  It's not particularly accurate, but at this stage I'm looking for changes on the order of 10Mbps, so I don't care.

 

Anyway, results:

 

2.6.33: ~27 Mbps

2.6.34-rc5: ~11.6 Mbps

 

So is this the difference that checksum off-loading makes?  What are the chances of topping 50Mbps?

 

Thanks

 

Joshua

 

Command lines:

 

Microblaze:

 

 

/ # cd /tmp
/tmp # tftp -g -r sendfile_tx_uB 192.168.200.207
/tmp # chmod a+x sendfile_tx_uB 
/tmp # ./sendfile_tx_uB 
Usage: ./sendfile_tx_uB server port file
/tmp # mknod /dev/urandom c 1 9
/tmp # dd if=/dev/urandom of=rand64k.dat bs=64k count=1
1+0 records in
1+0 records out
/tmp # md5sum rand64k.dat 
31187f52cd26e4fe1617e1046005d43e  rand64k.dat
/tmp # ./sendfile_tx_uB 192.168.200.177 8000 rand64k.dat 
Sent 46 packets, 65536 bytes (out of 65536).
/tmp # ./sendfile_tx_uB 192.168.200.177 8000 rand64k.dat 
Sent 46 packets, 65536 bytes (out of 65536).
/tmp # ./sendfile_tx_uB 192.168.200.177 8000 rand64k.dat 
Sent 46 packets, 65536 bytes (out of 65536).

 

 

Server (some fedora box):

 

 

[jpl@tux ~]$ nc -u -l 8000 > test_20100430_002.dat
^C
[jpl@tux ~]$ md5sum test_20100430_002.dat 
31187f52cd26e4fe1617e1046005d43e  test_20100430_002.dat
[jpl@tux ~]$ nc -u -l 8000 > test_20100430_003.dat
^C
[jpl@tux ~]$ md5sum test_20100430_003.dat 
31187f52cd26e4fe1617e1046005d43e  test_20100430_003.dat

 

 

The tcpdump output:

 

 

[jpl@tux ~]$ sudo /usr/sbin/tcpdump -nn -i eth0 'port 8000'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
17:15:26.306301 IP 192.168.200.215.34471 > 192.168.200.177.8000: UDP, length 1450
17:15:26.307227 IP 192.168.200.215.34471 > 192.168.200.177.8000: UDP, length 1450
.........
17:15:26.352637 IP 192.168.200.215.34471 > 192.168.200.177.8000: UDP, length 286
^C
46 packets captured
46 packets received by filter
0 packets dropped by kernel
[jpl@tux ~]$

 

 

 

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
5,882 Views
Registered: ‎09-10-2008

I can't say for sure, but yes I would presume that this is the checksum offload.

 

I kind of doubt 50 Mbps myself.  The TCP numbers with iperf are lower than your numbers.  The last TCP numbers I saw (unofficially) were less than 20 Mbit, but there was some kernel work being done to try to help that.

 

Thanks for getting it running. I was afraid those are the results you'd see since checksum offload is not in the driver correctly yet.  There is some code, but it doesn't work right from my preliminary testing.

 

 

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
5,880 Views
Registered: ‎09-10-2008

I surprised on that timeout change.

 

What's the speed of your processor and bus clock?

 

Maybe that needs a change in the driver.

0 Kudos
Highlighted
Adventurer
Adventurer
5,876 Views
Registered: ‎10-28-2007

As far as I know the Virtex-4 is a fairly high-speed FX60 (I don't have access to the exact part number right now).  We are running the microblaze at 100MHz.  Perhaps this has more to do with the Soft TEMAC?  As far as I know, the bus and CPU have the same clock speed.  I'll confirm when I next have a look at the mhs file.

 

Joshua

0 Kudos