06-27-2008 10:49 AM
I'm having another problem with the TEMAC :-)! Here's the setup :
1> Top level logic wraps an EDK "PowerPC" system (i.e. I instantiate the EDK project in ISE).
2> The device is a Virtex4 FX12 on a Memec mini-module. I am using the LocaLink FIFO <-> TEMAC component.
3> I have to admit I am doing something strange. I added a component called "break" to the Tx side
of the local link bus. This allows my custom logic to stream UDP data directly into the MAC w/o CPU intervention.
4> Tools : I have tried the following with ISE/EDK10.1.1 as well as the "final" releases of ISE/EDK 9.2.
Now, I know #3 might alarm some people, but this is a known working widget I created. I had it working on a Virtex5
TEMAC/LLINK (this system was Microblaze based though). To test this "split", I had the user logic path stream UDP
encapsulated video while Webserver was running. This worked just fine. I don't think it is the "split" (I *really* wish Xilinx
would support something like this, but the person my FAE put me in contact with had not a clue what I was trying to do...
I have seen others request this feature as well). For a test, another engineer is porting my design to a Virtex 5 FX
dev. kit. I pray the V5 FXs aren't the disaster I saw with earlier Virtex 4 FXs. I still wake up at night sweating getting that
PCIe core working in 2005...what was supposed to take 2 months wound up mutating into an 8 month bleeding ulcer.
On the Virtex 4, what I am seeing is my ethernet LEDs flickering on both the mini module and the host, but it seems that
the host is not receiving a single packet! Etherreal (in promiscuous mode) is not reporting any CRC errors or anything. I have to
think that something is being lost due to some unreported timing error or something. The timing reports from ISE indicate that
everything is OK.
Another disturbing thing....Chipscope is not working in this setup. I can insert the cores and probe signals, etc....but when I
go to look at any waveforms, all signals are low. I can set the trigger to '1' on a single signal, hit "wait for trigger", and
get a display with all zeroes :)! Something is goofed up there.
One other question...in the Virtex 4, is the local link bus running @ the PLB frequency, or is it running at the TEMAC
frequency? On Microblaze, it was running at PLB frequency? In my timing report, it looks as if the 125MHz GTX clock
has a rather light load (i.e. it seems to only be driving the MAC and nothing more).
Anyone ever see something similar....even when not using the "split"?
PS, Again, I really wish someone @ Xilinx could hear me out regarding this "split" I am talking about. I'd be willing to send my peripheral to someone there who might be able to help....it'd really give FPGAs an advantage over standalone processor based
systems (i.e. Atmel, etc.) since you can get some significant bandwidth and still have the CPU handle TCP/IP related stuff.
All Xilinx would really need to do is modify the TCL script for the LLTEMAC software generation...that's all. Make it a "use
at your own risk core" since you'd still need an arbiter, etc. to drive the LL bus externally.
06-27-2008 03:26 PM
As a sanity test, I downloaded a WebServer reference design from Avnet for this V4FX12 mini module.
What's strange is that the simple "hello" screen on the console indicates that the link is set to 10Mbps.
The link LEDs indicate 1Gbps.
Now, if I try to access the server in 1Gbps "mode", nothing happens.
If I force my PC to 10Mbps, I can access the Webserver.
So, I know my cables, etc. are OK.
Another engineer is having problems @ 1000Mbps with known working designs that used 9.1, but
broke after migrating to 9.2 and beyond. If he forces the mode to 10/100, we can get data transfers. However, my problem is that
I typically stream video over ethernet, so 100Mbps isn't enough. Moreover, I love how resource utilization is
growing with each release of the tools!!!
Another equally frustrating thing is that I have an older design where I directly instantiate the
MAC myself in my VHDL (as opposed to using this rotten EDK nonsense). I don't configure a
single thing in the MAC aside from Coregen settings....that design works fine on the V4FX12
Has anyone seen anything remotely like this?!?!?! This is downright strange. It's like everything says that
Gig-e mode is OK, but as soon as I go to use ethernet, it is almost as if nothing is happening for the most part
regardless of what the LEDs are telling me! Is there a chance there is some kind of bad setting in the MAC
or something? I did see that the Broadcom PHY is used on the mini module while Xilinx boards like to use
Marvell. The PHY I like the most is the National one on the S3DSP board as you can get data for that one
without a silly NDA....however, we even see 1000Mbps issues on that using the soft core GEMAC (that might
be related to crossover stuff though....I haven't dug into that one as this V4 is in my critical path).
Man this stuff is frustrating anymore :-)! I spend more time fighting with tools than I do my designs. My SOPC
buidler friends are not seeing 1/10th the issues I am seeing with this EDK based garbage....and this is coming
from a Xilinx user of 10 years that downright HATED FPGAs @ the other end of the alphabet until recently.
06-27-2008 04:58 PM
I can answer atleast part of your question: The lwIP autodetect code only works for Marvell PHY's on Xilinx dev boards. If you use a different PHY, then you have to tell lwIP what the speed of the connection is using a parameter. So you should be able to tell lwIP that your speed is 1G, and it should work at 1G. Check lwIP documentation for phy_link_speed parameter.
06-30-2008 09:29 AM - edited 06-30-2008 09:41 AM
Thanks. Unfortunately, I searched for that string in the 3.00.a verison of the document and nothing came up.
The strange thing is that I have hardware that handles these transfers (lwip is not needed) and it will not work on
Virtex 4s using EDK/ISE 9.2 and 10.1 now. With 9.1 everything seems OK, so I guess the initial values in the MAC that
was used in 9.1 are what is needed. My problem now is that I don't have access to 9.1 at all. This is actually impacting all
of our projects that use these Virtex 4 FXs.
Can't Xilinx provide a changelog for each revision of their cores??? I don't mean a hidden one where I have to wade through
search results....I mean a link to a changelog in the documentation or something with MEANINGFUL descriptions of anything
07-01-2008 04:58 AM
MDIO baby learn your phy chip its has delay registers(RGMII) download and print the data sheet. Also if you are going to use Lwip you need 10.1 Lwip likes to crash every 10-26 hours in 9.1 (I used TRECK). Stay away from anything that Xilinx Calls Auto because AUTO never works unless you completely match the one test case that xilinx created. I have 4 phy interfaces and believe me AUTO never worked and as for Gig using RGMII don't focus on that until you get everything else working on your board(run with MII it just works on every build). Every time you make a huge change to something else you need to recalculate the delay setting(Or try planahead). Once you have everything working change to RGMII if you have GMII then it should just work on every build once you figure out MDIO. AVNET along with Xilinx have many example on how to write to the PHY using MDIO.
Learn MDIO cold
GMII/MII IT JUST WORKS
RGMII SUCKS and if you used this interface wait until you finish making changes. But it will work!
07-01-2008 05:47 AM
I am using the mini-module as well and observed similar problems when switching to 1000 Mbps. With 100 Mbps all my tests (including the Memec TEMAC_TX application) worked fine. What scared me most was the fact the temperature of the PHY increased within a few seconds dramatically when connect to a 1000 Mbps network. In fact, you could burn your finger when touching it. Cooling it down using some cooling spray made it work again, but the documentation of the Broadcom PHY did not mention that water cooling is required for 1000 Mbps operation...
We repeated these test with 3 different mini-modules, 2 different baseboards, 2 different host pcs, using a hub: always the same result, the PHY gets hot as soon as the 3 LEDs indicate 1000 Mbps link speed and 20 seconds later the frame transmission fails.
May be somebody at Broadcom has an idea?
07-01-2008 10:28 AM
Thanks for the replies. OK, now I know I am not going crazy :-)! I had this same project running in a Virtex 5 w/o issues (only
difference was Microblaze vs. PPC + the V4 TEMAC vs. V5 TEMAC (and Broadcom vs. Marvell PHYs, but I didn't have to run any
init code at all on the V5).
For the PHY configuration (MDIO) ... I cannot get access to Broadcom data until tomorrow, so I am screwed there for now :)!
As far as the Broadcom PHYs getting hot....try touching the Virtex 4 when all guns are blazing logic wise and you are clocking the
PowerPC @ 200MHz :-)! The Broadcom PHY is probably getting hot seeing that the entire package is the die itself :)! Moreover,
it isn't like gigabit transceivers aren't exactly power friendly.
As a test, I used a slightly modified version of Webserver to run on my platform and it worked, so I now need to take Webserver's
software and merge it with mine. Again, on the Virtex 5, I didn't have to initialize the MAC or the PHY....it just worked.
Also, I did a project where I instantiate the V4 TEMAC in HDL (i.e. no EDK) and drove packets in directly....that worked w/o any
initialization software as well. That was back in 9.1 though.
I'll post what I see next...this is frustrating, but Webserver shed some light on this issue :)!
07-02-2008 07:29 AM
at least I found a 'solution' for all of my problems. Everything works fine now with the TEMAC in 1000Mbps. Have a look on the attached photo. I know. it looks ridiculous and of course it is ridiculous, but adding a small cooler and a fan made the system work. I really can not believe that this the only solution, but it works! I do not loose any more frames and all my applications seem to work OK.
07-02-2008 10:24 AM
I am not a board expert, but I have designed quite a lot of ethernet realted stuff in FPGAs over the years :-)!
I wonder if there is not enough copper plane area to sink all of the heat that the PHY throws off. It really
does get hot!!!! We're running in 10/100 mode now just to save power and the device's life :)! Try touching the
Virtex 4 as well!!!! I can almost light a cigarette off it when running with all guns blazing.
I still cannot drive the local link interface with my custom logic (again, my logic worked fantastic in 9.1, but
flat out doesn't work in later versions of the tools). The downright bloody crazy thing is that I can't even
get chipscope to capture what is occuring on the bus!!!! I can see the nets I'd like to see using the core inserter, but when I
go to capture those nets, everything is returned as zeroes regardless of what I use for triggers, etc. In fact,
I can set a trigger bit to a '1' and Chipscope triggers for some reason...all bits are still 0s....neat tool, huh :)!
It's fun being a debug engineer for Xilinx these days, isn't it? Working for free is fantastic. On any given
project, I seem to spend about 75% of my time fighting with these now horrible tools, and about 25%
getting real work done. Moreover, you can't even attempt to innovate anymore without breaking some
bloated, precanned, underverified and non-regression tested IP!!! Yeah, I know I can file a webcase, but
when I asked about the ethernet split a while ago, I was told that it wasn't supported....great, I cannot use
the FPGA fabric to accelerate packet transfers (on the Tx side only....looking at lwip, Tx packet transfer irqs are
masked anyway)....what good is it to have an FPGA then?
Oh well, time to implement a horrible software/FIFO hack to work around this issue. 5 weeks of runaround has been more
than enough. Maybe I'll just upgrade to ISE/EDK9.1 from 10.1 and be done with this.
I hate doing cheesy hacks to get crap to work, but that seems to be the way of the world anymore.
07-17-2008 12:37 PM
06-24-2009 12:08 PM
Hello, I have exactly the same problem. But for me it's for a big industrial project. I'm in the **bleep**. Do you have a solution? Please help me. Why 1 Gb don't work on minimodule ?
06-24-2009 05:16 PM
I have an Avnet FX12 minimodule running at 1G speeds and that side of things hasn't presented any problems. Perhaps the minimodules are shipping with a later revision of the broadcom chip which isn't as susceptable to heat-related problems as earlier versions. That seems a bit odd though. I have certainly had my hardware pushing out 1Gbps for hours on end and although it and the FPGA gets hot it hasn't frozen. This is under EDK 10.1.3. I didn't have to do anything overly special to get the PHY to work at 1G either - for initialisation I think I'm only calling xemac_add(), netif_set_default(), and netif_set_up(). lwip is set up with a single call to lwip_init(). If using lwip you may want to try the 1.3.0 version which ships with EDK 10.1 since I've heard that in some situations that can work better than the older 1.2.0-based version which is utilised by default. Note that lwip won't sustain 1Gbps throughput - but that's another issue.
In terms of driving the TEMAC directly using a custom locallink implementation you may want to take a look at
Initially I was experiencing similar symptoms to those described in thread 404. The solutions outlined in the fifth post fixed things for me and I subsequently I verified that like others only the second one was really needed in my case. Other posts in this thread suggest that things worked without this hack in earlier EDK versions and that it only became necessary in 10.1. I can't personally verify this, but it seems consistent with what has been described in earlier posts of the present thread. I do not have EDK 11.1 so I don't know if the problem has been addressed in that release.
I realise this might not necessarily solve the issues pertaining to the present topic but I offer it in case it is of assistance in some way.
06-25-2009 02:42 AM
06-26-2009 12:36 AM
I have seen it written that the Locallink interface is extremely picky about timing. If your custom locallink state machine isn't quite right that might give rise to the things you're seeing. Having said that, I followed the spec in sp006.pdf and the notes in the previously linked forum thread and have had no trouble getting reliable communication at 1G speeds. In fact my development system has been sitting next to me all day today and has sustained over 900Mbps for nearly 8 hours now.
I can't comment on timing troubles between the FPGA and the PHY. Our project uses the Avnet minimodule which takes care of these issues for us, so as a result I've never had to look into such things myself. Depending on your PCB layout and routing within the FPGA perhaps there is a timing issue. I suspect it will be difficult to pin down though - you'll probably need a logic analyser or mixed-signal CRO along with the data sheet of the PHY that you're using in your design. You'll then have to compare what the datasheet says it needs with what your system does and see if you can see any glitches. It could take a while to track down. Before going to that extent though I wonder whether it's worth thoroughly checking your locallink logic out since any timing glitch there could also render the TEMAC "dead".
I don't believe it's possible to slow the TEMAC down unless you run the entire link at a slower speed (eg: 100Mbps). Even then though the logic will probably run at full speed so you might not prove anything. You also can't slow the TEMAC clock down or else the data it puts out onto the wire will be out of spec.