UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Observer amwamw
Observer
11,447 Views

Zynq DDR ECC error count register never incremented

Can some one tell me if the ECC support on the Zynq has been tested and works and/or has any known faults?

 

In particular I am trying to test if it works on the Zedboard and believe I have it enabled (the width set 16bits, ECC enabled).

Booting up the kernel with ecc=on :

....

 

[ 0.000000] parse_tag_cmdline: tag cmdline (@c000011c)=root=/dev/mmcblk0p3 rootwait
[ 0.000000] setup_arch: boot_command_line= ecc=on default_command_line=console=ttyPS0,115200n8 rw earlyprintk lpj=1000 rw root=/dev/mmcblk0p3 rootwait
[ 0.000000] bootconsole [earlycon0] enabled
[ 0.000000] Memory policy: ECC enabled, Data cache writealloc
[ 0.000000] PERCPU: Embedded 7 pages/cpu @c06ad000 s4832 r8192 d15648 u32768

....

I am testing it by not clearing a patch of memory and when I read that memory under Linux I get a fault:

 

 

   [ 3539.090000] Unhandled fault: external abort on non-linefetch (0x1018) at 0x4009eff0

 

I can then write to that location and read back what I wrote with any faults afterwards - so that looks good.

 

**BUT** the register CHE_ECC_STATS_REG_OFFSET at 0xf80060f0 is always zero - i.e. the count of ECC errors is not increasing. 

 

As this is the Zedboard it's running a development chip so if there is a hardware fault it might be fixed in the production version - but I haven't seen any mention of a problem in the Errata.

 

Also does any one have any info on what support there is in the Linux kernel for DDR ECC?

It appears it just generates a fault (perhaps clearing the above register?) but not reporting it anywhere. 

 

Finally if this hardware has been validated can some one tell me the procedure used? As correctable errors are automatically fixed - how do you create them in order  to confirm they are fixed?

I certainly don't want to wait around for a cosmic ray to hit.

Am I supposed to pull out a radioactive source or something?

 

Thanks

 

0 Kudos
38 Replies
Scholar austin
Scholar
11,435 Views

Re: Zynq DDR ECC error count register never incremented

a,

 

You should ask Avnet, or Digilent (as they design and sell the boards).

 

Or, post on zedboard.org

 

Xilinx manufactures the ZC702 pcb, so we could answer questions on what is tested, what is not.

 

 

But, since the zedboard is designed and tested by others, you will need to go ask them.

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
11,426 Views

Re: Zynq DDR ECC error count register never incremented

Thank you Austin for your reply I was glad to see the prompt response.  But unless I am mistaken the DDR ECC error count register (0xf80060f0) is manufactured by Xilinx and they should know if it was tested and worked. If so they will need to have a method of testing it which they could share with a customer to confirm their coding has configured it as well.

 

If not perhaps you could tell me who would know? I suspect AVnet / Digilent have never run ECC on any of their boards and the only support is in the Xilinx EDK/SDK tools.

 

Incidentally I am using the Xilinx FSBL and u-boot to boot up the digilent kernel (with patches in all of them) so it is possible that some interrupt driven code is jumping in and clearing out the register with out reporting it. But I haven't seen any sign of that code. I will be looking at this interrupt code today.

 

What I want from Xilinx is to know if they have tested the ECC support and they can confirm it works (and how I could test it) so that I can use it in my companies product. Or if it doesn't work,  whether we should drop this work on the Zynq chip.

 

Thanks

0 Kudos
Scholar austin
Scholar
11,412 Views

Re: Zynq DDR ECC error count register never incremented

a,


As far as I know, everything has been verified.

 

We just had ARM here yesterday discussing items similar to yours (various registers that are not documented for use by the customer, but used for test).  We do not spend millions of $ on a mask set until the RTL is 100% verified, and then the verification and characterization team has a great deal of work to do when the silicon gets here.  That is almost two years ago, now.


If it isn't in the errata, then it is working.

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
11,406 Views

Re: Zynq DDR ECC error count register never incremented

That's what I would have thought - so how about sharing an example validation code?

 

It doesn't have to compile or even be documented just enough to  show how you prove that ECC is working. Ideally  a

sequence of ECC register read/writes that would demonstrate the basics and I can build on it from there.

 

At the moment:

 

  1. I can't get the ECC error count register to ever change - perhaps because it doesn't work or because the test of reading uninitialised memory won't trigger it. 

 

   2. I always get a data abort - despite registering an interrupt handler for IRQ 92 (listed as Parity / SCU in TRM v1.5 Table 7-3 of Interrupt Chapter).

 

Can Xilinx offer me any help with this?

0 Kudos
Scholar austin
Scholar
11,398 Views

Re: Zynq DDR ECC error count register never incremented

a,

 

Yes.  You may file a webcase (fastest way to get a response).  You may request a visit from your distributor or Xilinx FAE by contacting the distributor, or the local Xilinx sales office.

 

If there is something standing in your way to build your systems (and place orders for our parts) we are all ears!

 

If this is an academic project, then help is requested through your professor who is registered with the XUP.

 

I have asked the verification team here specifically about the DDR ECC error count.  As I said, we have just finished our radiation testing of Zynq, so I am interested to get the answer (even though we did not use the register -- we only used the exceptions (interrupts) to note what failed).

 

I would start by simplifying what is running.  For example, if you are trying to see this under the public linux build, there are so many things that are probably preventing you from seeing it, that the list is too long to even start to describe.  For one, with 96 interrupt types, and 7 exceptions, practically none of those is handled properly by the linux build (it is as small as they could make it, and as generic as possible, dealing with only what it needed to in order to work).

 

http://forums.xilinx.com/t5/PLD-Blog/Zynq-Exception-and-Interrupt-Handling-in-Safety-Critical-Systems/ba-p/300703



Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
11,392 Views

Re: Zynq DDR ECC error count register never incremented

Thank - you.

 

Sounds like a web case is the way forward.

 

No this is not an academic project it is a commercial system. Our hardware team wants robustness and has required ECC and I have gotten pretty much everything demonstrated on the Zedboard and we believe ECC should work if we have the right configuration.

 

I have used the Xilinx tools to make a new FSBL with ECC, patched u-boot (it needs to write to the DRAM - the FSBL DMA initialisation routine appears to fail that operation with "fatal errors").

 

I can read the relevant ECC registers under Linux and they look okay: But the error count never changes and I get a data abort exception (no parity interrupt).

 

I suspect there are some undocumented setup required for proper ECC. e.g. Some enabling for the Parity interrupt and ECC counters OR ECC just doesn't work is not officially supported - (no method for a customer to develop / test an ECC system.)

0 Kudos
Scholar austin
Scholar
11,378 Views

Re: Zynq DDR ECC error count register never incremented

Got the confirmation,


Yes, it has been verified, and it does increment.


We have asked further clarification, and I will post what I hear back.

 

A webcase is always the fastest way to get things solved:  the hotline employees are graded (paid if you will) by how well they handle a case.  That means fast, clear, and solved.

 

So, they are highly motivated.

 

Cases that linger are automatically brought to their manager's attention, and that sort of attention is bad for them.

 

Also, with a webcase, I can go into the system and see all the notes taken on the case, and if necessary, intervene.

 

I act as the Xilinx ombudsman, representing the customer if a customer finds themselves in a situation where they are unhappy, and the support system isn't working.  If you will, I am a lightning rod.  It isn't a recognized position -- I just took it.  In this way I have learned what is working, and what is broken, and helped fix things.  For me, it is part of what a professional at my level should do (so Xilinx may be even more successful).  It will be 15 years this July, and I think I have worked to help Xilinx be successful, and will continue to do so (as long as they will have me).

 

If you wish to discuss the latest SEU testing of Zynq (the ARM Cortex(tm) system has been radiation tested), send me an email at austin@xilinx.com.  We now have FIT rates, AVF, etc.

 

 

 

 

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
11,365 Views

Re: Zynq DDR ECC error count register never incremented

Thanks Austin, I have taken your advice and raised a WebCase #9644504

 

I took the trouble to verify the same issues are valid with out Linux running by accessing the memory under u-boot and got

the same results except under u-boot it hangs as the u-boot doesn't handle the data abort.

 

But writing to the uninitialised location prevents it from happening (you can read back the data after the write) and the various ECC registers as described in section 10.8 Error Correction Code of the TRM have the correct values as dumped out from u-boot.

 

I am guessing not many people are using ECC in their Zynq designs so far, even though the Xilinx tool has an option to generate parameters for that set up.

 

As an additional piece of issue I note that  DDREcc_Init() isn't running to completion (I suspect this is why I had to modify u-boot to initialise all the DRAM for it to work under ECC mode).  Here's some debug from fsbl running with DEBUG enabled showing that the DDREcc_Init() routine isn't working - see the PCAP_DMA_TRANSFER_FAIL message below.

 

Xilinx First Stage Boot Loader
Release 14.3 Apr 10 2013-10:12:30
Devcfg driver initialized
Silicon Version 1.0
et the loopback bit
PCAP MCTRL F8007080: 00000010
...................................................................................................
FATAL errors in PCAP A8131012
PCAP MCTRL F8007080: 00000000
DDR Init done for ECC
Check_ddr_init - wrote 0xAA55AA55 to 0x100000 and got 0xAA55AA55
Check_ddr_init - wrote 0xAA55AA55 to 0x200000 and got 0xAA55AA55
Boot mode is SD
SD: rc= 0
SD Init Done
Flash BaseAddress E0100000
Reboot status register 0x60000000
ImageStartAddress = 00000000
PartitionNumber = 00000000
flash read base addr E0100000, image base 0
image move with partition number 0
mageAddress = 0x0
Partition hdr for 0: 9C0
Header dump:
Image Word Len:0000DA3C
Data Word Len: 0000DA3C
Partition Word Len:0000DA3C
Load Addr:04000000
Exec Addr:04000000
Partition Start:000042A0
Partition Attr:00000010
Section Count:00000001
Checksum:F7FD2C4A
Partition Start 000009C0, Partition Length 0000DA3C
Source addr 00010A80, Load addr 04000000, Exec addr 04000000
Start transfer data into DDR
Get next partition header
mageAddress = 0x0
Partition hdr for 1: A00
Next Header dump:
Image Word Len:00000000
Data Word Len: 00000000
Partition Word Len:00000000
Load Addr:00000000
Exec Addr:00000000
Partition Start:00000000
Partition Attr:00000000
Section Count:00000000
Checksum:FFFFFFFF
There are no more partitions to load
end of partition move, reboot status reg 60000000, Next partition 0
In FsblHookBeforeHandoff function
SUCCESSFUL_HANDOFF
FSBLStatus = 0x1


U-Boot 2012.04.01-svn2293 (Apr 08 2013 - 02:21:42)

U-Boot code: 04000000 -> 040315E8 BSS: -> 04074368
monitor len: 00074368
ramsize: 10000000
TLB table at: 0fff0000
Top of RAM usable for U-Boot at: 0fff0000
Reserving 464k for U-Boot at: 0ff7b000
Reserving 4160k for malloc() at: 0fb6b000
Reserving 36 Bytes for Board Info at: 0fb6afdc
Reserving 120 Bytes for Global Data at: 0fb6af64
New Stack Pointer is: 0fb6af58
RAM Configuration:
Bank #0: 00000000 256 MiB
relocation Offset is: 0bf7b000
WARNING: Caches not enabled
clearing 05000000 (83886080) bytes from 00000000
monitor flash len: 000368F0
Now running in RAM - U-Boot at: 0ff7b000
MMC: SDHCI: 0
Using default environment

In: serial
Out: serial
Err: serial
Setting bi_boot_params=00000100
Net: zynq_gem
Hit any key to stop autoboot: 0

0 Kudos
Observer amwamw
Observer
11,364 Views

Re: Zynq DDR ECC error count register never incremented

Sorry I forgot to include the source code to the problem I am talking about which generates the FATAL error message. It turns out that the failure of the ClearPcap_Status() call in DDREcc_Init() is not checked for an so the rest of that routine is run and it ends up booting up (but requires the u-boot to clear the memory).


u32 ClearPcap_Status(void)
{

volatile u32 StatusReg;
u32 IntStatusReg;

IntStatusReg = XDcfg_IntrGetStatus(DcfgInstPtr);

/* Clear it all, so if Boot ROM comes back, it can proceed */
XDcfg_IntrClear(DcfgInstPtr, 0xFFFFFFFF);

/* Fix for #672779 */
if (IntStatusReg & FSBL_XDCFG_IXR_ERROR_FLAGS_MASK) {
fsbl_printf(DEBUG_INFO,"FATAL errors in PCAP %x\r\n",
IntStatusReg);
return XST_FAILURE;
}
0 Kudos
Scholar austin
Scholar
7,635 Views

Re: Zynq DDR ECC error count register never incremented

a,

 

From the engineer:

 

"The scenario looks okay. In any case if you try to access the un-initialized memory locations you should see non-zero value in the register CHE_ECC_STATS_REG_OFFSET."

 

He did not see your code in your last post, so I will send it to him,

Austin

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Scholar austin
Scholar
7,632 Views

Re: Zynq DDR ECC error count register never incremented

Try:

 

-   try to make Linux not use all DDR; keep some portion of it reserved for the purposes of this experiment

-   run Linux

-   use XMD to connect to the CPU. This will freeze Linux.

-   from XMD do "mrd <uninitialized_ddr_addr>"

-   read CHE_ECC_STATS_REG_OFFSET now

 

The code you posted no one understands.  It is likely is is not doing anything useful.

 

 

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
7,619 Views

Re: Zynq DDR ECC error count register never incremented

Thanks Austin it's good to know my scenario is valid. I can confirm that I don't see this register CHE_ECC_STATS_REG_OFFSET at 0xf80060f0 is always zero hence my concern. In fact I don't think I have ever seen it non-zero....

0 Kudos
Observer amwamw
Observer
7,618 Views

Re: Zynq DDR ECC error count register never incremented

That's a brilliant idea Austin - I boot up Linux with the Xilinx JTAG probe connected and fire up XMD and connect to it.

 

I think you want mrd_phys command as the mrd doesn't work because of Virtual Memory (unless you have a way around that).

 

It demonstrates that the register at 0xf80060f0 (CHE_ECC_STATS_REG_OFFSET) is always zero still.

 

XMD% mrd 0xfffffec
ERROR: Cannot Read from target


        Cannot translate from virtual to physical address
        
XMD% mrd_phys 0xfffffec 1
 FFFFFEC:   DEDEDEDE

XMD% mrd_phys 0xf80060f0 1
F80060F0:   00000000

XMD% mrd_phys 0xffffff0 1
ERROR: Cannot Read from target


        AP transaction error (DP CTRL_STAT=0xfc000020)

XMD% mrd_phys 0xf80060f0 1
F80060F0:   00000000

  Note: I have arranged ECC memory to be initialised up to 0xfffffec but not  the next address 0xffffff0. So I can read the early one but get an error on the uninitialised 0xffffff0.

 

But if I write to the uninitialised memory I can read it happily. Note CHE_ECC_STATS_REG_OFFSET always remains zero...

XMD% mwr_phys 0xffffff0 0xdeadbeef
XMD% mrd_phys 0xffffff0 1
 FFFFFF0:   DEADBEEF

XMD% mrd_phys 0xf80060f0 1
F80060F0:   00000000

0 Kudos
Observer amwamw
Observer
7,617 Views

Re: Zynq DDR ECC error count register never incremented

Hi Austin, I just had the thought to do this under u-boot instead of Linux - and once you get the exception under u-boot xmd doesn't recover - so basically similar results:

 

 

XMD% connect arm hw

JTAG chain configuration
--------------------------------------------------
Device   ID Code        IR Length    Part Name
 1       4ba00477           4        Cortex-A9
 2       03727093           6        XC7Z020

--------------------------------------------------
Enabling extended memory access checks for Zynq.
Writes to reserved memory are not permitted and reads return 0.
To disable this feature, run "debugconfig -memory_access_check disable".

--------------------------------------------------

CortexA9 Processor Configuration
-------------------------------------
Version.............................0x00000003
User ID.............................0x00000000
No of PC Breakpoints................6
No of Addr/Data Watchpoints.........1

Connected to "arm" target. id = 64
Starting GDB server for "arm" target (id = 64) at TCP port no 1235
XMD% mrd 0xfffffec                                                                                                                                                                 
 FFFFFEC:   DEDEDEDE

XMD% mrd 0xf80060f0 1
F80060F0:   00000000

XMD% mrd 0xffffff0 1
ERROR: Cannot Read from target


        AP transaction error (DP CTRL_STAT=0x0ffff090)
        Error Address = 0x0ffffff0, Size = 0x00000004


XMD% mrd 0xf80060f0 1
ERROR: Cannot Read from target


        AP transaction timeout (ACK=0x01, Expected=0x02)


XMD% mwr 0xffffff0 0xbeefdead
ERROR: Cannot write to target


        AP transaction timeout (ACK=0x01, Expected=0x02)

0 Kudos
Observer amwamw
Observer
7,601 Views

Re: Zynq DDR ECC error count register never incremented

Hi no reply today - on holiday?

 

Andrew

0 Kudos
Scholar austin
Scholar
7,583 Views

Re: Zynq DDR ECC error count register never incremented

Nope,

 

I'm here.  I have passed it along, and passed back their suggestion.  They have verified it, and it works.

 

I suggest you file a webcase (in fact, you should have done that first if this is something that is preventing you from getting your design done).  The forums are useful, but there is not time clock on them, and no one has to answer.

 

We discovered that we had not enabled the parity error interrupt on the L1 in some test code we were running recently.  Like most really difficult software bugs, it took over a month to find it.

 

In theory it all works.  In characterization it is all working.  In practice one has to write code (and perhaps depend on other code, too).

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
7,578 Views

Re: Zynq DDR ECC error count register never incremented

I already have a web case - 964504 - still awaiting a response to my offer of three options on how to proceed.

 

I think in practise the only thing that counts is working code and how do we know the code is working if we can't test it ...

 

That said I think you already told me that "people have tested it and it works" but can they tell me how to use the registers to test ECC?

 

Or is that secret magic stuff and they don't think it is needed by Xilinx customers?

 

Even better would be to supply me with their working code and I can check it for my self....

 

 

Thanks

 

0 Kudos
Scholar austin
Scholar
7,571 Views

Re: Zynq DDR ECC error count register never incremented

Email the link to this thread back to support with your case number,

 

If the response is slow (lacking), request the case be escalated.


You are the customer, hence, you are always right.

 

No need to passively wait for a support response!

 

If you are not asking, the person handling your case is likely to assume you found your problem and went away....don't let that happen.

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
7,561 Views

Re: Zynq DDR ECC error count register never incremented

Is anyone working on this issue?

 

I have uploaded a SD card image on the web case 964504 :

https://xapps11.xilinx.com/amdocs_prod/eSupport/eSupportJSP/CaseShow.jsp?id_number=964504

 

which contains details and files that allow you to reproduce the issue under Linux on a Zedboard.

 

So the question remains "How do you verify ECC RAM works on the Zync chip?"

 

 

0 Kudos
Scholar austin
Scholar
6,713 Views

Re: Zynq DDR ECC error count register never incremented

I have asked the case be escalated on your behalf.

 

You could have asked for escalation yourself.

 

I will let you know if I hear anything.

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
6,705 Views

Re: Zynq DDR ECC error count register never incremented

Thank-you Austin.

Could you confirm that the Zync actually implements ECC and not just Parity Error detection?

 

The TRM says the Zynq implements an ARM cortex-A9 but looking at the ARM TRM shows only the A8 with an ECC option where as the A9 only has Parity error dection.

 

e.g.

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0414i/index.html - Cortex-A8 TRM

  - see Section "8.7 Parity and error correction code" - says it can be implemented with ECC support.

 

  **BUT**

 

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388i/index.html Cortex-A9 TRM Section 7.7 "Parity error support" which only talks about an optional parity error support but no mention of Error Correction.

 

So if the Zynq does support it perhaps this is a Xilinx enhancement, which would mean the ARM guys are not going to

be helpful, you would really need to talk with an internal engineer.

 

Thanks

 

Andrew

 

 

0 Kudos
Scholar austin
Scholar
6,697 Views

Re: Zynq DDR ECC error count register never incremented

ECC on DDR DRAM,

 

Parity on L1, L2 and OCM.



Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
6,693 Views

Re: Zynq DDR ECC error count register never incremented

Thanks - I think that ECC is Xilinx addition.

 

If you interested I added a suggestion to the Web Case of how the ECC could be tested once we get the ECC error stats register incrementing. Jangi has raised some sort of CR against it so perhaps we might get this working.

 

0 Kudos
Observer amwamw
Observer
6,685 Views

Re: Zynq DDR ECC error count register never incremented

Jangi on the Web case they are looking at building some code to test this - will keep posted if they

make a break through  - which would be great for me.

 

0 Kudos
Observer amwamw
Observer
6,663 Views

Re: Zynq DDR ECC error count register never incremented

From the Web case I understand there is an issue and they suggest closing the Web case and waiting for the CR.

 

Unfortunately I still don't know any details of what the CR is - software/hardware/configuration/???

Perhaps they think I know or perhaps it's secret - I have asked perhaps I might be lucky and be told something to help me ... one day... :-(

0 Kudos
Scholar austin
Scholar
6,656 Views

Re: Zynq DDR ECC error count register never incremented

Unacceptable,

 

Request that it be escalated.  Tell them you need an answer.  You are the customer, and you are always right.  If you let them close it, they are "off the hook."

 

There are no "secrets,"  if it is broken, we have to post an errata, and tell folks when it will be fixed.

 

I will ping some folks again.  It should not be this difficult to use an error counter....

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Scholar austin
Scholar
6,648 Views

Re: Zynq DDR ECC error count register never incremented

Follow below steps.
 
  • Initialize DDR with ECC from CPU0.
  • Issue read to couple of DDR locations(make sure that requests should reach DDR, these requests should miss the L2 Cache)
  • Read ECC counters from CPU1, since you can't do anything with CPU0.

 

I am being told this works.  An example with code is something you have requested from support, and that may be what is taking the time.  There is no technical issue that anyone is aware of (among the folks who know -- i.e. the designers/verifiers).

 

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos
Observer amwamw
Observer
6,645 Views

Re: Zynq DDR ECC error count register never incremented

Thanks for this Austin, I only just noticed your reply (was expecting a web case response first).

 

I will have to look at this next week.

 

I think I am missing a key part of the puzzle - likely this distinction between reads from one CPU versus another or probably something to do with the L2 cache (which has another ECC level) getting in the way or something.

Perhaps disabling the L2 cache could avoid the issue. Not sure how to read from one CPU versus another via the kernel (runs both CPUs) or uboot (only runs one CPU) .

 

Thanks

 

Andrew

0 Kudos
Scholar austin
Scholar
6,640 Views

Re: Zynq DDR ECC error count register never incremented

a,


Stay tuned!  It appears that all my questions about this may have uncovered an issue!  They are now really looking at it in the design team (the folks responsible for the design).  Seems they were convinced it had all been checked out (box checked in the verification spreadsheet as 'done').  But, when they go to look at exactly what was checked out, they try to repeat it, and it doesn't work....

 

So, I stand by everything I have already posted (as they have told me that is how you do it).  Now it is a questions of them doing it again, and telling us how they did it.

 

And, do not let the webcase folks off the hook, either.

 

 

Austin Lesea
Principal Engineer
Xilinx San Jose
0 Kudos