Embedded Development Tools

[wheeda]

BACKGROUND

I have a system with a MicroBlaze, some custom PLB and AXI cores and an AXI master bridge to an ARM processor on a custom board.  I'm using shared access.

CONCEPTUAL IMPLEMENTATION

The ARM has an nWAIT pin.  This allows me to create a simple statemachine that makes the processor wait until ARREADY and finally RVALID have been asserted.  To create ARVALID and RREADY I used:

ARVALID <= (not nOE) and (not nCS);

RREADY <= (not nOE) and (not nCS);

PROBLEM

Implemented this way results in the ARM hanging because nWAIT is asserted indefinately because ARREADY and RVALID are never asserted.

DEBUG TRIED

Putting chipscope on the MicroBlaze in the system I see that there are about 7 clock cycles (100MHz, not that is should matter) between the assertion of ARVALID and ARREADY.  Similarly there are about 7 clock cycles between the deassertion of ARREADY and the assertion of RVALID.

I tried putting edge sensitive pulse stretchers on ARREADY and RVALID.

I setup timeouts on nWAIT so that the processor wouldn't hang.  I let nWAIT stay asserted for 31 clock cycles.  This caused the expected behavior of a 31 clock wide assertion of ARVALID and ARREADY, but sadly never an assertion of ARREADY, or obviously RVALID.

I got rambunctous,

ARVALID <= '1';

RREADY <= '1';

This is helpful.  The ARM is now able to read from stuff on the AXI bus.  Sadly it seems that the MicroBlaze is hung.  Additionally, I'm pretty sure this isn't going to be a brilliant idea for getting write to work.

Tieing RREADY high and letting ARVALID toggle didn't get data onto the bus, and RVALID never went asserted.

RANDOM OBSERVATIONS

-With ARVALID setup the way they should work, RRESP is 3, or decode error.

-I'm using shared access, which should be fine.  I haven't tried Crossbar recently.

[wheeda]

Here is what ISim looks like with ARVALID and RREADY driven by nOE and nCS.

[wheeda]

I noticed that it works immediately after a reset, but then stops working.  I was guessing that maybe the MicroBlaze in the system wasn't sharing with my external master correctely, so I held the uB in reset.  This didn't help.

Does anyone have recommendations for what to look for with a a hung AXI-lite bus?

[johnmcd]

It's not clear to me if you are designing a master or slave.

ARVALID is a master output and slave input and RREADY is also a master output and slave input. You state that you are tieing both of them high so that makes me believe you are designing a master.

I'm going to take a guess that you are designing a bridge. If that's the case, them I'm a bit worried about:

ARVALID <= (not nOE) and (not nCS);  -- I presume this is slave to master of a bridge

You probably will not have the next address ready so ARVALID should drop once you receive ARREADY.

An easy nomenclature to use is M_ARVALID and S_ARVALID. That way, it's easier to understand if you are talking about the master out or slave in.

[wheeda]

Correct, I'm making an ARM CPU to AXI bridge.  The ARM chip conviently has an nWAIT signal.

I interpreted the following:

"The master can assert the ARVALID signal only when it drives valid address and control information.  When asserted, ARVALID must remain asserted until the rising clock edge after the slave asserts the ARREADY signal."

as, "as long as ARVALID is asserted until ARREADY gets asserted, that is fine".  I previously wondered about my interpretation of this and so I previously tried making ARVALID deassert after ARREADY asserts.  The documentation I've read on the AXI bus has nice diagrams for the assertions on the bus, but not the deassertions.  I've attached a picture of the deasserting on ARREADY behavior, which looks fine.  However it only does this immediately after reset, and then behaves as noted above.

I have a ChipScope monitor on the ARM-AXI bridge part of the AXI bus, and at the AXI bus to a slave.  The slave sees the S_ARVALID signal intitially, but only immediately after reset.  After that M_ARVALID is still being asserted, but S_ARVALID isn't.

It seems like the AXI bus is getting hung up on something, but I can't figure out where.

[wheeda]

I'm currently thinking it is a memory space vs register problem.  A custom slave I made has memory space, but the trouble slave doesn't.  The trouble core doesn't have any address lines coming in, only chip selects for the registers.  Perhaps the handshake lines aren't being propogated through the AXI interconnect when no memory space is detected.

I haven't explored this in detail yet.  This would makes lots of sense if the core just plain didn't work, instead of working immediately after a reset, but then hanging.

[johnmcd]

The axi interconnect relies on address ranges in order to determine how to route accesses from masters to slaves.

[wheeda]

I tried reimporting my IP into an AXI peripheral to see if I had gummed something up the first time.  I'm getting the same behavior.  Immediately after reset multiple reads are sent to the slave.  The slave responds correctly, and the handshaking and data is succesfully passed all the way back to the master. 

After the four reads (as far as I can tell with ChipScope, four makes sense based on code in ARM) the handshaking from the master (ARVALID, RREADY, and ARADDR) never reaches the slave, however I can see them being generated on the master side by my ARM to AXI master bridge.

Communicated differently....

Immediately after reset

M_ARVALID gets through to S_ARVALID, and S_ARREADY gets through to M_ARREADY

M_RREADY gets through to S_RREADY, and S_RVALID gets through to M_RVALID

M_ARADDR gets through to S_ARADDR, and S_RDATA gets through to M_RDATA

After approximately four reads

M_ARVALID stops driving S_ARVALID

M_RREADY stops driving S_RREADY

M_ARADDR stops driving S_ARADDR

I think this behavior means my idea that maybe it was my implementation of the userlogic in regards to registers vs memory space was incorrect is not actually the problem.

It almost seems like there is a reset polarity problem with the AXI bus.  I've wondered previously if the MicroBlaze, that is in the system also, was hogging all the AXI bus (it shouldn't have been since both the uB and ARM are at priority 0 on the shared bus).  I tried holding the uB in reset to test this idea.  That didn't help.

[wheeda]

My debugging of this behavior was reaching a dead end.  An alternate problem I was seeing is that the JTAG console would often not work.  I've read that ChipScope and one of the serial debug methods don't work together easily, so I didn't take this failure seriously.  I decided to debug this problem.  I removed all of the IP from the design except the MicroBlaze and memory and added IP back in one at a time to find the problem IP.  Once I added everything back in, it all worked, except I forgot to update the AXI clock for one of my cores from the default clock name to the system clock I was using.  Once I did this, the design started having the problem where the JTAG console (previously unreported by me problem) didn't work, and the AXI interconnect would pass handshaking signals immediately after reset, but then stop passing handshaking and data.

The system clock is at 100MHz (Spartan 6).  I use 100MHz on the uB as well as the AXI cores.  This causes no timing errors.  Based on the above behavior I decided to run the AXI core in question at 25MHz.  This resolved the problem where the AXI interconnect would only pass handshaking and data immediately after reset.  It also resolved the problem with the console not getting data.

Further Questions:

1.  What could I have done to debug this faster?

2.  Is there documentation that I failed to read that requires the AXI cores to run slower than the uB, or alternately an external master?  My external master is running at 72MHz and connecting to the ARM-AXI bridge with the standard nWR, nOE, nCS, data, address signals. 

3.  This seems like a marginal timing problem this didn't create an error.  What can I do to keep this from happening again as I add additional IP?

[wheeda · Accepted solution]

It appears that the real problem was that my custom AXI master wasn't generating strictly synchronous control signals.  This caused strange behaviors from the AXI bus:

1.  Failing to read from AXI unless ARVALID and RREADY tied high (as described in this thread already).

2.  The AXI bus locking up, such that no transaction from the other master on the bus would happen.

3.  Intracommunication channel skew where there were multiple clocks between WVALID being asserted at a slave and WDATA finally coming through.  This was happening while WDATA and WVALID were timing correctly entering the AXI interconnect.

At this point I don't believe any of this is a problem with any of the Xilinx IP.  These are just symptoms of not strictly synchronous handshake signals being fed to the AXI core.  I'm assuming that my previously accepted solution of slowing the clock down was just a red herring.  I'm recompiling now.

There weren't timing errors while compiling the design, or simulating the custom master.