UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Scholar jg_bds
Scholar
1,267 Views
Registered: ‎02-01-2013

Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

We have a design that contains 2, 4-GB DDR's, as well as other resources. Our MicroBlaze is configured with 36 address bits and 64 data bits.

We're trying to verify the interfaces using XSDB/XSCT, and we're not getting far. We've tried 2018.2.1, 2018.3, and 2019.1.2. Nothing seems to work. The snaps below are from the 2019 try.

This is the current (preliminary) system:

2019-09-02_15-12-43.jpg

And here are the address assignments:

2019-09-02_15-13-40.jpg

We've added a (64-bit addr/64-bit data) JTAG-to-AXI IP for a sanity check, and created some simple, memory-access commands:

2019-09-02_15-14-22.jpg

We can write to the BRAM using the JTAG, and read it back:

2019-09-02_15-16-16.jpg2019-09-02_15-15-47.jpg

However, we can't access that same memory correctly using the MicroBlaze. In fact, trying to write anything to the memory results in only 0's getting written:

2019-09-02_15-18-21.jpg

What might we be missing?

-Joe G.

 

 

 

0 Kudos
1 Solution

Accepted Solutions
Xilinx Employee
Xilinx Employee
712 Views
Registered: ‎10-08-2010

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

The issue appears to be Windows specific. The C_MASK calculation is done with Tcl in Vivado, and on Windows the Tcl interpreter apparently only has 32-bit integer support in the format command used by the calculation, even when running on Windows 64-bit Operating System.

Previously I only tested on Linux RHEL-7, where Tcl uses 64-bit integers by default.

Currently the only solution appears to be manually setting the C_MASK on Windows.

23 Replies
Scholar jg_bds
Scholar
1,122 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

We've refined the problem down to a simpler one: we cannot access memory that sits above 4GB, using XSCT & a 64-bit MicroBlaze.

We've got a simple design that's now targeted at the KCU105:

2019-10-03_16-05-20.jpg

The MB is configured to have a 64-bit (data-port AXI) address width:

2019-10-03_16-06-32.jpg

The BRAM is placed just above the basic (32-bit) 4GB address space:

2019-10-03_16-07-36.jpg

We can access the BRAM fine using a JTAG-AXI IP:

============================================================================================================

(TCL window transcript, here. The "DDR" labels below are leftovers. The commands actually generate BRAM accesses.)

create_hw_axi_txn rd_txn_DDR0_00 [get_hw_axis hw_axi_1] -type READ -address 0000000100000000
rd_txn_DDR0_00
create_hw_axi_txn rd_txn_DDR0_08 [get_hw_axis hw_axi_1] -type READ -address 0000000100000008
rd_txn_DDR0_08
create_hw_axi_txn rd_txn_DDR0_10 [get_hw_axis hw_axi_1] -type READ -address 0000000100000010
rd_txn_DDR0_10
create_hw_axi_txn rd_txn_DDR0_18 [get_hw_axis hw_axi_1] -type READ -address 0000000100000018
rd_txn_DDR0_18
run_hw_axi [get_hw_axi_txns rd_txn_DDR0_00] [get_hw_axi_txns rd_txn_DDR0_08] [get_hw_axi_txns rd_txn_DDR0_10] [get_hw_axi_txns rd_txn_DDR0_18]
INFO: [Labtoolstcl 44-481] READ DATA is: 0000000000000000
INFO: [Labtoolstcl 44-481] READ DATA is: 0000000000000000
INFO: [Labtoolstcl 44-481] READ DATA is: 0000000000000000
INFO: [Labtoolstcl 44-481] READ DATA is: 0000000000000000
create_hw_axi_txn wr_txn_DDR0_00 [get_hw_axis hw_axi_1] -address 0000000100000000 -len 1 -data 00aabbccddeeff00 -type write -force
wr_txn_DDR0_00
create_hw_axi_txn wr_txn_DDR0_08 [get_hw_axis hw_axi_1] -address 0000000100000008 -len 1 -data 9988776655443322 -type write -force
wr_txn_DDR0_08
create_hw_axi_txn wr_txn_DDR0_10 [get_hw_axis hw_axi_1] -address 0000000100000010 -len 1 -data 7766554433221100 -type write -force
wr_txn_DDR0_10
create_hw_axi_txn wr_txn_DDR0_18 [get_hw_axis hw_axi_1] -address 0000000100000018 -len 1 -data 55aa66cc33ee77ff -type write -force
wr_txn_DDR0_18
run_hw_axi [get_hw_axi_txns wr_txn_DDR0_00] [get_hw_axi_txns wr_txn_DDR0_08] [get_hw_axi_txns wr_txn_DDR0_10] [get_hw_axi_txns wr_txn_DDR0_18]
INFO: [Labtoolstcl 44-481] WRITE DATA is: 00aabbccddeeff00
INFO: [Labtoolstcl 44-481] WRITE DATA is: 9988776655443322
INFO: [Labtoolstcl 44-481] WRITE DATA is: 7766554433221100
INFO: [Labtoolstcl 44-481] WRITE DATA is: 55aa66cc33ee77ff
run_hw_axi [get_hw_axi_txns rd_txn_DDR0_00] [get_hw_axi_txns rd_txn_DDR0_08] [get_hw_axi_txns rd_txn_DDR0_10] [get_hw_axi_txns rd_txn_DDR0_18]
INFO: [Labtoolstcl 44-481] READ DATA is: 00aabbccddeeff00
INFO: [Labtoolstcl 44-481] READ DATA is: 9988776655443322
INFO: [Labtoolstcl 44-481] READ DATA is: 7766554433221100
INFO: [Labtoolstcl 44-481] READ DATA is: 55aa66cc33ee77ff

create_hw_axi_txn rd_Core_00 [get_hw_axis hw_axi_1] -type READ -address 0000000080000000
rd_Core_00
run_hw_axi [get_hw_axi_txns rd_Core_00]
INFO: [Labtoolstcl 44-481] READ DATA is: 0000000010060001 (<- This is the correct response from a read-only status register.)

============================================================================================================

Results using XSCT are problematic:

2019-10-03_16-21-40.jpg

Has ANYONE had success using XSCT (controlling a 64-bit MicroBlaze) to access memory space when there are more than 32 address bits? In 2018.3?

Thanks.

-Joe G.

 

 

 

0 Kudos
Xilinx Employee
Xilinx Employee
1,077 Views
Registered: ‎10-04-2016

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi @jg_bds ,

That first access in XSCT to 0x8000_0000 worries me. The returned data implies that the AXI Interconnect cannot decode and forward the address to the proper MI.

I'm wondering if M_AXI_DP is outputting the address that you are specifying. It looks like you have ILAs on the interfaces of the interconnect. Could you post a waveform of what you see there?

Have you tried running a little C-program on the MB processor that access 0x8000_0000 and/or 0x1_0000_0000 to see if the behavior is the same there? I'm not certain if this is a XSCT problem or a MB configuration problem. 

Regards,

Deanna

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Scholar jg_bds
Scholar
1,032 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Deanna,

Thanks for looking into this for us.

We've whipped-up a modified "Hello World", that reads bytes from the module at 0x8000_0000 and then prints-out values to the console. Those AXI transactions appear to work correctly:

2019-10-07_12-00-13.jpg

A comparable read using XSCT, however, does not work correctly--the address bits above A[31] are all 1's:

2019-10-07_12-02-33.jpg

Unfortunately, our code is much more complicated than the "Hello World" program, and we rely on XSCT to debug it.

-Joe G.

0 Kudos
Xilinx Employee
Xilinx Employee
1,017 Views
Registered: ‎10-04-2016

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi @jg_bds ,

Yikes, the XSCT path is filling the upper DWORD of the read address with 1s. No wonder you are getting the DECERR. 

If you do a mrd 0x0000000080000000, do you see the same failure? I'm suggesting this as a workaround for now, not as the "right" way to use XSCT.

Regards,

Deanna

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
Scholar jg_bds
Scholar
998 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

Note that our system has been tweaked a bit, to use only 36 bits of address now. (64 MicroBlaze address bits was just a trial configuration, arrived-at whilst flailing around trying to characterize our problem.)

Putting an explicit 0 in front of the 32-bit address 0x8000_0000 had no effect. This is what we got with an XSCT command of "mrd 0x080000000":

2019-10-07_16-34-20.jpg

There was an interesting development, though. We moved the structures that had been located at addresses 0x8XXX_XXXX, to addresses 0x4XXX_XXXX. (Another BRAM was also added at 0x8000_0000, which was targeted in the access, above.) That resulted in some successful AXI accesses:

2019-10-07_16-32-58.jpg

So it appears that bit 31 of the read address is "sign-extended" through the remaining higher address lines, when a transaction is placed on the MicroBlaze's M_AXI_DP interface.

We still cannot access anything above 0xFFFF_FFFF using XSCT. An XSCT read to location 0x1_0000_0000 doesn't even generate a M_AXI_DP bus cycle.

This dilemma is closer to our original problem--which dealt with accessing locations (i.e., DDR memories) above the normal 32-bit address space.

-Joe G.

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
963 Views
Registered: ‎10-08-2010

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Here is a simple example for KCU105 board with Vivado 2019.1 that works for me. You can implement the design with:

vivado -mode batch -source project_kcu105_2019.1_bd.tcl.

I have sucessfully manually accessed the AXI BRAM at 0x1_0000_0000 in xsct and run the memtest application in SDK (a fix is necessary to print the full 36-bit address, see the attached memorytest.c).

 

956 Views
Registered: ‎07-23-2019

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

I'd say there is some 32-bit type somewhere that gets cast to 64-bit and extends bit 31

0 Kudos
Scholar jg_bds
Scholar
916 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

Thank you all for the input.

One thing that we've realized with these latest suggestions, is that EVERYTHING should be brought-up to at least 2019.1: Vivado, XSCT and (here's the tricky part:) the firmware on our SmartLynq. It appears that as we've bounced around from 2018.3 to 2019.1(.2) and back, our problems have sadly just morphed into other problems. The good news is that while 2018.3 might have its problems, we can avoid them all by staying with 2019.1--and dealing only with its problems.

Unfortunately, 2019.1 XSCT still isn't working for us.

We've tried the design provided by @stefana, above. This is what we encountered:

2019-10-08_16-52-59.jpg

You can see that the accesses targeted at the high-address BRAM, appear to be diverted to address 0--the MicroBlaze's instruction memory BRAM.

To confirm this assumption, we added a second BRAM--addressed at 0x2_0000_0000--along with an ILA.

2019-10-08_17-16-15.jpg

Here are the results of trying to access the 2 BRAMs:

2019-10-08_17-13-53.jpg

NOTE: The ILA was set to trigger on AWVALID (of the MicroBlaze's M_AXI_DP interface) going high. It didn't trigger for any of the mwr commands, above.

-Joe G.

 

 

0 Kudos
Scholar jg_bds
Scholar
905 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

Yeah, I know...  I dislike speculation, too.

--------------------------------------------------------------------

So to be sure, we built the design again, with another ILA on the D-LMB to the MicroBlaze's Instruction/Data BRAM.

We ran a write to the BRAM at 0x1_0000_0000:

2019-10-08_18-34-38.jpg

And a write transaction occurred on the LMB:

2019-10-08_18-33-48.jpg

Notice the LMB address is 36 bits wide--showing the complete, "correct" address--despite the BRAM being only 128 KB.

Nothing happened on the MicroBlaze's M_AXI_DP interface.

-Joe G.

 

 

0 Kudos
Xilinx Employee
Xilinx Employee
894 Views
Registered: ‎10-04-2016

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi @stefana,

Did you use SmartLynq with your KCU105 design?

Hi @jg_bds ,

Can you post the *.xci file for your MicroBlaze? I'm not sure what to make of XSCT forwarding the BRAM accesses to LMB.

Just wanted to check: when you run C code on your MicroBlaze, the accesses are still decoding properly and going to the correct targeted memory? These issues are still only occuring with XSCT.

Regards,

Deanna

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
Scholar jg_bds
Scholar
872 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

@demarco,

Sorry if my previous post was confusing.The SmartLynq wasn't used with the KCU105; that board has a built-in USB-JTAG device.

We're concurrently checking the test-changes we're making on the KCU105, on a system we're developing--the one our original problem was encountered on--and we use a SmartLynq on that one. It's on that system that we discovered that behavior changes depending on the firmware on the SmartLynq.

The .xci for the MicroBlaze is attached. It hasn't been changed since it was generated using the TCL script from @stefana.

To be honest, I can't remember how well (or not) our software was accessing DDR. (See way above; we have 2, 4-GB DDR interfaces that the MB must access.) The last time we worked on the SW was almost 6 weeks ago. I recall that something was going wrong, which led us to try XSCT, which (unfortunately) raised more questions than it answered--and led to this forum post and an SR.

I should be able to roust a SW resource tomorrow and re-visit that.

-Joe G.

 

0 Kudos
Xilinx Employee
Xilinx Employee
825 Views
Registered: ‎10-08-2010

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi @jg_bds ,

The behavior you see would occur if the DLMB BRAM Controller is not correctly configured, in particular if the C_MASK parameter is incorrectly set. This parameter is used by the controller as a mask for the address bits. If the controller determines that the address is intended for LMB, it will respond and no access will be done on M_AXI_DP. The parameter should be automatically calculated, and should have the value 0x101000000.

Did you actually get this behavior after simply implementing my design with no other changes?

Could you post the file "bd.tcl" produced by the Tcl command "write_bd_tcl bd.tcl" in Vivado for the design?

0 Kudos
Scholar jg_bds
Scholar
812 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

@stefana,

The bd.tcl has been attached for both the modified design, and a reconstituted virgin design.

We believe got the same behavior with and without modifying your original design. See above; initial accesses to 0x1_0000_0000 appeared to be aliased to 0x0--which is why we added items to the design, to investigate that.

Are you saying your system behavior is different than ours? If so, are you also running 2019.1.2?

The design has been modified only as stated above, in two tranches: an additional AXI BRAM and a System ILA IP were added, and then another System ILA IP was added:

2019-10-09_7-45-01.jpg

The original TCL was downloaded here at ~11:30A on 10/8. You can see the two, latter design-change tranches in the date-sorted IP directory listing:

2019-10-09_7-40-34.jpg

The two LMB BRAM Controllers' MASK fields (grabbed below from the current, modified design) look different from each other, and from your expectations:

2019-10-09_7-50-10.jpg

2019-10-09_7-57-59.jpg

A new, virgin design was created using the original TCL file that you uploaded, and the LMB BRAM Controllers' Mask fields are identical to those shown above.

So are you saying that all MB data fetches go to the LMB first, and unsuccessful ones get re-tried on the M_AXI_DP bus? That seems rather inefficient. I would have expected the MB to have a better understanding of where to find resources on its different data interfaces.

-Joe G.

 

0 Kudos
Xilinx Employee
Xilinx Employee
800 Views
Registered: ‎10-08-2010

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

I used 2019.1, but I get the same (correct) behavior with 2019.1.2.

If I unzip and implement the design in bd_tcl.zip with "vivado -source bd.tcl", the DLMB BRAM Controller C_MASK parameter is set to 0x301000000, which is the expected value, since you also have an AXI BRAM at address 0x2_0000_0000. The reason for your issue is the incorrect C_MASK.

I have no idea why the automatic calculation is not working as expected in your case, but you can fix it by setting it manually (click on the AUTO button and then enter 0x301000000 in the dlmb_bram_if_cntlr IP customization dialog).

Data fetches are simultaneously tried on LMB and in the data cache (if you have one). If LMB or the cache does not claim the access, an M_AXI_DP fetch is initiated the next clock cycle.

0 Kudos
Scholar jg_bds
Scholar
760 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

It's interesting that the C_MASK attribute in the test design has been adjusted by Vivado to account for the devices addressed below 0x1_0000_0000 (i.e., 0x8100_0000 and 0x8300_0000 yield 0x0100_0000 mask bit), but not above (for 0x1_0000_0000 and 0x2_0000_0000).

Changing the C_MASK attribute by hand makes the operation of the KCU test design work now. We'll next try that change on our original development system.

After that, I suppose we need to figure out why that attribute is not being updated properly by Vivado on our build computer--so we don't have to manually update that field all the time. I don't suppose you notice anything 'wrong' with the versions we've installed--on a Windows 10 Pro 64-bit machine?

2019-10-09_13-39-33.jpg

-Joe G.

 

0 Kudos
Explorer
Explorer
736 Views
Registered: ‎05-04-2014

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi,

I am also facing the same problem with 64-bit Microblaze. I tried to access bram and mig , but it failed. According to the previous reply, it seems C_MASK parameter may be incorrect. How to set proper C_MASK parameter on my case?

 

擷取1.PNG

C_MASK.PNG

 

Sitting

0 Kudos
Scholar jg_bds
Scholar
722 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

@sitting,

If the C_MASK attribute is used as we've been told above, I'm not sure why its value is not simply the inverse of the HIGH_ADDRESS of the dlmb_bram_if_cntlr. In your case, I would expect it to be 0xFFFF_FFFF_FFFF_0000--so any access above the 64K BRAM gets 'masked'.

Your current value is 0x20_0000, which is sufficient to block-off accesses to the axi_uartlite_0 module--your sole, 'low-address' module at 0x4060_0000. (The "2" within your current value represents the least significant of the "1" bits in the address "406".)

Now you'll need to identify all of the high-address blocks, and update the C_MASK attribute. Based on your currently used addresses, you'll need to manually change your C_MASK attribute to 0x0000_0014_0020_0000. The new portion "14" is the unique-bit combination necessary to identify accesses to adresses 0x4_0000_0000, 0x10_0000_0000 or 0x12_0000_0000.

BTW: Which version of Vivado are you using? (Updates, OS, etc.?)

-Joe G.

0 Kudos
Scholar jg_bds
Scholar
718 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

Here are a couple more data points:

1. We uninstalled 2019.1 from our build computer (Windows 10 Pro, 64-bit, Version 1903, 64 GB), and then re-extracted the installer and re-installed only 2019.1--without any updates.

Running the original script (project_kcu105_2019.1_bd.tcl) on a fresh, new KCU105 project, we still get a corrupted C_MASK in the dlmb_bram_if_cntlr module.

2. We downloaded a fresh version of the installer archive on a second computer (Windows 7 Pro, 64-bit, SP1, 48 GB), extracted the installer and then installed 2019.1 for the first time--again, without any updates.

The results were the same: bad C_MASK.

-Joe G.

 

Xilinx Employee
Xilinx Employee
713 Views
Registered: ‎10-08-2010

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

The issue appears to be Windows specific. The C_MASK calculation is done with Tcl in Vivado, and on Windows the Tcl interpreter apparently only has 32-bit integer support in the format command used by the calculation, even when running on Windows 64-bit Operating System.

Previously I only tested on Linux RHEL-7, where Tcl uses 64-bit integers by default.

Currently the only solution appears to be manually setting the C_MASK on Windows.

Scholar jg_bds
Scholar
677 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

@stefana,

Thank you for helping get to the bottom of this issue.

Please add a reply to this thread when an AR becomes available.

Hopefully, the AR will contain a description of the methodology needed to manually update the C_MASK attribute in Windows designs. I was 20%-guessing when I replied to @sitting, above. :-)

-Joe G.

0 Kudos
Scholar jg_bds
Scholar
673 Views
Registered: ‎02-01-2013

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

 

@demarco,

Just to follow up: I haven't been able to scrounge-up a software person to check-out our old code, but I did manage to tweak a Hello World program enough to show that the problem @stefana identified occurs with (2019.1) MicroBlaze application code, too.

It's not just an XSCT thing--like some of the other issues we experienced when using 2018.3 were.

-Joe G.

 

0 Kudos
Explorer
Explorer
516 Views
Registered: ‎05-04-2014

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

Hi, 

My system is 64 bits win10 19H1 and vivado is 2019.1.  After I changed system to linux 16.04.06 and recreated my test project, there was no problem now.

Thank you so much!

 

Sitting

0 Kudos
Xilinx Employee
Xilinx Employee
293 Views
Registered: ‎10-04-2016

Re: Can't properly access memory using a 64-bit MicroBlaze

Jump to solution

There is a patch available to resolve this issue. Please see this AR:

https://www.xilinx.com/support/answers/72944.html

 

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------