UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer kg1
Observer
337 Views
Registered: ‎04-03-2016

The application has run out of memory : Labtools 27-3266

Hi,

I am having an issue when running Vivado 2017.4 in conjunction with the hardware manager to communicate with a System ILA within the FPGA. I will try to explain it to the best of my ability below:

Hardware Description:
The hardware design is quite simple, a DDR4 MIG, a VDMA connected with a FIFO in a loop and a Microblaze for communication between the blocks. My aim is to determine the latency, throughput and responsiveness of the system and hence the ILA is connected to the VDMA MM2S and S2MM, and to the AXIS ports of the FIFO. The data is not a concern for me at this stage and hence I disabled it in the ILA IP to conserve BRAM utilization. The ILA depth is 16384 since I issue multiple transactions and want to see them in full.

Software Description:
In SDK, I simply program the VDMA to read some lines and then write them back.

The issue:
1.I program the FPGA through the SDK, and wait till the program halts at the 'main' breakpoint.
2.I auto connect to the FPGA through the hardware manager and set the trigger for the ILA.
3.I execute the program from SDK,
4.The ILA is triggered but when the Hardware manager starts processing the data from the ILA, the RAM utilization on my PC increases to a 100% (16GB) and due to a lack of RAM it starts disk caching into the C: Drive which also ends up in 100% utilization (256GB SDD PCIe). After some time, around 5 minutes, Vivado issues a warning in the TCL console :
  "The application has run out of memory"
  "Warning: [Labtools 27-3266] Failed processing protocol of interface ... The application has run out of memory"
5.At this stage processing has already elapsed around 5-10 minutes, and this keeps going even up to 15minutes, and often other programs start crashing, such as SDK suddenly closing, or Vivado suddenly closing, or Task Manager suddenly closing. 
6.I am then forced to halt Vivado. Re-open and try again.

I should note that this does not always happen, when everything runs smoothly, the ILA data is displayed within seconds and PC RAM utilization remains nominal (around 30%) - for the same FPGA hardware configuration.

However when this does happen it is very annoying as it disrupts the workflow, and I am also concerned for my system because of the significant C: drive utilization which often reduces the free space on the C: Drive by 20GB.

I know Vivado is RAM hungry, which is why I have 16GB. I also know that more is better, but I do not expect this behaviour whilst using the Hardware Manager since it is only reading and processing a waveform. From the behaviour of the program, it seems that it is some sort of bug in Vivado, which is why I am posting this.

The PC Platform:
OS: Windows 10 Pro 64-bit
CPU: i7-7740X 4.3Ghz 
RAM: 16GB DDR4 

Software:
Vivado 2017.4 and SDK

Hardware:
ZCU104 Development Board : Zynq Ultrascale+

I do not know if any reports can be presented which can substantiate this issue further. If so, please let me know if I can present any more information.

Regards.

0 Kudos
1 Reply
Moderator
Moderator
257 Views
Registered: ‎02-09-2017

Re: The application has run out of memory : Labtools 27-3266

Hi @kg1 ,

 

Thank you for contacting about this issue.

We haven't really seen that issue before. It looks like it could be some issue with Vivado.

The first test I would ask would be to try the latest release of Vivado (2018.2 or 2018.3). There could be an issue with 2017.4 that has already been fixed in the newer versions.

You've mentioned that this issue does not happen all the time. Is it exactly the same design and sometimes you try it and it works and sometimes it doesn't?

Also, when you see such issue, could you plese go into your project folder and collect the following log files:

  • vivado_<pid>.jou
  • vivado_<pid>.log
  • hs_err_<pid>.log
  • vivado_<pid>.str

In addition, could you start the HW Server (in TCL, before opening the HW Manager) wwith the command exec hw_server –Lhwserver.log –ljtag2,slave  in order to capture the JTAG logs.

Finally, if you are able to, could you please share the whole design project? If you don't feel confortable in sharing it publicly in the Forums, I can send you a private message with the e-mail to send it to.

Thanks,

Andre Guerrero

Product Applications Engineer

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos