cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
nrabenold
Newbie
Newbie
4,890 Views
Registered: ‎01-27-2016

Vivado Crashing during synthesis after upgrading to 2015.4

Hi-

 

I have recently upgraded to Vivado 2015.4.  We run automated builds on a virtual machine.  With one project in particular, the synthesis spuriously crashes right after finishing "Part Resource Summary".  The last message in the readme.log file is:

 

INFO: [Synth 8-5580] Multithreading enabled for synth_design using a maximum of 2 processes.
Start Parallel Synthesis Optimization  : Time (s): cpu = 00:01:15 ; elapsed = 00:01:20 . Memory (MB): peak = 785.145 ; gain = 613.141

 

I am using a .tcl script to run the synthesis, implementation, and bistream generation.  I have two implementation runs within the project, and each uses a pre.tcl command to set a top-level generic value within the shared source files.  Initially, we had this configured to run a maximum of 4 simultaneous jobs with max multithreading at the default value of 2 (we have a lot of IP that is nice to run synthesis in parallel using max of 4 jobs).  The virtual machine has 4 processors.

 

I took the following troubleshooting steps and the problem still occurred in all attempted fixes:

1.  Changed max number of multithreading processes to one, since this appears to be where the synthesis run dies per the readme.log excerpt above.

2.  Following this, I thought maybe it was a problem with the pre.tcl file changing the generic value when the 2nd run starts in parallel with the first one already running (even though source files aren't changing I read that this could knock synthesis out of data for the run that started first).  I changed the .tcl script to no longer call a batch run and instead call each of the two runs individually (all the way through bitstream generation before commencing the next run).  The same failure was observed.  Note I did revert back to multithreading enabled since we should have enough processors available; the only time we utilize all four are during synthesis of IP before in-context synthesis even starts.

 

Interestingly, when building on local machines via the GUI (where max jobs and multithreading take on default values), we have never had this issue.  Also, this problem never occurred in Vivado 2014.2, and only occurs ~50% of the time in 2015.4

 

 

It is a tremendous nuisance to have 50% of automated builds fail for an unknown reason.  Do you have any insight into what may be causing this and what we can do to mitigate the problem?

 

I also have crash dump files from the synthesis run that I can provide via email if needed.

 

Thank you,

Nate

0 Kudos
Reply
9 Replies
vuppala
Xilinx Employee
Xilinx Employee
4,861 Views
Registered: ‎04-16-2012

Hello @nrabenold

 

Can you attach the synthesis log file and crash dumps here?

 

Thanks,

Vinay

--------------------------------------------------------------------------------------------
Have you tried typing your question in Google? If not you should before posting. Also, MARK this is as an answer in case it helped resolve your query/issue.Give kudos to the post that helped you to find the solution.
0 Kudos
Reply
nrabenold
Newbie
Newbie
4,823 Views
Registered: ‎01-27-2016

Yes.  The runme.log file and both crash dump files are attached.

 

Thanks.

0 Kudos
Reply
vuppala
Xilinx Employee
Xilinx Employee
4,812 Views
Registered: ‎04-16-2012

Hi @nrabenold

 

Try running synthesis with different synthesis strategies and check if it helps.

 

Thanks,

Vinay

--------------------------------------------------------------------------------------------
Have you tried typing your question in Google? If not you should before posting. Also, MARK this is as an answer in case it helped resolve your query/issue.Give kudos to the post that helped you to find the solution.
0 Kudos
Reply
nrabenold
Newbie
Newbie
4,798 Views
Registered: ‎01-27-2016

Hi-

 

I updated all Vivado strategies to the 2015 defaults.  I went back to multithreading enabled and max number of jobs set to 4 and still received the same error (this is the same multithreading and number of jobs we were using in 2014.2 with no problems).

 

The readme.log and dump files are attached.

 

After this failure, I configured it to run with the updated strategies and no multithreading, one job at a time, and not calling batch runs.  I have run two successful builds with no problems so will continue to monitor it with this 'single-job' script.  However, we ultimately would like to identify the source of this problem, as increasing the number of jobs and allowing multithreading is desireable to speed up the build time.

 

I will also attach the build.tcl script we are using to launch the batch runs.  We pass in the project directory, name, and maximum number of jobs (4) to this script.

 

Thanks,
Nate

0 Kudos
Reply
nrabenold
Newbie
Newbie
4,442 Views
Registered: ‎01-27-2016

@vuppala,

 

 

0 Kudos
Reply
nrabenold
Newbie
Newbie
4,211 Views
Registered: ‎01-27-2016

One other interesting note about this issue is that it never seems to occur after rebooting the virtual machine.  In other words, every time we have a failed build (as described above), we can restart the virtual machine and get a successful build.  After that, however, the problem typically returns.

0 Kudos
Reply
kpiter
Observer
Observer
2,563 Views
Registered: ‎04-03-2013

I have the same issue with new motherboard (GA b85m-d2v). I updated BIOS. Change all motherboard\cpu features to enable. Even example project (Menu -> Open Example Project -> CPU(HDL)) crashes. But "Base MicroBlaze" example project is ok.

 

Vivado crashes at:

INFO: [Synth 8-5580] Multithreading enabled for synth_design using a maximum of 2 processes.
Start Parallel Synthesis Optimization : Time (s): cpu = 00:00:43 ; elapsed = 00:00:47 . Memory (MB): peak = 935.340 ; gain = 763.188

 

hs_err_pid8744.log has:

#
# An unexpected error has occurred (EXCEPTION_ACCESS_VIOLATION)
#
Stack:
no stack trace available, please use hs_err_<pid>.dmp instead.

hs_err_pid8744.dmp has 102KB of data.

0 Kudos
Reply
nrabenold
Newbie
Newbie
2,549 Views
Registered: ‎01-27-2016

Xilinx was able to help me resolve this issue a few months ago.  Per their instruction, I issued the following tcl command before running synthesis:

 

set_param synth.elaboration.rodinMoreOptions "set rt::enableParallelFlowOnWindows 0"

 

I do not know the details of what this does, but I have been running for a few months now with no additional problems.  We have a tcl script to run the build, so I just simply issue this command at the beginning.

 

Nate

0 Kudos
Reply
hmsjopa
Newbie
Newbie
2,350 Views
Registered: ‎08-25-2016

Thanks a lot for the answer.
Seems to help us as well with crashing synthesis on build servers.

 

Regards Jörgen

0 Kudos
Reply