01-27-2016 01:53 PM
I have recently upgraded to Vivado 2015.4. We run automated builds on a virtual machine. With one project in particular, the synthesis spuriously crashes right after finishing "Part Resource Summary". The last message in the readme.log file is:
INFO: [Synth 8-5580] Multithreading enabled for synth_design using a maximum of 2 processes. Start Parallel Synthesis Optimization : Time (s): cpu = 00:01:15 ; elapsed = 00:01:20 . Memory (MB): peak = 785.145 ; gain = 613.141
I am using a .tcl script to run the synthesis, implementation, and bistream generation. I have two implementation runs within the project, and each uses a pre.tcl command to set a top-level generic value within the shared source files. Initially, we had this configured to run a maximum of 4 simultaneous jobs with max multithreading at the default value of 2 (we have a lot of IP that is nice to run synthesis in parallel using max of 4 jobs). The virtual machine has 4 processors.
I took the following troubleshooting steps and the problem still occurred in all attempted fixes:
1. Changed max number of multithreading processes to one, since this appears to be where the synthesis run dies per the readme.log excerpt above.
2. Following this, I thought maybe it was a problem with the pre.tcl file changing the generic value when the 2nd run starts in parallel with the first one already running (even though source files aren't changing I read that this could knock synthesis out of data for the run that started first). I changed the .tcl script to no longer call a batch run and instead call each of the two runs individually (all the way through bitstream generation before commencing the next run). The same failure was observed. Note I did revert back to multithreading enabled since we should have enough processors available; the only time we utilize all four are during synthesis of IP before in-context synthesis even starts.
Interestingly, when building on local machines via the GUI (where max jobs and multithreading take on default values), we have never had this issue. Also, this problem never occurred in Vivado 2014.2, and only occurs ~50% of the time in 2015.4
It is a tremendous nuisance to have 50% of automated builds fail for an unknown reason. Do you have any insight into what may be causing this and what we can do to mitigate the problem?
I also have crash dump files from the synthesis run that I can provide via email if needed.
01-27-2016 07:31 PM
Can you attach the synthesis log file and crash dumps here?
01-28-2016 07:36 AM
Try running synthesis with different synthesis strategies and check if it helps.
01-28-2016 12:24 PM
I updated all Vivado strategies to the 2015 defaults. I went back to multithreading enabled and max number of jobs set to 4 and still received the same error (this is the same multithreading and number of jobs we were using in 2014.2 with no problems).
The readme.log and dump files are attached.
After this failure, I configured it to run with the updated strategies and no multithreading, one job at a time, and not calling batch runs. I have run two successful builds with no problems so will continue to monitor it with this 'single-job' script. However, we ultimately would like to identify the source of this problem, as increasing the number of jobs and allowing multithreading is desireable to speed up the build time.
I will also attach the build.tcl script we are using to launch the batch runs. We pass in the project directory, name, and maximum number of jobs (4) to this script.
02-11-2016 08:15 AM
One other interesting note about this issue is that it never seems to occur after rebooting the virtual machine. In other words, every time we have a failed build (as described above), we can restart the virtual machine and get a successful build. After that, however, the problem typically returns.
07-31-2016 10:57 PM
I have the same issue with new motherboard (GA b85m-d2v). I updated BIOS. Change all motherboard\cpu features to enable. Even example project (Menu -> Open Example Project -> CPU(HDL)) crashes. But "Base MicroBlaze" example project is ok.
Vivado crashes at:
INFO: [Synth 8-5580] Multithreading enabled for synth_design using a maximum of 2 processes. Start Parallel Synthesis Optimization : Time (s): cpu = 00:00:43 ; elapsed = 00:00:47 . Memory (MB): peak = 935.340 ; gain = 763.188
# # An unexpected error has occurred (EXCEPTION_ACCESS_VIOLATION) # Stack: no stack trace available, please use hs_err_<pid>.dmp instead.
hs_err_pid8744.dmp has 102KB of data.
08-01-2016 05:29 AM
Xilinx was able to help me resolve this issue a few months ago. Per their instruction, I issued the following tcl command before running synthesis:
set_param synth.elaboration.rodinMoreOptions "set rt::enableParallelFlowOnWindows 0"
I do not know the details of what this does, but I have been running for a few months now with no additional problems. We have a tcl script to run the build, so I just simply issue this command at the beginning.