UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 

Vivado Synthesis Crash Debugging Guide

Moderator
Moderator
6 0 860

To resolve any synthesis crash, the start point should always be to understand which phase of synthesis the crash is occurring in, and if there is any indication from the tool pointing to a particular module, assignment, declaration or inference.

If the below article doesn’t help to resolve your query, please share the hs_pidxxxx.log file generated in the runs folder along with the synthesis log file under the project_name.runs/synth_1/ directory.

There will be some scenarios where the logs will not be sufficient and the RTL design will need to be shared with Xilinx in order to debug the issue further.

Tool Crashes in the Elaboration Phase:

 

This means that the tool crashed while elaborating the RTL design and the synthesis log will look like the following:

    ---------------------------------------------------------------------------------

    Starting RTL Elaboration : Time (s): …

    ---------------------------------------------------------------------------------

    …….

    Abnormal program termination (11)

    Please check '..hs_err_pidxxxx.log' for details

    Parent process (pid xxxxx) has died. This helper process will now exit

    OR

    ---------------------------------------------------------------------------------

    Starting RTL Elaboration : Time (s): …

    ---------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------

    Finished RTL Elaboration : Time (s): cpu = …

    ---------------------------------------------------------------------------------

    RTL Elaboration failed

 

In the synthesis log, you should be seeing a crash related message similar to "Abnormal program termination (11)" although it can be the case that there are no messages. These issues are crash issues.

For the elaboration crash issues, most of the time the issue lies in the RTL or some part of RTL which is not handled correctly in the tool.

For such scenarios, please share your project with us so that we can add a fix in the tool to avoid these issues in the future.

In order to debug such issues at your end, the below steps could be useful:

The goal here would be to narrow down the crash to the "Design Module" then -> "RTL File" then -> "Part of RTL" then -> "Line of RTL Code"

Getting to the RTL file is one of the toughest jobs. To find the problematic module, you can use the following method:

Using the black box method: 

Assume a design structure and hierarchy as below:

Hier.PNG               

The goal here would be to eliminate the other three modules by using the black_box attribute.

We need to assume that one module has a problem.

Assume A has an issue: For this scenario, keep the A module intact and make B, C, and D a black_box in their RTL file:

(* black_box *) module B (

…)

end module

 

(* black_box *) module C (

…)

end module

 

(* black_box *) module D (

…)

end module

 

Run Synthesis and check whether the tool crashes or not.

If the tool crashes, then the issue is either in module A or in its submodules (A11, A12…A21, A22…).

In this case, continue to debug by making A1 active and A2 a black_box and so on.

If the tool is not crashing then try to keep the B module intact and mark A, C, and D as a black_box.

Here is a simple example project:

hier2.PNG

While synthesizing the above hierarchy, the tool crashes with the below error in the synthesis log:

….

….

Parameter break bound to: 8'b11110000

An unrecoverable error has occurred, synthesis canceled.

---------------------------------------------------------------------------------

Finished RTL Elaboration : Time (s): cpu = 00:00:05 ; elapsed = 00:00:07 . Memory (MB): peak = 3278.059 ; gain = 194.688 ; free physical = 20020 ; free virtual = 114523

---------------------------------------------------------------------------------

RTL Elaboration failed

INFO: [Common 17-83] Releasing license: Synthesis

4 Infos, 5 Warnings, 0 Critical Warnings and 1 Errors encountered.

synth_design failed

ERROR: [Common 17-69] Command failed: Synthesis failed - please see the console or run log file for details

INFO: [Common 17-206] Exiting Vivado at Thu Feb 21 23:37:22 2019...

 

As you can clearly see in the log, the crash occurred when the tool was trying to elaborate the design.

In order to debug this issue, we need to start the black boxing of modules.

After trying several combinations, the tool passes synthesis when ps2_to_ascii1 is disabled using the black_box attribute, and similarly, the tool is crashing when only pas2_to_ascii2 is enabled as below:

attribute black_box : string;

attribute black_box of Debounce : component is "yes";

attribute black_box of sinchronizer : component is "yes";

attribute black_box of SIPO : component is "yes";

attribute black_box of Driver : component is "yes";

 

Now, as we have identified the RTL file/module, the goal is to find the part of the RTL which is resulting in a crash.

This can be due to big loop iterations, long and complex assignment, complex bit slicing, or an incorrect coding style/unsupported constructs.

In the above case, the tool was crashing due to an unsupported coding style:

process(…)

begin

case dummy is

when S1 =>

    if rising_edge(clk) then

      ...

        end if;

    end if;

when S2 =>

    if rising_edge(clk) then

       ...

        end if;

    end if;

when S3 =>

    if rising_edge(clk) then

        ...

        end if;

    end if;

when S4 =>

  ...

end case;

end process;


**The same debugging is possible by making an individual module a "top-module" for synthesis to see if the tool crashes.

For example, the above project will crash with the same error when pas2_to_ascii2 is set as the top-module.

Crash in the Cross Boundary Optimization Phase:

 

What is Cross Boundary Optimization?

There are many optimizations that the tool will try to perform on any given design based on the synth_design command.

Block-level settings and attributes can define whether the tool can perform the optimizations "within" the hierarchy (boundary) or across the hierarchies (boundaries).

For default settings of Vivado, where '-flatten_hierarchy' is set to 'rebuilt', the tool will try to flatten the hierarchy and will perform cross-boundary optimizations.

The tool tries to find optimization possibilities across the boundaries and may crash(in the worst case) due to a tool bug or due to a design issue.

In these scenarios, the log can appear as below:

---------------------------------------------------------------------------------

Start Cross Boundary Optimization

---------------------------------------------------------------------------------

Abnormal program termination (6)

Please check 'hs_err_pid5649.log' for details

Parent process (pid 5649) has died. This helper process will now exit

 

To prevent such crashes, one simple work-around would be to disable cross-boundary optimization by changing the global setting -flatten_hierarchy to "none". The disadvantage of using "none" might be a bad QoR and so might it not be acceptable for all designs.

A better way would be to find the level of hierarchy by using the black_box method and then use the "KEEP_HIERARCHY/DONT_TOUCH" attribute on it to disable any cross-boundary optimization within/from that hierarchy.

There is also a possibility that the tool will crash when inferring/optimizing BRAMs.

In such scenarios you can disable the BRAM inference to see if that is the real reason. To achieve this, the -max_bram options need to be set to '0'. If the tool completes synthesis after setting 'synth_design -max_bram' to 0 then the crash was related to BRAM inference.

As disabling the BRAM inference is not an acceptable solution, the user will need to find the module in which the BRAM register exists and based on the customer usage, a "DONT_TOUCH" attribute can be given to the memory register to block the BRAM inference.

Another approach would be to mark that module as OOC with -max_bram as '0' and run synthesis of the top module with the default values, which is -max_bram "-1".

If you are still seeing the crash please post your log files (hs_pidxxxx.log and runme.log) and design details here: Xilinx Forums: Synthesis Board

Crash in Timing Optimization:

 

Vivado is a timing driven synthesis engine, and for all designs the tool will try to see if there are any possible optimizations that can be performed to meet the timing requirements of the design.

There are a few scenarios where the tool can crash with the below log in the Timing Optimization phase:

---------------------------------------------------------------------------------

Start Timing Optimization

---------------------------------------------------------------------------------

Abnormal program termination (11)

Please check 'hs_err_pid15514.log' for details

 

In these scenarios, the first check would be to disable the constraint (.xdc) file and run synthesis. If the tool passes synthesis then the issue might be related to the way the design is constrained (user issue) or the way tool is handling the design constraints (tool issue).

One approach to debugging such issues would be to find the constraint/set of constraints which are resulting in a crash. This can be done by commenting constraints and then enabling them one-by-one to see when the tool crashes again. Once you have the set of constraints which are failing, please refer to UFDM and validate the constraints, its usage, and then modify them accordingly to avoid such crash issues.

If you are still seeing the crash, please post your log files (hs_pidxxxx.log and runme.log) and design details here:

Xilinx Forums: Synthesis Board

Additional important debugging steps and their analysis results:

 

FSM Inference Crash Issues:

We have also seen a few issues where the tool was crashing while synthesizing the FSM, and use of the global setting -fsm_extraction set to "off" helped to resolve the FSM based crash issues.

Crash issues due to stack overflow:

The tool can crash when it runs out of stack memory and might not report anything.

Also, the tool might not even generate a hs_pidxxxx.log in the runs directory.

For stack related crashes, please invoke the Vivado tool with the command below:

 "vivado -stack 2000".

This will help the tool to get more stack memory to perform synthesis, and also to avoid stack related crash issues.

Windows OS Specific Crashes:

There can be Windows-specific crashes for several reasons, so to be sure that the issue you are seeing is not Windows OS specific, please run it once on any Supported Linux OS.

OOC (Out of Context) vs non-OOC flow:

We have seen a few cases where the whole design passes synthesis when the project is set to full non-OOC (no IP and the RTL module as OOC). The opposite case also worked in some scenarios, by generating the output product of the IPs in OOC mode instead of "Global" mode.