cancel
Showing results for 
Search instead for 
Did you mean: 

Using the Report QoR Assessment Command

Xilinx Employee
Xilinx Employee
4 0 598

What is Report QoR Assessment?

 

Report QoR Assessment (RQA) details how you can achieve your design’s QoR goals. It will analyze methodology, as well as characteristics of the design and give you the following details:

  • A score between 1 and 5 detailing how likely you are to meet the design's QoR goals
  • Whether you need to correct methodology issues impacting QoR
  • Whether you should use Report QoR Suggestions to enhance the design
  • Flow guidance recommending when to take advantage of tool features such as ML Strategies or Incremental compile

It is a text based command that can be run at any stage of the implementation flow in the Tcl console or within scripts after synthesis.

Assessment Score

The assessment score predicts how likely a design is to meet its QoR goals. It can be generated at any implementation phase but as it is forward looking it is intended to be used before a design is fully routed and has the highest value when generated post opt_design.

The assessment score is broken down into 5 levels:

Assessment Score

Prediction

1

Design will not complete implementation

2

Design will complete implementation but will not achieve timing goals

3

Design runs have a small chance of success

4

Design should achieve goals if run with a few directives

5

Design will easily meet timing

 

The following table details the accuracy of the score for 41 designs:

 RQA_1.png

It can be seen when we compare the opt_design score to the route_design score:

  • 59% of designs are predicted accurately
  • 98% are predicted to within +/- 1

This level of accuracy allows us to guide the user to run further steps in the implementation flow or for example, use Report QoR Suggestions to improve the design and improve the likelihood of meeting timing.

After place_design, the assessment score is increasingly accurate:

  • 76% of designs are predicted accurately
  • 98% are predicted to within +/- 1

As you can see if further accuracy is required you can run the extra place_design step, but it should be noted that there is less benefit to the command when run at this stage.

The assessment score can be used to determine if you should apply more effort improving the netlist or running the implementation tools.

The following table outlines typical next actions:

Assessment Score

Typical Next Action

1

Redesign HLS modules

Review design features

Review target part / speedgrade

2

Review constraints

Review HLS/RTL modules

3

Review clocking and design.

Use report_qor_suggestions

Run ML Strategies

Use incremental compile for final mile timing closure

4

Use report_qor_suggestions

Run ML Strategies

Use incremental compile for final mile timing closure

5

Run implementation

 

The QoR Assessment score can be improved using  Report QoR Suggestions (RQS) but not for all designs. In order to move a design from score 2 (or low 3), a more significant effort might be required such as optimizing HLS modules / HDL recode / IP configuration changes etc. 

If we focus solely on the implementation tool flow, the impact of applying automatic QoR suggestions on the RQA score is positive for many designs. After running Report QoR Suggestions, the impact on the RQA score can be seen in the following graph.

Snag_514c73ea.png

Whether a design’s score improves or not can be down to the type of suggestion, the number of suggestions and the number of paths impacted.

Although not all designs improved enough to move the score, all improved their design performance characteristics such as WNS or congestion, so it was a step in the right direction. Clocking and congestion based suggestions show the largest improvements. 

Methodology Report

When you make a design improvement but QoR regresses, you are presented with a difficult decision. Should you stick with the fix or regress and find new solutions to the problem?

Here is a good tip… fixing methodology issues makes this task easier.

For more details on this claim, check out this blog Solving Methodology Issues Makes Timing Closure Easier 

There is a cut down version of Report Methodology integrated with RQA. The RQA report only details failing methodology checks that will impact QoR and tool variation. For a full report, run the report_methodology command.

Using the QoR Assessment Report with QoR Suggestions

The QoR Assessment Score is generated by looking at a number of lower level design metrics and forming an overall design score. Even when using automated suggestions, if you need to improve the design score, it helps to see a breakdown of the issues in the design and see how you improve from one iteration to the next.

The details table is broken down into the same categories as the Report QoR Suggestions (RQS) command. It works really well by putting a summary of design issues together in one place. In addition, there is a status column showing which areas must be reviewed and what the ideal thresholds to meet are.

The following figure is an example of the QoR Assessment Details table:

Snag_4e1dced7.png

An item is marked for REVIEW whenever the threshold is exceeded. The threshold numbers are not hard limits but guidance. They offer insight into when you can expect a design to start seeing QoR degradation. If you are a little over the threshold on a single item you can expect your design to show little impact. However, if many factors are a little over, or if any single issue is significantly over the threshold you will almost certainly see issues.

The QoR Assessment Details table is also a handy overview you can use to examine resource changes after you have used QoR Suggestions to improve the design. Its concise nature makes comparing it side by side with a previous version very easy.

In the QoR Suggestions report, you will find suggestions organized such that the ones at the top have the highest impact on the RQA score for your design. By comparing this table with the suggestions, you can see where the tools will try to improve the design.

Flow Guidance

Flow guidance is given by RQA detailing what a user should do next. It is great for novice users, but it can also be effective for experienced FPGA designers.

Typically it is great for catching:

  • Methodology violations that have not been addressed
  • ML Strategies or Incremental Compile as users might not be familiar with these flows and can overlook them
  • Identifying when report_qor_suggestions (RQS) should be used

Flow guidance is given in the Overall Assessment Summary table. Here is an example of the report:

Snag_4e1f9639.png

Flow guidance will determine if the design needs more methodology fixes, if critical implementation suggestions need to be implemented or whether it is ready for the ML Strategy or Incremental Compile implementation flows. None of the items are hard requirements or prohibited, but the QoR metrics of the design are unlikely to meet your desired goals if it is not compatible.

For a design to be ML Strategy Compatible, the following criteria must be met:

  • Implementation is complete and run with the following stages: opt_design, place_design, phys_opt_design, and route_design
  • Design is run with all directives set to Default or all directives set to Explore.
  • Critical design modifications have been made. If the design is not compatible and the above criteria has been met, you should run RQS to identify these.
  • Supported families are UltraScale and UltraScale+

For a design to be Incremental Compile compatible, it should:

  • Be within striking range of timing closure. WNS > -0.500 ns
  • Have an RQA score of 4 or 5
  • Have some incremental friendly RQS suggestions

Supported families are UltraScale and UltraScale+

Note: there are critical paths that the incremental flow cannot solve, for example fixed cascade paths in DSPs/BRAMS.

The next recommended flow stage looks at all of the information and decides what is the best course of action for the user. When a design is both Incremental Compatible and ML Strategy Compatible, the tool will recommend the best option to you.

Summary

In this blog we have shown how you can use Report QoR Assessment to identify how likely your design is to meet timing and where you can improve it.

We have demonstrated how the Details Table is a concise handy overview of the design and finally how to use the Flow Guidance features to get the most out of Vivado’s tool flows. 

An example of using this on an actual design will be shown in our next blog:

https://forums.xilinx.com/t5/Design-and-Debug-Techniques-Blog/Design-Closure-with-RQA-and-RQS/ba-p/1118594