cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Visitor
Visitor
390 Views
Registered: ‎03-22-2019

System Verification Methodologies - Tools & Capabilities

Jump to solution

Hi all,

I need to verify a system of 2 or more FPGA boards (VCU118). I'm totally new to verification, so I have a few questions:

1) Apart from UVM, UVVM and OSVVM, are there any other verification frameworks?

2) Is it possible to migrate a design from Vivado that uses Xilinx IP cores and import it in GHDL or Modelsim to verify it?

3) Are there different capabilities between these three methodologies? I.e. is some superior to others?

 

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Scholar
Scholar
314 Views
Registered: ‎08-01-2012

Usually, you would verify smaller blocks and then do the high level verification on the board itself as this is a much faster process. Are you expecting to do simulation on an entire two board system? or can you do unit tests?

To answer questions directly

1. The other main ones will be VUNIT and CocoTB. All the listed frameworks and methodologies are based in differenct languages:

UVVM and OSVVM are VHDL verification frameworks

UVM is written in SystemVerilog

CocoTB and VUnit are python based with Verilog and VHDL libraries.

Some can easily interplay with others. Eg. you can easily use VUnit and OSVVM together, because they may provide different capabilities in certain areas.

I recommend you read up on verification. Simply adopting a mothodology if you dont have an understanding of verification wont get you very far. All the frameworks apply similar methodologies and it is this theory you need to understand. Which one you adopt can hugely depend on what code skillsets you have. All of them will require a more software based coding approach. While they are all open source, the abilities of tools to use them will vary. GHDL is VHDL only so that immediately rules out UVM. It should work just fine with OSVVM and UVVM and likely Vunit too. The issue with VUnit is that it will also require some python skills. UVVM and OSVVM can be used with purely VHDL knowledge.

While modelsim will support all of them, UVM will not come for free. Usuaully UVM support comes with a higher tier of licence. While Vivado does now support UVM, the issues I have read seem to imply support is not great.

2. If you can output a VHDL netlist, then you can export it to GHDL. Modelsim should support mixed language. Many Xilinx IPs are moving to verilog only output.

3. All of the methodologies provide tools, to provide excellent levels of verification and coverage. It will really depend on what your goals are and how much time you are willing to learn. IMO:

- UVM is a huge task to learn and get set up in. Many companies have a dedicated team of verification engineers who write pure UVM tests. I would expect 1-2 years training before being proficient. While some FPGA using companies have taking to using UVM for verification, a lot are taking up UVVM and OSVVM. UVM has traditionally been used in ASIC verification.

- OSVVM provides tools like a tool box - you can add as many or as few items as you like and easy to integrate parts bit by bit into existing testbenches. It does provide a recommended way to create a test framework but all the tools can be used without doing this (Note: I am mainly a user of OSVVM).

- UVVM is a bit more heavyweight than OSVVM. It requires you to set your testbenches up in a specific way (a bit like UVM) but is very powerful. It provides a lot of similar features to OSVVM.

I have little/no experience with VUnit and CocoTB. My understanding is that VUnit can also provide some CI features as well as verification.

If you are totally new, then all of these tools can be rather overwhelming. I highly suggest you look into the basics of testbenches first. There are many tutorials/examples if you search. Writing using the above frameworks will require a change in mindset from pure HDL development.

View solution in original post

3 Replies
Highlighted
Scholar
Scholar
315 Views
Registered: ‎08-01-2012

Usually, you would verify smaller blocks and then do the high level verification on the board itself as this is a much faster process. Are you expecting to do simulation on an entire two board system? or can you do unit tests?

To answer questions directly

1. The other main ones will be VUNIT and CocoTB. All the listed frameworks and methodologies are based in differenct languages:

UVVM and OSVVM are VHDL verification frameworks

UVM is written in SystemVerilog

CocoTB and VUnit are python based with Verilog and VHDL libraries.

Some can easily interplay with others. Eg. you can easily use VUnit and OSVVM together, because they may provide different capabilities in certain areas.

I recommend you read up on verification. Simply adopting a mothodology if you dont have an understanding of verification wont get you very far. All the frameworks apply similar methodologies and it is this theory you need to understand. Which one you adopt can hugely depend on what code skillsets you have. All of them will require a more software based coding approach. While they are all open source, the abilities of tools to use them will vary. GHDL is VHDL only so that immediately rules out UVM. It should work just fine with OSVVM and UVVM and likely Vunit too. The issue with VUnit is that it will also require some python skills. UVVM and OSVVM can be used with purely VHDL knowledge.

While modelsim will support all of them, UVM will not come for free. Usuaully UVM support comes with a higher tier of licence. While Vivado does now support UVM, the issues I have read seem to imply support is not great.

2. If you can output a VHDL netlist, then you can export it to GHDL. Modelsim should support mixed language. Many Xilinx IPs are moving to verilog only output.

3. All of the methodologies provide tools, to provide excellent levels of verification and coverage. It will really depend on what your goals are and how much time you are willing to learn. IMO:

- UVM is a huge task to learn and get set up in. Many companies have a dedicated team of verification engineers who write pure UVM tests. I would expect 1-2 years training before being proficient. While some FPGA using companies have taking to using UVM for verification, a lot are taking up UVVM and OSVVM. UVM has traditionally been used in ASIC verification.

- OSVVM provides tools like a tool box - you can add as many or as few items as you like and easy to integrate parts bit by bit into existing testbenches. It does provide a recommended way to create a test framework but all the tools can be used without doing this (Note: I am mainly a user of OSVVM).

- UVVM is a bit more heavyweight than OSVVM. It requires you to set your testbenches up in a specific way (a bit like UVM) but is very powerful. It provides a lot of similar features to OSVVM.

I have little/no experience with VUnit and CocoTB. My understanding is that VUnit can also provide some CI features as well as verification.

If you are totally new, then all of these tools can be rather overwhelming. I highly suggest you look into the basics of testbenches first. There are many tutorials/examples if you search. Writing using the above frameworks will require a change in mindset from pure HDL development.

View solution in original post

Highlighted
Visitor
Visitor
230 Views
Registered: ‎03-22-2019

Thank you very much for your answer!

There are, of course, unit tests for each module, but the interconnection of these modules must be also verified (i.e. integration testing). There is a need to be able to simulate arbitrary designs and verifying them directly on the silicon is complex and very time-consuming due to the connections that need to be made between different sub-systems via fibers. That is why there is a preference for simulation (even if it runs during the whole night for example) over real world verification.

It's definitely a large and new (for me) area, so I'll have to read a lot to get into it.

0 Kudos
Highlighted
Visitor
Visitor
159 Views
Registered: ‎08-25-2017

Here are some additional comments/corrections regarding VUnit.

VUnit is not based on Python but rather Python-aided. The difference is that with cocotb you write your testbenches in Python but with VUnit your tests are still in VHDL or SystemVerilog. If you want you can use the VHDL utility packages in VUnit to create testbenches and not write a single line of Python

The primary purpose of Python is to provide the things you cannot do in VHDL/SystemVerilog, for example

Automatic compilation. VUnit scans you project for the code to compile. Code dependencies are analyzed and only code that depends on code that have changed is (re)compiled (incremental compilation)
All your testbenches or a selected subset are simulated. A failing test doesn't prevent the remaining tests to be simulated. 
All failing tests are reported, not only anticipated errors found by asserts in the testbench. Examples are indexing arrays out of range, division by zero, null poiner access, simulator internal errors etc.
Multi-threaded test execution to reduce simulation times
Optional integration with continuous integration tools

While VUnit introduces Python you really don't have to know Python to get started. This script will give you all the things above.

from vunit import VUnit

# Create VUnit instance by parsing command line arguments
vu = VUnit.from_argv()

# Create library 'lib'
lib = vu.add_library("lib")

# Add all files ending in .vhd in current working directory to library
lib.add_source_files("*.vhd")

# Run VUnit function
vu.main()

The Python parts of VUnit also works with the utility packages of UVM, UVVM, OSVVM. OSVVM is also shipped with VUnit since it provides randomization functionality not provided by VUnit's own utility packages.

VUnit supports GHDL, ModelSim/Questa, RivieraPro, ActiveHDL and Incisive.