UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Contributor
Contributor
650 Views
Registered: ‎07-09-2014

cheapest and easiest way to use uvm

Jump to solution

hi all,

i still use vhdl for both rtl and verification. however, when searching about verification one sees always the same word: uvm.

arter researching i see a bunch of companies support uvm with their own eda tools but not xilinx at the moment.

i wonder why xilinx does not support uvm? i dont want to buy and learn a New software. but if i have to, what are your opinions about uvm verification softwares? which iş easier to adopt and learn, which is cheaper . pros and cons ?

thanks in advance 

0 Kudos
1 Solution

Accepted Solutions
Scholar richardhead
Scholar
604 Views
Registered: ‎08-01-2012

Re: cheapest and easiest way to use uvm

Jump to solution

UVM is just a verification library written in System Verilog. So if you want to use UVM you need a very good understanding of SV. And that is a massive investiment if you only know VHDL. UVM is very complicated. I have worked in a department where there was a dedicated verification team of 3 people writing nothing except UVM. It is very hard for VHDL engineers to understand UVM.

Vivado 2019.2 apparently has UVM support.

UVM support on ActiveHDL or Questa usually comes at an extra cost.

 

But have you had a look into the VHDL verification libraries? Both UVVM (Universal VHDL Verification Methodolody) and OSVVM (Open source VHDL Verification Methdolody) are free and open source, and hence come for free with a single language VHDL licence for Modelsim or ActiveHDL, or you can use the free and open source GHDL. Vivado lacks support because it doesnt have VHDL 2008 support in the simulator.

There is also Vunit if you want to setup a python CI environment.

(there is also CocoTB, but I dont know if it is still actively developed)

If you are only a VHDL guy, expect to spend at least 12 months getting up to speed with and comfortable with UVM.  It is a different way of thinking from standard VHDL and even plain SV. It is probably overkill for the majority of unit tests and will require very good test requirements capture. It will also require a good chunk of time to maintain.

View solution in original post

7 Replies
Scholar richardhead
Scholar
605 Views
Registered: ‎08-01-2012

Re: cheapest and easiest way to use uvm

Jump to solution

UVM is just a verification library written in System Verilog. So if you want to use UVM you need a very good understanding of SV. And that is a massive investiment if you only know VHDL. UVM is very complicated. I have worked in a department where there was a dedicated verification team of 3 people writing nothing except UVM. It is very hard for VHDL engineers to understand UVM.

Vivado 2019.2 apparently has UVM support.

UVM support on ActiveHDL or Questa usually comes at an extra cost.

 

But have you had a look into the VHDL verification libraries? Both UVVM (Universal VHDL Verification Methodolody) and OSVVM (Open source VHDL Verification Methdolody) are free and open source, and hence come for free with a single language VHDL licence for Modelsim or ActiveHDL, or you can use the free and open source GHDL. Vivado lacks support because it doesnt have VHDL 2008 support in the simulator.

There is also Vunit if you want to setup a python CI environment.

(there is also CocoTB, but I dont know if it is still actively developed)

If you are only a VHDL guy, expect to spend at least 12 months getting up to speed with and comfortable with UVM.  It is a different way of thinking from standard VHDL and even plain SV. It is probably overkill for the majority of unit tests and will require very good test requirements capture. It will also require a good chunk of time to maintain.

View solution in original post

Highlighted
Adventurer
Adventurer
553 Views
Registered: ‎07-16-2009

Re: cheapest and easiest way to use uvm

Jump to solution

Hello bbinb,

UVM is a set of systemVerilog classes for verification. Their main advantage is that it provides high level of abstraction for the verification engineer. The basic idea is, that you have reusable components which connects to the tested component to the object oriented testing environment. Therefore you can write your tests in object oriented paradigm which can be done faster. 

For example, lets suppose you have ALU, which has axi_stream input and axi_stream output. On the input ALU receives configuration packets which specifies what operation it should perform and data packets, which contains data for the operation. To verify this component, you need to write testbench which will provide such stimulus, that you will test all mode of operations (all operation at least once?, tests corner cases such as empty or full fifos?, ...) or check that all lines of codes are excecuted (code coverage) or any other verification metric you chose to use. This is generaly a lot of stimuli and it is hard to prepare. If you use UVM, you will create (or reuse from previous component) axi_stream agents which are components  that accepts transaction object and toggle DUT pins accordingly. This is big bonus if most of your components has the same interface.

Now you are writing your test in object world, so you do not need to deal with timing. So you are able to prepare configuration packet followed by data packets and observe datapacket with the results. But UVM can help you here as well. The systemVerilog constraints can be use to specify what are valid for of configuration/data packet and you can use one line of code to fill whole packet with random data according to these constraints (this is called constraint random verification). So by this way you are able to send random (but correctly formed) data to your DUT. 

But now you have hundreds of thousands packets coming to your complex ALU  and houndreds thousands of potentially correct results. And because you have randomly generated inputs, you have to have correct way to compute results and verify that your ALU works correctly. So you must implement the ALU functionality once again but this time it does not have to be synthetisable, nor it needs to meet any other of your design requirements, it just have to provide correst results and after that you have to implement component which will take results from your golden model and compared the with the results from your ALU. Luckily, we are not on the RTL level here but you can just compare the transactions, which are SV objects so we can just write if A.compare(B) and you have result of the comparisson of the two results. However, general DUT may switch the order in which it sends the results. UVM provides support for effective writing of all these things.

No more specifically to your questions:

Xilinx provides support for UVM since VIVADO 19.2. I am playing with it not and it seems to work. However, Xilinx still do not support full SystemVerilog.

To use and work with UVM, you do not need to lear new software, you need to learn verification techniques, SystemVerilog language and object oriented programming. You do not need to be expert in any of this to start. If you want to learn about UVM, I would recommend www_edaplayground_com. Which allows you to experiment and simulate simple testbenches. If you want to work offline, I would use VIVADO 19.2 since I expect you are most familiar with it and do not have access to Questa or Aldec. Do not focuss on tool but on the language and on the methodology. I have experiences with VIVADO, Questa, Aldec Riviera Pro and in all casese it is more about how good I am in SystemVerilog, and object oriented design  and  the UVM itself. 

As for the price, first you must remember that you pay licence per core not per PC or anything. And you have many testcases which can be run in parallel (as in you run 40 simulations on your cloud). So you have to have enough UVM simulation licences. As to cost of one licence, I once wanted UVM simulator from our management, I do not remember exact numbers but in the end it was about as much as myself (like all monay company was spending on me) for one year. 

To conclude, If you want to increase your personal skills, you may go with VIVADO or the eduplayground. But if you want to incorporate UVM into your company, first hire someone who has experience with it and take their advice. 

 

Jan

Scholar dgisselq
Scholar
501 Views
Registered: ‎05-21-2015

Re: cheapest and easiest way to use uvm

Jump to solution

A quick look at the CocoTB respository reveals that issues were closed as little as 15 days ago, and commits made to the repository as few as 3 days ago.

I'd say it was actively developed.  Whether or not it is sufficiently developed to be useful for a given purpose would be a different question.

Dan

0 Kudos
Contributor
Contributor
460 Views
Registered: ‎07-09-2014

Re: cheapest and easiest way to use uvm

Jump to solution

Hi,

Woow, that was one of the best answers I got in this forum. Thanks all for your very beneficial guides.

I have already made some generic process blocks for testbench in VHDL to stimulate commonly used communication interfaces like UART, SPI slave, I2C slave, AXI Lite, AXI Full. I will look for UVVM and OSVVM, I think instead of trying to learn SV and UVM, finding better testbench solutions in VHDL will be more convenient for our team as we don't design huge logic like IC prototyping etc.

@lordgalloth 

I will kindly request for an explanation about the paragraph you wrote:

But now you have hundreds of thousands packets coming to your complex ALU  and houndreds thousands of potentially correct results. And because you have randomly generated inputs, you have to have correct way to compute results and verify that your ALU works correctly. So you must implement the ALU functionality once again but this time it does not have to be synthetisable, nor it needs to meet any other of your design requirements, it just have to provide correst results and after that you have to implement component which will take results from your golden model and compared the with the results from your ALU. Luckily, we are not on the RTL level here but you can just compare the transactions, which are SV objects so we can just write if A.compare(B) and you have result of the comparisson of the two results. However, general DUT may switch the order in which it sends the results. UVM provides support for effective writing of all these things.

If I don't understand wrong, I need to implement another ALU functionality to test my RTL ALU. How can I be sure that my new implementation of ALU to compare the correctness of RTL design is working correct ? Isn't it possible that my RTL ALU is working correctly but the new ALU to test RTL ALU has functional problems?

Thanks,

Burak

0 Kudos
Scholar richardhead
Scholar
445 Views
Registered: ‎08-01-2012

Re: cheapest and easiest way to use uvm

Jump to solution

What @lordgalloth talks about are verification techniques that can be used in any language.  UVM is an industry standard but is extremely heavy weight, but is very effective. But if you're an fpga house most (if not all) of it can be achieved with good planning and verification techniques in vhdl.

I'm currently using osvvm to test a packet headset extractor.   With the coverage tools available (osvvm calls it intelligent coverage) I can get 100% functional coverage with a test bench that takes about 5s to run. This uses randomized packets using randomised axi transactions. Each tb run gets a new seed and coverage its achieved every time. While this is only a small example,  it scales to whatever you can plan for.  Aldec have been promoting osvvm and uvvm alot in recent years.

0 Kudos
Adventurer
Adventurer
424 Views
Registered: ‎07-16-2009

Re: cheapest and easiest way to use uvm

Jump to solution

Hi,

first, thank for praise. Secondly, as @richardhead wrote, I mix the UVM as libraries with the verification techniques itself. From this point of view, I would recommend for you to focuss more on verification techniques (constraint random verification, different types of coverage, etc..) and less on implementation UVM, UVVM, OSVVM, etc... The knowledge of SV necessary for basic UVM tutorial (with the goal of understanding how coverage works) is minimal. I am not sure about others but for UVM, you can try to visit verificationacademy<dot>com

You mentioned 5 (maybe seven if AXI are both master and slave) intrerfaces. This would be OK for UVM because you would need to implement 5 or seven agents which would be reused for all your components.

Now for your question about the meaning of the paragraph. Lets suppose that our ALU is little more complex than just sum of 2 integers. Let it work in matrices with operations such as (matrix sumation, matrix multiplication, inversion, diagonalization, etc). Let there be about 15-20 possible operation. Now the ALU supports limited pipelining (i.e.) it accepts next operation, or next data while performing the current operation. As a side effect, if for example current operation is diagonalization and following is sumation (much faster operation), the unit will provide the result for the sumation before result of diagonalization, even if the sumation was send later. However, for the resource efficiency, some computation units inside the ALU are shared (such as division, CORDIC, etc..). 

Now for this ALU we need to specify coverage. Which is what we want to achieve by our verification. We can specify the amount of code coverage (lets say 100% of line of code must be run during our tests). Together with this, we can specify some functional coverage. So we start with requirement that we check every operation. But what if there is some mistake which causes undefined state of the system after one operation. So we may consider requirement to check that every possible combination of operation was checked. (Just about 400 coverage points here) Now what will happen if the consumer is not able to accept results (of course we have some output fifo on axi stream and input fifo on input axi_stream). So we can specify another requirement, that  the enery FIFO must be at least once full and at lest once empty). And this though process will follow until we, as a verification engineer decides that the tests will be good enough.

So we have DUT and we know what we want to test. Now we can manually take every possible situation and prepare stimuli  for it, generate expected results and run the test. However, this will take too much of our effort for this ALU. So instead we go for constrain random verification. We create constraints to specify whart is correct data packet, what is correct control packet and create sequences to provide required operations (like that data packet has correct/incorrect length, etc..). And now we let the testbech run and wait until the all our coverage points are covered. Maybe some our point can not be covered (maybe we have a dead code in our DUT, or our testbench does not allow malformed packets but we check if malformed packets arrived into our DUT). If our verification metric is easy to achieve, test may be finished  in few seconds, if not, tests may run for days (I personally worked on projects where whole test suit run about month of processor time). 

So now, you have large amount of stimuli and you have to decide if the result provided from the DUT are correct. The easiest way how to do this, is to have another implementation to which you can compare results. 

So yes you have to implement the functionality again. This is one reason why the verification engineer is typically different person from the design engineer because it is probable that different persons will make different errors. It is however true, that when the error is detected, it may be the error inside the verification environment instead of error in DUT. However keep in mind, that when you are writing the golden model, you are not limited to synthetisable code. So in our case, you will have to implement all matrix operation again but this time, when you want to divide two numbers, you simply write A / B instead of instantiation of division component. Moreover you are not limited by time or by resources so you do not need to implement the sharing of computation elements again etc. You have whole object oriented paradigm to simplify your design. Moreover, you can use C binding to some matrix computation lilbrary.

I hope that This somehow cleared the confusion. It is still very simplistic answer but I hope it demonstrates the power of UVM. But do not forget that while it is possible to kill a sparrow by machinegun, there may be easier way to protect your crop. 

Contributor
Contributor
389 Views
Registered: ‎07-09-2014

Re: cheapest and easiest way to use uvm

Jump to solution

thank you very much for detailed explanations you made. it is difficult if one needs to both design and verify. both are different discplines, needs years to expertise. 

regards

Burak

0 Kudos