Open Source Verification Whitepaper - 2024 - v1.0.3.0
Open Source Verifcation Whitepaper
Clara Harvig Bjerrum, Kasper Hesse, Stefano Minigutti, Aamir Sohail Nagra, João Carvalho, Jacob Sander Andersen & Andrei Lavric
In recent years, many open-source projects have emerged making hardware design
and verification possible without the need for the common proprietary SystemVerilog
tools. This paper presents a detailed implementation of a testbench using PyUVM, an
implementation of the Universal Methodology Framework in Python. The use case is a
testbench for a memory arbiter, verified using reusable Universal Verification Components
and Constrained Random Verification to ensure all corner cases are hit. When building
Universal Verification Methodology (UVM) testbenches that utilize Constrained Random
Verification, the Universal Verification Component is a key element to achieving high
reusability and flexibility.
To connect the testbench with the Device Under Test a Python class was implemented
which provides an abstraction layer between the Universal Verification Component and
the Device Under Test. An API for the Register Abstraction Layer was also developed
on top of the existing Register Abstraction Layer in PyUVM for register programming.
The use case also presents how a C-based reference model may be integrated with a
Python-based testbench.
To investigate the state of open-source verification, a detailed comparison of the open-
source Python libraries for constrained randomization PyVSC, cocotb-coverage, and con-
strainedrandom is presented. Here, we consider both randomization speed and the quality
of the randomization result. The cocotb-coverage and PyVSC libraries also support col-
lecting coverage which serves as an evaluation matrix for stimulus. The paper presents
the current state of the coverage collection capabilities.
In the end, we conclude on the current state of open-source verifications and address
the outstanding challenges in performing CRV with open-source tools.
Keywords— cocotb, pyuvm, UVM, UVC, coverage, testbench, randomization, memory arbiter,
verification, open-source.
Download
and verification possible without the need for the common proprietary SystemVerilog
tools. This paper presents a detailed implementation of a testbench using PyUVM, an
implementation of the Universal Methodology Framework in Python. The use case is a
testbench for a memory arbiter, verified using reusable Universal Verification Components
and Constrained Random Verification to ensure all corner cases are hit. When building
Universal Verification Methodology (UVM) testbenches that utilize Constrained Random
Verification, the Universal Verification Component is a key element to achieving high
reusability and flexibility.
To connect the testbench with the Device Under Test a Python class was implemented
which provides an abstraction layer between the Universal Verification Component and
the Device Under Test. An API for the Register Abstraction Layer was also developed
on top of the existing Register Abstraction Layer in PyUVM for register programming.
The use case also presents how a C-based reference model may be integrated with a
Python-based testbench.
To investigate the state of open-source verification, a detailed comparison of the open-
source Python libraries for constrained randomization PyVSC, cocotb-coverage, and con-
strainedrandom is presented. Here, we consider both randomization speed and the quality
of the randomization result. The cocotb-coverage and PyVSC libraries also support col-
lecting coverage which serves as an evaluation matrix for stimulus. The paper presents
the current state of the coverage collection capabilities.
In the end, we conclude on the current state of open-source verifications and address
the outstanding challenges in performing CRV with open-source tools.
Keywords— cocotb, pyuvm, UVM, UVC, coverage, testbench, randomization, memory arbiter,
verification, open-source.
Download
DVCon EU 2020
Does It Pay Off To Add Portable Stimulus Layer On Top Of UVM IP Block Test Bench?
Xia Wu, Jacob Sander Andersen, Ole Kristoffersen
The work presented is the outcome of deploying portable stimulus to an existing UVM block level test bench. The target is to verify a highly configurable filter chain system which has extensively used generics and flexible run-time configuration. Accellera Portable Stimulus Standard Domain Specific Language (PSS-DSL) is used to create the abstract PSS model. The paper investigates the effort to create a PSS based solution on a traditional UVM test environment, and if it contributes to better verification quality. We present five concrete challenges that we have experienced when creating a PSS model: Compile-time parameters, Run-time configuration, Inheritance, Partial Configuration and Semantics equivalence, and solutions are proposed to overcome or mitigate these challenges. In the end we conclude a few guidelines and consideration for future projects use.
Download
Download
DVCon EU 2019
Random Stimuli Models for UVM Registers
Jacob Andersen, Laura Montero, Lars Viklund
In this document we are targeting issues which become relevant when using the UVM register layer. Typically, randomization of the registers inside an instance of a UVM register model using constraints will lead into issues such as poor reusability and/or maintenance and redundant source code due to the way the UVM register models are constructed. Thus, to overcome these issues then the verification engineers typically implement small stimuli models which are randomized and then apply them to the registers afterwards. These models are almost identical to the register model except they can be randomized independently of the register model and constraints can be easier applied. The stimuli models can be generated since the register model and the stimuli model are almost similar and the meta data used for generating the UVM register model contains enough information to also allow generation of the related stimuli model. This document presents a generalization of these small stimuli models based on some base classes. The reader is assumed to have basic knowledge on the UVM register layer.
Download
Download
DVCon EU 2018
Generating Bus Traffic Patterns
Jacob Andersen, Lars Viklund, Kenneth Branth
During the block level verification of the modules in an ASIC using SystemVerilog with the UVM methodology we were required to generate specific traffic patterns for a bus protocol as requested by the architect (design engineers, system architect, etc.). Particularly the specification of the traffic pattern was problematic, as it was ambiguous and could be interpreted differently by the architect and the verification engineers. For improved reusability across various verification environments and platforms the traffic patterns could instead be accurately expressed by using a domain specific language (DSL). The DSL could then easily be converted into executable code for producing the expected traffic patterns.
Download
Download
DVCon EU 2016
Agnostic UVM-XX Testbench Generation
Jacob Andersen, Stephan Gerth, Filippo Dughetti
Code generation or model driven software development has always had his place within the field of ASIC verification due to the obvious advantages with respect to time savings, complexity reduction, less bugs/errors etc. Typically, model driven software development has been used for generating RTL implementation for registers, register documentation, self-contained register tests from abstract specifications such as IP-XACT. Over the last couple of years generation of testbenches implemented in UVM have been widely introduced within the field by several contributors. This paper tries to leverage all of this previous work and introduce a layered abstraction for UVM testbenches which makes it possible to generate UVM-SystemVerilog (UVM-SV) and UVM-SystemC (UVM-SC) based testbenches from the same abstract specification. Especially UVM-SystemC enables the reuse of testbenches, e.g. from concept level down to Hardware-in-the-loop (HiL) approaches.
Download
Download
DVCon EU 2014 & DVCon US 2015
Versatile UVM Scoreboarding
Jacob Andersen, Peter Jensen, Kevin Steffensen
All UVM engineers employ scoreboarding for checking DUT/REF behavior, but only few spend their time wisely by employing an existing scoreboard architecture. Main reason is that existing frameworks have inadequately served the user needs, and have failed to accelerate the user efficiency in the debug situation. This paper presents a better UVM scoreboard framework, focusing on scalability, architectural separation and connectivity to foreign environments. Our scoreboard architecture has successfully been used in UVM testbenches at various architectural levels, across models (RTL, SC) and on physical devices (FPGA/ASICs). Based on our work, the SV/UVM user ecosystem will be able to improve how scoreboards are designed, configured and reused across projects, applications and models/architectural levels.
Download
Download
DVCon 2005
Modeling a Highly Generic Processing Unit Using SystemVerilog
Wolfgang Ecker, Volkan Esen, Peter Jensen, Lars Schönberg, Thomas Steininger
The past decades of system design have shown that generic modeling is an efficient instrument for reducing the overall design time and allows IP-reuse more easily. In this paper we describe the usage of the lately evolved hardware description and verification language (HDVL) SystemVerilog with regard to implementing a highly generic processing unit. Our analysis shows that SystemVerilog allows writing generic designs in a more convenient manner. However it is shown that some of the useful features are still not supported by the current stage of EDA tool development. Nevertheless workarounds for the unsupported features are presented
Download
Download