Debugging and analysis with SystemVerilog test bench
Some minor additions to OVM libraries can open a new world of visibility for design verifiers.
By Bindesh Patel and Rex Chen, SpringSoft -- EDN, February 4, 2010
The verification component of SystemVerilog has dominated the rapid adoption of the language. The new verification syntax in the language allows for dramatic productivity gains in the verification cycle, which itself is now the major portion of the entire design and verification cycle. However, the debugging process of such environments brings about additional complexities. First, unlike with RTL (register-transfer-level) logic, no callback-driven “dumpvars”-type, dump-everything schemes exist for the object-oriented, software-like SystemVerilog verification constructs. Using these capabilities, engineers have access to the entire history of events that occur during a simulation. Once available, this data can drive basic capabilities, such as waveforms and overlay of the trace data onto familiar source code and schematic views, and automation, such as multicycle tracing to the root cause with one command or mouse click.
Second, it is unclear what “access to all the data” would mean in the first place for SystemVerilog object-oriented test-bench code, which creates and destroys objects at runtime. Would the code record local variables or object-creation time? More important, how would this information be useful to users even if it were available? It is clear that the dump-all paradigm that is applicable to RTL logic is not meaningful for test-bench code for these reasons.
Early adopters of SystemVerilog test-bench syntax used crude logging of useful information to text files using the built-in $display or sprintf constructs. The users then inspected the resulting text files during debugging and manually correlated the text-file data with the RTL-dumped data to get a picture of what happened. In the extreme case, users invoked the simulation in interactive mode because this mode can potentially allow for full introspection of objects. This approach was less than ideal, however, because interactively running a simulation raises obvious performance issues, is inefficient, and is not friendly to today’s simulation-farm, batch-based environments.
More recent advances have leveraged the notion of logging by recording the user-specified information into a proper debugging database, such as SpringSoft’s FSDB (Fast Signal Database) format. Once users load such a specialized database, they can employ an automated debugging system that allows them to analyze the resulting data in a waveform or a table view that synchronizes with the rest of the system (Figure 1).
With this structured logging approach, a user can specify the data to record using a special system task, $fsdbLog (Figure 2). This task can record types of data and class local variables, thereby providing maximum flexibility. This structured logging approach comprises a message label—“success” in the example—and other optional attributes or properties of that message, such as a severity value. This breadth gives a more flexible capability to users than the previous string-only text-based approach. Users can debug and analyze using the resulting messages within the same environment as the design RTL or gate-level dumped-signal data, thereby significantly boosting productivity. The availability of such a logging task that dumps data into a proper debugging database is a key foundation for further refinements in the flow and eventual automation.
Logging schemes
Someone must be responsible for the instrumentation of the described logging tasks. The recording happens whenever a user encounters any such task during the execution of the test-bench program during simulation. The brute-force method is to replace or augment all uses of the built-in $display construct with a corresponding call to the logging task, $fsdbLog. Even this primitive use model leads to a more productive user experience than before. A further refinement would be to bury the logging code into the underlying OVM (Open Verification Methodology) or VMM (Verification Methodology Manual) libraries, which are foundation base-class libraries that most SystemVerilog verification environments use. Listing 1 shows how you can add the logging task in boldface to the `vmm_error macro that you use to log error messages in environments employing the VMM library.
You could also extend the newer vmm_log_catcher and vmm_log_format classes with the logging instrumentation. Similarly, you could extend the OVM reporting functions (Listing 2) for the ovm_report_info function. Once users have performed this underlying coding, those who use these library macros or functions to record information will also be able to record into a proper debugging database with all the resulting benefits.
Automation
Using the basic $fsdbLog task to record messages into a proper database as a foundation, the next logical step is the complete automation of transaction recording in OVM- and VMM-based test benches. To achieve this goal, you can leverage the well-defined structure of OVM and VMM to watch transactions flowing up and down the test bench and to automatically record this activity.
|
Like VMM, OVM promotes a transaction-based approach to writing test benches. In this context, a transaction is a higher-level abstraction of data. The basic components of an OVM-based transaction-level verification environment are a stimulus generator or sequencer to create transaction-level traffic to the DUT (device under test); a driver to convert these transactions to signal-level stimuli at the DUT’s interface; a monitor to recognize signal-level activity on the DUT’s interface and convert it into transactions; and an analysis component, such as a coverage collector or scoreboard, to analyze the transactions (Figure 3).
VMM-based environments are similar. Regardless of the underlying library, the most interesting and useful data for engineers are the transactions between the sequencer and the driver and those between the monitor and the analyzers. You could apply these techniques to record the transactions of interest. The ultimate goal, however, would be to automatically record all the transactions between the sequencer and the driver. You can use the structure of OVM test benches to achieve this goal.
The OVM and its library require that the sequencer and the driver communicate with each other in specific ways. For example, the sequencer sends requests to the driver using the ovm_sequence::send_request() function. This scenario is true even when you use the `ovm_do and `ovm_do_with macros to generate the sequence data because these macros call the same send_request() function. The driver itself sends the response to the sequencer using the ovm_seq_item_pull_port::put() or ovm_seq_item_put_port::item_done() function. It then follows that you can embed your logging function—sps_generic_log(), for example—in these OVM-standard functions and in the process be able to log all transactions into your debugging database. You can provide a modified OVM library with these changes for end users. This modified library can include the logging function sps_generic_log() in the ovm_sequence_item class. As a starting point, assume that you simply include a basic call to the $fsdbLog task in the ovm_sequence_item::sps_generic_log() to record a label, a string message, and a severity for each transaction (Figure 4).
You can provide hooks to map actual labels, strings, and severities pertaining to an environment using virtual functions that the user can then easily code. From a user’s perspective, this additional coding is trivial and involves overriding a function, as Listing 3 shows in boldface. The end result will be meaningful, intuitive transactions that the debugging system will capture and display (Figure 5). Even without these user modifications, the system would still record all the traffic between the sequencer and the driver, albeit with nonintuitive labels.
Advanced visualization and analysis
You can also instrument the modified OVM library to automatically log information connecting a sequencer that makes a request and a driver that sends back the corresponding response. This approach allows you to associate each request transaction with a response transaction. Although this information is useful to the user even in the waveform and table views, the availability of such complete information can inspire other views that can more intuitively show the test-bench activity.
One avenue for pursuing this inspiration is the UML (Unified Modeling Language), which provides intuitive graphical representations of systems, including software systems. The UML sequence diagram shows components and the interactions between these components in their temporal, sequential order. Although UML’s primary use has been to document a program or system, you can use a sequence diagram to represent the dynamic behavior of a SystemVerilog test-bench program. The components in this case would be the OVM or VMM, and the interactions would be transactions passing between these components.
You now have practical schemes for extracting all the transactions and even for getting the additional details about which components are providing the transactions and which components are receiving them. Capturing all this rich data in the debugging database during simulation can drive the sequence diagram, thereby providing an additional, more intuitive and appropriate view of the test-bench behavior. A sequence-diagram view representing test-bench behavior working alongside and in synchronization with traditional hardware-behavior views, such as waveforms, can provide an ideal system for engineers to debug, understand, and analyze their entire design and verification environment. A prototype sequence-diagram view shows test-bench activity for a sample XBus transaction (Figure 6).
In a sequence view, the time axis is vertical, increasing downward. The components appear horizontally at the top, and the interactions or transactions between them appear as horizontal arrows at the respective simulation times. You can synchronize this view with other views, such as waveforms, so that the selection of a transaction causes the time cursor to move to the same time in the waveform. Other useful features can include tracing forward and backward. In the sample, tracing the “write” from the upper_sequencer causes the diagram to highlight the “write” and corresponding “response” between the sequencer and driver.
Using the foundation of a flexible logging function that records messages into a debugging database, you can instrument a modified OVM library to automatically log transaction-level activity between a sequencer that makes a request and a driver that sends back the response. This approach allows the debugging system to record each request transaction and associate it with its appropriate response transaction. Although this information is useful to the user even in the context of the waveform and table views, the easy availability of such complete data allows you to introduce a sequence-diagram view that can more intuitively show the test-bench activity.
Future work will focus on refining the use model for automatic logging so that you can eliminate the need to provide a modified library. This advance would remove dependencies in keeping in lockstep with OVM or VMM library revisions. Another focus area is to explore other UML representations, such as the component diagram for design and verification.
