Portable stimulus smooths path to SW-driven verification, eliminates duplication
SoC designs are becoming ever more complex, and increasingly include a processor – often multiple processors. Because the processor is an integral part of the design, it’s important to verify the interactions between software running on the processor and the rest of the design.
Verification and validation of the hardware/software boundary cannot reasonably be deferred until prototype bring-up in the lab, because software is so critical to the operation of today’s systems. Or, at least, verification teams do so at their own peril.
I’m sure we’ve all heard the nightmare scenarios where, for example, a team discovered in the lab that the processor’s bus was connected to the design in reverse order, or the processor was unable to power up again from low-power mode.
HW/SW stepwise refinement
An obvious solution, of course, is to do more verification around the hardware/software boundary during the traditional verification process. However, we can’t just jump from hardware-centric verification to attempting to run a full application stack. With the complexity and voluminous debug logs that result from attempting to execute a large amount of software, tracking down simple bugs gets really complicated.
A productive approach is to verify everything we can in the simplest verification environment that reasonably allows us to exercise the target functionality, has the greatest visibility, and has the least activity that is extraneous to the test intent.
In this article we’ll look at a simple example involving register-access verification. Verifying that registers of an IP can be correctly written and read by the processor is a key integration-verification task. Even simple SoCs contain hundreds of registers, and creating the tests to verify that the processor can read and write each of them can be time consuming.
Figure 1 shows a simple SoC with flash and DDR memory, a tightly-coupled memory, and a UART and DMA engine whose registers are accessed via a low-speed peripheral bus.
Figure 1 Simple SoC
While we will eventually need to verify that code running on the processor can access IP registers, we can focus the verification task just a bit more by starting with UVM-based verification. Verifying the memory subsystem in UVM first will give us more confidence when bringing up software on the embedded processor. Using a portable stimulus tool will allow us to retarget the same test intent to a UVM and embedded software environment, saving test-development time.
Use graphs to describe registers
A declarative graph-based input description provides the power of constraint programming and adds the ability to specify decisions in an iterative fashion. This ability to make decisions iteratively is very helpful when it comes to specifying the constraints for accessing registers.
We start by capturing the core attributes of a memory-test operation: address, access size, write data, and a write mask. The write mask specifies which bits are read/write, and which must be ignored for the purposes of checking.
An action is a unit of behavior to be carried out in the target verification environment. Later on we’ll see how changing the implementation of the body action allows us to easily retarget the register-access test intent to UVM and embedded-software environments.
The register access descriptor shown in Figure 2 doesn’t include any details of the IPs in our system.
Figure 2 Core register access struct
We next need to add these restrictions in. Our DMA engine (the Wishbone DMA Core from opencores.org) has a set of core registers, and then an array of channel-descriptor registers. Using a graph-based description allows us to describe the register address iteratively.
The graphical description in Figure 3 shows the process for selecting a DMA-register address:
- Select a core register or the array of channel-control registers (dma_reg)
- If the channel-control registers are selected
- Select which channel (dma_ch)
- Select which channel register is being targeted (dma_ch_reg)
Figure 3 DMA register address selection
Figure 4 shows the textual description of this process.
Figure 4 DMA Register Address Selection Rules
There are, of course, more detailed constraints for specifying the register address and write-mask which are coded in the constraints shown in Figure 5. Note that this constraint is marked as ‘dynamic’, which means that it only applies once activated within the graph – in this case, once we’ve decided to access a DMA register. While these constraints were created by hand, they could also be generated from a machine-readable register description, such as IP-XACT, SystemRDL, etc.
Figure 5 DMA Register Address Selection Constraints
Once we integrate the DMA, UART, and TCM portions of our register-access graph, we arrive at the graph shown in Figure 6. We can view this as a flow chart: first, we decide which IP to access, then the graph branches based on the selected IP and specific choices are made. For example, the left-most graph branch contains DMA-specific decisions.
Figure 6 Simple SoC Register Access Graph
Even our small SoC has over 250 registers, and a real SoC would have many times more. We need to ensure that we access all of these registers, and that we access them in some random order. We might also want to focus our testing a bit – for example, just test the UART registers.
A rules-based approach allows a combination of constraints and graph-coverage goals to be used to focus testing on the goals to be achieved. Constraints specify which tests are allowed to be generated, while coverage goals prioritize systematic generation of some tests. For example, a constraint could be used to specify that the UART isn’t currently ready to be tested. A coverage goal is used to specify that the focus of a given test is the DMA engine, but other available IPs can be targeted once that goal is achieved.
The coverage goal (the blue-shaded region) shown in Figure 7 focuses test activity on the DMA registers for the current test. The portable stimulus tool will generate tests that systematically access the DMA registers in a pseudo-random order, then generate a mix of accesses to DMA, UART, and TCM.
Figure 7 Coverage goal focused on DMA
Mapping to a UVM environment
Thus far, the description of register-access testing that we’ve created is independent of any verification environment. We now need to provide a little glue logic to connect our test graph into a subsystem-level verification environment.
In our UVM subsystem-level environment, we’ve replaced the RTL model of the processor with a bus functional model (BFM) to access registers across the interconnect. Our register-test graph is encapsulated in a UVM virtual sequence that will access registers via the BFM.
Figure 8 Subsystem-level verification environment
Our base ‘CPU’ virtual sequence provides access to the BFM via a class API that supports reads and writes of various sizes. We’ve added a specific memcheck sequence that carries out write, read-back, and check inside a do_memcheck task, which is shown in Figure 9.
Figure 9 UVM do_memcheck task
The mapping information needed to connect our register-test graph to the do_memcheck task in the virtual sequence is collected in a file – target.rseg in this case. The mapping information is shown in Figure 10.
Figure 10 UVM sequence target mapping
The mapping information specifies:
- That our graph should be encapsulated in a class that extends from or1k_memcheck_vseq
- That we should use the or1k_memcheck_c class to represent the or1k_soc_regacc struct in the UVM environment.
- That the do_memcheck task should be called when the ‘body’ action within or1k_soc_regacc executes.
With just these few lines of code, we can now run our register-test graph in our UVM environment. This allows us to verify the basics of register connectivity, and debug any issues using our standard UVM and SystemVerilog debug tools.
Mapping to embedded SW
But, of course, verifying that our BFM can access register memories doesn’t definitively ensure that our processor will be able to. Consequently, it’s still important to run register-access tests as embedded software on the processor. In its simplest form, this full SoC verification environment might look like what is shown in Figure 11.
Figure 11 Software-Driven Verification Environment
In this case, we will create a series of tests written in ‘C’ that write and read-back registers using the processor. Our tests will use a do_memcheck function to perform the actual write, read-back, and check. This function is shown in Figure 12.
Figure 12 Embedded software do_memcheck function
Just as with the UVM environment, we need to specify how the graph maps to the embedded software environment. The mapping information is shown in Figure 13.
Figure 13 Embedded software mapping
The mapping information specifies the following:
- That the header file or1k_memcheck.h must be included. This file declares the do_memcheck function.
- That the function do_memcheck must be called each time the body action executes, passing the values of the addr, size, wr_data, and wr_mask fields.
Because we’ve described a graph-based model of the register tests to perform, we now have a lot of flexibility in how we generate specific sets of tests. Figure 14 shows one example of a generated test. In this case, we’ve allowed the portable stimulus tool to target all three IPs, but restricted the test to just five iterations through the graph.
Figure 14 Example Register Test
As you can see, the generated test is a directed test with randomized values filled in by the portable stimulus tool during the generation process. While this specific test will always do the same thing every time it is run, we can achieve more randomness across regression runs by regenerating the test with a different seed each time the regression runs.
In order to achieve coverage of the register-access goals that we described earlier, we will need to generate more specific tests. Because all of these tests are generated from a single declarative model, it’s easy to simply change the options to the test-generation program in order to move between running many tests that each make a few calls to do_memcheck per test to running a few tests that each make many calls to do_memcheck per test. It’s also easy to adjust targeted coverage goals to, for example, generate a series of tests that only focus on DMA register tests.
Taking a step-wise approach to verifying interactions between embedded processors and the IPs in the rest of the design saves time by finding bugs earlier in the verification process when they’re easiest to debug and correct.
Portable stimulus allows high-quality tests to be generated from test intent that is described once, and retargeted to multiple environments. This addresses the major challenges of the stepwise approach: the amount of duplicated effort to execute test intent in multiple places, and the need to create embedded software tests using low-productivity directed tests.
In this article we’ve seen how a single description of test intent, for testing access to SoC registers as a part of SoC integration testing, can be easily targeted to both UVM and embedded-software environments.
Next time you start planning for SoC integration testing, consider how portable stimulus and a stepwise testing approach can benefit your verification process.
—Matthew Ballance is a Product Engineer and Portable Stimulus Technologist at Mentor Graphics, working with the Questa inFact Portable Stimulus product.