Bringing automation to debugging in the SoC verification struggle

The growing proportion of SoC design time devoted to verification has attracted surprisingly little investment from the EDA industry compared to the magnitude of the problem. The work that has gone into refining the high-profile design tools-synthesis, place & route, extraction, and analysis-seems to vastly outweigh the work that has gone into automating the as yet mostly manual verification process. This seems particularly odd because studies repeatedly show that verification is taking around two-thirds of the total project time, and in some cases much more. Nor is it a well-bounded problem. Often verification goes on until the schedule runs out, not until the team is convinced they have identified the major bugs and repaired them.

A similar situation exists within the verification process itself. Debugging-tracing unexpected results or triggered assertions back to their root cause and repairing them-according to some experts consumes half of total verification time. This seems on the surface startling in that debug should in principle involve only a tiny subset of the project’s design and test-bench code. But reality is that debug remains almost a manual process for most teams, and at that a process uncomfortably partitioned between the verification team that identifies issues and the design team that usually has to diagnose and repair them. There are important tools to assist in the process, such as SpringSoft’s Verdi, which many design teams find indispensible. And lately, formal-verification vendors-notably Jasper with their root-cause analysis-have been pioneering a new approach to debug by using formal methods to prove what parts of the code could possibly be responsible for an aberrant behavior.

Another tool, so far little-known in the US but in production use in Japan, has taken a similar approach of formal analysis to identify possible causes of a bug. But rather than applying a formal verification tool in a new way, Vennsa’s OnPoint is designed from the ground up for debug. It’s not a formal tool with which you can prove assertions about causes, but rather a debug tool that happens to have code analysis techniques inside. OnPoint’s claim is to start with an assertion counterexample or a suspect simulation waveform and, using only the files that already exist in your verification flow, automatically produce a rank-ordered list of “suspects”: RTL lines, stimulus code, constraints, and assertions that logically could be the source of the problem.

The tool works, according to Vennsa CTO and vice president of engineering Sean Safarpour, by asking a logical question: what lines in the design or the test bench can influence this result? In essence, the tool is automatically building a logic cone, just as debug engineers today would do by hand. Safarpour notes that it is vital during this process to include the test bench as well as the design RTL, since there is no guarantee that the problem is actually in the design. Test benches have bugs too.

But building a logic cone by itself would be of minimal usefulness. Usually the cone would be huge. So there is a critical next step: OnPoint removes from the list any suspects that have alibis: that is, items that, had they acted incorrectly, would have caused other problems that were not observed. This drastically prunes the list. “Often we will report only 15 or 20 suspects,” says Vennsa president and CEO Andreas Veneris. If there are multiple assertion firings, OnPoint can analyze them simultaneously, using the fact that the problems occurred in the same test to further narrow the search for suspects. Using an internal heuristic, the tool ranks the remaining suspects by probability that they are the root cause of the problem. Generally, Veneris says, the actual bug will turn out to be one of the few highest-ranking suspects.

The tool presents data not only as a text list, but with its own viewer/GUI, allowing you to view the suspect Verilog, System Verilog, or waveforms. In a rather novel additional step, OnPoint will show you a suspect waveform complete with fixes-changes to the waveform that would remove the observed error. From there it is up to the designer to come up with code changes, or the verification team to come up with test-bench corrections, to remove the bug.

OnPoint is designed to drop into existing verification flows, taking ordinary Verilog design files and SystemVerilog assertions and testbench code. The tool works directly with Mentor and Synopsys logic simulation environments, reading the simulation output files and in some cases working with the simulators at API level. Veneris said that working with Cadence simulation requires a few minutes of setup to transfer files. Vennsa has the tool available now, and will be showing it at DAC.