Ideas from ISQED: how Denali beats verification bloat
The International Symposium on Quality Electronic Design is one of those almost quirky small conferences so rich in surprises that it always seems to be worth the time. The organizers clearly interpret the conference theme freely, so topics range from verification to fault- or variation-tolerant design to conceptual subjects like defining architectural quality.
One of this morning’s keynotes was of particular interest to SoC design teams. Denali CTO and CFO—surely one of the most interesting titles in the industry—Mark Gogolewski spoke on dealing with the cost of verification. Gogolewski opened with a powerful generalization: "The cost of quality is creating havoc in the industry." He went on to document his claim, and to offer by way of solution a recent design experience at Denali. Remarkably for keynotes, Gogolewski’s solution didn’t involve licensing anything from Denali.
First the bad news. About 85 percent of SoC designs miss schedule, Gogolewski said. And anywhere from the typical industry estimate of 70 percent to in some reported cases 90 percent of the engineering time goes into verification. Not only does this make verification stick out like a sore thumb in a risk analysis, but as Gogolewski remarked after slipping on his CFO hat, verification is overhead: it doesn’t create the differentiating features that increase product margins.
What to do? Gogolewski related a recent Denali design experience, for the most recent revision of their PCI express gen-3 controller IP block. First, he put some numbers around the problem. The PCIe specification is now about 1,000 pages of non-machine-readable stuff. For gen-3 the spec is still changing, frequently. There are so many operating configurations and modes that it requires about 75 parameters to specify one instance of the controller. Denali’s completed design comprised about 150,000 lines of RTL, generating typically about a million gates of logic in the largest possible configuration.
Gogolewski said that Denali’s approach to this project was reminiscent of one of the controversial now-and-again fashions in software development, called Agile Development. The idea is that in an uncertain environment the impact of changing requirements is the most serious risk to the design, so the project should be managed to respond quickly and accurately to change. I doubt many experienced designers would dispute the diagnosis, although in the software world there has been a great deal of debate about the prescription.
The Denali version of Agile Development starts with small team size. No, really small. Gogolewski said that the entire PCIe team comprised less than ten people, all with very strong skill sets, able to recognize the state of the design and respond accordingly without outside guidance. In order to free the team to exercise their initiative, Denali eliminated most internal processes and documents from the design flow. That meant, for instance, no formal design phases with reviews and sign-offs. In their place, the team had clear goals and milestones so that the individuals could understand where they were and what they needed to do next. To make sure the goals were clear, the team built them systematically. Gogolewski said that the design started out with the team writing the user’s guide for the IP, and then the configuration guide. These, together with the giant PCIe specification, became the requirements document for the project.
Gogolewski said the next step was to have a coverage-guided verification plan from the outset. The team used an OVM-based test bench that included the entire design, all its features and signals, and all its configuration, initialization, and dynamic configurability functions, rather than building up test cases for subsets of the design. The bench was set up so that an IP configuration file configured not just the design, but also the test plan and the coverage database automatically. So the team could pick a configuration and automatically generate not only an instance of the IP block, but the test bench for that instance. This was particularly valuable, Gogolewski said, because configuring the IP turned out to be one of the strongest ways of stressing the design. "Every night the team would pick a set of configurations and launch a regression suite of directed and pseudo-random tests," he explained.
Another vital point, Gogolewski said, was that the team used third-party verification IP (VIP) to check the functioning of the controller instances against the PCIe spec. As it happened, Denali naturally used their own VIP, but Gogolewski emphasized that the important things were reuse—saving the design team’ s time—and the fact that the VIP was not produced by the design team, so it didn’t reflect the design team’s vision of how the IP was supposed to work. Instead, it contained the vision of an independent verification group. The time savings were considerable. The VIP for PCIe gen-3 contained nearly 10,000 compliance tests in about 250,000 lines of code—a larger body of code than the RTL design itself.
The result of this approach, Gogolewski claimed, was a significant impact on design time. In fact, he said, verification consumed 43 percent of the engineering time instead of 70 percent. How much of this was due to the ability to reuse existing VIP, how much was due to the experience of the team on PCIe designs, and how much to the novel team organization and management are all points of debate. What does not seem to be arguable is that at least in some situations, the metastasis of verification time is not incurable. It is subject to measures, albeit perhaps drastic ones, by senior engineering management. But those managers will require a detailed, hands-on grasp of the design and the methodology, or they could reap even more havoc as their reward.
