May 21, 1998
Optimize ASIC test suites using code-coverage analysis
Performing code-coverage analysis of HDL code before synthesis saves time
and prevents costly design errors.
By the time you make silicon-processing masks, a single
defect introduced at the behavioral-coding stage of a design can cost thousands of dollars
if left undetected. What's worse, once you reach production silicon, an undetected defect
can cost millions of dollars. To help detect behavioral-code problems, designers have
borrowed a tool from software engineers' toolbox: code-coverage analysis.
Software engineers originally developed code coverage to
help programmers profile code and thus find out how to make it run faster. Many C
programmers use code-coverage technology to figure out how effective software test cases
are in really exercising the software they test. Good software code-coverage tools include
features such as statement, branch, branch-subcondition, and code-path coverage criteria.
Code-coverage techniques applied to HDL-based chip designs can help you check the
effectiveness of your chip's testbench and can lead to more efficient chip test suites.
HDLs and functional verification
To obtain the functional verification of a
behavioral-circuit description written in VHDL or Verilog, you write an accompanying
simulation testbench to model the design's intended application. To guard against error,
design engineers must develop a testbench that completely exercises the behavioral-circuit
description.
Defects can occur if you have not tested a circuit for all
expected behavior. In addition, because of unanticipated situations, you may have a
combination of events occur that the testbench does not check. Use behavioral VHDL and
Verilog, because they allow you to rapidly produce a very complex model from which you can
automate all subsequent design processes. Rapid coding of your design may also result in
unintentional circuit behavior. Unfortunately, side effects can be the most difficult to
detect, because testbench development focuses primarily on validating that the behavioral
model works according to the design specification and with "nice" stimuli.
 Statement
coverage tracks assignments. You apply this technique to the signal and variable
assignments for HDL code coverage. Statement coverage measures how many times a simulation
executes each assignment. When a simulation never executes an assignment, a design error
may remain undetected. A good verification tool should highlight nonexecuted assignments
in the code to show that they are a source of potential design errors. For example,
simulation has not executed the statement in Listing 1.
The design or test engineer should now create extra tests to check the assignments that
the verification tool highlights.
Further options for code include branch coverage and
branch-subcondition coverage. Both options are helpful to engineers because they offer
diagnostics to help identify why a simulation did not execute an assignment statement.
 Branch
coverage measures how many times a simulation executes each branch of an IF or CASE
statement. This information adds value to code coverage because not every branch contains
an assignment. The branch may be empty; it may be missing, as in an IF statement with no
ELSE clause; or it may contain further nested branching constructs. In Listing 2, branch coverage shows exactly which
branches a simulation executes and which branch conditions prevent access to the
assignments contained at the most nested branch. From the execution count of zero for
"if b = '1' and c = '0' then," it is clear the branching signals b and c need to
be verified with additional tests.
 Branch-subcondition
coverage, often abbreviated as "condition coverage," measures the combinations
of branch-selection subcriteria for complex branches occurring in simulation. This
operation can reveal if one or more sub-conditions have not triggered the branch or,
conversely, have always caused a simulation to enter this branch. Condition coverage acts
as a diagnostic aid in helping to locate which subcondition is causing problems with the
simulation run. This coverage also provides a higher level of control-structure test than
branch coverage provides. For the code example in Listing 3,
condition coverage shows what values of B and C a verification tool tested at this point
in the code.
Branch and condition coverage are higher level metrics than
statement coverage. These operations provide useful diagnostics, such as why a simulation
has not reached an assignment statement, and they give quality measurements that are more
accurate than those obtained by simulation alone. Path coverage is another essential
verification feature needed to encourage higher quality testing.
You apply path coverage to sequential-code blocks only in
an HDL. For example, a VHDL process may contain paths. A path occurs where one complete
branching statement follows another branching statement in the code. A branching statement
can be an IF or CASE statement. Path coverage calculates how many first-construct branch
combinations and branches in the second construct could be entered. In addition, path
coverage measures how many of the combinations execute during simulation. For example,
note the two IF statements, each with one branch, in Listing
4.
Four possible paths exist through this block of code. The
first path is to enter the first IF to enter the second IF statement. The second path is
to enter the first IF and execute the ELSE clause of the second IF. Path coverage measures
how many times a simulation executes all four paths.
In Listing 4,
neither statement nor branch coverage would provide a reliable measurement of how many
possible outcomes occurred, because a later assignment would overwrite an earlier
assignment in the process. Thus, knowing the paths taken through the code is a
prerequisite for knowing that all possible outcomes have been simulated. In this example,
the values of signals x and y cause the different paths to execute. Path coverage
effectively measures the combinations of x and y that occur during simulation.
In summary, you apply the four code-coverage criteria to
concurrent signal assignments, processes, procedures, functions, and operator bodies in
VHDL or to continuous assignments, procedural blocks, tasks, and functions in Verilog. The
criteria measure how well the simulation tests execute various control structures in HDL
code. Coverage metrics are particularly effective in measuring how well your verification
tool tests complex controllers, decoders, and multiplexer blocks.
HDLs create more challenges
An ASIC-design engineer typically faces challenges unknown
by his software-design colleagues. The first challenge is the concurrent nature of HDLs.
There may be many sequential blocks executing simultaneously in an HDL-model simulation.
Hardware-code-coverage tools assist the designer with simulation problems. Signals in VHDL
or variables in Verilog are the means by which sequential blocks interact.
There are several ways a code-coverage tool can measure
signal or variable activity. Process-sensitivity-list coverage exists in a VHDL
code-coverage tool. This coverage tests that each signal in the sensitivity list has
triggered the process and records how many times each signal has been the only signal to
trigger the process. Process-sensitivity-list coverage guards against one or more signals
in the sensitivity list having no value change during simulation.
It is important to measure the unique signal that triggers
a process. The triggering shows whether an event on one signal has no effect on the
process outputs because another signal masks that signal. For example, if a register bank
has clock, preset, and clear inputs in the sensitivity list, tests should reveal whether
the clear signal masks the preset signal or, conversely, preset masks clear. Furthermore,
tests should reveal if the register behaves as though the preset and clear are synchronous
or asynchronous.
To find out what complex interactions can occur between two
state machines in a design, you can use signal or variable tracing to construct a picture
of the joint-system states that the two state machines create. More important, the
designer can verify if he has tested all of the joint-system states. This move reveals
whether the design has avoided all possible deadlock conditions.
Signal tracing (VHDL) and variable tracing (Verilog) allow
you to nominate a selection of signals critical to the design. The tracing monitors
simulation results and presents a table containing all of the signal-value combinations
that occur. This information ensures that a design's state machines interact as designed.
You also use this data to debug the design and prove that simulation results are correct.
Thus, signal tracing has become a valuable debugging aid for ASIC designers.
A simpler form of testing the way signals behave in a
design is signal-toggle coverage, which identifies that all bit-level signals in the
design change from 0 to 1 and from 1 to 0. Additionally, tristate-signal-toggle coverage
can also examine the changes to and from a high-impedance (Z) value. Using signal-toggle
coverage reveals whether simulation has not activated some signals. You can use this
information to show if the HDL model has declared unused signals, signals that remain
constant throughout the simulations, or signals that have too few value changes. For
example, if a simulator resets a register bank at the beginning of a simulation and then
changes the register only once during testing, you may not need that register.
Alternatively, you may find that your test vectors may cause inadequate activity in a
register.
Subdesigns may differ
Design modules are often instantiated many times. This
instantiation leads to a concern that each module instance may behave differently from
other instances of the same module. The problem does not arise in software testing,
because a procedure called from several places in the source code still runs on the same
processor CPU, and all instances are sure to behave similarly. For pure HDL test coverage,
all module instances behave in the same way, so collecting code-coverage results across
all instances is acceptable.
With HDL-based designs, engineers still want to review
coverage of each instance. The reason for a separate review is that when a silicon vendor
fabricates the design, each instance is on a separate piece of the silicon chip.
Manufacturing tests thus need to separately test each instance. Where you use
functional-simulation vectors for manufacturing tests, RTL-code coverage on an
instance-by-instance basis becomes the starting point for manufacturing-test-coverage
planning.
Code coverage is a practical way to measure how well you
verify an ASIC design for correct functionality. Because HDL code coverage supports VHDL
and Verilog, you can easily add code coverage to an existing design flow. Many companies
recognize the benefits of ASIC-design code coverage: Designers have found redundant code
using this technique, and, in other cases, code coverage has discovered untested code that
was later found to contain design errors. Companies using code coverage have improved
design quality and achieved early testing-problem diagnosis. Thus, by embracing code
coverage, these companies have simultaneously shortened project design cycles and reduced
project costs. |
