March 14, 1997

Roughly right or precisely wrong?

Bill Schweber, Technical Editor

It's easy to be swayed by the apparent precision of data and forget to think carefully about its accuracy.

Recent news reports have stated that the US government may change the way it calculates the Consumer Price Index. Federal Reserve Board Chairman Alan Greenspan summarized his concerns with the current methods by saying, "It's better to be roughly right than precisely wrong." Despite all the data gathering that the government uses to gauge price changes, Greenspan was expressing the concern of many that the price-analysis methods used don't reasonably reflect changes in quality, performance, or features of purchases. His concern about the reliability of the final result should be a lesson for designers.

It's easy to be swayed by the apparent precision of data and forget to think carefully about its accuracy. We're conditioned to speak precisely, with numbers, even when it makes little sense: Market researchers are the worst offenders, making predictions to ludicrously fine precision ("The market for these widgets will grow 98.3% in five years"). Today's instruments are capable of performing sophisticated calculations based on high-resolution measurements, data manipulation, and advanced algorithms. But, when you step back and look at possible sources of errors, you soon realize that it is pointless to show answers to three, four, or more significant figures when there are uncertainties of 10% or more in the underlying assumptions and other factors.

When you have your results, you should step back and do a "sanity check." Be a little skeptical. Do these answers make sense? Are they roughly in line with what you expected? If not, what could account for the differences? Have you overlooked some factors? What assumptions did you use, and are they valid or at least recognized? Did you take enough data points for meaningful analysis? Conversely, did you take so many data points that interesting anomalies are averaged out? Or, is the fundamental data that you've taken correct but the subsequent analysis based on logical errors? If you make small changes in some of your setup parameters, do you get the expected changes in results? Can you meet the true test of any analysis and, based on your reasoning, predict the results you'll get with different initial values?

Historically, the analog domain has been less precise than the digital domain in a mathematical sense and more qualitative in the sense of cause and effect in circuit behavior. Analog designers look at questions such as: If I increase the bias current here, what effect does that have on distortion over there? At what point does this amplifier saturate and become nonlinear, and how does it depend on temperature and supply? Digital systems are inherently more precise, with resolutions of 8, 16, 32, or more bits, but sometimes they can lead designers to mistake precision for correctness. Keep these factors in mind as you try to interpret the results from tests on your initial development or prototype units.

Bill Schweber, Technical Editor Let me know what you think. Send me your comments via fax at (617) 558-4470 or over the Internet at bill.schweber@cahners.com.

| EDN Access | Feedback | Table of Contents |