The confidence interval: How it relates to serial data links
I've become alarmed recently at the number of young engineers (i.e. those with less than 5 years of work experience), who seem to have missed the college course on applied probability and don't know how to quantify their certainty in an estimation. In high speed serial communications, this takes the form of estimating the bit-error ratio (BER) of a communication link and quantifying one’s confidence in that estimate. I’m hearing more and more young engineers say things such as:
"Well, I’m pretty sure it’s running below 1E-12 BER," or worse, "I’m about 99% certain in that BER estimate I gave you."
Often, that second statement is made without any basis for the 99% claim. That number seems very attractive because it suggests near certainty, while leaving the claimant with a 1% plausible deniability in case things go South and he needs an exit strategy. None of this marketing-esque thinking belongs in any engineering group!
The confidence interval
Let's first review the very precise (and elegant, I think) mathematical definition of the Confidence Interval:
where m’ is a variable integer, m is the number of errors actually observed, BER is the actual bit error ratio of the link, and is the estimate being offered.
In English, Equation 1 says, "my confidence in my estimate is simply the probability that I would have observed more errors than I did, if the BER were any worse than I'm claiming."
The Binomial Distribution given in Equation 2 defines the probability of observing a certain number of errors, m, in some number of bits, n, given the probability of any single bit being erroneous, p.
While exactly correct, Equation 2 is not practical, due to the term, n!, which is, typically, very large; too large, in fact, for most digital calculators or computers to handle. Instead, we must find an approximation to Equation 2, which our devices can handle.
In the realm of serial communication, and assuming well designed links, both m and p in Equation 2 are, typically, very small quantities, while n is very large. When this is true, we can make the following two simplifying approximations:
Making these two substitutions in Equation 2 yields the well known Poisson Distribution:
Note that all large factorials have been eliminated from Equation 7 making its calculation possible using digital computers.
Confidence interval vs. error free observation time
With a usable expression for P(m) at our disposal, let’s now try to evaluate Equation 1 for the case in which we've observed a link run error free for some number of bits, n. Expanding Equation 1 in a straightforward fashion yields:
Now, while some of these infinite sums have closed-form solutions—via clever algebraic tricks—this is not one of them. We are, however, in luck because we can draw upon the following general truism of probability:
which states that the probability of an event is equal to one minus the probability of its complement. Using this, we can rewrite Equation 8 as:
Because m is almost always quite small, Equation 10 is usually tractable for well-designed serial communication links. In this particular case (error free observation time), m = 0, and we have:
Table 1 gives the calculated confidence intervals, for several different values of the quantity, np. The quantity np can be thought of as the error free observation time, normalized to UI/, where UI is the unit interval. In other words, if we are estimating the BER of our link at 1E-12, then a value of np = 1 means we observed 1E12 bits, and a value of np = 5 means we observed 5E12 bits.
Table 1 Confidence interval vs. error-free observation interval
As you can see from the data in the table, it requires an error-free observation interval of 5E12 bits, before one can claim that a link is running with a BER at or below 1E-12 with better than 99% certainty.