Finding clock glitches

It was a dark and stormy night. It was dark because we had turned off all the lights in the lab as we anxiously studied the bright-blue-green glow of the oscilloscope, and it was stormy because our prototypes were not cooperating with our troubleshooting efforts, and the boss was getting antsy. (Note that “antsy” is an anagram of “nasty.”)

The previous Friday afternoon, all had been working great. Four of our brand-new babies were finally debugged, tucked safely into their card slots, and obediently passing error-free bits back and forth over the backplane and across the optical links. We went home with big smiles and plans for a weekend celebration, thinking that the project was near completion.

On Monday morning, the bit-error counters had climbed to record numbers and were steadily increasing. All the optical-link SNRs (signal-to-noise ratios) were great, but each card insisted on periodically, randomly, and independently flipping bits. The following days of troubleshooting efforts concentrated on the analog optical links. Who ever heard of digital circuits giving this kind of trouble? Digital circuits either work, or they don’t, right?

Cranking up the timebase and vertical scale to position the brightest trace portions off screen (but still with a phosphor glow from reflected electrons), we were able to just discern the ghost of something that was not right. Just after the clock’s falling edge, we observed an occasional small pulse, made very faint by its infrequent occurrence. Bingo!

Today, a digital scope set to “runt trigger” or “infinite persistence” would have immediately found this problem, but analog scopes from back then did not have those features. Finding problems like this took a lot of effort and perseverance, combined with good eyesight and a darkened lab at night.

It turned out that placing the card’s backplane interface buffers 3 in. away from the connector formed stubs. As more and more of these cards loaded the parallel backplane bus, the problem got worse. The backplane clock edges were ringing under the stub loading to the point at which the ringing occasionally passed the buffer’s digital threshold and added an extra clock cycle.

We respun the card, and it worked great.

Glen Chenier is a former telecom engineer and is now a design consultant. You can reach him at glen@teetertottertreestuff.com. Like Glen, you can share your Tales from the Cube and receive $200. Contact edn.editor@reedbusiness.com.