The case of the stolen capacitor
Tales From The Cube: An engineering change order erases a design's input stabilizing capacitance, with costly results.
By Glen Chenier, Teeter Totter Tree Stuff -- EDN, September 4, 2008
It was the usual team project: a group leader with overall responsibility for the end product, with assistant designers working on specialized subsystems. I was designing the fiber-optic-communications links for the product, and, not wanting the noisy 5V digital-VCC supply to contaminate my sensitive analog circuits, I elected to linearly regulate the 12V down to a reasonably clean 5V to keep my analog circuits happy. The data sheet instructed me to include 10 µF of stabilizing capacitance at the input to my voltage regulator. Read on to learn the very good reason for this inclusion.
In the week before shipping the product, a horrendous problem emerged. Hot-plugging a card into the live backplane caused a glitch on the system’s 5V rail and caused the main processor card to restart. Oops! I managed to arrive at a Band-Aid solution to that problem by attaching tantalum capacitors and fuses across the 5V rail on the backplane at every card slot. I then suggested that we should perhaps also look at the hot-plug effect on the 12V rail.
Surely enough, a similar problem resulted; hot-plugging caused a glitch on the 12V rail, in turn causing another colleague’s PLL (phase-locked loop) to hiccup. This glitch did not bother my 12 to 5V regulator because it had lots of filtering on the output side.
|
So, the team lead came up with a “fix” that resulted in the insertion of a resistor that effectively separated the capacitor from my regulator and stole my regulator’s input capacitor for use in the PLL-filtering function. When I saw his ECO (engineering-change order), I told this colleague that this fix was not doable: I needed that capacitor for my regulator. His reply was that our prototypes worked without the capacitor on my regulator and that it was too labor-intensive to attach another capacitor during the assembly rework. (Another “Oops!” ensued.) I did not press the point; after all, it was his circuit card, and, as group leader, he made the final decisions.
Six months went by with no problems. Then reports suddenly started coming in from the field that new-production cards would not work when customers installed them into systems. The clincher occurred when a field engineer called us to ask whether it was normal to have a 10-MHz sine wave riding on the 12V rail. We investigated, and found that the production department had changed to a new vendor of the voltage regulators. The new units were some no-name brand for a few pennies less and with no vendor logos or markings on the parts. What’s worse, the production department did not keep the necessary records to indicate the original manufacturer. Nevertheless, these regulators really did require some input stabilizing capacitance.
The result was the addition of capacitors that we should have included during the initial rework, even though it was too labor-intensive. Yet, it would have been far less-labor intensive than the resulting recall of the cards.
I learned a good lesson from that experience. Even though it was ultimately not my responsibility, I should have insisted that the team leader include the additional capacitor. I have since applied this lesson to later team-design issues, and some people now think of me as an ornery old curmudgeon, but the resulting designs always worked properly.
You can reach design consultant Glen Chenier at glen@teetertottertreestuff.com. Like him, you can share your Tales from the Cube and receive $200. Contact edn.editor@reedbusiness.com.
-
There is an old saying that goes along with this: There is never enough time or money to do the job right, but there is plenty of both to do it over (an over...) again. I learned this in my first few months as an engineer. In over 40 years I have not forgotten it.
Harvey Altstadter - 2008-11-9 10:34:00 PDT -
If it was a 10MHz oscillation then it could probably have been tamed with a very small capacitor, say 0.1uF at the regulator input. Your use of a large cap on the power rail of a hot-pluggable board is problematic. When that cap is suddenly dropped across the power rails of the backplane, very large and uncontrolled currents can flow. You experienced one consequence of this. Another is damage to the mating contacts. I believe that you can find ICs that are made to solve this exact problem in hot-pluggable systems.
George Pontis - 2008-11-9 10:32:00 PDT -
I agree. If the datasheet requires a stabilization cap, it had better be there in the application. That cap value has been researched by the IC design engineer and is not there for grins and giggles. However, the problem in this situation was more to do with the purchasing agent that knows absolutely nothing about semiconductors. How on God''s green earth does anyone expect to have a stable reliable product if the purchasing department can stick any off-brand semi into a socket without some level of qualification?
No changes in vendor selection should be made by folks that are clueless as to how the circuit works. No two ICs are the same. Especially when you consider that the vendor didn’t even want to put their logo on the package… They were probably either counterfeit parts run on a minimally stable process in some third world country or they were line rejects stolen from a legitimate supplier.
There should be a procedure in place that forces some level of verification that the new vendor’s component will work in the application.
In automotive applications this verification is to the level of the bizarre. And rightfully so, when you consider that a $0.15 IC can cause a $35k vehicle to, say, stall in an intersection.
If it were my company, I would smack the purchasing agent on the nose with a rolled up newspaper and say, “Bad buyer”.
David Swanson - 2008-11-9 06:25:00 PDT
