Many years ago, I was involved in an electromechanical project that included hydraulics, a microprocessor, custom chips, power circuits, software, dc motors, solenoids, and the like. I was responsible for overseeing the electronics and software that a supplier designed. Initial design and testing went well, but our team decided that we needed some early customer testing before starting production.

We wanted to continuously monitor and gather data on the operation and performance of the system. Therefore, I designed a field-data logger for the system that would communicate with the system via a serial-data link. We installed systems including these data loggers at a couple of customer sites and soon began getting reports of erratic system operation. Upon examination, we learned that one of the primary sensor signals that the system reported to the data logger was erratic. But, when we tested the systems, they all worked fine.

Nothing we did could make them operate erratically. The system needed to operate from –40 to +125°C outdoors and in harsh environments, but, in extensive lab testing at –40, 25, and 125°C, the testing showed no erratic operation. Production wasn’t far away, and, if we couldn’t resolve this problem, we would have to delay the start date, which would be expensive.

We concluded that the system electronics were corrupting the signals. Examining the data from the data loggers showed what time of day the erratic operation was occurring. We compared the erratic-incident time with the lab-test results and found an interesting phenomenon: The erratic operation was occurring over a narrow temperature range, inconsistent from system to system. One system showed the problem at 40 to 80°C, and another showed it at 60 to 90°C. Each system that displayed the problem had a different starting temperature and a somewhat different range; no problems were occurring at the nominal ambient of 25°C or at the temperature extremes. We quickly narrowed the culprit down to the signal-processing chip in the system’s electronics.

The custom-built IC had both positive- and negative-temperature-coefficient components that determined the sensor-signal thresholds and hysteresis. Because the internal component’s temperature coefficients were not matched, at certain temperatures over a narrow range, the signal threshold would rise to or slightly above the sensor’s output, and the needed signal hysteresis would disappear. We redesigned this custom IC and expedited production. We barely made it into production on time, but we had no further problems.

We learned a number of lessons and had a bit of luck. We designed the system with the ability to report incoming and internal signals and status, without which the problem would have taken a lot longer to detect and fix. We opted for early customer testing well ahead of production. The biggest lesson we learned was that you must monitor temperature-sensitive components, parts, or subsystems with temperature-sweep testing over the full specification range.

Consultant Clark Robbins is a software-application engineer. Like him, you can share your Tales from the Cube and receive $200. Contact edn.editor@reedbusiness.com.