My first job out of college was for a military-electronics company. We built rectangular metal boxes full of electronics that go into aircraft and space vehicles. Both plug and socket types of keyed circular connectors covered the boxes. We knew that bad things could happen if we mismated the connectors, so we went to great lengths to make sure to plug together only those connectors that were intended for each other. We varied the sizes of the shells and also varied whether a given connector was a plug or a socket. We avoided having two plugs or sockets of the same diameter. If we absolutely had to have two shells of the same size and type, we made sure that each one had unique keying configurations that were incompatible with any other connectors on the box.

So, in accordance with Murphy’s law, an aircraft-maintenance technician somehow managed to mate two plug-type connectors together, despite all of the size, keying, and other preventive measures that we had designed in. The result of this configuration was a large flash and loud boom when he powered up the aircraft. The electronics unit was destroyed, as were other circuits in the aircraft.

Although we were impressed with the technician’s ingenuity and perseverance, the officers in his chain of command were not amused. The general in charge of the air base got directly involved.

He first ordered us to determine what would occur if any connector signal on the box were to be connected to any other connector signal on the box, regardless of whether it was physically possible. After determining that outcome, he instructed us to make whatever circuit changes were necessary to make such a mismating innocuous. He didn’t go as far as to tell us to make it work in that configuration, but he did insist that no damage occur.

Our managers, duly chastened after their visit to the general’s office, immediately set to work. They put together a task force to start the cross-connection analysis. The task facing us was enormous. The box had 20 connectors, each with about 30 pins. When we began, it took several hours to analyze the effect of shorting a pair of signals together. As we got used to the work, that time dropped to about one hour per pair. This phase was only for analysis, however; it didn’t include redesign. After several days, we saw how little progress we were making. We decided to determine the scope of the task.

The math wasn’t hard: 20 connectors of 30 pins each totals 600 pins. The combination of 600 items, two at a time, is 179,700 combinations. At an hour each, that job would take 179,700 man-hours, or 86.4 man-years. The four people on the team could count on spending the next 21+ years doing these analyses.

We presented our figures to our management. They didn’t say much, but we saw some eyebrows rise. Management took our figures to the general. We never heard what the general said, but the next week we were taken off the connector analysis and returned to our interrupted projects.

The lessons I learned from this experience were that you should always define the scope of a job in the beginning, and, whenever you think you have made something idiot-proof, remember, along will come a better idiot.