Leibson's Law

The Hubble Space Telescope’s SIC&DH (Science Instrument Command and Data Handling) unit is once more operational after the switch to the Side B communications channel was followed by two failures. The Advanced Camera For Surveys (ACS) boot-up sequence detected that the low-voltage power supply had not achieved nominal voltage (see here) causing the ACS to shut itself down and the main flight computer (a 33 MHz Intel 80486 microprocessor, also called the “housekeeping” computer) safed the entire on-board scientific instrument package after it failed to observe a “keep alive” signal from the SIC&DH unit's NSSC-1 processor, aka the “payload computer” (see here).

NASA’s Independent Review Team looked at the events leading up to the shutdowns and has concluded that the ACS boot sequence contained a software race condition that caused the ACS processor run a limit-checking algorithm on power supply data that had not yet been collected. This classic software race condition appears to have been caused by software updates and clock changes that made the timing between the limit-checking and measurement routines “too tight.” Appropriate changes are being made.

The Independent Review Team attributes the loss of the NSSC “keep alive” signal to a transient short or open, which is within the realm of probability for electronics that haven’t been powered on in 19 years. The SIC&DH unit is now back online. If it stays online, NASA plans to reactivate the Widefield Planetary Camera (WPC-2) on Saturday morning. If things continue to go well, the ACS will be reactivated later.

It’s getting a bit hard to keep track of all the different processors involved here so I thought I’d provide a list of the processors that I “think” are involved:

• The Main Flight Computer, a 33-MHz Intel 80486 (aka the “Housekeeping” computer)
• The Science Instrument Command and Data Handling Computer, one of two redundant NSSC-1 processors (aka the “Payload” computer)
• The ACS processor, a 16-MHz Intel 80386 (hardly ever discussed explicitly)

With three processors in this particular chain, the Hubble is akin to a very large, heterogeneous MPSOC and it’s displaying some of the same sort of programming problems that you might run into when designing such a product.