Boeing postpones test flights again: how's your tape-out looking?
Boeing–giant aircraft manufacturer, equally giant but much lower-profile defense contractor, and one-time paragon of complex project management–has hit the newspapers twice in recent weeks in the worst possible way. The company has announced that it is taking financial charges because its two premier commercial projects—the 787 Dreamliner and the 747-8 cargo version—are both delayed, again. Neither will meet its most recent schedule for first test flight. This is a direct concern for suppliers providing electronics into those programs, of course. But it is also a cautionary tale for any chip design team engaged in a complex project.
The sadness of it all is the irony that this should happen to Boeing. In the late 1960s, Boeing pretty much astonished the aircraft industry by creating arguably the world’s most complex jetliner to date—the initial 747—on a tight schedule and at enormous risk. The company bet essentially its net worth on the 747 project, on a schedule so tight that the first planes for delivery were on the running production line right behind the plane for flight testing and FAA certification. If something had gone seriously wrong, the company might have had to rework a year’s worth of deliveries in parallel on the factory floor. But the plan, and the plane, worked. The 747 project became a case study in project management, and forever altered the landscape of the commercial aviation market.
That was the 60s: limited use of computers for simulation, project management, or resource planning, and manual project-management skills learned over decades by engineers trained in the must-do environment of the Second World War, the Cold War, or the Apollo program. This is now: everything from design databases to simulation to project management to interpersonal communications is computerized, often supervised by managers whose proven expertise was in getting an MBA, not getting a project out the door. And this is the era, we are insistently told, of outsourcing.
According to published reports, these changes lie at the heart of Boeing’s problems. To paraphrase one report, the 787 is an entirely modularized design. The modules were outsourced. It was intended that final assembly would comprise simply snapping together functionally completed modules—already carrying the necessary electronic subsystems, cabling, plumbing, and mechanicals–in Boeing’s final-assembly building.
But Boeing found when the modules started arriving that they had somewhere lost control of the module designs. Some things didn’t fit. In some cases module designs exceeded the capabilities of the subcontractors, and so they arrived with subsystems missing. Modules required further assembly after delivery.
Perhaps more disturbing, software simulations of the system-level mechanical behavior of the composite materials of which the modules were constructed turned out never to have been reality-tested at a full-system level. There is a tragic photo of a complete 787 airframe on the Boeing assembly floor trussed up like an accident-victim in traction, with ropes and weights, attempting to simulate the static loads on the airframe.
By now I’m sure this is beginning to sound relevant to chip design managers. We, too, live in a world of pervasive outsourcing, increasing dependence on simulation at several different—and unlinked—levels of abstraction, and no chance at a reality check until someone assembles the complete system. We too live in the shadow of that possibility that a system-level design error will only become visible when we flatten the design for physical extraction and DRC. Or at silicon debug.
In some ways our problem is more controllable. There are fewer degrees of freedom in even a challenging mixed-signal block than in a physical chunk of a jumbo jet. We have put a lot of investment into tools to manage those degrees of freedom we must accept. And unlike Boeing, no chip design team is the only one in the world doing what it is doing. (Intel R/D people are welcome to dispute this point—you have an argument.) So there is a body of experience, however imperfectly we share it.
Yet the caution is still there. It only takes a tiny bit too much confidence in the tools and abstractions; just a little too much casual dealing with the company, language, and time barriers between the core team and the subcontractors; one step too far down the path of partitioning; to turn a challenging project into a catastrophe. And unlike Boeing, which is limited in its ambition by the realities of mechanical design and the needs of its customers, we are all servants yet of Moore’s Law. To cite Intel once again, in paraphrase this time, only the paranoid are likely to survive.
CATeng commented:
His_wife80 commented:
JohnZ commented:
BobGroh commented:
ipse commented:
20 year EE commented:
A new day commented:
KAM commented:
Been there B4 commented:
Been there B4 commented:
Been there B4 commented:
kg5q commented:
Medical Device OEM commented:
A new day commented:
Linn commented:
