According to semiconductor-market-research company IC Insights, electronic content made up 23% of the total price of an automobile in 2004, and it will make up 40% of the price by 2010. Many of the earliest in-car electronic systems replace mechanical systems, and they provide more efficient operation of the vehicle for lower cost and higher reliability than the mechanical systems they replace. More recent in-vehicle electronics manage safety systems, such as air bags and intelligent restraint systems, which automatically attempt to adjust and change the environment in the vehicle to minimize the risk of serious injury and death to drivers and passengers while a crash is occurring.

In addition to new safety features, automobiles are incorporating more convenience features for assisting the driver and for providing passengers with information and entertainment. Car manufacturers usually isolate the electronic systems for passengers from the convenience and safety systems that assist the driver. This article focuses on those electronics that are available or are in development that improve the driver's ability to direct and operate the vehicle.

An increasing number of emerging safety features, such as object detection around the vehicle, begin as convenience features in high-end vehicles (Figure 1). The feature supports adaptive cruise control, in which the system enables the vehicle to adjust the cruising speed based on the road position; distance; and relative speed of objects, such as other vehicles, without explicit driver intervention. A different implementation of the feature enables parking assistance and blind-spot-warning systems that can alert the driver when there is an object or vehicle in a risky position relative to the driver's likely goal position.

An explicitly acknowledged value of convenience and safety systems for drivers is that they can make driving safer, easier, or less stressful. The implied value of these systems is that they might be able to alert and direct a driver's attention to an important detail, thereby avoiding an accident. Peter Schulmeyer, director of strategy for Freescale's Transportation and Standard Products Group, points out, "A main aim of in-car electronic systems currently in development or early deployment is to reduce the load on the driver."

In 2004, police reported approximately 6.2 million motor-vehicle accidents that killed 42,636 people and injured approximately 2.8 million (Reference 1). According to an NHTSA (National Highway Traffic Safety Administration) report and the VTTI (Virginia Tech Transportation Institute), driver inattention is the leading factor in most crashes and near-crashes. The report found that 80% of collisions and 65% of near-collisions involved driver inattention, such as from drowsiness or cell-phone use, within three seconds before the event.

The development and deployment of passive safety systems, which improve a vehicle's crash worthiness and survivability, are experiencing diminishing returns on the effort to improve them. Examples of preconditioning systems include tightening the tension in seat belts, moving the seats into a position best for the safest air-bag deployment, and closing the windows and sunroof in response to an imminent crash. The findings of this report support the significant value potential for electronic systems that can bridge the driver-inattention gap. This realization is pushing the focus of new systems for active roles that can alert the driver or actively affect how the vehicle is operating to avoid an accident in the first place (Figure 2).

As automobiles become more autonomous, the driver's focus will be able to shift toward directing the vehicle to the destination and away from the mechanics of operating the vehicle. Adaptive cruise control, for example, uses radar (radio detection and ranging) to maintain a safe distance from another vehicle while traveling on the road, thus offloading some of the cognitive load from the driver and filling in for the driver during moments of inattentiveness. The system can more quickly and more precisely respond than the driver to changes in speed of the leading vehicles.

Electronic stability control combines antilock brakes, traction control, and yaw control to assist the driver to maintain control of the vehicle. The system operates by comparing the direction the driver wants to go with the vehicle's actual response; it accomplishes this task by correlating the steering-wheel and braking sensors with sensors measuring vehicle yaw and roll acceleration, as well as the rotation of each wheel. When the system detects differences between the driver's commands and the vehicle's behavior, the system intervenes by applying braking forces to the appropriate wheels to correct the path of the vehicle for understeering and oversteering. The system may also include a connection to the vehicle's power-train controller to reduce the engine torque as needed.

To provide capabilities that help avoid crashes, systems need to be able to recognize potential crash indicators seconds before an accident occurs. It is important for a safety system to be able to infer the driver's intent and correlate that intent with the vehicle's behavior so that the system can avoid taking predictive action based on false and incorrect conclusions. Active safety systems that intervene as a crash is beginning become active within a fairly narrowly defined set of conditions and are associated with the vehicle's acting in an unstable or a distressed manner.

For systems that predict and recognize potential threats with time to avoid the threat, the need to accurately infer the driver's intent is more important and difficult. First, false alarms immediately erode and quickly destroy the value of the warnings because the driver either turns off the system or ignores the warning. For these systems to be effective, they must alert few or no false or irrelevant conditions to the driver. For example, for a driver that follows a consistent sequence of starting the vehicle first and immediately putting on a seat belt, the seat-belt-warning system offers marginal value because it always alarms when the car is turned on.

The challenges facing a blind-spot-detection system illustrate the need to infer the driver's intent. If the system warns the driver whenever an object was in the blind spot, the driver would quickly disable the system. To be useful, the system should warn the driver of a blind-spot condition only when the driver intends to change the vehicle's position such that a collision with the detected object is a real possibility. This requirement complicates the task of notifying the driver of a blind-spot condition beyond just detecting the object. Examining a single driver control, such as the turn signal, is insufficient to accurately determine that the driver intends to change lanes.

In addition to the sensors detecting the environment outside the vehicle, the sensors to infer the driver's intent can include collecting inputs from multiple points, such as the steering wheel, the brake pedal, the turn signal, the vehicle's inertial and acceleration-measurement systems, and even examining the driver's body language through an in-cab vision system. No industry-standard algorithms exist for inferring the driver's intent to change lanes; such algorithms are the result of independent, proprietary research and development by the Tier 1 automotive suppliers.

To support the need for high accuracy when acting on a risk condition, these systems are receiving inputs from multiple sensors and types of sensors. Each type of sensor has strengths and weaknesses, and, by combining different types of sensors to cross-correlate redundant information, the systems can make better decisions. Manufacturers are deploying or exploring such technologies as radar, LIDAR (light detecting and ranging), infrared, ultrasonic, and video for perceiving the environment (Figure 3). Each of these sensor technologies overlaps with the others, but each one provides unique strengths in different regions of the vehicle.

Some car makers are mounting cameras in and on vehicles. The cameras point not only outward from, but also inward to the vehicle. Outward-pointing cameras can complement the other outward-pointing sensors by providing details that enable the system to perform not just object detection, but also object recognition to support behavior prediction. Inward-pointing cameras provide the system with occupant information, such as size, position within the seat, body language, and even facial expressions. Inward-pointing cameras support functions such as smart air-bag deployment, as well as detect when the driver experiences drowsiness or intoxication.

The camera sensors continue to improve. For example, cameras that can support HDR (high dynamic range) can collect better image details across a wider range of lighting conditions than can cameras with a linear range. HDR-capable sensors are important for use with cars because they must operate in dark and bright environments. An HDR sensor, in contrast to a linear sensor, can detect details that bright environments wash out, and it misses fewer details in dark environments (Figure 4). HDR shines in situations in which both dark and light regions exist in the same scene. The linear mode of operation affords greater differentiation between brightness levels if the scene maintains one range of light level.

Other relevant sensor inputs capture information about the vehicle's behavior. These sensors include gyroscopes, accelerometers, steering-wheel- and brake-pedal-position detectors, and tire-rotation-rate systems. Another source of information includes in-vehicle knowledge of what the environment should contain through the use of digital maps with positioning systems such as GPS (global positioning system). Devices embedded in the road can wirelessly provide additional local and real-time information to the vehicle.

"Data fusion" involves combining these sensors and correlating their inputs. These inputs can enable the vehicle's electronic systems to identify its position and path relative to other objects near the vehicle, but, as these systems collect data from more types of sensors, there is an increase in the amount of needed real-time processing performance and of data that must traverse the in-vehicle networks (see sidebar "In-vehicle networks").

The vehicle network not only decreases the cost and weight of the vehicle by reducing components, but also enables designers to reduce or eliminate redundant sensors. For example, the air-bag and chassis system can share a gyro to sense the vehicle's stability and a rollover or impact condition. Sensor or data fusion may even enable designers to eliminate sensors or measure variables that are difficult to place a sensor near with virtual sensors. Possible opportunities for virtual sensors include measuring tire pressure and road-surface type and friction.

The biggest challenge for electronic assist, convenience, and safety features may be in implementing the HMI (human-machine interface). The HMI must ensure that each of the predictive, active, and preventive systems in the vehicle operates according to the expectations and the ergonomic and cognitive limitations of the driver. These features support the driver, and that support can have unintended consequences in how the driver's behavior may change in different situations.

As these systems become more autonomous, the driver has less control of the vehicle. Systems that are partially autonomous may make driving more dangerous because they may contribute to more inattentiveness from the driver; the driver's response time could suffer in those events that require a decision. For example, if a collision is likely, which of two vehicles should the car hit to avoid hitting the other? What if the choice is between two pedestrians? Manufacturers cannot leave all of the decisions to the vehicle's control system until it can accurately recognize these concepts and make appropriate decisions based on other external factors at the time of the incident.

The HMI must consider the cognitive load on the driver. The way it presents data to the driver is an important consideration. The idea of presenting everything to the driver on a display is a tempting idea, except that using a display demands the driver's attention and constant polling to interpret the presented data. The display could become a distraction from the road because the cognitive burden may be too much if the display is the mechanism to deliver warnings while driving. The more information available on the display while driving demands the driver to perform more cognitive processing to filter and interpret the data.

Telematic systems can offer a lot of convenience and value to a driver, but how the driver interacts with these systems is important. Merely switching radio stations proves to be a significant cognitive burden or distraction to a driver's maintaining attention on the road. VoiceBox's Telematics Navigator is an approach to reducing the cognitive load on a driver to navigate radio stations, as well as other telematic services that will become available to drivers. The system promises to allow the driver to use conversational language to search and navigate capabilities. Click here for a four-minute video demonstrating the system.

As the cost of the underlying components for these types of systems continues to decrease, the opportunities for new in-vehicle-electronics convenience and safety features will increase. Moving in lock step with the decrease in component prices is the building of the software layers for precursor functions that support more complex functions with more intelligence. Enabling software reusability and verifying increasingly complex systems at an industry level are keys to the forward momentum of these systems. AUTOSAR (Automotive Open System Architecture) is attempting to address the basic infrastructure for the management of functions for future applications and software modules.

One last hurdle for these emerging systems is the cultural acceptance of letting the automobile take over more of the tasks of driving. A complaint from driving enthusiasts about the existing systems, such as electronic stability control, is that they make driving less fun. Also, as these systems become more interdependent, the opportunity for self-adjusting the vehicle will decrease as such changes could adversely affect the performance of the entire system. The underlying concept of these types of systems is appropriate for any type of vehicle, such as planes, trains, and boats—not just automobiles. The success of one system in one type of vehicle could translate to acceptance in another type of vehicle.