Feature

Real-time software reigns in post-PC products

An RTOS is essential to untangle the complex software that you need to drive the new generation of interactive and networked embedded products. Review the real-time fundamentals before you pick the product to jump-start your next design.

By Warren Webb, Technical Editor -- EDN, 2/17/2000

Embedded designers are flooding our lives with dedicated, non-PC devices. The average person interacts with hundreds of embedded processors every day in phones, automobiles, home appliances, toys, cash registers, entertainment electronics, security systems, environmental controls, and personal electronics. And now the second wave is under way, as many of these devices connect to the Internet or a local communications network. The common link between all of these products is their ability to react in real time to the user, external events, and the communications channel.

If you have ever watched a teen-ager interact with a video game, you know the definition of real time. Imagine writing the code to capture all the inputs indicating directional movement or weapon fire and simultaneously updating the display to show the action without the perception of a computer delay. You need a real-time operating system (RTOS) to ensure that your routines execute on time in response to external events. You set up the priorities and data dependencies, and the RTOS manages your application software even with a flurry of external, real-time activity. You can write each of your software routines independently without getting bogged down with intertask timing problems. Messages that pass between routines give the user the perception that everything is running simultaneously.

An RTOS, like a high-level language, should ease your development job and shorten your schedule, but selecting the right RTOS for an embedded product can be a harrowing experience. Unlike the limited choice in desktop systems, there are nearly 200 operating systems to choose from, each with subtle differences (see sidebar "Embed the desktop"). But the RTOS-vendor landscape constantly changes. Two of the most popular RTOSs, VxWorks and pSOS, now reside under the same roof with Wind River Systems' acquisition of Integrated Systems. Before settling on an RTOS, you should understand scheduling algorithms, signaling features, memory requirements, latencies, tool support, and pricing models. Alternatively, you may be tempted to entirely bypass a commercial RTOS and write your own real-time routines.

Hard or soft?

Real-time systems automatically execute software routines or tasks in response to external events. RTOS vendors have invented terms such as "hard" and "soft" real time to describe the operation of their systems. Hard real-time systems are scheduled so that tasks are guaranteed to start within a precise length of time from an external event. Hard real-time systems are deterministic. Soft real-time systems generally list the average length of time to start the routine, but a small probability exists that the maximum time will be much longer. Mission-critical applications must be deterministic. For example, an air-bag controller, antilock brakes, and even arcade games must react in a known time. Soft real-time applications usually respond within a few seconds, but an occasional slow response is not serious.

The basic architecture of a multitasking RTOS includes a program interface, the kernel, device drivers, and optional service modules (Figure 1). The kernel is the core of the operating system and provides an interrupt handler, a task scheduler, resource-sharing flags, and memory management. Calls to the kernel's application-programming interface request the kernel's services (see sidebar "Posix: application transporter"). The kernel is active continuously during real-time operation and must remain memory resident. An alternate approach in the Neutrino operating system from QNX Software implements a microkernel architecture incorporating traditional kernel components as optional modules.

One of the kernel's primary functions is to process interrupts that external or internal events cause. When an interrupt occurs, the processor transfers control to an interrupt-service routine that logs in the interrupt, sends a message to the scheduler, and returns to the active code. You normally write interrupt-service routines in assembly language; they must be short because most RTOSs disable global interrupts during their execution.

The kernel also includes the scheduler that sets up the order of execution of your application code. Your software consists of many individual tasks, each with an entry point, a priority, and a stack space. The priority ensures that a higher priority task can pre-empt a lower priority task to maintain a deterministic response.

RTOSs vary in their scheduling techniques. They can schedule threads of equal priority by a first-in, first-out, or round-robin, method. This method works if no task has a deadline that is shorter than the time it takes to execute all tasks combined. This type of scheduling is similar to software polling all interrupt inputs for activity. A simple modification of the round-robin scheduling technique adds time slicing, guaranteeing each task some processor time even if some tasks take a long time to complete. However, time slicing equally allocates processor time between tasks, and there is no way to ensure that the RTOS performs time-critical tasks first.

The most popular scheduling technique is pre-emptive, prioritized scheduling. Tasks can pre-empt a lower priority task and keep the processor until they are finished or until a higher priority task pre-empts them. This technique requires careful assignment of priorities to ensure that tasks are completed on time. Most RTOS vendors include tools to analyze priority scheduling and view the results. For example, SurroundView for NucleusPlus from Accelerated Technology provides live application-level data monitoring and task profiling (Figure 2).

On time, every time

The ultimate scheduling technique is to prioritize tasks depending on their execution deadline. This approach allows the scheduler to dynamically change priorities of other tasks to ensure that each task is completed on time. The drawback of this approach is that it is time-consuming to compute the deadline of each task before scheduling each new task. So far, dynamic scheduling has been restricted to research projects and robotics.

Even the best scheduling algorithms have problems. The Mars Pathfinder mission made RTOS priority-inversion problems famous. To illustrate this concept, assume that a long-running, low-priority task calls a high-priority task. The kernel changes the priority of the running task to that of the highest priority subelement. Unless the scheduler resets the priority when the high-priority routine is complete, the low-priority task can block medium-priority tasks; thus, a priority inversion occurs. One technique for solving the inversion problem is priority inheritance, in which the scheduler automatically reassigns the high-priority task the same priority as the calling task.

In addition to priority scheduling, a real-time kernel also provides objects for task synchronization. Look at the operation of a printer, for example. If several tasks wanted to print at the same time and had to wait only for the printer to be ready to accept the next character, the print-out would be garbled. Characters from each task would be intermingled on the page. RTOSs handle this situation with a semaphore. The first task to use the printer locks the associated semaphore until the task is finished printing. Other tasks must wait for the semaphore to be unlocked to begin printing. Then the highest priority task (or the one waiting the longest) gets the semaphore. Semaphores may also allow more than one task to simultaneously use a resource. In this case, the programmer initiates a counting semaphore and specifies the maximum number of simultaneous users.

Memory allocation is also a kernel service, and it impacts the performance of your real-time system. As each new task begins execution, the memory-allocation module searches for a free memory block adequate for the task. Depending on fragmentation, the memory-search timing is variable and contributes to task-switching latency. Fortunately, most RTOSs provide for preallocateing memory in fixed locations to bypass this variable time search. In the Integrity 2000 RTOS from Green Hills Software, tasks run in their own memory segment, requesting kernel services via a trap mechanism that prevents them from directly accessing or corrupting critical kerneldata structures (Figure 3).

The kernel design is also important in minimizing the latencies that can degrade the performance of your real-time system. Interrupt latency is the worst-case delay between an external event, such as a switch closure, to the first instruction of your interrupt routine. If the processor interrupts are enabled, the hardware delay is usually nanoseconds, but this delay can vary from processor to processor. The processor needs only to complete the current instruction before jumping to the requested interrupt location. If the processor has only a single interrupt line, the time required to poll inputs to determine which interrupt routine to call is part of the interrupt latency. You can also affect interrupt latency in your application software by disabling interrupts for a time-critical section of code. For example, if you disable interrupts to generate a known pulse width on an output pin, the interrupt latency will be at least as long as the pulse.

Out of context

Another latency that the kernel contributes is due to the time it takes to context-switch between tasks. Each task has a program counter, a data-area pointer, register data, and other state information that the kernel must save for the current task and restore for the pre-empting task. Some operating systems also require that the system call return to the application software before the context switch is allowed. You can easily measure the average timing of these latencies with an oscilloscope or a logic analyzer. However, worst-case combinations that create long and potentially fatal latencies may not show up until your product is in a customer's hands.

Besides the kernel, RTOS vendors are adding a variety of optional modules to entice you to apply their products in your next application. For example, almost every RTOS includes Internet communications protocols, such as Transmission Control Protocol/Internet Protocol (TCP/IP). Most of the major RTOS vendors also offer graphical-user-interface routines. The user can add or delete these modules as needed, depending on the application. In another recent example, QNX Software has added multimedia extensions to its RTOS to participate in the gaming and entertainment markets. Optional modules provide decoding support for popular audio or video formats and interfaces to 3-D-effects cards. RTOS vendors often team with third-party software developers to expand their optional module offerings.

If your application involves heavy data processing, you should investigate RTOSs that can easily scale up to multiple processors. You can spread tasks across several processors to gain a significant performance boost. Applications such as radar- and sonar-signal analysis use hundreds of processors. The Virtuoso DSP RTOS from Eonic Systems allows designers to ignore the number of processors in the system and how they are connected. The RTOS provides the communications and synchronization services to make the multiprocessing transparent.

Most RTOSs are designed for 32-bit processors. The application complexity and multiple real-time inputs result in large software-development teams, an ideal situation for a commercial RTOS. As you step down to 16-bit processors, you find that most users are in the automotive industry. European manufacturers have created a slimmed-down RTOS standard called OSEK to serve automotive applications. (The OSEK initials are from the German phrase for "open systems and interfaces for in-car electronics.") Although 8-bit microprocessors handle most real-time applications, few 8-bit RTOSs exist. Many of the systems are small enough that you can incorporate real-time processing into the application code. However, you can find commercial RTOSs to support the 8051 and 6800 processors.

Roll your own

Homegrown operating systems still occupy a large percentage of our real-time products. Sometimes, a small system grows over the years, and a lot of legacy code exists, which is difficult to convert to a commercial product. Designers often insist on total control over the hardware and software, claiming that only they can produce the smallest, cheapest, and fastest product. Writing a real-time application is also a great engineering challenge. Nonetheless, shrinking budgets and shorter schedules are helping RTOS vendors sell their software. See Reference 1 for a complete discussion of the RTOS buy-or-build decision.

The development-tool chain is another big issue in the selection of an RTOS. You will spend most of your software-development and -debugging efforts interacting with the integrated development environment (IDE) to gain quick access to the editor, compiler, linker, downloader, and runtime tools. Consider the costs before changing your IDE because a new set of tools may require a significant learning curve. Tools from another vendor may also work successfully with your IDE. For example, the XRay debugger from Mentor Graphics is popular and has been integrated into a broad set of tool chains (Figure 4). Reference 2 provides a summary of debugger features and vendors. Software vendors also supply software-performance-analysis tools to help you profile and visualize the real-time activity in your software. Many of these analysis tools are optional and add to your overall tool cost.

Another purchase decision that you must make is whether you need or want the source code for the vendor's RTOS. Some vendors automatically supply the source code when you purchase their software. If you need more memory space, you can tweak their software to remove every line of unused code. The source code can also help you understand those subtle bugs in your application code. However, vendors that supply only object code claim that if you change the source code, you create a unique and unsupportable RTOS. You can always purchase a copy of the source code to document your system or to deliver to your customer. Source code comes with Accelerated Technology's Nucleus Plus and Embedded Systems Products' RTXC operating systems.

You may be surprised to find that the prices of commercial operating systems vary widely. Because there is essentially no cost of goods with software, vendors can adjust their pricing models to generate revenue from various sources. Vendors can charge for an initial license fee, development seats per engineer, development tools, per-unit royalties, continuing support, or major upgrades. To fairly compare prices, you must compute the overall cost of ownership for each RTOS that you are considering. Your calculation must include the number of units that you plan to deliver over the life of the product. For example, Green Hills Software includes its royalty-free VelOSity RTOS for free with the purchase of its compiler and development tools.

You can look forward to real-time software as the norm in future embedded products. Customer demand for faster responses, complex functions, and instant data access will increase the challenge of embedded design. Typical projects will includea 32-bit processor, a large software-development team, and an RTOS. Fortunately, real-time vendors will offer plenty of off-the-shelf software modules to keep you from reinventing standard functions. So visit our Web site, pick your favorite RTOS, and join the real-time revolution (see sidebar "Looking for an RTOS? Start here.").
   

Embed the desktop When you start a development project, it is tempting to consider a familiar operating system, such as Windows or Linux. Simply select one of the many PC-compatible single-board computers for your embedded system, and install the same software that you have on your desktop computer. You have the immediate advantages of a well-known application-programming interface, familiar tools, a ready pool of development talent, built-in networking support, a graphical user interface, and plenty of third-party extensions. But this warm feeling starts to fade when you look at the real-time performance of desktop operating systems. Recognizing the problems with adapting desktop operating systems, Microsoft introduced Windows CE and Windows NT Embedded for the embedded market. Both products retain many of the advantages of desktop development. Windows NT Embedded allows a user to configure a custom subset of modules and supports headless (no-display) operation. A minimum configuration system still requires more than 16 Mbytes of memory. Although standard Windows NT Embedded has the same real-time characteristic as the desktop system, you can add a real-time subsystem from VenturCom to deliver worst-case response times of less than 20 msec. For systems of around 1 to 16 Mbytes of memory range, Microsoft's Windows CE gives a response time of 50 msec. The next upgrade promises much faster responses. The lure of an open-source, royalty-free operating system has designers scrambling to modify Linux to fit their embedded products. This craze has spawned a number of companies that provide Linux-customization services and software support to embedded-system manufacturers. Zentropix Computing offers a modification of the Linux kernel that promises deterministic, real-time operation.

Posix: application transporter Sometimes, nightmares come true. You finally get all the bugs out of your embedded software, and you find out that the manufacturer plans to stop producing your processor. Good grief! You have to transport your application source code to another processor and operating system. This scenario could be a real-life nightmare unless your code conforms to a standardized interface. Posix (portable operating-system interface) is a set of standard program interfaces to ensure portability between operating systems. Posix was derived from the Unix applicationinterface. Many of the major RTOSs offer Posix as an alternative interface to their kernels. The Posix interface is more cumbersome than the proprietary interface and requires additional memory for the translation software. Unfortunately, few developers opt for the Posix interface unless a contract or company policy mandates it. Posix standards are defined by a decimal following the Posix. For example, Posix.1 is the standard for an application-programming interface in the C language. You can obtain copies of the Posix standard from the IEEE Standards Association (http://standards.ieee.org).

 

Looking for an RTOS? Start here. EDN is building an online database of real-time software. This database will be a direct link between product vendors and you. If a vendor adds a feature or module to its product, EDN will update the database. As the database grows, you'll be able to search for specific technical or cost criteria. Because the data is dynamic, you can check back any time to find the latest information for your next real-time, embedded project. To get to the real-time database, visit our embedded-system Web page .

---

Author info

You can reach Technical Editor Warren Webb at 1-619-513-3713, fax 1-619-486-3646, wwwebb@cts.com.

 

REFERENCE

1. Webb, Warren, "Embedded operating-system software—build or buy?" EDN, Aug 3, 1998, pg 77.

2. Webb, Warren, "Squash your embedded debug time" EDN, Oct 14, 1999, pg 113.