Operating systems (OSs), large and small, are evolving. Monolithic structures are giving way to modular designs. The OSs' modularity, in turn, allows designers to readily adapt them to new environments, prune off and graft on needed services, and plant commercial OSs into application niches in which they could never before thrive.

A growing adoption of microkernel architectures is driving this OS evolution. Ref 1 defines a "microkernel" as a program that provides four minimal services:

• Interprocess communications,
• Low-level input and output capability,
• Limited scheduling and process management, and
• Basic memory management.

All other traditional OS services, such as file and directory management, reside as user-level servers outside the microkernel.

The microkernel architecture debuted in Unix OSs. Unix developers had, over time, added many functions to their monolithic kernels. The ultimate results were multimegabyte OS kernels. To make those kernels smaller, the designers of OSs such as Chorus Systems' Chorus Nucleus, the Open Software Foundation's Mach 3.0, and QNX Software Systems' QNX turned to the microkernel architecture. Designers reduced the kernel to the bare minimum-10 to 100 kbytes. The term "microkernel" reflects the system's size relative to the monolithic original.

The microkernel architecture, a bare-bones core with a collection of serverlike system-resource providers, is inherently modular. This modularity, in turn, provides the microkernel OS with tremendous advantages over monolithic designs. These advantages include simplified maintenance and modification, portability to new processors, flexible operation, an ability to run while distributed over multiple processors, and compact code.

The microkernel approach's modular nature simplifies the problems of code maintenance and modification. In contrast, many traditional monolithic OSs grew without structure, as designers repeatedly added code over years. As a result, monolithic designs developed internal coupling, allowing modifications in one section to affect the operation of other-apparently unrelated-sections. Thus, designers must take care when attempting to fix bugs or extend a monolithic OS's capability, making maintenance and modification difficult. End users who modify a commercial OS to meet their needs do so at some risk.

With the microkernel approach, designers can add and debug new system-service modules with less risk of system crashes because these services run in protected memory at the user level. Errors within the modules do not affect the OS kernel, and adding a service does not introduce conflicts with existing services. The microkernel approach also allows service-module designers to employ the range of debugging tools available to application designers. The system servers are, after all, running at the user level, just as an application program does.

Microkernel designs can simplify porting an OS to new processors. The microkernel encapsulates most of the processor-dependent software in a small, manageable package. Once a designer ports the microkernel to the new processor, the designer can simply recompile the remainder of the OS, rewriting only those services that the new processor does not physically support.

Flexibility is another hallmark of a microkernel OS. A microkernel system's personality traits, such as file formats and user interfaces, reside in servers. The user needs only an appropriate set of servers to obtain any OS personality. In addition, the OS can support more than one personality. If the OS contains a collection of servers for different personalities, the system can appear as a Unix machine for one user, a DOS machine for another, and so on. Further, the microkernel OS allows dynamic modification. You can add or delete system resources while the system remains running; you need not reboot. This feature makes the microkernel OS perform the software equivalent of hot-swapping boards.

The system-resource servers that compose the OS interact with the microkernel by exchanging messages. The messages need not pass through registers or shared memory; they can just as effectively move over a network. A microkernel OS, therefore, readily maps to a multiprocessor configuration. Servers can reside on any of the various processors and still exchange messages with the microkernel. However, although such distributed operation is structurally easy with any microkernel architecture, the microkernel's design must incorporate such distribution. Unless the interprocess communications protocol responds appropriately to lost messages, distributed systems can crash after one node fails.

The use of message-passing to link the microkernel with the rest of the OS is the architecture's major drawback (Fig 1). The architecture slows the system down. Part of the slowdown occurs as the messages propagate. The rest of the slowdown comes as a result of context-switching between microkernel and servers when the application calls upon an OS resource.

In a monolithic architecture (Fig 1a), a context switch turns system control over to the kernel. The kernel services the application's request and then returns control to the calling application through another context switch. In a microkernel architecture (Fig 1b), additional context switches occur because the kernel must call the resource server to satisfy the request. The result is a doubling of the context-switching overhead of system calls. As a result of this potential overhead penalty, some Unix OS vendors serving the embedded-system market choose to remain with monolithic architectures. Lynx Real Time Systems' LynxOS, JMI Software Systems' PSX, and VenturCom's Venix, for example, all operate as integrated, monolithic OSs to achieve top performance.

The magnitude of the overhead penalty is a subject of much debate in the Unix community. Some researchers claim that the performance can degrade as much as 15%. Others claim that the penalty is insignificant if the context-switching and message-passing time is substantially shorter than the service requires. Still others find that the microkernel's modularity allows them effectively to fine-tune service routines and show an overall performance improvement of 10%. Lacking consistent results, the microkernel-overhead-penalty debate has become a religious, rather than scientific, argument.

The penalty notwithstanding, vendors of non-Unix real-time OSs for embedded applications have embraced-although not fully-the microkernel concept. To avoid the penalty, most real-time vendors stopped short of a full microkernel architecture in new designs. Instead, vendors have built scalable OSs using a microkernel-like collection of independent functional modules surrounding a core of basic services.

The vendors' marketing literature sometimes refers to core services as a "nanokernel" and to the entire OS as a "microkernel." The assembled OS, however, is still monolithic, in that OS services run in supervisor mode along with the kernel. Nonetheless, by building the OS from modules rather than as an integrated whole, vendors inherit the microkernel approach's scalability and can offer extremely compact code.

At first glance, a modular design would seem most likely to result in larger, not smaller, code. The OS would need additional code, for example, to handle message-passing. Also, independent modules might duplicate some functions. A glance, however, doesn't encompass the entire picture. Further examination shows that modularity can help save code. The designers of Microware's Atomic OS-9, for example, found that modularity allowed them to optimize their code more fully. Because the modules were independent, the designers could tune modules without complicating another function. The result is a 15% size decrease.

Modularity also yields compact code because it lets end users assemble only the minimum set of OS functions their projects require. Trimming the fat from commercial OSs isn't practical with monolithic designs; it's too risky. Integrated monolithic designs, therefore, leave designers with an all-or-nothing choice. A microkernel-like OS, on the other hand, lets a user scale the OS's overall size to better fit a system's memory resources.

This scalability is allowing microkernel-like OSs to become viable commercial alternatives to custom OSs in deeply embedded applications. Such applications need both OS and application programs to reside entirely within a microcontroller's on-chip ROM. Commercial-OS vendors are achieving this goal. Microtec Research's VRTXmc, for example, can produce a basic kernel that occupies approximately 4 kbytes of ROM. The Integrated Systems pSOSelect scales down to 1800 bytes.

The main difference between scalable OSs and true microkernels is that scalable OSs are still monolithic. Once you assemble the OS, you cannot add or delete system functions to or from the OS using programs in user space. Instead, you must reassemble the OS whenever you make a change. The modular approach still simplifies the task of adding unique functions to the OS kernel, however, because the new module is relatively independent of the others. The modules no longer have memory protection, however, because they all operate in the supervisor mode. Thus, design errors in the new module can crash your system.

Vendors of real-time OSs differ in their approaches to leveraging the modularity inherent in their new OS designs. For example, Microtec Research offers two options. It created VRTXmc, which allows you to select from more than 33 services to build a monolithic OS. The services are so fundamental that the OS requires no core nanokernel. You simultaneously assemble both the equivalent of the nano-kernel and the remainder of the OS. For more substantial needs, Microtec Research also offers VRTXsa, a true microkernel with modular-OS services.

Integrated Systems offers a single user-scalable OS, pSOSelect, a scalable version of the pSOS+ OS that is fully upward-compatible with its parent. Projects that outgrow the scalable version can readily migrate to pSOS+. The company also offers a version of pSOS, pSOS+m, that operates while distributed over multiple processors.

Rather than offer users the mix-and-match capability of a scalable OS, Spectron Microsystems offers its SPOX DSP OS as a product family. The family members are all based on the same SPOX nanokernel but offer combinations of services for various applications. Because the family members are standard products, not users' own kit productions, users don't have to debug the OS. The family's common core makes the products compatible with each other, however, so designers need not learn a new OS with each new application.

Microware also offers a product family, which is based on its Atomic OS-9 kernel. Each family member builds on the one before by adding utilities and I/O capability. The base version is small enough to fit into microcontrollers, and the high-end member of the family is the full-scale workstation-based OS-9, Version 3.0.

One drawback that application developers face when using scalable OSs is that paring the OS may cut links to debugging tools. Many RTOS vendors put "hooks" in their kernels that enable debugging tools to provide information on the timeliness and logical flow of code execution (Ref 2). The nanokernels of scalable OSs lack these hooks. Thus, if you want to keep the OS small, you may lose the ability to monitor the code's execution. In such cases, you must turn to hardware-based debugging tools to trace code execution. ROM emulators are one such tool, and, if your processor supports on-chip debugging, you have other options (Ref 3).

Another possibility is to debug your code on a memory-augmented version of your final product using more than a bare-bones OS. When it comes time for production, you can then scale the OS to fit your normal memory limits. Spectron Microsystems' SPOX, for example, offers a debugging module that you can remove from the OS for production.

Even though the scalable-OS approach may limit tool selection, commercial OSs offer a broader tool selection than homemade designs. If you're used to creating custom OSs for embedded projects, then you might reconsider purchasing an OS to gain tool support. The commercial OS gives you the freedom to focus on your application without first becoming an OS expert. Purchasing a microkernel or modular OS also provides some assurance that you'll have ready access to future innovations in OS design-just add modules.

1. Tannenbaum, Andrew S, Modern Operating Systems, Prentice Hall, Englewood Cliffs, NJ, 1992.

2.Quinnell, Richard A, "Debugging real-time systems," EDN, Nov 23, 1994, pg 48.

3. Strassberg, Dan, "On-chip debug: Panacea? No. Useful? Yes," EDN, April 14, 1994, pg 71.