Tcl adds extensibility to design tools

John Ousterhaut -September 02, 2006

Increases in processor speed and memory sizes are making it possible to incorporate scripting in embedded systems themselves. This offers the potential of reducing deployment costs and allowing a new class of value-added applications to be created for embedded systems.

There are many scripting languages-such as Tcl, Java-Script, Perl, Visual Basic and Python-that differ greatly in their syntax and features. However, they are all targeted in one way or another for integration applications.

What these languages do best is to connect and coordinate existing resources to automate processes, customize applications and create new functionality out of existing (large) building blocks. The resources being integrated could be graphical controls in a GUI, pieces of hardware, existing applications, network protocols, component frameworks and many other things.

Integration applications present different problems from traditional build-from-scratch applications. Algorithms and data structures don't typically play a large role in integration applications: They are typically handled inside the resources being integrated. Instead, the problems with integration applications have to do with connecting a variety of different resources and managing rapid evolution with non-expert programmers.

System programming languages such as C, C++ and Java are poorly suited to integration applications. The combination of compilation and strong typing makes system programming languages static and inflexible. Applications must be planned carefully in advance and it is difficult to have them evolve in ways that weren't planned.

Since applications are shipped in binary form, it may be impossible to have them evolve once they are deployed. Furthermore, the sophisticated features of system programming languages, such as objects and threading, make it difficult for non-expert programmers to use them.

In contrast, scripting languages have properties that make it easy to develop integration applications. They are interpreted rather than compiled, which decreases turnaround time during testing and also makes it possible to modify and customize applications in the field.

Scripting languages are also weakly typed or string-oriented. They try to make all things look and behave similarly, which makes it easy to mix and match resources that weren't originally planned to work together.

Tcl is a particularly simple and powerful example of a scripting language. Essentially, it was created to serve as a command language for applications. The goal was to incorporate Tcl into an application and then combine the features of the application with those of Tcl to form a command language.

This resulted in two key properties. First, Tcl is embeddable: Its interpreter is a library package that can be linked into an application. Second, Tcl is extensible: It provides a set of APIs that can be used to create new features in the language.

Tcl was designed from the start to be easily extensible. A new Tcl command can be created by writing a command procedure and then telling the Tcl interpreter to call it whenever a command by a given name is invoked. Other API functions allow extensions to access all of the internals of the Tcl interpreter, such as variable values. Extensions can create new features, define new data structures and even create new control structures.

That Tcl can be embedded and extended makes it relatively unique among scripting languages. Those properties have allowed Tcl to be used for a huge variety of tasks, both inside applications and as a standalone programming platform.

For example, an extension called Tk provides facilities for creating graphical user interfaces; one of the main reasons for Tcl's early success is that Tcl and Tk together made it dramatically easier to create GUIs on Unix.

Tcl has been used for applications ranging from automated testing to Web site management to financial applications. It now runs on Windows and Macintosh platforms in addition to Unix, so it provides an excellent cross-platform development tool.

There are more than 500,000 Tcl developers worldwide; moreover, Tcl is an open-source package, which means that its source code is available freely on the Internet, with no restrictions on usage and no licensing or royalties required.

In some cases it makes sense to test the software of an embedded system directly, rather than using test hardware to measure the overall response of the system. In this case, the goal is to invoke an API in the software and then determine whether the API behaved appropriately by invoking other APIs to examine the state of the system.

With an extensible scripting language like Tcl, extensions can be created that connect directly to low-level internal APIs in the software. That allows the software to be tested in small pieces, thereby giving the most control over the tests. Without the extension mechanism, tests would have to be invoked using only the external accessible APIs for the software, which would reduce the control of the test writer and make it harder to test all of the code in the system.

Inside the environment
The interpreted nature of scripting languages provides additional benefits for automated testing. First, interpreters provide dramatically faster test development by eliminating lengthy compilation cycles. Second, they allow tests to be stored in databases, since the tests are just strings containing scripts in the test language.

The uses of scripting described so far are all outside the embedded systems themselves. However, a more radical alternative is to incorporate scripting languages directly into the run-time environments of embedded systems. This opens up some intriguing and powerful opportunities.

Until recently it would have been unthinkable to add the interpreter for a scripting language into an embedded system. On-board memories were too small to accommodate the hundreds of kilobytes needed for an interpreter, and the processors were so underpowered that an interpreted language would have executed too slowly to carry out meaningful tasks. Also, the functionality of embedded systems tended to be simple and fixed, so scripting's power was not often needed.

Today, however, the situation has changed.Embedded systems often have fast 32-bit processors and several megabytes of memory. In-tegrated circuit technology will drive additional improvements in memory sizes and performance in the future. Thus, hardware limitations are no longer an impediment to using scripting languages.

At the same time, the complexity and functionality of software used in embedded systems has been increasing rapidly, and much of this software has an integration flavor. Scripting languages offer the potential to increase the power and decrease the development and deployment costs for embedded systems.

Onboard scripting languages would make it dramatically easier to customize embedded systems. Embedded systems are often used by service providers that sell or rent the devices-such as pagers and cellular phones-to end users as part of their services. The service providers would like to extend the base functionality of the devices to add features that differentiate them from their competitors.

For example, a cellular telephone service might wish to incorporate a white-pages-style number lookup service into its phones so that users could enter a name on the keyboard, see the corresponding phone number or numbers on the display, choose one number and have it automatically dialed.

Customizing an embedded system today requires coding in a low-level language such as C, which exposes programmers to a great deal of complexity. Even worse, it gives the customizers complete control over the devices, so errors in the customization could compromise the operation of the device.

On the other hand, if the device had an onboard interpreter for a scripting language, these problems would be reduced.First, scripting languages provide a higher-level interface, so it would be much easier to develop the customizations.

Second, the device designers can choose which facilities they make available in the scripting language, so they can ensure that only a safe subset of the device's features are available to customizers. Third, the safety of scripting languages means that errors in the customization are less likely to crash the entire system.

The Tcl scripting language has several properties that suit it for onboard use in embedded systems. The embeddable nature makes it easy to incorporate into embedded systems: The Tcl library package can simply be linked into the run-time system of the device.

Tcl's extensibility makes it easy to expose selected features of the device in the Tcl interpreter, and its APIs for extension are comprehensive and well-documented and provide considerable power to the extension writer.

In comparison, most other scripting languages were written to be standalone virtual machines; they can't easily be embedded and their extension mechanisms, if they exist at all, are not as well-defined or as comprehensive.

Most of Tcl's functionality is encapsulated in its command procedures, not as part of the core interpreter. If it becomes necessary to reduce Tcl's memory footprint, this can be done relatively easily by deleting commands. Thus, system designers can choose whether to optimize it for functionality or memory consumption. Tcl is available freely in open-source form, which makes it easy to make these or any other changes to the system.

In recent years there has been considerable discussion of the role that the Java programming language might play in embedded systems. Sun Microsystems and others have suggested that Java interpreters should be incorporated into embedded systems, with Java used for many of the functions I have described above for scripting.

Unfortunately, Java is caught in a neither-here-nor-there situation for embedded devices. Java's real strength is in creating software components, the sort of things that C programmers typically do today for embedded devices. But the low-level programming for embedded devices requires a level of execution and memory efficiency that Java cannot achieve.

It might also be possible to use Java for integration and customization functions for scripting, but unfortunately the Java language is not well-tailored to these tasks. A scripting language like Tcl allows these tasks to be carried out much more easily and by less sophisticated developers.

Furthermore, a full-function Java virtual machine is many times larger than the interpreter for a scripting language. The cost-effectiveness of Java will always lag a few years behind scripting languages, which have already proven their value at several points in the life cycle of embedded systems, such as design tools and testing.

The next step is for scripting languages to move onboard, becoming part of the systems themselves. This will enable new applications to be created by a much larger programming community, and will increase the power and sophistication of embedded systems. It will also make the systems easier to deploy and manage and could reduce the total cost of ownership.

John Ousterhout, creator of the TCL scripting language, is currently professor of computer science at Standfor University and chairman of Electric Cloud, Inc.

Loading comments...

Write a Comment

To comment please Log In