Design Feature:July 20, 1995

Anyone following the PC industry has most likely heard about Intel's native-signal-processing (NSP) technology. NSP theoretically lets an end user take advantage of multimedia features, such as wave-table audio, by sacrificing some CPU cycles for signal-processing tasks rather than some dollars for a hardware-based sound card. A Pentium processor alone cannot shoulder much of the signal-processing load in a state-of-the-art multimedia system. NSP introduces valuable concepts, however, such as real-time processing and a unification of multimedia subsystems.

The first designers to develop a cohesive architecture that can manage audio, telephony, and, eventually, video subsystems will gain a substantial advantage over their competition. Designers may find that NSP mixed with programmable DSPs or fixed-function ICs based on DSP technology will yield such a cohesive architecture. According to Intel, NSP moves some signal-processing tasks to the host µP and enables a standard set of multimedia capabilities on every Pentium-based system. Intel says the technology raises the baseline of capabilities that application developers can expect in a Windows-based desktop PC.

Intel is targeting audio, telephony, and, eventually, video subsystems as candidates for host-based signal processing. The company has so far delivered only on audio via the Native Audio Developers Kit that Intel announced at DSPx in May. Next on the list will likely be a 28.8-kbps, V.34 modem with support for digital simultaneous-voice-over-data (DSVD) capability. Intel has demonstrated such a product with the signal-processing functions still running on a DSP processor but with control functions, such as the Hayes AT command set, data compression, and error correction, running on the Pentium processor.

PICTURE 1

The idea of signal processing on a standard µP is not new (see box, "µP-based DSP abounds"). NSP, however, entails more than host-based signal processing, and these additional features can be compelling to designers of multimedia PCs. For example, with Spectron Microsystems, Intel has embedded a Pentium port of Spectron's real-time Spox kernel into the Windows architecture. Called IA-Spox (Intel Architecture Spox), the kernel resides at Ring 0 in the Windows architecture and adds real-time, pre-emptive multitasking for signal processing.

Real-time Windows extension

Real-time response can be key to functions such as an NSP-based modem. Even though the control code that moves to the host doesn't perform signal processing, the control functions require real-time response to keep pace with the fast data stream a V.34 modem produces.

PICTURE 2

Another compelling benefit of NSP is that the native-audio subsystem unifies media types. Native audio is a virtual-audio system, just as a disk drive is a virtual-memory system. The native-audio subsystem handles all Windows-based audio calls, processes some audio functions on the host Pentium processor, and passes others to "signal-processing accelerators," which are either programmable DSPs or fixed-function ICs based on DSP technology. These accelerators enable two or more Windows applications to open virtual-audio channels simultaneously.

Native audio is implemented in Windows' MMSystem dynamic-link library (DLL). This is the same DLL that a sound-card vendor would supply to handle the multimedia-audio Wave interface and musical-instrument digital-interface (MIDI) application-programming interfaces (APIs). On the hardware side, native audio interfaces to a 16-bit, full-duplex stereo codec via a device driver. To understand the merits of these technologies, consider how you can use them in a specific application, such as telegaming.

Target telegaming

Telegaming presumes that two players, each with a multimedia PC, connect via dial-up modem. The systems require a well-defined set of capabilities, which should become standard in multimedia PCs in 12 to 18 months. Those capabilities include a V.34 or faster modem to ensure timely response for the game and because support of DSVD requires a part of the bandwidth (probably around 25%). The DSVD capability allows the gamers to shout at each other during the heat of battle, a capability that also requires a full-duplex speaker phone.

Telegaming also requires audio support, including two audio subsystems. Most new Windows-based application programs rely on high-quality wave-table audio and Windows sound system as a baseline for compatibility. System or add-in card vendors, however, still need to support SoundBlaster applications. SoundBlaster applications require a fixed-function IC and expect a specific register set, a DMA controller, a bus interface, and the Yamaha OPL2 or OPL3 FM synthesizer chip. You have two choices in implementing support for SoundBlaster-compatible sound: combining the Yamaha chip with a codec or buying a chip that embeds OPL3-compatible functionality.

Crystal Semiconductor, for example, offers the 16-bit CS 4232 audio codec, which includes the expected register set, DMA controller, bus interface, and plug-and-play capability. The codec sells for $20.70 (10,000) and requires the use of an OPL3 that sells for less than $12 in similar quantities. Intel ships a driver for the CS 4232 in the Native Audio Developer's Kit. Another choice is a codec that includes OPL3 functionality. Analog Devices' AD1812 SoundPort controller, for example, combines a plug-and-play codec, a bus interface, DMA, and OPL3 functionality. The single chip sells for around $25 (100,000). ESS Technology also offers NSP-supported codecs with and without OPL3-like capabilities.

PICTURE 3

Baseline native audio

However you choose to implement SoundBlaster-compatible support, the cost of that implementation is the cost of baseline native audio. Crystal Semiconductor and Analog Devices provide native-audio drivers for their codecs, and Intel provides free native audio and the IA-Spox runtime software on NSP-compliant, Pentium-based PCs and motherboards.

After you solve the audio problem, which is largely a DOS need, you can turn to native audio to address the other aspects of the telegaming application under Windows. The modem, telephony, and audio functions break down to four computation-intensive signal-processing tasks and housekeeping functions, such as control and mixing. Moreover, all of the functions must operate concurrently for telegaming to work. The four signal-processing tasks are a V.34 data pump, a DSVD voice codec, a full-duplex speaker phone with echo cancellation, and wave-table synthesis.

In partitioning these signal-processing tasks, you might first examine the limitations of existing fixed-function, add-in cards in a telegaming application. Assuming that you can find a V.34 DSVD modem with a speaker phone, the speaker phone likely won't work concurrently with another audio application. The game application would open an audio channel on the sound card. When the modem speaker-phone application tries to open a channel on the same sound card, the application gets a device-busy message. Native audio can resolve such conflicts because it lets both applications open audio channels. The native-audio mixer then combines the game sounds with the competitive banter from the speaker phone.

Multifuntion, multimedia cards most closely match the needs of tele-gaming applications. These cards are based on programmable, general-purpose DSPs or on combinations of fixed-function subsystems on one add-in card. Generally, these cards combine audio and telephony functions, but none support V.34 modems or DSVD.

The products evidence the fact that no signal-processing resource can simultaneously handle all of the telegaming functions. Designers can hit the jackpot by being the first to discover the most cost-effective mix of NSP, DSP, and fixed-function ICs and an architecture to meld the media types.

PICTURE 4

Data pump requires DSP

Consider tasks that are beyond the capabilities of a Pentium processor, such as the modem data pump. The data-pump algorithm would require virtually all of the available cycles in a 100-MHz Pentium. Many 16-bit, fixed-point, programmable DSPs can handle the V.34 algorithm. These ICs, which cost $15 to $30 (OEM), include Analog Devices' ADSP-2100 family, IBM's Mwave family, Motorola's 56000 family, and Texas Instruments' TMS320C50 family. These DSPs offer 25- to 40-MIPS performance, and MIPS ratings for DSPs are relatively good indications of comparative performance.

The IBM 2780 Mwave chip, for example, performs 25 MIPS. Generally, you may see metrics such as 30 MIPS or more required to implement a V.34 data pump. IBM, however, has demonstrated the 2780 handling V.34 algorithms and using only 20 MIPS in the Mwave software environment for multimedia products. Still, this leaves only 5 MIPS for other functions and points out the need for other signal-processing resources in a telegaming application.

Analog Devices has also considered how the telegaming application might fit the capabilities of its ADSP-2181 IC, which delivers 33 MIPS. The V.34 function, including the controller function, requires around 25 MIPS. The IC, in fact, includes the on-chip static RAM for data-pump applications. Application Engineer Jeff Stevens from Analog Devices also estimates that the speaker phone with echo cancellation would require around 10 MIPS; the voice codec, 10 to 12 MIPS; and robust wave-table audio, 20 MIPS. The bottom line is that telegaming requires two DSPs.

Motorola makes similar estimates for the 56002. You can implement today's 80-MHz ICs, including memory, for around $25, and they can handle a V.34 modem and a rudimentary speaker phone with limited echo-cancellation capabilities. Motorola DSP Marketing Manager Garth Hillman predicts that, in a year, a 56000-family device will be able to handle the telegaming application at substantially lower price than that of a current dual-DSP implementation.

Texas Instruments offers the widest range of general-purpose DSPs in the industry. The products include the 320C10, 320C20, and 320C50 fixed-point devices; 320C30 and 320C40 floating-point devices; and the 320C80 that integrates four DSPs on a single IC. The TI C50 family includes products in the price/performance range of DSPs from Analog Devices, IBM, and Motorola. TI's products, however, probably offer more design flexibility because you could choose to combine several C20-family DSPs that can cost less than $5 (100,000).

The high-end C80 costs more than $400; as a result, it does not fit into the PC market segment. Don't dismiss the chip as a possible design target, however, because Texas Instruments predicts that the price will drop to $150 in large volumes by 1997. Moreover, the chip can handle features such as Moving Pictures Experts Group decoding or other video processing in addition to the telegaming requirements, making the price reasonable in high-end PCs.

Fixed-function telephony

You can also use a fixed-function DSP for the V.34 function. Market leaders in modem ICs AT&T Microelectronics and Rockwell both offer V.34 modem data pumps with or without companion chips that implement the control function. AT&T recently introduced the four-IC Catamaran chip set that can concurrently handle a V.34 data pump and control, DSVD based on DSP Group's TrueSpeech voice codec, and speaker phone with echo cancellation. The RAM-based chip set includes two devices based on the company's 1600 DSP core and allows field updates to accommodate emerging international DSVD standards.

Whether you choose a fixed-function or a programmable DSP to support telegaming, you still must execute audio functions in the system and, perhaps, offload some of the telephony functions to other resources. The Pentium processor and native audio may be good choices to offload. Intel provides estimates on the load that audio tasks might place on a 100-MHz Pentium. Mixing eight 22k-sample/sec stereo channels, for example, requires 9% of the processor's cycles. Housekeeping tasks, such as sample-rate conversion, require 5% or less of the cycles. A TrueSpeech codec, on the other hand, requires 32% of the cycles.

Wave-table synthesis is perhaps the most compelling of the native-audio functions because it replaces for free what today is still a premium-priced add-in card. Intel estimates that wave-table audio requires approximately 1% of the processor per voice (per instrument). Intel points out that games typically use only 10 or 12 voices. Conversely, supporting the full capability of a dedicated wave-table add-in card might require more than half of the CPU cycles.

Augmenting NSP

Designers that choose to augment NSP capabilities have several implementation choices. Intel, for example, has devised a simple scheme through which independent hardware and software vendors can enhance native audio. Hardware vendors can add functions, such as hardware wave-table audio, by providing a MiniDevice driver to make native audio automatically take advantage of the hardware. Likewise, software vendors can develop MiniFilters that execute on the host Pentium processor and operate on audio bit streams in the native-audio subsystem. Vendors can use MiniFilters to implement a voice codec between a modem and the audio subsystem or to add special effects to an audio bit stream.

For the telegaming application, you must augment native audio with telephony functions. For anything more than extending the native audio with MiniDevice drivers, you must develop a software link between the IA-Spox streaming I/O interface and your DSP. For this task, you need to purchase the IA-Spox software developer's kit from Spectron Microsystems. You can add either a hard-coded, fixed-function DSP, such as a modem chip, or a general-purpose, programmable DSP. The programmable device itself can run a real-time, signal-processing operating system, such as Spox, Mwave, or AT&T's VCOS or can use dedicated control software.

For a fixed-function DSP, you must develop an IA-Spox device driver that can interface to the DSP and pass higher level commands from the API level to the device. The same scenario holds for a general-purpose DSP running dedicated software. For a DSP running an operating system, you must develop device drivers for both IA-Spox and the operating system running on the DSP. You have to adapt operating systems such as IBM's Mwave to the Spox streaming I/O interface. Although IBM hasn't revealed its plans to support NSP, the company will likely develop an NSP reference design for the Mwave environment.

If Spox resides on the DSP, the drivers in each subsystem are virtually identical at the source-code level. Moreover, the source code for any signal-processing task, whether the task runs on the host processor or on the DSP, is nearly identical. Only the computation-intensive, assembly-coded algorithms, such as an FFT, differ. The code that configures and manages the interworkings of a signal-processing task would most likely be in C and be portable among environments.

Tasks on the host and DSP subsystems connect and share data via the Spox streaming I/O interface. You use this interface to link multiple tasks running either strictly on the host IA-Spox subsystem or on multiple tasks running strictly on the DSP. Once you complete the device drivers, the streaming I/O interface eases the partitioning of tasks between IA-Spox and an underlying DSP. Spectron will most likely offer turnkey drivers for popular DSP platforms in a future version of the IA-Spox software developer's kit.

The questions that designers considering NSP must answer is how much of the host processor a user will willingly sacrifice and how much of the tele-gaming application will have to be handled with hardware. Intel estimates that a user will never notice a 30% depletion in host horsepower. Intel Product Marketing Manager John Stewart points out that users are already giving up that 30% when using products such as a SoundBlaster card. Stewart claims that the context switches within Windows and data transfers to the sound system add the 30% overhead.

Thomas Clarkson, division vice president of multimedia marketing at Brooktree, agrees with Stewart. Clarkson claims that the ISA bus implementation of audio-carrying, high-bandwidth stereo audio is a significant bottleneck in today's systems and, therefore, believes you could move to host-base wave-table audio with no effect on users. Brooktree's Media-Stream chip set removes the ISA bus bottleneck and moves support for SoundBlaster applications onto the PCI bus. This approach frees the host processor for wave-table duties whether in Intel's native audio or in the company's own WaveStream software that can run in 486 environments.

Creative Labs, which has shipped more multimedia products than any other company in the industry, has a different opinion. In each iteration of its sound cards, the company moves more functions to the host processor, as the host platforms get more powerful. Rich Sorkin, Creative Labs' director of audio and communications, believes that occupying 30% of the host processor would be acceptable only if you could be sure that the customer would use a single application at any given time. Sorkin claims that the system could quickly become unreliable if 30% of the host is occupied with wave-table audio when a fax comes in. To ensure reliable operation in environments beyond its control, Creative Labs limits the signal-processing drain on the host to less than 10% of the available cycles.

Finding the right mix

It is virtually impossible for a multimedia designer to plot a perfect course to telegaming. You may want to consider a few facts and expert opinions, however.

Creative Labs' Sorkin claims that a combination of fixed-function ICs will always be the implementation method of choice when a product goes mainstream. Creative uses programmable DSPs to quickly get low-volume, leading-edge technologies to market. As those leading-edge technologies go mainstream, the company's subsequent product generations migrate to fixed-function implementation. Sorkin doesn't believe that general-purpose DSPs will ever break through this business model because they don't achieve the correct mix of price, performance, and feature set.

Intel lists several codec, DSP, motherboard, add-in card, software, and system vendors that support NSP and native audio. No company is dissenting, although it's unclear how many of the vendors believe in NSP and how many just want to make sure they don't miss the next industry bandwagon. Several NSP-enabled motherboards will emerge in the next several months because the vendors are essentially adding extensible sound support, and baseline sound is becoming a requirement in the PC market.

NSP can serve in tasks such as synchronization of multiple audio streams, sample-rate conversion, and mixing. Even if signal-processing hardware handles all other audio tasks, native audio and its virtual-audio capability somewhat unifies multimedia subsystems.

NSP, however, is far from a complete answer to telegaming or other multimedia applications. Intel has yet to publicize implementation details on NSP modems. Moreover, NSP does not address resource allocation, which ensures that some signal-processing resources are available to handle multimedia system calls. Companies such as IBM with Mwave have extensive resource-management expertise, which could even allow users to select either audio quality or application performance. For now, however, Intel is leaving to designers the task of integrating resource management into NSP-based products.

You should approach all multimedia designs with an eye toward video support. Just as telegaming needs robust audio and telephony, many new games use video, and most rely on software to compress video streams. Video introduces a new set of problems, such as lip/audio synchronization. As a result, the desktop-PC industry may need a new hardware architecture to support different media types and new software that can unify media types.