September 1, 1998

Media processors target digital video roles

Several IC vendors are touting processors, generally based on VLIW architectures, that can handle media-processing chores for applications from PC multimedia to high-definition digital TV. The proponents believe that only a programmable device offers the flexibility necessary to serve in such volatile and evolving applications. Ultimately, media processors may prosper, but system designers so far have preferred fixed-function ICs and ASICs.

Maury Wright, Technical Editor

Digital multimedia tasks—communication algorithms, digital video or audio coding, 2- and 3-D graphics, and others—all share some characteristics. All are computationally complex and require similar types of resources whether the implementation is on a programmable processor or in hard-wired logic. The similarities have long enticed IC companies with the possibility that they could design a programmable device to concurrently host several of these tasks. Presumably, such a multifunction device would cost less than multiple fixed-function devices and offer the flexibility of programmability's easy updates. Despite these promises, media processors have failed to proliferate. Still, a long list of IC vendors either have announced products or have new designs poised to enter the market, and programmable cores already lie at the heart of ubiquitous products, such as modems.

Potential applications for media processors abound. One such device, Mpact, developed by Chromatic Research and sold by IC partners LG Semicon, Toshiba, and STMicroelectronics, is perhaps the most recognizable. The IC can host a variety of PC multimedia tasks. Alas, at press time, Chromatic announced that it wouldn't continue with the planned Mpact 3, effectively ending the development of the architecture (see sidebar "What happened to Mpact (in my opinion)"). Chromatic does plan to continue selling and supporting Mpact 2. Moreover, at press time, it appeared possible that one of the semiconductor partners might try to assume a lead role in assuring a future Mpact road map.

Other natural media-processor applications center on digital video. Digital TVs will need horsepower to perform video and audio decoding, format conversions, and other functions. In fact, high-definition-television (HDTV) systems will need to support a dozen or more video formats as well as include algorithms that can maximize NTSC or PAL video quality on the 16X9-aspect-ratio screens.

You will probably hear more about media processors in these video and multimedia roles than in other applications, mainly because of the allure of the markets. Actually, some of the processors can serve equally well in other areas, including telecommunications, data networking, printers, and cameras. Down the road, C compilers for media processors may advance to the point at which embedded designers can harness the available horsepower even in one-of-a-kind systems.

Before considering the current menu of media processors, you might want to look back and realize that they're not new. Moreover, a historical perspective clearly shows the advantages of programmability and reveals roadblocks that derailed earlier media processors.

About a decade ago, Texas Instruments developed the 32010 and 32020 programmable graphics processors. Early on, the ICs proved superior to hard-wired graphics chips in 2-D Windows-acceleration tasks and won significant market share in high-end PC graphics. Unfortunately, the 2-D-acceleration task didn't change much for several years. The IC vendors incrementally improved performance, but the real changes took place in the move from the ISA to the VL bus (VESA local bus) to PCI. The programmable ICs from TI had no sustainable perform-ance advantage, and the hard-wired ICs eventually won out based on much lower cost. The story could have been different, however, if the 2-D graphics tasks hadn't stagnated for several years before vendors added on-chip support for video scaling, 3-D graphics, and even audio. A programmable IC could possibly have more easily supported an evolving set of features than the hard-wired ICs ultimately did.

Lucent Technologies (www.lucent.com), meanwhile, was promoting its DSP32 family as a multifunction processor for the PC multimedia subsystem. The company developed a multitasking real-time DSP operating system, the Visible Caching operating system (VCOS), and claimed that a single DSP could operate as a modem, an audio synthesizer, a speakerphone, and a speech-recognition system and could handle other evolving multimedia tasks. The Lucent demos were impressive, but the DSP made no headway into the media-processor market. Several roadblocks derailed the effort. Coding a DSP for such an application required hand-tuned assembly language, and few qualified programmers were available. Ensuring that one processor could handle multiple real-time media streams in worst-case conditions proved more challenging than Lucent and others before and since had estimated. And the cost of fixed-function ICs for each task dropped more quickly than expected.

Despite the failure by Lucent, TI, and others to deploy a multifunction media processor, the same group, along with Rockwell (www.nb.rockwell.com), converted the modem market to programmable DSP-based ICs. Until the era of 1200- to 2400-bps modems, designers used hard-wired digital-filter ICs along with microcontrollers in modem designs. Around that time, however, designers from several companies realized that, with the rate of change in modem- and fax-modulation standards and adjunct compression and error-correction schemes, only a programmable approach made sense.

Menu of choices

Only time will tell how successful media processors might be in emerging applications, such as digital-versatile-disk (DVD) players and digital TV, but a growing list of players plans to offer such processors (Table 1). Very-long-instruction-word (VLIW) architectures dominate the list, although most vendors also offer some single-instruction multiple-data (SIMD) capabilities. (Figure 1 depicts the MCP, a representative VLIW processor from Matsushita.) The list includes processors from Matsushita, Mitsubishi, and Fujitsu that have debuted only in Japan and most likely were designed primarily for use by the companies' own consumer-electronics divisions. Sharp's event-driven, clockless Data-Driven Media Processor (DDMP) is the most unusual architecture.

The table also lists mainstream µPs, such as the Pentium, Pentium II, and compatible families with multimedia extensions (MMXs); Sun's UltraSPARC with visual instruction set (VIS); and Motorola's PowerPC with AltiVec. These µPs all include an SIMD-based vector-execution unit with instructions optimized for graphics and other multimedia applications. Moreover, experience shows that as horsepower goes up, such CPUs take on tasks that once required dedicated hardware. In most new PCs, for example, the host processor can perform the modem-data-pump task. This fact prompted Chromatic to readily admit that its success depended on staying one to two years ahead of Intel in multimedia capabilities and constantly moving to new functions or higher levels of performance.

Deciding the viability of a media processor for your application should be relatively straightforward. You can consider several questions when making such an evaluation.

• Can a media processor match or undercut the cost of using separate hard-wired ICs or combining fixed-function cores in an ASIC?
• Is the application at hand likely to undergo a steady stream of changes or upgrades that makes programmability a significant advantage?
• Can a media processor successfully handle the concurrent tasks an application requires?
• Can a media processor supplant a microcontroller or another CPU, or does it have to act as a coprocessor?
• What development tools exist, and how easy is it to do custom software in C?
• What applications do the media-processor vendors support with assistance in firmware and software development?

Working from the bottom of the list, you might wonder why it matters which applications the media-processor vendor targets. In a perfect world, it wouldn't matter, but it's likely you will need some help in coding a media processor. The programmability of the ICs is both a blessing and a curse because the programming task historically has been somewhat herculean. For example, Chromatic admits that programming Mpact is a step more difficult than programming a DSP. To achieve suitable performance, the company relies on hand-tuned assembly language for all performance-critical paths through the code. In essence, you can use Mpact in only the applications—graphics, video, audio, modem, and DVD for the PC—that Chromatic supports. Chromatic has begun to expand the software-development community and claims that more than a dozen customers have licensed a software-development kit. The company claims that a few customers are working on applications other than PC multimedia subsystems. It's doubtful, however, that Mpact programming skills will extend beyond the currently trained software engineers, now that Chromatic has announced that it won't follow with an Mpact 3 offering.

From the table, you can see the other applications that media-processor vendors are targeting. Unfortunately, most of the list will be hard to test and verify in actual applications because the products are still in development or the vendors are restricting access to strategic customers and even internal consumer-electronic projects. Philips' TriMedia is an exception. The company has customers shipping ICs into videoconferencing applications and has impressive demos in DVD and digital TV, including HDTV. You can rely on Philips to supply optimized code for tasks, such as video scaling and format conversion, MPEG-2 video and audio decoding, AC-3 audio decoding, voice encoding and decoding, and MPEG-1 and JPEG video encoding.

Programming a media processor

So where do you turn if your application is slightly or perhaps significantly different from applications that the media-processor vendors support? Can you write you own code or contract custom programming from the vendor or a third party? Presumably, media-processor vendors are moving toward more friendly programming models in which programmers can do most of the work in C. Realize, however, that it's not a simple step. Even with relatively simple fixed-point DSPs, many OEMs have found over the last few years that programmers can write less than 50% of the code in C. DSP vendors now claim that compiler technology allows C coding for more than 85% of a project. Even at that percentage, the hand-coded assembly tasks can dominate the hours programmers spend on the project. Media processors are typically more difficult to program than standard DSPs because efficient use of the resources requires essentially DSP-type instructions and algorithms to be scheduled and executed in parallel.

Several media-processor vendors understand that their compiler and RTOS expertise is more important than the ICs themselves. Philips recognized this fact early on and spent six years working on compiler technology using simulators before moving to a relatively quick, two-year IC-design cycle. Philips, among others, is touting the ability to program its IC using standard C. Philips claims that the TriMedia design, with 27 execution units, targets an average completion rate of five instructions per clock. The company further claims that it wrote an MPEG-2 decoding routine 100% in standard C and realized 4.5 instructions per clock. Philips does admit that some of its code has segments of hand-tuned assembly, but the company is moving close to a 100%-C world in which OEMs can easily roll their own code.

Start-up Equator Technologies is even more bold about using 100% C for its as-yet-undetailed media processor. The company's founder and president, John Setel O'Donnell, claims that his history as a designer of VLIW computers at now-defunct Multiflow Computer validates Equator's still-undemonstrated software expertise. O'Donnell claims that Multiflow systems hosted 50 million lines of user code in C and Fortran without hand-tuning. Moreover, the company developed a library of math routines using C that outperformed an assembly-coded library because of the superiority of the Multiflow C compiler.

Ultimately, you will have to judge for yourself how efficient any compiler, RTOS, and media-processor combination can be. It's most likely that you can always extract some additional performance by hand-tuning certain loops. On the other hand, if a C implementation meets your application requirements, then you needn't worry about optimization. When you choose a media processor, however, you have to project performance for your application based on a sample from the IC vendor. And remember that a vendor can skew a compiler to optimize code for an application or type of algorithm.

Not all of the media-processor vendors are quite so worried about some assembly-language programming. Bill Rehbock, vice president of third-party development at VM Labs, fully expects his customers to do some assembly-language programming. VM Labs targets 3-D games and claims game authors invariably will turn to assembly to create content that outshines competitive offerings. Rehbock points out that VM Labs will offer a C compiler and that authors will use C to prototype titles and perhaps author first-generation games. He claims that the authors will later move to assembly in performance-critical sections of the code, such as inner loops (see sidebar "Start-ups take different slants").

Still, most of the vendors tout C support as critical, and you can find one indicator of potential C efficiency in the media-processor architecture. Moreover, the same architectural details can sometimes determine whether a given media processor fits your application. The spectrum of VLIW architectures, for example, ranges from ICs with multiple identical or generic execution units to ICs with highly specialized execution units. TI's 320C6000 family, which TI calls a DSP rather than a media processor, includes two sets of four execution units with each of the four types optimized for specific math operations (Figure 2). A VLIW front end feeds the processor, and VLIW is simply the mechanism that provides software access to the parallel DSP architecture, points out Ray Simar, TI fellow. TI contends that software engineers have an easier time writing efficient compilers that take advantage of the parallelism on chip with relatively generic execution units.

At the other end of the spectrum, Chromatic integrates a hard-wired 3-D graphics engine on its Mpact 2. The company feels that the 3-D unit is necessary to compete in the PC market but is one hardware feature that can complicate programming and compiler design. Philips also includes some dedicated functions on the TriMedia IC (Figure 3). For example, the IC includes a variable-length decoder optimized for MPEG decoding, a video-format-converter block, and a DVD encryption block. Such blocks not only complicate software design, but also can go unused in some applications. On the other hand, a video application may benefit greatly from the dedicated functions.

The general-purpose CPUs from Intel, Motorola, and Sun are even more architecturally generic than the TI C6000 family. Compilers from multiple sources provide considerable support for the SIMD vector units, and OEMs won't have trouble finding experienced programmers. Unfortunately, the ICs cost more than the other media processors and—perhaps of greater importance—consume way too much power for some consumer applications, such as DVD players or digital-TV decoders. Power can be a critical issue in a sealed consumer device. Most of the VLIW architectures run at relatively low clock speeds to conserve power. The designs essentially trade off processor complexity for the required clock speed to handle a given task.

Taking a different approach from the general-purpose and VLIW processors, Sharp's DDMP design employs a data-flow architecture with multiple signal-processing cores. The cores don't use a clock but rather execute instructions based on the arrival of an instruction or data. An ARM-based RISC controller coordinates the clockless cores using a message-passing scheme for interprocessor communications. The elimination of the clocks in the cores means that the IC uses little power when no instructions or data are present, presumably making the architecture a perfect match for consumer devices. On the other hand, the non-von Neumann architecture could prove more difficult to program than any of the other media processors: You have to program a DDMP IC using assembly language.

Sharp has developed a graphical editing tool, Data Flow Graph, that allows designers to develop assembly code and visually map the application to parallel cores. The editor allows manipulation of objects that can be as simple as a single assembly-language statement or as complex as a macro block of code, such as an audio decoder.

Sharp's DDMP may also be tougher than normal to evaluate for any application. The company is sampling two versions of the IC with eight and nine processor cores, although the ICs are more for evaluation than for production and deployment. The company ultimately envisions developing custom implementations of the DDMP architecture for specific applications—presumably matching cost and performance requirements of the application at hand.

Scaling for varied applications

Potentially, Sharp's plan provides more scalability than most other vendors can offer. Moreover, customers would benefit from the scalability by reusing some code across products with varying levels of performance. On the other hand, it's unclear how quickly Sharp can produce variants of the DDMP. The company has two ways to vary the family's performance and capabilities. It can increase or decrease the number of cores it uses, and it can add dedicated hardware in some cores to support tasks in the same way Philips integrated a decoder block for MPEG. Sharp has added MPEG-2-decoding hardware in one core on its sample ICs.

Other media-processor architectures may not have the flexibility in scalability that the DDMP has but still could meet customers' needs. Equator, for one, states that it will offer versions of its media processor for applications from camcorders to HDTV. Matsushita has new versions of the MCP coming. In Mitsubishi's case, the lower perform-ance D10V for audio applications, including cellular phones, preceded the D30V (see table). These companies may be able to offer more software portability to VLIW ICs than Sharp can with its DDMP, which requires hand-tuned parallel coding.

Surprisingly, µPs such as the Pentium and PowerPC family might offer the ultimate in scalability. The problem with a general-purpose CPU as a media processor emerges when a company such as Intel constantly strives to obsolete older parts and forces OEMs to use the newest design. The latest and greatest µPs cost too much for many media-processor applications. This phenomenon eliminates the chance of reducing cost and power in parts for an application such as set-top boxes. Even though Intel has ignored this avenue, other vendors of Pentium-compatible ICs, such as STMicroelectronics, Cyrix, and Integrated Device Technology may follow this path. Moreover, Motorola vows to deploy its AltiVec technology in lower end PowerPC chips that target embedded applications, even though the first AltiVec ICs fall at the high end.

The general-purpose ICs offer one other advantage: They will never require another CPU in the system. Some media processors can't make the same claim. Chromatic, for example, designed Mpact strictly as a coprocessor. Most of the other media-processor ICs can handle user-interface tasks and simple system management but perhaps can't handle a user-accessible operating system. Closely evaluate the processor-versus-coprocessor angle relative to your application requirements.

Ultimately, the gating factors for any of these media processors come down to cost and how the programmable devices stack up against fixed-function implementations in hardware. Most of the target applications require a single-chip heart to meet consumer cost expectations. Media processors can answer that need for most consumer-electronics products but, increasingly, so can the so-called system-on-chip ASICs that might combine a microcontroller with hard-wired video and audio decoders and other functions. Harking back to TI's late-'80s graphics processor, you might expect media processors to be first to market, but can they sustain an advantage?

The volatility of target applications will go a long way toward determining the success of media processors. Were an application such as a set-top box or DVD player to stagnate, the ASIC approach would likely offer lower cost. The media-processor vendors, however, are positive the opposite will happen. Equator's O'Donnell points out that digital TV will combine with Internet delivery of voice, video, fax, and other data to place the industry at the beginning of a new era with new standards emerging every six to nine months for many years. O'Donnell claims that no one has a grasp on all the things consumers might ask of their set-top boxes over the next decade.

You can see some evidence to support O'Donnell's claims in Philips' TriMedia demos. The company's digital-TV reference design includes an amazing number of options for video formats. The programmable approach even allows the system to offer varying ways to enhance NTSC or PAL signals on high-resolution displays. The media processor also can enable Internet access in a digital TV or set-top box without the need for another processor. Thus far, Internet access has bombed in the living room, but broadcast video content, such as sporting events over the Internet, could change things. Philips even ported the popular game Doom to TriMedia to show the range of what you can do without another processor in the system. A fixed-function IC could be at a big disadvantage in this market in the short term, perhaps changing the fate of media processors for the better.

References

1. Gwennap, Linley, "Mitsubishi Designs DVD Decoder," The Microprocessor Report, Dec 9, 1996, MicroDesign Resources, www.chipanalyst.com.
2. Itoh, Motoaki, "Media Processor Cuts Time to Market," Nikkei Electronics Asia, October 1997, www.nikkeibp.com/nea/oct97/specrep/.

What happened to Mpact (in my opinion)

Chromatic Research, its semiconductor partners, and Mpact fans should now be celebrating. In late June, Chromatic claimed that more than 8000 Mpact ICs were shipping in PCs each day. Gateway (www.gateway.com) was using the IC in a number of systems, Micron (www.micron.com) had announced an Mpact-based product, Compaq (www.compaq.com) was rumored to be on the verge of an Mpact announcement, and a dozen smaller add-in card and motherboard vendors offered products. It appeared that Chromatic was headed for profitability and a public stock offering. Then, in mid-July, the company suddenly announced that it would not continue the Mpact architecture with an Mpact 3 chip. Chromatic will continue to sell and support Mpact 2, but it's unlikely that the architecture will win new designs.

What happened? The short answer is that Gateway, the biggest Mpact customer by a large margin, decided to move away from Mpact. But Gateway's decision was a symptom of many other problems that plagued Chromatic and ultimately its silicon partners.

The reasons behind Chromatic's troubles start with the architecture and business model. Mpact was architecturally a media processor but may well have been a multifunction IC based on hard-wired functional blocks. The architecture was practically closed; only Chromatic engineers could develop software. Therefore, the IC was limited to the PC multimedia function that Chromatic supported.

To make matters worse, Mpact was a beast to program, and Chromatic struggled to deliver the Mediaware code modules on time with both Mpact 1 and 2. The programming problem should have been under control by the time Mpact 2 debuted, considering that Chromatic had tested and deployed Mpact 1 code. Inexplicably, however, Mpact 2 differed significantly from Mpact 1 and required a rewrite of the key multimedia algorithms.

The business model was also awkward. Chromatic didn't sell the ICs but instead got royalties from its silicon partners. The company also derived revenue by licensing Mediaware modules to OEMs. Historically, however, OEMs don't like paying extra for such code modules, and the business model most likely scared off potential customers.

Ultimately, however, Chromatic's biggest battle was the one it picked with Intel when it decided to offer a programmable multimedia resource for the PC. Intel views the Pentium and Pentium II as the ultimate media processors. Clearly, Chromatic had limited time to sell audio, modem, and digital-versatile-disk decoding functions before Intel moved those tasks to its µP. Intel, however, hastened Chromatics' exit from PC multimedia by drastically dropping prices of the new 740 graphics ICs and by fiercely lobbying customers away from Mpact.

Start-ups take different slants

Even as some media-processor vendors struggle, two US start-ups are poised to enter the market: Equator Technologies and VM Labs. The companies, however, have vastly different business plans and slightly different target markets. Still, the two may later go head to head trying to stake out a market.

Neither Equator nor VM Labs has yet detailed their media processors, but expect to hear from them soon. Equator will employ a very-long-instruction-word (VLIW) architecture, whereas VM Labs says only that its chip will use multiprocessor technology. VM Labs also believes that processors such as Mpact or TriMedia with fixed functions in hardware are the wrong avenues to take in the future. The company claims it will deliver a processor that is comprised totally of programmable resources.

Equator is targeting digital-video applications, including digital TV and set-top boxes. The company, however, plans a range of software-compatible ICs that target any product through which a digital-video stream passes. VM Labs, meanwhile, is squarely focusing on a combo digital-versatile-disk (DVD) player/3-D-game console. The company believes customers might offer other products such as set-top boxes with DVD and 3-D-game functions, but it believes that the DVD/3-D-game combination will drive demand. According to VM Labs' plans, DVD customers will choose a Project X player because it also plays games, and gamers will do the same because of DVD support. The company also claims that its undisclosed architecture will offer game authors 3-D features that far surpass those now on the market.

Although lacking a fab facility, Equator intends to sell ICs along with software tools, including a C compiler and an RTOS. VM Labs has signed a nonexclusive manufacturing license with Motorola, and the semiconductor giant will make and sell media processors compatible with VM Labs' Project X. VM Labs will receive royalties from Motorola and future semiconductor partners and from the sale of development tools and licenses.

VM Labs will also receive royalties from game authors, just as Nintendo (www.nintendo.com) and others do. Unlike Nintendo, however, VM Labs has no plans to publish its own titles. Nintendo receives royalties from independent authors and offers its own titles that compete with the independents for shelf space.

It appears that VM Labs is ahead of Equator in hardware development. Whereas VM Labs doesn't disclose its architecture, it received first silicon at the end of 1997. Software developers are already developing titles. Moreover, Thomson Consumer Electronics (www.thomson-multimedia.com) (manufacturer of RCA, GE, and Proscan brands) and Toshiba have licensed VM Labs' technology for consumer-electronic products, which should appear early next year. If successful, VM Labs will be the first designer of a game platform that multiple vendors will manufacture.

Table 1—Representative media-processor vendors

Maury Wright, Technical Editor

You can reach Technical Editor Maury Wright at 1-619-748-6785, fax 1-619-679-1861, maury-wright@home.com.

| EDN Access | Feedback | Table of Contents |

Copyright © 1998 EDN Magazine, EDN Access. EDN is a registered trademark of Reed Properties Inc, used under license. EDN is published by Cahners Business Information, a unit of Reed Elsevier Inc.