The new PCI (Peripheral Component Interconnect) Express spec provides the biggest improvement in more than a decade in I/O performance for computation systems, significantly improving graphics in desktop PCs and workstations. Intel initially launched the spec in its chip sets in mid-2004, and the technology has become mainstream in high-end systems. But PCI Express is far more than an avenue to better games or video. As have many other PC innovations, PCI Express will enable significant applications, such as medical imaging, and serve in industrial control and many other embedded-system roles.

Serial PCI Express is usurping the parallel AGP (accelerated graphics port) in graphics as just one part of a broad industry trend toward serialization. For example, USB replaced the parallel and serial ports, and SATA and SAS (serial attached SCSI) are replacing parallel ATA and SCSI in storage interfaces. And, like parallel PCI, PCI Express will handle far more than graphics, providing a flexible and scalable data highway for all types of performance-centric add-on functions.

You might ask whether a replacement for PCI-derived AGP is necessary. AGP8× pushed the evolutionary limits of what was possible with a parallel bus—at least at a price point sustainable for volume PC production. Clock skew was the problem plaguing AGP, like other parallel buses touting higher performance. A single clock defines the data-valid period across AGP's 32 data lanes. But the faster the clock runs, the narrower the data-valid window becomes, and the tougher the design challenge becomes (see sidebar "History of the graphics pipe").

So, the graphics pipeline became serialized with PCI Express. This serialization means that data carries with it an embedded clock, which a PLL recovers in the receiver circuits. Multiple lanes can carry data in parallel, providing scalable performance, but each lane has its own embedded clock (Figure 1). At the receiver end, the data resynchronizes. PCI Express can tolerate as much as four symbols of skew between lanes, dramatically easing the constraints for motherboard and graphics-controller design.

PCI Express is more than just the follow-on to AGP. PCI Express is also the migration point for applications currently on conventional PCI and PCI-X as they move to higher speeds and performance. These legacy buses will not go away overnight, but, like ISA (Industry Standard Bus), they eventually will. PCI Express reunites the forks that broke away from the original PCI with a common protocol and electrical interface. Moreover, PCI Express technology will make its way to the embedded-system world in CompactPCI implementations.

For graphics implementations, the PCI Express bus comprises 16 lane pairs. Each lane pair has four data wires—a transmitting differential pair and a receiving differential pair. Because PCI Express is dual-simplex, data can flow in both directions unimpeded by data going the other direction, unlike AGP (Figure 2). AGP was point-to-point like PCI Express, but AGP is full-duplex: Data can flow in only one direction at a time. Furthermore, AGP never offered symmetric bandwidth in its implementations, even though nothing about the AGP protocols precludes symmetry. AGP8× achieves 2.1-Gbyte/sec peaks in data flows from the host CPU to graphics controllers, which is the primary direction for graphics traffic. However, in typical system implementations, the real available "back-channel" bandwidth is about one-tenth of that figure. In contrast, PCI Express graphics offers 4-Gbyte/sec peak bandwidth simultaneously in both directions and, in typical implementations, delivers the bulk of that bandwidth in both directions. This highly symmetric bandwidth leads to some interesting new capabilities for graphics based on PCI Express and in many other I/O applications.

AGP implementations yielded weak back-channel bandwidth for several reasons. To achieve the best CPU-to-graphics controller performance, AGP worked from uncached address spaces (the AGP "aperture"). For reads, this assumption makes perfect sense, because caching requires snooping, degrading performance. However, chip-set designers simply did not optimize their chip sets for large data-set writes to this space from the graphics controller because the cost of providing a large number of write-posting buffers was prohibitive. This trade-off made sense to the chip-set architects, because graphics traffic is primarily in the forward direction.

A second reason for the relative weakness of AGP's back-channel bandwidth was a limitation in the GART (graphics-address-remapping-table) memory-management system that AGP provided to assist in the graphics controller's task of managing physical- and virtual-address translations for access to uncachable system-memory space. Again, the theory sounded great, but practical design considerations led to suboptimal graphics performance, because real chip sets never implemented enough TLBs (translation-look-aside buffers). Each 4 kbytes of memory requires a new TLB because 4 kbytes is the default page size in Windows. But even two dozen TLBs support only about 100 kbytes of memory before the onset of TLB "thrash." A cache thrashes when its miss rate is too high, and it spends most of its time servicing misses. Thrashing, particularly virtual-memory thrashing, is bad for performance, because the relative cost of a miss is so high: It may slow a machine down by a factor 100 or more.

In the definition of PCI Express, the graphics industry was unanimous in the opinion that it didn't want similar "help" in the new I/O interface. With PCI Express graphics, all memory management occurs within the graphics controller. Memory-management performance is under the control of the graphics vendors, who are more economically motivated in general than are chip-set vendors to spend gates on graphics performance.

At this point, it is reasonable to ask why continually increasing the bandwidth of the pipe from the host CPU to the graphics controller is so important. More prosaically, an end user may reasonably want to know what he can do with a notebook, desktop PC, or workstation with PCI Express graphics that is not possible with AGP8× or, for that matter, what PCI Express offers in embedded-system roles. Only a few of today's applications are starting to top the limits of what AGP8× can deliver. It is not difficult to create a demo application that uses the full bandwidth of PCI Express, and the suppliers of graphics controllers use this approach to show off their latest products. But broadly available commercial applications rarely show any advantage just from the fatter pipe when the pipe enters.

The PC world still needs PCI Express graphics, and end users have good reasons for desiring them. To understand this concept, consider the perspective of a developer of graphics-intensive software, such as that for video games or CAD. These developers write their programs to the capabilities of the least-common-denominator hardware in their target customer base. For leading-edge video games and high-end-workstation applications, these hardware units are highly capable, recently introduced systems, but more generally the target is likely to be systems the vendors released over the last two to four years.

The only way to get the software community to continuously raise the bar in graphics capabilities is by continuously increasing the capabilities, including bandwidth, of the client platforms. The impact of AGP8× on software development is happening now. The work to bring PCI Express to market will have its major impact on software applications in the future.

But the end user still has plenty of incentive to purchase PCI Express graphics today. A buyer of PCI Express graphics is well-prepared for the getting the most from the new applications that emerge over the life of that PC. And, second, the suppliers of graphics controllers are focusing their latest developments on PCI Express products. So, the client with PCI Express graphics will likely significantly outperform previous generations of products, even if PCI Express cannot, in general, now take credit for that performance advantage.

The performance advantages of PCI Express graphics may emerge more quickly than with previous transitions because of the tremendous increase of CPU performance and memory bandwidth just coming to market. For example, the latest Intel workstation platform exploits as much as 64 Gbytes of fully buffered DIMM with a peak bandwidth of 21 Gbytes/sec. Conjoin those features with additional capabilities in the most recent graphics controllers, and you can more fully exploit the bus.

PCI Express affords system designers an extremely broad range of capabilities. In the graphics world, PCI Express dual-graphics controllers are popular with gamers. Some designers are even contemplating externally cabled PCI Express subsystems (see sidebars "Express outside the box" and "Dual controllers accelerate rendering").

PCI Express is a key enabler in some lifesaving applications. For example, consider Vital Images, a leading provider of enterprisewide advanced visualization and analysis software for use in disease-screening applications, clinical diagnosis, and therapy planning. The company's technology gives radiologists, cardiologists, oncologists, and other medical specialists timesaving productivity and communications tools for easy use in the day-to-day practice of medicine. Vital Images' software products include a medical-diagnostic tool that allows physicians to use PCs or notebook computers to gain remote access to 2-, 3-, and 4-D advanced visualization. The software enables users to measure, rotate, analyze, and segment images.

One technical challenge for this medical application stems from the size of data sets that volumetric visualization requires. According to Karel Zuiderveld, PhD, director of technology research at Vital Images, "PCI Express is especially beneficial when dealing with large data sets that do not fit into graphics memory. The size of modern medical data sets, [which the company obtained using computer-tomography], range from hundreds of megabytes to several gigabytes. In addition to a vast amount of CPU and GPU [graphics-processing-unit] resources, fast rendering of such data sets also requires high transfer speeds to the GPU." With PCI Express graphics, medical professionals can view 3-D images from alternative perspectives with reasonable response times.

In the past, the graphics bus has not been the bottleneck. When uploading textures to OpenGL, the driver usually "swizzles" the texture—that is, swaps the pixels around so that they are stored in the same way as the original format. It then creates a copy in system memory, according to the OpenGL spec, and writes a copy to the graphics memory. Until recently, the CPU usually performed texture swizzling, resulting in low texture-upload speeds. Each texture requires a read-swizzle-write memory DMA to the graphics card; this approach involves at least two reads and one write to system memory. With the latest workstations, Vital Images' target applications may fully exploit the bandwidth of PCI Express, says Zuiderveld. Though medical imaging may be a leading application for taxing the PCI Express bus, he believes that the trend toward using resources such as the virtual texture maps that Microsoft's DX10 supports, will drive steeply increasing usage of PCI Express's graphics bandwidth into mainstream applications.

One application that can now take advantage of PCI Express' unique capabilities is video editing. In video applications, PCI Express affords dramatically better back-channel bandwidth than AGP. Back-channel rates are important, because main memory must store intermediate and temporary video-processing results. The files are just too large for the graphics controller's local store. However, the memory must preserve this data without lossy compression, because the cumulative effects of repeated compressions would visibly degrade the end result. With AGP, these writes of uncompressed data back to main system memory are major performance bottlenecks that PCI Express relieves. Watch for video editors that take advantage of PCI Express to enter the market.

PCI Express is backward-compatible with PCI protocols but offers numerous features that go beyond the PCI protocols. One feature—isochrony, or equality in length of time—promises to further aid video editing and other heavily multithreaded applications. PCI and AGP provide no guarantee for worst-case latency. Particularly in commercial-scale video editing for broadcast and film, this lack of guarantee for data delivery creates difficult challenges when trying to maintain output at a given frame rate. Isochrony could also ensure that systems running many multithreaded concurrent applications don't drop display frames. As chip-set and device-hardware vendors and operating-system upgrades add isochrony support, PCI Express will provide this guarantee that AGP could not.

Some graphics vendors have already figured out how to exploit PCI Express' back-channel bandwidth to create a new class of products. Before PCI Express, graphics memory was either in system memory—for chip sets with integrated graphics—or on an external card with an external graphics controller. Though AGP's developers intended it to support heavy use of system memory, especially for textures, actual AGP implementations failed to deliver sufficient performance. The performance gap was too great between implementations with local frame memory on the graphics card and implementations using the AGP bus to access system memory. So, either you integrated graphics in a chip set, or you put a lot of graphics RAM on the external controller. A significant price and performance difference exists between these two approaches. Nvidia and ATI with their TurboCache and HyperMemory technologies, respectively, use the PCI Express bus back channel to effectively cache their local memory (Figure 3).

This method provides lower performance than that of a large local memory store on the graphics card, although the performance decrease does not approach the degradation that would occur on AGP. Still, these caching technologies allow the removal of significant amounts of RAM from the graphics-controller card. Instead of, say, eight 8-Mbit×16-word DDR DRAMs for a traditional, state-of-the-art graphics controller, the controller card using caching over PCI Express could use just a single 4-Mbit×32-word DDR DRAM. Memory costs would drop from approximately $16 in today's prices to $3.50, and performance would still be better than that of integrated graphics.

The future of PCI Express graphics does not end here. The PCI-SIG (special-interest group) has announced work on a Generation 2 version of PCI Express. Though the specification was under development at press time, the PCI-SIG has announced key aspects of the new version. It will double the clock rate to 5 GHz. The group doesn't plan significant protocol enhancements over PCI Express 1.1, and Generation 2 will be backward-compatible with PCI Express 1.1. The PCI-SIG suggests that Generation 2 could be in production in 2007. When it does arrive, Generation 2 PCI Express will continue the grand tradition of regularly expanding the graphics pipe between host and graphics controller that you have witnessed from ISA to PCI to AGP and now to PCI Express.