Fabric technologies reach the mezzanine level

Telecom-equipment engineers are transforming their industry from proprietary systems with long, expensive development cycles to standards-based designs using low-cost, off-the-shelf products. The first step was a new board, backplane, and software standard dubbed the AdvancedTCA (Advanced Telecom Computing Architecture) with a large-board form factor, high-availability features, a distributed-power architecture, system management, and switched-fabric performance. Their latest innovation, the AdvancedMC (Advanced Mezzanine Card) standard, extends many of these same features onto replaceable plug-in modules or daughtercards to form a complete system-design package for telecom and other applications requiring high performance.

Designers use mezzanine modules to increase the flexibility and scalability of a board design. For example, these plug-in cards may host a variety of I/O ports or memory devices so that they can use a single baseboard in a variety of applications. Mezzanines may also contain rapidly changing technology, such as the processor section, so users can upgrade to the latest and fastest model without affecting their baseboard design. Because baseboards or carriers usually have provisions for multiple mezzanine slots, designers can often satisfy future expansion requirements by adding another module. Also, mezzanines allow you to simply increase density by providing additional board space.

Mezzanine cards got off to a rocky start in the early 1980s with cooling problems, unreliable connectors, and proprietary interfaces. Industrywide efforts to address these problems involved standardizing mechanical and electrical interfaces, resulting in several module configurations, including IndustryPack, M-module, PC-MIP, and PMC (PCI Mezzanine Card). Designers have used these cards, with their wide selection of available functions, with many systems, including PCI, CompactPCI, VMEbus, and proprietary baseboards.

PMC, which its developers based on the PCI (Peripheral Component Interconnect) bus, is currently the most popular mezzanine standard. The PMC specification, IEEE P1386, defines both mechanical and electrical properties. The card measures roughly 3×5 in., provides 32- or 64-bit datapaths, and uses one or two 64-pin interface connectors to the baseboard. The VITA (VMEbus International Trade Association) standards organization later extended the protocol to include PCI-X, support processor PMCs, and include switched-fabric interconnections. Although some AdvancedTCA-board manufacturers have adopted PMC modules, the modules have limited thermal, I/O, and power capabilities to support the new architecture. PMC modules cannot support hot-swap functions, because they plug into the interface connectors vertically and are accessible only when you remove the baseboard from the chassis.

AdvancedMC modules include single- and double-width and half- and full-height form factors. The basic single-width module measures approximately 74×183 mm and, as such, four module sites fit across an AdvancedTCA carrier card. To simplify maintenance, users can remove and replace mezzanine cards from the front panel while the carrier card remains in the chassis. AdvancedMC modules include a faceplate for I/O connectors; displays; and a module latch, which has an enclosed hot-swap switch. Using half-height modules, a single AdvancedTCA carrier card can support as many as eight AdvancedMC mezzanines. This modularity allows a designer to populate a carrier card with only the capabilities it needs at deployment and still provide expansion for future requirements.

AdvancedMC employs a subset of the same IPMI (Intelligent Platform Management Interface) that AdvancedTCA carrier cards require. Intel, Hewlett-Packard, and others defined this management-interface specification. It allows local and remote monitoring of equipment for power management, cooling, electronic keying, and hot-swap transactions. IPMI interacts with a shelf-management controller that works on its own in case the host processor is defective. With platform management, operators can actively monitor equipment for marginal operation or potential problems and correct them before they turn into system failures. Management enables the hot-swap feature of AdvancedMC modules to reduce the cost of spare-part inventories and also lower the system MTTR (mean time to repair). A switch on the module's front panel activates a hot-swap sequence to suspend operation, remove power, specify operator action, and reactivate the replacement module. Redundancy and hot-swap capability for active modules allow systems to achieve and even exceed "five-nines" availability (99.999%).

AMC.0 specifies multiple connector arrangements depending on the module's form factor and complexity. A single-width module requires 170 gold-plated card-edge pads, 85 on a side, to mate with the carrier connector. All 170 pads must be present, even if unused, to protect the mating connector. The carrier's card-edge connector segments into a basic connector and an extended connector. The basic connector is mandatory and includes pins for power, clock, management, ground, control, and fabric connections. The extended connector, which is optional, includes additional ground pins, extended fabric connections, and debugging signals. A two-slot mating connector and a cutaway carrier board allow two half-height AdvancedMC modules to occupy a single mezzanine position. Four levels of sequential mating ensure a safe electrical connection sequence during module removal and insertion.

Individual mezzanine modules use dc/dc converters to generate local voltages from a 12V supply that the carrier provides. This approach reduces the likelihood that a single point of failure will bring down multiple modules. Although a single AdvancedMC connector can handle as much as 60W, single-width modules have an approximate 30W thermal envelope. Compared with today's PMC specification, which is limited to 7.5 or 12W, AdvancedMC provides more than a 100% power increase over its surface area. A double-width module can use the full 60W through one connector.

On the interconnect side, AdvancedMC supports high-speed serial interconnects that will eventually include all of the switched fabrics AdvancedTCA allows. The base specification defines a fabric interface with as many as 21 ports or 42 differential pairs providing full-duplex, point-to-point connectivity between modules or to the baseboard. At 12.5 Gbps per port, AdvancedMC can handle multiple lanes of the modern protocols, such as Ethernet, PCI Express, RapidIO, and InfiniBand. Subsidiary specifications map the ports to specific fabric requirements. For example, AMC.1 defines ports for PCI Express and Advanced Switching environments, including guidelines for various lane-width implementations. Jeff Durst, director of product marketing at Artesyn Technologies, says, "On the surface, it looks like you have too many fabric choices with AdvancedMC, but if you listen closely to your customers, they will narrow it down for you. One sure bet is Ethernet. We are transitioning from Gigabit Ethernet to 10-Gbit Ethernet, and there will be plenty of headroom for advanced systems. On the other hand, if everything goes well in the laboratory, and silicon emerges to support it, serial RapidIO looks to be an important second thread."

SBS Technologies recently announced the Telum1001-DE network adapter for applications requiring full-duplex DS3/E3 ATM wide-area-network connectivity, such as line-access multiplexers, wireless networking, router/gateways, streaming video, and protocol internetworking (Figure 1). The module complies with the AMC.0 base and AMC.1 PCI Express specifications. An IPMI microcontroller subsystem initializes board-level parameters, monitors board voltage and temperature, maintains system status, and tracks hot-swap operations. Rubin Dhillon, general manager of the SBS Technologies Communications Design Center, says, "Our customers are demanding the high-bandwidth capabilities of PCI Express, manageability with IPMI, and hot-swap capabilities that existing mezzanine approaches, such as PMC, cannot deliver." The Telum1001-DE comes with carrier-grade Linux, VxWorks, and Windows drivers, as well as with a driver-development kit. The module is available now and sells for $1836.

Kontron's AT8001 server blade is one of the first AdvancedTCA boards to support the AMC.0 and AMC.1 specifications and offers space for two single-width, full-height AdvancedMC modules (Figure 2). The AT8001 integrates the low-voltage, 2.8-GHz Xeon processor with a 1-Mbyte L2 cache, an 800-MHz system bus, and PCI Express technology. The board features dual Gigabit Ethernet and optional dual Fibre Channel connections, 8 Gbytes of DDR-II 400 SDRAM, plus support for MontaVista carrier-grade Linux. The AT8001 complies with the IPMI 1.5 standard and includes a controller for management and implementation of diagnostic functions.

Featuring the ADSP-TS201S TigerSHARC processor, BittWare's B2-AMC AdvancedMC module supports universal baseband processing for wireless-communications infrastructures, such as 2G, 2.5G, 3G, WiMax, and software-defined radio (Figure 3). A full-height, single-width mezzanine module, the B2-AMC attaches to AdvancedTCA carriers or other cards with AMC bays and is fully hot-swappable. The board features four TigerSHARCs, an Altera Stratix II FPGA, a variety of front- and back-panel I/O interfaces, IPMI system management, and a configurable 4× network interface. It also provides a 10/100 Ethernet interface and a Gigabit Ethernet interface for command, control, and reprogramming, as well as flash memory for booting the DSPs and FPGAs.

With all the high-power, hot-swap, switched-fabric, and management features of AdvancedMC, it is understandable that designers considered using these modules to plug directly into a backplane for small, stand-alone systems. In fact, PICMG has formed a working group to develop the MicroTCA, a specification based on the concept; the group expects to have a prototype by midyear. MicroTCA provides a stand-alone chassis with a backplane that directly accepts AdvancedMC cards, thereby eliminating the AdvancedTCA carrier board. The smaller form factor makes the concept viable for lower budget applications in telecom and a range of embedded projects. Although PICMG has not yet released it, a proposed chassis configuration includes a 4U-high×19-in.-wide unit that provides 12V payload power, management, and cooling for as many as 48 half-height, single-width AdvancedMC modules. It is also considering a smaller cube design—approximately 8 in. on a side. Designers are shooting for chassis-hardware costs of approximately $500. Eran Strod, product-marketing manager at Mercury Computers, says, "MicroTCA is a great idea. It allows us to take advantage of the latest processors and fabric technology like serial RapidIO in a smaller, lower cost form factor. It will be a good fit for a lot of lower end applications that don't need all of the board real estate that AdvancedTCA provides."

Whether designers use it as a mezzanine card or transform it into a new board-level standard, AdvancedMC provides one of the essential building blocks to update the telecom industry from mostly proprietary to mostly purchased products with the same high-performance and high-availability characteristics. AdvancedTCA and AdvancedMC promise to reduce the cost of system ownership with shorter development schedules, off-the-shelf hardware, and lower maintenance expenses.

Author Information

You can reach Technical Editor Warren Webb at 1-858-513-3713, 1-858-486-3646 (fax), and wwebb@edn.com.