April 9, 1998

Fibre Channel finally fills the gap in I/O subsystems

Maury Wright, Technical Editor

After a decade of development and waiting in the wings, Fibre Channel appears poised for deployment--delivering 100-Mbyte/sec data rates in storage and networking subsystems. Vendors now offer designers a plethora of building-block products, starting with ICs, and including connectors, cables, modules, boards, and disk drives.

After a tempestuous marketing battle and a lengthy development process, Fibre Channel is finally ready to prosper. The technology offers compelling features that start with 100-Mbyte/sec data rates and include a unique flexibility that can serve storage and networking applications and even replace a computer bus in some cases. Moreover, vendors now offer a variety of products that becomes more affordable every day. To understand how you can deploy Fibre Channel in general-purpose computers or embedded systems, you should examine the basics of the technology, speculate about what might be a killer application, and identify the key products that enable high-speed subsystem designs.

The concept of Fibre Channel emerged more than a decade ago and, in retrospect, posed a daunting challenge to semiconductor vendors. After all, typical SCSI implementations then attained 1.5-Mbyte/sec rates, with a few vendors offering 4-Mbyte/sec synchronous products. Ethernet and other LANs operated even slower. Yet, Fibre Channel's founders envisioned that a serial interface could ramp to 100-Mbyte/sec rates. When you realize that no interface enjoys wide success until vendors ship single-chip implementations, it's unsurprising that Fibre Channel took more than a decade to mature.

The ANSI committee working on Fibre Channel, however, didn't just concentrate on unheard-of data rates; it also conceived a switched architecture that could provide dedicated, full-duplex connections between two devices connected via a fabric (Figure 1). The standard leverages separate transmitting and receiving circuits to enable concurrent 100-Mbyte/sec communications in each direction (Figure 2). Theoretically, a Fibre Channel fabric can support a virtually unlimited number of nodes. The addressing scheme allocates 16 million individual addresses. The ports on the fabric are "F_ports," and the ports in each device, or node, are "N_ports." The physical distance any point-to-point link can span varies based on the type of cable, and the standard provides for multiple copper and optic cable choices (Table 1).

FC-AL storage evolves

As the Fibre Channel standard progressed, it became clear that dedicating a switch port to every device might be neither economically feasible nor necessary in many applications. For example, a single host might need to connect with a number of disk drives, and that scenario cannot justify a switch. The committee developed the Fibre Channel Arbitrated Loop (FC-AL) for just such applications. FC-AL connects the transmitting port from one node into the receiving node of the next port (Figure 2). Data transfers proceed in one direction around the loop. At any time, one node on an FC-AL loop can arbitrate for the media, receive access, and use the loop as a dedicated link to another node on the loop. Fabric ports that connect to a loop are "FL_ports," and loop ports on devices such as disk drive are "NL_ports."

An FC-AL configuration can connect as many as 126 devices, although a typical loop connecting disk drives might comprise only 10 to 20 nodes. In cases requiring greater bandwidth or reliability, you can deploy dual counter-rotating loops in which each node is dual-ported. Disk-drive vendors plan to ship dual ports as a standard feature. Similar to the Fiber Distributed Data Interface (FDDI), a dual-FC-AL implementation doubles aggregate bandwidth when all nodes are working. Moreover, the second loop provides a way for the loop to heal automatically and form a single loop when any node fails. Fibre Channel implementations can mix and match looped and switched connections.

As you can see, the Fibre Channel committee developed a capable physical interface but also sought to make that interface protocol-independent. This Fibre Channel feature differentiates it from other interfaces, allowing network protocols, such as TCP/IP, to operate over Fibre Channel just as readily as do SCSI disk protocols, which are called Fibre Channel Protocols (FCP) when you map them to the serial interface. (See boxes "Data storage meets network layers" and "Fibre Channel standards and organizations" and References 1 through 3. For more information, check the Web sites of the Fibre Channel Loop Community (FCLC) and the Fibre Channel Association (FCA).)

Fibre Channel application path

Given a basic understanding of Fibre Channel capabilities, you can envision an application-migration path in which the new technology slowly supplants other storage interfaces and networks. Fibre Channel proponents fought a lengthy marketing battle with another serial interface, the Serial Storage Architecture (SSA), which IBM championed. Today, it's clear that SSA isn't a roadblock and will disappear over time, despite the fact that IBM has shipped SSA in much greater volume than total Fibre Channel shipments to date. In data storage, Fibre Channel must replace the venerable SCSI, although you can expect the two to coexist throughout the next decade. Fibre Channel is also unlikely to replace Ethernet but will carry network traffic in some cases.

Early on, the opportunity for Fibre Channel lies in solving problems that SCSI can't address. For example, Fibre Channel can span far greater distances, both between nodes and in total. The long cables make Fibre Channel a natural for box-to-box links. For example, such companies as Sun (www.sun.com), Silicon Graphics (www.sgi.com), and even Compaq (www.compaq.com) have widely deployed Fibre Channel as a link between computers and disk subsystems and arrays.

Within a Fibre Channel disk subsystem--either a RAID (redundant array of inexpensive disks) or JBOD (just a bunch of drives)--the actual disk drives can use the SCSI or FC-AL interface. Bridging an FCP stream to a SCSI channel within a subsystem is a relatively trivial task.

Inside the box, SCSI remains attractive because vendors have pushed data rates to 80 Mbytes/sec in wide Ultra 2 drives that use low-voltage-differential signaling (LVDS). SCSI drives also have a significant sales volume advantage that translates to lower cost, ensuring that the drives will be around for a long time in most applications. Even at the high end, SCSI drives will remain inside Fibre Channel subsystems as long as SCSI drives keep pace with the raw data rate from the disk media and maintain a cost advantage over FC-AL drives. Today, only Seagate ships FC-AL drives in volume, and it proclaims that SCSI and FC-AL drives have reached price parity. Realistically, FC-AL drives still cost 20% more. This premium will drop as Seagate ships products in higher volumes and as Hitachi, Fujitsu, and Quantum ship drives in volume.

Fibre Channel proponents, especially those such as Hewlett-Packard that offer no SCSI products, proclaim a number of advantages--that may offset a price premium--resulting from using FC-AL drives. Some of these proclamations hold water, and some don't. For example, the small-diameter serial cable and small connector attributed to serial Fibre Channel presumably offer cost and convenience advantages over parallel SCSI cable and connectors. However, in the space that FC-AL and SCSI compete, either interface runs along backplanes rather than cables. The SCSI backplane may be marginally more complex, but the effect on cost is negligible, especially considering SCSI volumes. The two also use similar backplane connectors, as Revision 2 of the Single Connector Attachment (SCA) spec defines.

FC-AL boosts reliability

FC-AL does offer some potential advantages in reliability and fault tolerance. Designers using SCSI drives have developed backplanes and arrays that support hot-swapped drives and fault tolerance, but FC-AL supports such concepts even in a loop configuration. FC-AL vendors have also taken reliability even further. For example, a number of vendors have developed Fibre Channel hubs that work in a similar way to Ethernet hubs. A hub allows individual drives or small FC-AL configurations to connect with a large loop via the hub in a star topology. By making the hub the center of the star, vendors can add fault-tolerant features in the hub yet maintain a logical loop. In turn, the hub can bypass any failed nodes and manage maintenance and repair operations.

In the not-too-distant future, it may be possible to connect a rack of FC-AL drives via a backplane with a hub residing directly on the backplane. The local hub could greatly improve reliability and maintenance. Moreover, the hub could be modular, providing users with a choice of using a hub or a switch to connect the drives, thereby offering a trade-off of cost vs aggregate bandwidth. These ad-vancements should enable FC-AL drives to supplant SCSI drives within subsystems.

In the short term, the most promising opportunity for FC-AL drives comes in single-channel JBOD subsystems or FC-AL configurations that connect more than the 15 drives that wide SCSI supports on a channel. Applications such as video editing can require more than 15 drives and benefit from the slightly higher data rates that FC-AL drives afford.

You may think that none of these scenarios applies to you, so you needn't worry about Fibre Channel. However, no one can predict all the applications that Fibre Channel will find. Suffice it to say that designers have deployed the technology as often in embedded applications as in MIS departments. For example, DY 4 offers the $2500 PMC-640 Fibre Channel module for its VMEbus single-board computers and targets the product as a replacement for Mil-Std-1553 links. Moreover, you would be well-advised to consider Fibre Channel whenever you need a low-latency, high-speed interface for point-to-point, bus, or network topologies.

The killer application

An imminent killer application--storage-area networking--that only Fibre Channel can address will benefit all potential Fibre Channel  applications by creating more products and lower prices. This emerging application links multiple servers and data-storage devices on a private backbone network that lies behind one LAN or more (Figure 3). On a storage-area network (SAN), the servers communicate with the storage devices using the SCSI protocol, which is generally more efficient than TCP/IP or another networking protocol.

SANs can mix and match Fibre Channel loops and fabrics using switches and hubs. The SAN might feature FC-AL-to-SCSI routers and bridges and, potentially, even bridges between FCP or SCSI and TCP/IP traffic. Regardless of whether FC-AL or SCSI drives prevail in SAN environments, the environment requires numerous Fibre Channel interface cards and ICs. The SAN promises to ramp volumes of these key Fibre Channel components, quickly making them more attractive for other applications.

Vendors have recently been jumping on the Fibre Channel bandwagon in great numbers. More than 70 vendors have announced Fibre Channel products, and a dozen or so more vendors will emerge this year (Table 2). Several vendors serve every category of products with key areas, such as interface cards and ICs, enjoying the greatest support.

A single-chip implementation of an interface or network is the key for widespread acceptance, and that gate to Fibre Channel success has opened. QLogic has begun shipping its ISP2100 Fibre Channel protocol IC that integrates a 64-bit, 66-MHz PCI bus on one side and a direct interface to copper Fibre Channel media on the other. The IC costs around $150, but a PCI-interface card requires little extra other than passive components and software drivers. QLogic's ISP2100 can serve only in FC-AL--not switch or hub--applications.

Not surprisingly, competing IC vendors don't think QLogic developed the best Fibre Channel implementation, given the current state of semiconductor technology. QLogic is the first company to build a CMOS IC that integrates the mixed-signal transceiver, including the serializer/deserializer (SERDES) function that sits between the outside serial world and the internal 8B/10B encoder/decoder. Competitors such as Hewlett-Packard and Emulex feel strongly that an IC vendor could better realize the transceiver in bipolar or GaAs technology. These companies assert that QLogic's CMOS SERDES may work in some applications but may create reliability problems in noisy environments or as cables get longer. Your media choice can also affect transceiver issues (see box "Media choices complicate physical interface").

Both Emulex and Hewlett-Packard have developed ICs that target FC-AL interface cards, integrate a PCI interface, and provide a 10-bit parallel interface to an external SERDES. Note that QLogic's IC also offers a parallel interface for external transceivers. The company claims that the auxiliary interface will be useful to designers who need to connect optical transceivers or who want to use a more robust bipolar or GaAs transceiver in copper applications. Several companies, including Hewlett-Packard, make Fibre Channel transceivers. Perhaps Vitesse offers the most extensive set of choices, which it makes using GaAs technology. Vitesse asserts that a stand-alone SERDES IC adds reliability and that volume prices of such ICs will reach approximately $3 by the end of this year.

In other features, Hewlett-Packard's $150 TachLite matches QLogic's offering with a 64-bit, 66-MHz PCI implementation. Emulex's IC includes only a 32-bit PCI interface but supports simultaneous full-duplex operation, whereas the ICs from QLogic and Hewlett-Packard can only transmit or receive at any moment. Emulex plans to offer the IC not as an off-the-shelf product but integrated on the company's $600 LP3000 PCI-interface card. Emulex has, however, provided access to its ICs, so interested parties might be able to negotiate an IC deal and use the LP3000 as a reference design. Emulex's LP3000 adapter costs approximately the same as cards from QLogic and Hewlett-Packard, so the IC could have a price similar to that of ICs from those companies.

Hewlett-Packard, Emulex, and Q-Logic have also taken different approaches to the internal IC architecture. Hewlett-Packard uses a hard-wired, state-machine implementation, whereas the others include onboard processors. Emulex includes dual parallel processors that enable full-duplex support. It's tough to judge the performance merits of the IC designs because no benchmarks exist. Moreover, the maximum Fibre Channel rate exceeds the capabilities of most PCI hosts. For the short term, you may have to base your IC choice on purely theoretical performance arguments or on logistics issues, such as the SERDES or software drivers. For example, QLogic's IC uses the same processor core it uses in its SCSI ICs, and the company asserts that its FC-AL driver uses 90% of the same code as its SCSI driver. Designers familiar with the SCSI IC could quickly scale the FC-AL learning curve.

Now and in the near future, you face a plethora of other IC choices. Adaptec and Symbios have announced ICs, although Adaptec has also announced plans to buy Symbios. It's unclear which ICs will emerge as mainstream products from those companies. Also, expect ASIC houses, such as LSI Logic and VLSI Technology, to offer Fibre Channel support in their portfolios.

Should you need an IC to develop switch fabrics or hubs, you will have a more limited menu of choices. Emulex has a three-chip set that it uses on its $1495 LP7000 adapters and in its own hubs and switches. The chip set relies on a Strong ARM µP and supports Fibre Channel profiles other than FC-AL.

Hewlett-Packard offers by far the most popular choice in its Tachyon IC, which has been available for several years. The IC offers the full-duplex support that's the key differentiator between FC-AL and switched or point-to-point Fibre Channel links. Unlike the Tachlite FC-AL controller, the Tachyon has no PCI interface but rather a Hewlett-Packard-proprietary interface. Moreover, Hewlett-Packard doesn't publicly offer a companion PCI to the Tachyon IC, although you can buy one from Interphase.

Tachlite and QLogic's ISP2100 are a generation ahead of any IC that targets full-duplex links other than the Emulex ASIC that the LP3000 uses. Expect QLogic to offer a full-duplex IC soon, and expect Hewlett-Packard to announce a successor to the full-duplex Tachyon as well.

You probably should also take a brief glance at the future before you settle on an IC or a board-level product. Some Fibre Channel proponents are discussing 2-Gbyte/sec data rates, and Hewlett-Packard claims that its state-machine-based Tachyon architecture can easily move to the higher rates. Hewlett-Packard envisions a market much like the current 10/100-Mbps Ethernet market in which end users get 100-Mbps support even if they need only a 10-Mbps adapter. The company believes a number of vendors will ship 2-Gbyte/sec adapters long before most loops or switches support those speeds. Vitesse indicates that it plans to sample 2-Gbyte/sec transceivers this quarter. It remains to be seen whether Hewlett-Packard or others can essentially give away faster silicon. In many cases, dual 1-Gbyte/sec ports offer more advantages than single 2-Gbyte ports. Moreover, it's clear that 1-Gbyte/sec ports will dominate in switches and hubs for at least five years.

References

1. Dedek, Jan, Fibre Channel: The Basics, Ancot Corp, Menlo Park, CA, 1-650-322-5322.

2. Wright, Maury, "New peripheral interfaces: fast & full of features," EDN, Oct 12, 1995, pg 69.

3. Wright, Maury, "Disk drives at 40: lean, mean storage machines," EDN, Nov 7, 1996, pg 50.

| EDN Access | Feedback | Table of Contents |