FPGAs: Tilt or new game?
FPGAs' evolving roles in networking hardware may predict the future of the nearly ubiquitous ICs.
By Ron Wilson, Executive Editor -- EDN, March 4, 2010
| AT A GLANCE |
| FPGAs (field-programmable gate arrays) and networking have had a long, symbiotic relationship.The pressures that catapulted FPGAs into prominence in networking are reforming.It is now feasible to implement most 40-Gbps line-card functions into one FPGA.You will soon see networking applications profoundly change the nature of FPGAs. |
A long and intimate relationship exists between FPGAs (field-programmable gate arrays) and the networking-equipment industry, dating back to the days of the dot-com bubble. In those heady times, network-hardware vendors were under intense pressure to get new equipment out the door. With relatively slow transceivers and cabling holding back wire speeds outside the network cores, a features race developed among the vendors. The first one out the door with a new set of features claimed most of the pending orders, almost irrespective of price. If switch and router costs were to become ridiculous, the equipment vendors could just pass the cost along to networking start-ups, whose venture capitalists were always ready with another $20 million check, or to service providers desperate not to fall behind.
This climate was perfect for FPGAs because they can support rapid change in design requirements and have moderate operating speeds, little sensitivity to power consumption, and almost no sensitivity to price. FPGA vendors began to sell their largest parts—chips with prices of more than $1000 that had previously found use only a few at a time for prototyping or one-off projects—into the moderate-volume production networking boxes.
The impact on the FPGA industry was dramatic. The leading companies grew into billion-dollar businesses, creating the cash flow to fund much greater internal research and development. Altera and Xilinx moved from being conservative adopters of new process technology to being among the first few fabless companies at each new process node, which somewhat narrowed the performance gap between their chips and ASICs.
The way switch and router designers used the chips also changed. FPGAs had traditionally been convenient ways to implement glue logic. In the pressurized atmosphere of the Internet bubble, however, designers began to implement more complex functions, too—digital-signal-processing algorithms, framers, and mappers, for example. This trend, in turn, drove the FPGA vendors to reflect the new applications’ demands. They began seeding their logic fabric with powerful DSP engines, designing PHY (physical-layer) and MAC (media-access-control) hardware for high-speed serial I/Os onto the chips and enlarging on-chip memory structures to support greater data throughput. They also worked to make their design interfaces more powerful, recognizing that customers weren't just designing glue anymore.
But the dot-com bubble popped. Demand for network hardware plummeted and took FPGA orders with it. The FPGA-engineering work continued, however, even as the vendors scrambled to diversify their markets beyond networking. The result was that high-end FPGAs continued to include powerful features for switch and router applications, even though these features were often going into embedded-computing and control applications.
Here we go again
Now fast-forward to 2009 and a remarkably similar networking market. At every level—from the outskirts of the access network to the concentrators through which MAN (metropolitan-area-network) carriers route their traffic into the core—a scramble is under way to get more bandwidth and to support new payloads, such as video, and to move away from legacy networks to GbE (gigabit Ethernet).
No one calls the scramble a bubble this time. Once again, though, switch and router vendors are rushing to introduce new generations of equipment. This time, speed is an issue: Network operators are clamoring for 40 Gbps now and 100 Gbps as soon as possible. Complexity is spiraling, as CE (carrier-Ethernet) networks encapsulate and carry data from the old synchronous-network protocols that had maintenance and fail-soft features for carriers. New demands, such as traffic management and security, require more detailed packet inspection, classification, and processing. As a result, equipment designers are turning to a new generation of networking ICs and developing new SOCs (systems on chips) in advanced processes. Some are also looking again at FPGAs.
As the collapse of the dot-com bubble stopped the intense pressure for new features, networking-hardware developers no longer needed the fast time to market that FPGAs offer, and they could no longer pass along to their customers the cost of a boardful of $1500 chips. But FPGAs didn't disappear from routers. The chips could still be a good source of the high-speed transceivers the routers needed for interfaces, especially as moderate-speed transceivers began to appear in smaller, lower-cost FPGA families.
The chips are also useful for their flexibility in some difficult situations. “We use FPGAs only in critical time-to-market situations or to implement functions for which the requirements are still changing,” explains Martin Skagen, senior director and chief architect at Brocade. “Once the requirements are stable, we tend to lock down the design in an ASIC.”
The FPGA vendors recognize this reality. “Today, the established box makers have ASIC designs,” says Arun Iyengar, senior communications-business-unit manager at Altera. “They use FPGAs for modifying the function of the ASICs or for glue.” That state of affairs is what’s occurring now, but it may not be in the future. The new pressures for bandwidth and traffic management to cope with TV over IP (Internet Protocol), for example, are bearing down on the carriers. As a result, CE equipment is starting to experience accelerating, bubblelike change.
FPGAs in CE
“As speeds increase, we are seeing a transition in CE,” says Morteza Ghodrat, director of CE technology at Vitesse Semiconductor. “Packet processing is shifting from network processors to FPGAs as architects try to cobble together enough components to support CE functions at increasing wire speeds. This [scenario] is happening now, but it may not be a viable approach at 40 Gbps. The future is integrating functions into SOCs and then using smaller FPGAs as backplane or link drivers or to offload carrier-proprietary functions, such as Layer 3 services or DPI [deep-packet inspection].”
What SOC vendor Vitesse sees as a transitional form, however, Altera and Xilinx see as the leading edge of a permanent transition. “What’s changing people’s thinking is that wire-speed DPI just isn't possible for an NPU [network-processing unit] operating faster than 10 Gbps,” says Iyengar. “So architects of 40-Gbps systems look to something like an [Altera] Arria-sized FPGA to offload the inspection.” The next logical step is to ask what more you could do with more FPGA.
The answer, Iyengar says, is a great deal more. “Today’s 40-nm parts are fast and dense enough to bring in four lanes at 40 Gbps and handle packet processing and inspection at wire speed. So a large FPGA can handle the whole packet-processing task, not just one inspection stage,” he adds (Figure 1). This approach requires the FPGA to support both an external TCAM (ternary-content-addressable-memory) interface and a significant amount of DRAM. “You can use an algorithmic search engine and DRAM in place of the TCAM,” Iyengar says. “But, generally, an FPGA doing DPI, QOS [quality-of-service] functions, and policing is going to need about four banks of DDR-3 DRAM.” Ironically, the high-speed transceivers that made the FPGAs such good glue are now gluing memory and other engines to the FPGAs.
Gilles Garcia, business group director at Xilinx, agrees. In the short term, FPGAs may enter the architecture to increase differentiation, but, in the long run, “FPGAs can make ASSPs [application-specific standard products] irrelevant,” Garcia says. “All 40- and 100-Gbps designs are using FPGAs in one way or another.” To some extent, Garcia admits, the FPGAs are there because off-the-shelf parts aren't ready. Trials of 100-Gbps designs are going on, and some vendors are developing 160-Gbps devices. Only one commercial framer/mapper chip is available for that kind of speed, however, he claims. And if you put together the needs of a 100-Gbps product line, including packet processing, traffic management, QOS, and security, he says (Figure 2), “SOCs are just nonstarters.”
Analyst Bob Wheeler of the Linley Group doesn't see NPUs disappearing just yet. Although Wintegra and LSI are focusing on 10-Gbps or slower applications in the access network, EZchip, Xelerated, and Broadcom are all in the game for MANs as they move to 100 Gbps. The NPU and SOC vendors may have longer chip-development cycles than do FPGA users, but they also have enormous advantages in speed, density, and power.
|
A vendor that wishes to remain anonymous agrees with the FPGA suppliers’ view. This company uses FPGAs to do all the per-packet heavy lifting in an advanced Layer 3 flow-control system. The system manages to use a little of everything: ASSP switches on the front end, an NPU for setting up and taking down flows, and the FPGAs for inspecting individual packets. This arrangement allows a 1U box to handle 40 Gbps. The company doesn't see the FPGAs as interim solutions. By the time 100-Gbps links are common, the company expects to have access to FPGAs fast enough to do the work.
This vendor uses FPGAs much as Vitesse’s Ghodrat describes: as one component in a heterogeneous pipeline. But other design teams are moving in the direction Xilinx’s Garcia suggests: toward eliminating ASSPs and NPUs from their product lines. One example is Danish intellectual-property vendor TPack. Lars Pedersen, chief technology officer at the company, says that his organization is focusing on the rise of the OTN (optical-transport network) at the MAN level as well as the expansion of fast Ethernet. “We see both 40- and 100-Gbps Ethernet coming this year, as well as new MPLS (multiprotocol-label-switching) and OTN standards with more emphasis on packet transport,” he says. Pedersen sees the same issues all across the network, from access aggregation to MAN routing. As packet-based services, such as video, dominate, carriers start wanting all packets delivered in the order and with the timing with which they arrived. This level of service is far beyond conventional Ethernet switching, and this QOS must be in every node. “Operators want the same functionality on every box—from a 5-Gbps pizza box to a 5-Tbps rack,” he says.
TPack’s approach is to create the functions for a line card in RTL (register-transfer-level) logic and then synthesize the design into FPGAs for customers. The latest 40-nm Altera parts, Pedersen claims, allow TPack to put a full 40-Gbps Ethernet switch into one FPGA, with all the carrier-class features operators are requesting. Pedersen says that the 5-Gbps pizza-box implementation fits into one $50 FPGA. The plans don't stop there. TPack last year began designing a 100-Gbps, two-FPGA Ethernet device, which he expects to release in the second quarter of this year. One FPGA, the packet processor, connects through a 100-Gbps Ethernet CAUI (100-Gbps attachment-unit-interface) port to an optical module. The chip implements Ethernet/MPLS-TP (transport-profile) switching and tunneling; protection and timing facilities; and an Interlaken interface, a standard protocol for packet transfers between components in communications systems. The second FPGA, a packet manager, provides per-flow queuing and policing. This chip also has an Interlaken interface, and both FPGAs have private DRAM connections. Together, the two FPGAs will provide essentially the same level of CE wire-speed functions as planned 100-GbE ASSPs will.
TPack officials see FPGAs taking all the functions of the line card, but Altera’s Iyengar sees them going even further. “Some architects are looking at an FPGA with 48 transceivers and asking why they couldn't use it to build switch fabric,” he says. “Such a design would be easily upgradable to new speeds and might make a lot of sense. But it is a new area for us.”
Bracketing the future
Clearly, it is possible to implement an entire line card’s functions in FPGAs. For markets in transition—emerging 100-Gbps CE or new-generation OTN, for example—the flexibility of having everything in FPGAs may prove essential. Still, Brocade’s Skagen points out that, as standards mature, FPGAs quickly lose their luster. Being able to put all the functions for CE switching into FPGAs is one thing, but wanting to do so is another. “I can't agree that FPGAs are approaching SOC capabilities,” Skagen says. “For complex functions, a cell-based design will be 15 or 16 times denser than an FPGA. The speed will be much higher, as well. FPGAs are simply not fast enough to sit in our pipeline. And ASICs are an order of magnitude more energy-efficient. Keeping the power down on the chips makes life a lot simpler for everyone.”
The role for FPGAs, then, remains tightly circumscribed. On one boundary, the chips are helpful to network-equipment manufacturers only when they are available with fast-enough transceivers and logic fabric to keep up with wire speeds. On the other boundary, FPGAs become rapidly less competitive as soon as standards gel enough to implement with firmware-programmed ASICs or ASSPs. To stay in the networking market, then, an FPGA vendor must arrive early with fast-enough transceivers and fast-enough fabric. And the vendor must quickly drive these features down market to smaller, less expensive devices before alternative ways of implementing the line card start to appear. Only with less expensive devices can the FPGAs retreat into their customary special-function role instead of having vendors sweep them from their product lines.
This scenario may help explain why Altera last month began discussing its 28-nm generation—long before the products’ availability. The company is anticipating the next step in networking hardware beyond 100 GbE and weighing the strengths and weaknesses of its 28-nm-process capabilities against the coming task. “We see transmission going to 100 Gbps and then right on to 400 Gbps,” says Luanne Schirrmeister, senior director of component products at the company. “Today, it takes over 350,000 logic elements to implement the front-end block for 100 GbE: That includes MACs and the Interlaken interface,” she says. “To move this [speed] up to 400 Gbps is just impractical in 40-nm FPGAs. But even moving to 28 nm by itself doesn't solve the problem.” The transceiver isn't the issue. Schirrmeister says that Altera’s 28-nm chips will offer enough 28-Gbps transceivers to support 400-Gbps ports. The problems are the speed, density, and power of the programmable-logic fabric. There will not be that much more logic density or less power and little increase in speed in moving from 40 to 28 nm.
So instead of counting on scaling, Altera will offer Embedded HardCopy. Altera and its customers will be able to implement certain blocks using the company’s HardCopy metal-programmed ASIC capability: a step roughly halfway between implementing in programmable-logic fabric and doing a full cell-based ASIC implementation. Blocks that designers put into HardCopy will thus be denser, faster, and lower in power than in programmable logic but not as much so as cell-based portions of the FPGA, such as the DSP blocks. Altera will place these HardCopy blocks inside the FPGA chip for optimum routing to the other resources. The result will be an application-directed FPGA, with certain functional blocks hard-embedded in an otherwise field-programmable chip.
This move is necessary to meet the networking industry’s needs. It will also influence the way FPGAs relate to the rest of the electronics industry. Application-directed parts will move rapidly down the price curve, becoming attractive to users in other applications but with algorithmically similar problems. And the Embedded HardCopy design flow will be available to all customers. Once again, as with the long-ago appearance of DSP blocks and high-speed transceivers, features necessary to the networking industry will become available and affordable to everyone else. The wise designer will figure out how to apply them.
Altera www.altera.com
Broadcom www.broadcom.com
Brocade www.brocade.com
EZchip www.ezchip.com
Linley Group www.linleygroup.com
LSI Corp www.lsi.com
TPack www.tpack.com
Vitesse Semiconductor www.vitesse.com
Wintegra www.wintegra.com
Xelerated www.xelerated.com
Xilinx www.xilinx.com
