Keeping QPACE with quantum chromodynamics

This blog has covered several instances in the past year of FPGAs used in scalable supercomputing (oops, “HPC,” seems as though the supercomputing label is all but obsolete). One startup with roots from Convex Computer, called Convey Computer Systems, even built its business plan around FPGAs. But the use of Xilinx’s Virtex-5 in a network-management role in the University of Regensburg’s QPACE computer is somewhat unique in placing FPGAs in a communications and networking role in the data plane.

So what’s the big deal in using Virtex-5 for networking and the PowerXCell 8 (a spin of the Sony/IBM/Toshiba Cell architecture) as the primary processing engine? For one thing, power dissipation is low enough to put the QPACE in the Green 500 of environmentally-friendly HPC platforms. The system claims peak performance of 26 Tflops (single-precision, scaling to 56 Tflops double precision), with power consumption of 29 kW per rack.

The system is used for modeling “color” interactions between quarks and gluons in quantum chromodynamics, in particular the study of “Lattice QCD”, which has nothing to do with another FPGA vendor, and everything to do with modeling quark interactions without perturbations, by establishing a lattice or grid in space or time to describe the interactions of the color force.

This requires a lot of short messages among nodes in the QPACE architecture. Hence, the network processor is as important as the control plane architecture. Since network latency in excess of 10 microseconds might be common in some designs, but is unacceptable in QCD studies, Xilinx’s team working with the University of Regensburg took typical network latency down to 3 microseconds.

In a release on the QPACE design-in, Xilinx also cited its RocketIO transceivers as a critical factor in implementing a Rambus FlexIO interface for the QPACE cards. In the scaled, card-based HPC systems of the next decade, we should expect a strong presence by FPGAs in implementing this type of dual CPU/NPU architecture for problems as diverse as climate modeling, advanced 3D visualization, and subatomic physics.