Why FEC plays nice with DFE
At data rates above about 10 Gbits/s, the frequency response and impedance mismatches from the transmitting end of one SERDES (serializer-deserializer) to the receiving end of another SERDES causes eye-closing ISI (inter-symbol interference). The combination of pre/de-emphasis at the transmitter and equalization at the receiver fixes enough of that ISI to reopen the eye so it can operate at a reasonable BER (bit error ratio). The receiver usually employs two types of equalization: CTLE (continuous time linear equalization) at its input and DFE (decision feedback equalization) that feeds back ISI corrections following identification of 1s and 0s by the decision circuit.
The gross character of the channel (Figure 1), whether it's a trace between chips, traces from circuit to backplane to circuit, a module-to-host kind of thing, or cables, has the frequency response of a really bad low-pass filter. Correcting the response of a low-pass filter is easy: either crank up the high frequency components of the signal (pre-emphasis), attenuate the low frequency components (de-emphasis, strictly speaking), or both (CTLE and de-emphasis in real life).
Figure 1. "Typical" channel frequency response (source, Ransom's Notes).
DFE, on the other hand, corrects ISI by messing around with logic values rather than voltage levels so its frequency response is more or less flat.
Crank up the high frequencies
Murphy's Law (a.k.a., Sod's Law) guarantees that whatever you do to fix ISI will aggravate something else. If the most obvious effect of the solution is to amplify high frequencies, then that something else is likely to be crosstalk. After all, the crosstalk picked up by a victim is proportional to the slope of the aggressor's edge and that's all high-frequency content.
The hopeful, optimistic, naïve, soon-to-be unemployed engineer will point out that differential signaling has protected us from both generation and reception of crosstalk for years. But in the neighborhood of 10 Gbits/s, bit periods and rise/fall times drop so low that differential skew and nonzero trace separation let aggressors transmit crosstalk and can't prevent victims from absorbing it. Bummer.
Even though it's crosstalk agnostic, the real trouble comes from DFE. Left alone, pre/de-emphasis and CTLE would amplify crosstalk's high frequencies and the detector would experience a few errored bits, but when combined with DFE, Murphy and Sod conspire to cause burst errors.
FEC (forward error correction) is mathematical magic. At the transmitter, you take a bunch of data bits, add a few extra parity bits—really a generalization of parity or CRC (cyclic redundancy check) bits—and run the whole "frame" through a well-chosen binary polynomial. At the other end, you run the received frame through the inverse of the polynomial, wave a logical wand, and, voila! You both identify and correct some bit errors. There are restrictions, of course, it's only magic until it fails.
100 Gigabit Ethernet and OIF-CEI use Reed-Solomon FEC (forward-error correction) coding, RS(528, 514). First, let's think in terms of generic symbols, we'll convert back to bits in a second. The idea of RS(528, 514) is that you encode 514 data symbols and 14 parity symbols into a 528 symbol frame (Figure 2). The coding-decoding process can correct up to 7 errored symbols.
Figure 2. Reed-Solomon forward error correction (source: Ransom's Notes).
Now, back to bits. 100 GbE uses 10-bit symbols. So the 528 symbol frame is 5280 bits long, holds 5140 data bits, and employs 140 parity bits. RS-FEC can correct up to 7 symbols, no matter how many bit errors there are in each symbol. Let's look at a few cases:
At its most effective, RS-FEC can correct as many as 70 errors in 5280 bits provided that those errors come in very well placed strings of 10 in a row. That is, the errors have to fit perfectly in each of seven 10 bit symbols. At the other extreme, RS-FEC can always correct at least 7 bits, but if 8 bit errors are distributed across 8 different 10 bit symbols, RS-FEC won't be able to fix them. The statistics get messy.
Back to DFE and crosstalk
The "decision feedback" property of DFE means that it corrects ISI by assuming that each incoming bit has been interpreted correctly; it's called the "ideal decision" assumption and leads to DFE's Achilles' Heel: burst errors.
If we lived in an ideal crosstalk-canceling differential-signaling world, the DFE might never see an error. After all, it's protected by both pre/de-emphasis and CTLE. Instead, those very protectors ratchet up the crosstalk stress, increasing the probability of an error.
When the decision circuit gets a bit wrong, the ideal decision assumption fails, and the DFE feeds back the wrong signal. That is, instead of canceling out ISI, the DFE gives the decision circuit more noise and it's likely to generate another bit error. The DFE is then even farther out of control and burst or avalanche errors are likely to occur.
But RS-FEC loves burst errors! By encoding data into symbols and then correcting the symbols, RS(528-514) can correct a burst of up to 70 consecutive errors. Of course, the burst isn't likely to be organized to align in the symbol boundaries perfectly, but for an isolated burst, RS(528, 514) can stomach at least 61 consecutive errors.
After all the statistics have settled, the physical layer compliance test requires BER <5.0E-5 prior to FEC to assure a functional BER <1 E-15.
For the measly price of 14/528 = 2.65% extra bandwidth, you get a dramatically increased BER test budget. Cheap!