The scopes trial
Thanks for visiting "The scopes trial." Please click here for a zip file containing detailed results of the shoot-out described below.
Also, we encourage your comments. Please send any and all feedback to Senior Technical Editor Dan Strassberg.
The plans for a hands-on shoot-out between what were then the world's only two 6-GHz-bandwidth real-time-sampling digital oscilloscopes—Tektronix's TDS 6604 (Figure 1) and LeCroy's Wavemaster 8600A (Figure 2)—began to take shape in mid-August of 2002. Agilent had kept news of its Infiniium 54855A (Figure 3), under such tight wraps that the press would not get word of it until several weeks later. Only a couple of weeks earlier, on July 30, the day before the 8600A's public announcement, LeCroy had visited EDN's offices to demonstrate the new scope and, for comparison, had brought along a Tek scope. A 6604 wasn't available, so the company's director of product management, Mike Lauterbach, PhD, had brought an older, narrower bandwidth Tek unit, the 4-GHz-bandwidth TDS 7404.
Lauterbach readily conceded that, in generating complex displays that involve a lot of data processing—a key application for high-end scopes—the 6604 is much faster than the 7404. He said, however, that in creating such displays, the 8600A was still approximately an order of magnitude faster than the 6604. He also emphasized how important it is for a scope to rapidly put meaningful displays on the screen. Scopes have been a mainstay of EEs for more than half a century. A key reason for this enduring popularity is the highly interactive nature of using a scope.
You observe something on the scope display. If what you see is a problem, you hypothesize a fix for it. If it's unexpected behavior of the UUT (unit under test), you think up a new measurement to confirm the suspected cause. Either way, you can immediately try out your idea. Without this level of immediacy and interactivity, the lab experience takes on an entirely different character. LeCroy is hardly alone in recognizing the importance of scope interactivity; the need for this capability motivated Tektronix to incorporate the DPO (digital-phosphor-oscilloscope) mode in the TDS 7000 series, of which the 7404 but not the 6604 is a member. Unfortunately, some of the most valuable scope features are unavailable in the DPO mode.
Although its bandwidth is lower than the 8600A's and 6604's, the 7404, with deep-memory options, offers much greater memory depth than does the 6604. (The 7404's standard memory depth is 100k samples/channel in four-channel mode, optionally expandable to 32M samples in the single-channel mode.) In several respects, the deeper memory makes the 7404 more directly comparable to the 8600A, whose standard memory depth is 1M sample/channel in the four-channel mode and optionally as much as 48M samples/active channel in the two-channel mode. In the two-channel mode, both the 8600A and the 6604 acquire data in real time at a maximum rate of 20G samples/sec, whereas the 7404 can sample at that rate in the single-channel mode.
The 7404 and 8600A use true deep-memory architectures, but the 6604 uses an architecture known as FISO (fast in/slow out). In contrast with deep-memory scopes, which digitize in real time, FISO scopes acquire samples in real time but store the samples in analog form before digitizing them at rates that—at high sweep speeds—are usually slower than the acquisition rate. (Many FISO scopes, including the 6604, also offer a random-repetitive-sampling mode, which can further boost the effective rate of sampling repetitive waveforms. Random-repetitive sampling is also a feature of many deep-memory scopes. Both the 7404 and the 8600A offer it.) The 6604 provides a fixed memory depth of 125,000 samples/channel. This depth and the normal maximum real-time sampling rate (10G samples/sec on each of four channels), double when you operate the scope in the two-channel mode.
The game plan
After the 8600A's public announcement, a Tek representative was eager to demonstrate that LeCroy was wrong about its claims about the greater speed at which its new scope generates complex displays. The 8600A's predecessor product, the 5-GHz-bandwidth Wavemaster 8500, had been available for many months. Its architecture and the speed at which it generates displays are the same as those of the 8600A, and Tek had had ample opportunity to determine what the 8500 could do.
Each company proposed a list of tests (Table 1) and agreed to send two engineers each to meet at EDN's Newton, MA, office to use both the 8600A and the 6604 and perform both sets of tests to see which scope was faster at creating useful displays. With LeCroy's agreement, Tek later proposed bringing a CSA 7404 in addition to a 6604. The CSA 7404 is a version of the TDS 7404 that offers enhanced waveform-analysis capabilities. The CSA unit would enable Tek to demonstrate the responsiveness of its top-of-the-line deep-memory scope. As the arrangements moved forward, EDN's sibling publication, Test & Measurement World, agreed to participate in and co-sponsor the event.
At this point, the situation suddenly became more complex. A phone call from Agilent revealed that that company was about to introduce its own four-channel, 6-GHz-bandwidth DSO (Reference 1). I offered to include Agilent in the shoot-out, and the company initially expressed interest, but the logistical problems proved insurmountable. I would have had to inform Tek and LeCroy of Agilent's participation. In so doing, I would have revealed Agilent's plans to its competitors as much as six weeks before the company's planned Nov 1 announcement. This schedule proved unworkable, so we agreed that, if the company wanted us to do so, we could later stage a second round of the shoot-out and publish the results. Only Agilent would participate, but the tests would be the same as those we had run on the Tek and LeCroy scopes. Currently, it is unclear whether Agilent is interested in such a project, but if the company wants to proceed, you will read about Round Two in EDN.
As the plans progressed, Tektronix developed reservations about participating. Some people speculated that perhaps the shoot-out's premise was naive. Tektronix is the established market leader. It had gotten its 6-GHz scope to market approximately a year before LeCroy. Customers had enthusiastically received the product, and almost everyone who has used it respects it. (Even LeCroy has called the TDS 6604 a fine scope—though pointing out that "fine" is not a synonym for "best available.") So, with no guarantee that its product would solidly trounce the competition, what could Tektronix possibly gain from participating in a shoot-out?
In the end, Tek withdrew, saying that it had had no opportunity to evaluate the 8600A's brand-new SDM (serial-data-measurement) option, which is LeCroy's answer to the CSA 7404's advanced analysis capabilities. The option was too new for Tek to have tried it, and both companies had agreed to bring products only if the competitor could have evaluated them. LeCroy's proposed tests didn't require the option's presence, but Tek proposed several tests that involved serial-data analysis. LeCroy said that, for this purpose, it would use an option that Tek had had ample time to evaluate—JTA2 (jitter timing and analysis)—which had been available since LeCroy announced the Wavemaster series in March 2002. (On the Tek scopes, equivalent functions are available via the JIT3 option, which we used to compare the two manufacturers' scopes' speed of serial-data analysis.) Moreover, LeCroy said that it would disable SDM, which it could easily do from the 8600A's main menu, Still, Tek continued to object, perhaps because it misunderstood LeCroy's proposal.
What really happened
Despite Tek's withdrawal, the shoot-out took place on schedule. LeCroy owns a TDS 7404 equipped with JIT3 and as much memory (8M samples/channel) as the scope can accommodate. LeCroy also happened to have a rented TDS 6604 and was able to extend the rental period by several days. Unfortunately, JIT3 was not enabled on this scope. At 8 a.m., Tuesday, Oct 29, 2002, the two LeCroy representatives appeared with the two Tek scopes, a WaveMaster 8600A containing the JTA2 and SDM options and a carload of signal sources (Figure 4; see sidebar "Participants' biographies").
The project yielded much interesting and useful data, but the data would have undoubtedly been more interesting and useful had Tek at least sent an observer. (After Tek withdrew, EDN extended such an invitation—in time for a Tek representative to make the trip—but received no response.) A Tek observer likely could have shown where we were going wrong in using the Tek scopes. That advice might have kept the results from so overwhelmingly favoring LeCroy. Each company had proposed six tests, and, in two days, we ran four tests from each company's list on each 6-GHz scope. (One of Tek's proposed tests required using the 7404 rather than the 6604.)
We couldn't perform the full slate of 12 tests because time didn't permit it and because we eliminated any test that we couldn't perform on both the 8600A and at least one of the Tek scopes. We used the TDS 7404 for only one test, which the available 6604 wouldn't run because its JIT3 option wasn't enabled. The 8600A handily won every test related to the shoot-out's original purpose: assessing the speed at which the scopes presented data (see PDF-only sidebar "Summary of throughput results"). The 8600A's minimum victory margin was 2.1-to-1 in a test that LeCroy proposed. The maximum margin was 651-to-1 in a test that Tek proposed.
Note, however, that to achieve the 651-to-1 result, LeCroy used the SDM option. As indicated, had Tektronix been present, it would have objected to LeCroy's use of this option. Using JTA2 to make a similar measurement, LeCroy's victory margin was a still convincing but less dramatic 6.5-to-1. Click here to find detailed results, including screen images as well as analysis and commentary by LeCroy's Marty Miller, PhD.
The learning curve
What might have happened if Tek, instead of LeCroy, had been the only manufacturer that participated? No one can be sure, but Tek probably would have done better not only than it did, but also than it would have if it had participated along with LeCroy. That the outcome should favor the sole participant has nothing to do with cheating; as far as we know, none took place. Moreover, we have no reason to suspect that Tek would have cheated had it been the sole participant.
The issue is the learning curve for using extraordinarily sophisticated instruments. Both companies have done a superb job of creating user interfaces that simplify invoking the scopes' enormous range of advanced measurement and analysis capabilities. Still, nothing can substitute for hands-on experience with the model of instrument you are using. Although the LeCroy representatives had some experience with the Tek scopes, they were more familiar with their own products. Tek representatives would have been more familiar with Tek's scopes. Almost certainly, any company's representatives would be more adept at getting the best results from their own products than from a competitor's.
Therefore, after Tek withdrew, we told the company several times that, if it has any problem with our methodologies or reported results, we invite it to send us material stating its position so that we can publish the material on or link to it from our Web site (Reference 2). Our offer stands.
A baffling difference
Tek might help to resolve a troubling issue that continues to baffle us. Although both the 8600A and the 7404 measured nearly identical total jitter on a 622-Mbps PRBS7 data stream (repeating pseudorandom binary sequence of length 27–1 bits), the two scopes' assessments of RJ and DJ (the random- and deterministic-jitter components) differed significantly (see sidebar "Jitter discrepancies: not explained"). We don't know which, if either, of the scopes made correct assessments. LeCroy has offered a plausible—albeit not necessarily correct—explanation of the 8600A's results (Reference 2). By press time, Tek had ignored several entreaties for comment on why company officials believe that the TDS 7404/JIT3 assessments are correct and that the 8600A/JTA2's are incorrect. Tek's silence is probably not a concession that its instrument produced incorrect results, and, should Tek offer a response, EDN will post it here, along with reader feedback.
Under our original plan, Tek was to bring a 6604 equipped with the JIT3 option. If EDN had been able to use such a scope for this test, the testers could have applied a 2.488-Gbps PRBS to both Tek and LeCroy scopes. However, the only Tek scope with the JIT3- analysis capabilities was the 7404. Its 4-GHz bandwidth was, at best, marginal for the 2.488-Gbps stream, so the testers used a 622-Mbps stream and for consistency also applied this signal to the 8600A. It seems improbable that, with a 622-Mbps stream, the difference between the 7404's 4-GHz bandwidth and the 8600A's 6 GHz could account for the divergent RJ and DJ values.
One thing is certain, though: Jitter measurement is an immature science; some say that it remains an art. Many brilliant people are working in the field. Some of them are developing standards for measuring jitter, and several types of jitter-measurement instruments and software packages exist. If you work in design and your goal is to characterize and minimize jitter rather than quantify parameters, such as bit-error rate, which are mainly consequences of jitter—a good wideband scope with advanced analysis capabilities is likely to be your most valuable measurement tool. Even so, you may also want to use several other types of instrumentation.
|MANUFACTURERS OF 6-GHz-BANDWIDTH, REAL-TIME-SAMPLING DSOs:|
|MANUFACTURERS OF TEST EQUIPMENT USED:|
|LA Techniques Ltd
|Picosecond Pulse Labs
|MANUFACTURERS OF JITTER-MEASUREMENT HARDWARE AND SOFTWARE:|
|Amherst Systems Associates Inc