EEMBC's benchmarks help analyze processor performance

-- EDN, 4/13/2000

On April 11, the EDN Embedded Microprocessor Benchmark Consortium (EEMBC) celebrated the first release of embedded-processor-benchmark scores, which are free on the EEMBC Web site, www.eembc.org. EEMBC, an industry organization comprising more than 40 processor, RTOS, and compiler vendors, has five embedded benchmark suites with more than 35 benchmark tests that target the automotive/industrial, consumer, networking, office-automation, and telecommunications markets. This event was the pinnacle of almost three years of exhaustive benchmark development and establishment of the rigorous process for providing officially certified benchmark scores. It also represents the pioneering efforts of processor vendors Integrated Device Technology (IDT), Infineon Technologies, NEC, National Semiconductor, STMicroelectronics, and Toshiba, which make possible this first release of benchmark scores. And, although this write-up includes only benchmark scores and analysis for the EEMBC Networking benchmarks and a few processors, benchmark scores from the other application areas are available on the EEMBC Web site.

For this release of benchmarks, the EEMBC Certification Lab (www.embedded-benchmarks.com) certified results for more than 12 processors and compilers. Be careful when comparing the results from processors. For example, it's no surprise that a 400-MHz processor is faster than a 20-MHz processor, but what if you're comparing a 167-MHz processor with a 143-MHz processor? If you compare frequency alone, you'll find little difference. But you should weigh many other factors, including native data size, bus width and speed, cache size, single versus dual pipelines, and number of execution units. In addition, all scores in this release are out-of-the-box, or "unoptimized," scores, meaning that anyone running the benchmarks must use the unmodified source code from EEMBC. Although this code ensures portability across all processors and their corresponding compilers, strict ANSI C code doesn't always yield the highest performance code, even with all compiler optimizations active. Therefore, a mediocre benchmark score may indicate a mediocre compiler and may not reflect a processor's full capability. (In EEMBC's next release of benchmark scores on June 13 at the Embedded Processor Froum, vendors will probably also display optimized scores alongside the out-of-the-box scores.)

The networking benchmarks include the Open Shortest Path First (OSPF/Dijkstra) algorithm, the packet-flow routing benchmark, and the route-look-up algorithm. These benchmarks simulate some of the functions that a processor would perform in a networking application. Table 1 includes benchmark scores from several of the processors involved in this release. Although the score data requires more careful analysis to understand the pros and cons of a processor and the compiler, a cursory analysis points out the obvious benefits of higher operating frequency.

If you dig down to the next level of detail, you'll see that the IDT79RC32365 is a 32-bit RISC architecture with a scalar, five-stage pipeline (based on IDT's proprietary RISCore32300 core) and a 32-bit external bus and that the IDT79RC64575 and NEC VR5432 are 64-bit, superscalar devices with 64-bit external buses. Furthermore, both the IDT79RC64575 and the NEC VR5432 have 32-kbyte instruction and data caches, whereas the IDT79RC32365 has an 8-kbyte instruction cache and a 2-kbyte data cache. At the system level, both the IDT79RC32365 and the IDT79RC64575 have a 50-MHz bus, but the NEC VR5432's system uses an 83.5-MHz memory bus. You can find all other implementation details on the EEMBC Web site.

Integrated Device Technology, www.idt.com.

Infineon Technologies, www.infineon.com.

NEC, www.necel.com.

National Semiconductor, www.national.com.

STMicroelectronics, www.st.com.

Toshiba, www.toshiba.com/taec.

-Markus Levy