How We See Embedded Processing

Let me make it clear right away that I don’t want to start another debate about the merits of various benchmarks (or deal with the infinite thread of flaming comments that follows). Fact is, we often need a metric; a way to measure and compare the performance of the “apples” and “oranges” available to us. My focus is always on microcontrollers in embedded-control applications, so it happens that the apples and oranges I usually end up talking about are microcontrollers of different architectures, which are implemented in a seeming infinite variety of CMOS processes.

We all have the right to our opinions on the relative value of counting the cycles, MIPS or DMIPS of choice. I just wanted to bring to your attention the fact there is a new kid on the block. It is called CoreMark, and it’s a new offering from the guys at EEMBC that promises to improve on the…well, the “apples and oranges” thing! It is, in fact, meant to replace the old and dreaded Dhrystone benchmark, with a particular eye to embedded-control applications. (Have you ever tried to use a VAX machine in an embedded-control application?)   

I have to admit, I discovered this new benchmark quite recently, and I was understandably pleased that the Microchip PIC32 family had been chosen among the early “targets.” If you visit the CoreMark.org Web site, you will find a link to the Scores database, which is a freely accessible and easy to navigate source for all sorts of side-by-side comparisons and reports. Searching for your architecture/product of choice, you will actually obtain two kinds of benchmarks: an absolute maximum performance index, expressed in CoreMarks units (measured at the maximum operating frequency of the device), and an efficiency index, expressed in CoreMarks/MHz, which is typically computed at some optimal point of operation that is obviously device dependent. 

The PIC32, for example, operating at 80 MHz delivered a respectable 185CoreMarks and below 30 MHz would show a 2.6CoreMarks/MHz ratio. We should recognize that the performance of a core in this kind of benchmark is also affected by the efficiency of the compiler used, and the new Code Sorcery G++ compiler (based on the latest GNU GCC compiler release 4.3.3) seems to do a great job with the MIPS M4K core inside the PIC32.

But what do these new CoreMark indices measure? The documentation is quite clear; they are based on the combination of three simple algorithms that are meant to represent common tasks in embedded-control applications. They are: 1- List manipulation, 2- Matrix manipulation and 3- A simple state machine. Well chosen, indeed. I would agree most embedded-control programs contain a mix of these three tasks. But, that’s somewhat missing the big picture! In an embedded-control application, there is so much more. There are peripherals that require our attention. There are external events that we must respond to in a timely and deterministic way. And, perhaps more importantly, when we are making our microcontroller choice, we have to consider whether tools, software, and solutions are readily available to make the application development not only possible, but fast, efficient and economic.

Now, I am not saying the CoreMark didn’t do its job. On the contrary, it does very well what it claims to do (it’s right there in the name)—it provides us with a tool to perform a basic check: Is the performance of this microcontroller core sufficient for my applications? The question it cannot help us with is whether the rest of the system (hardware and software) around the core is up to the task!

Recently, I have been reading and hearing a lot of discussions about the virtues of this vs. that “core,” and it seems to me that most of these arguments are well… irrelevant!

Also I am sort of feeling a bit of a déjà vu. Fifteen years ago, I was involved in similar discussions when Microchip was an entry player in the 8-bit market with the PIC16 RISC architecture and its OTP technology. Back then, it seemed pretty much everybody else was “converging” on the INTEL8051-like cores (with some variations), and I was often asked how we would be able to compete with an “incompatible” core. My answer was short: the core must give you the performance necessary, but you will also need a low-cost set of tools, a choice of compatible peripherals and packages to choose from and software solutions (tons of application notes) to get your project off the ground and make it successful.

Fast forward to our times, and you see that Microchip has become very successful in the 8-bit market, all while adding as many as five different—yet compatible—8-bit cores (none of which is anything like an 8051). What we did not change—on the contrary, what we obsessed about—was maintaining compatibility on peripherals, software, packages and tools. Today, this obsession has been carried on to include all 16-bit and 32-bit products, creating a single PIC MCU platform that supports a large set of compatible solutions.

This is even more impressive when you consider that, in the old times, we used to program mostly in assembly; so, each programmer was directly exposed to the innermost, intimate details of the core—the machine instruction set. Nowadays, and especially in 16- and 32-bit applications, we program almost exclusively in C. Compilers take care of the details of the machine instruction set, which makes the whole discussion about the value of the core “standardization” ... moot.

All clues are pointing in one clear direction. Once it’s established that a core is giving us sufficient performance for the application, all of our attention should then be focused on the compatibility of the “stuff” around it! Are the peripherals compatible? Is the package (pin-out) compatible? Are the debugging tools compatible? Are there common software solutions available? Is the software tool chain, compatible with the Open Source community standards (GNU)?

When someone tries to make the case that core compatibility is the key issue, working engineers know better. When porting an embedded application (upgrading/updating/downsizing/rightsizing it), it’s the “stuff” around the core that dictates how long it will truly take and how much of the old hardware you will be able to keep (if any); and it’s what will eventually decide the project’s success!

-Lucio Di Jasio, Microchip Technology