SANTA CLARA, Calif. — Benchmarking multicore CPUs is a challenge, but there are some emerging approaches that show promise, according to Markus Levy, president of the Embedded Microprocessor Benchmark Consortium (EEMBC) and the Multicore Association. Levy spoke about the challenges of multicore design and benchmarking at the Synopsys Interoperability Developer's Forum here Thursday (April 26).
Multicore technology is "inevitable," Levy said, for several reasons. While single-core CPUs are providing "diminishing returns," he said, multicore CPUs open the door to increased compute density, functional partitioning, asynchronous multiprocessing, and parallel pipelining. But "nothing comes for free," Levy said, and multicore CPUs bring with them a host of design challenges.
Among these challenges are finding the right hardware and software interconnect, whether it be through shared memory or a network-on-chip approach. Inter-core resource management, distributed power management, load balancing, and algorithm partitioning all require a great deal of care. And providing a single view for debugging is very difficult with multiple, heterogeneous cores, Levy said.
In response to the debugging challenge, the Multicore Association is developing a debug API. Levy also discussed that organization's work in developing a communications API that targets "closely distributed" homogenous or heterogeneous multicore systems. This messaging API, he said, offers a small footprint, low latency, and high efficiency.
Meanwhile, EEMBC is tackling the challenge of benchmarking multicore devices. An important part of that effort, said Levy, is to allow a comparative evaluation of single-core and multicore implementations, so designers can see how much performance 2, 4, or 16 or more cores might provide.
Levy said that multicore benchmarking must also be scalable to hundreds of cores. It must measure memory bandwidth, he said, because that becomes a major issue as the number of cores grows. And it must also consider operating system support for scheduling, because different operating systems differ tremendously in how fast they dispatch and manage threads.
The easiest type of multicore system to benchmark is one that uses symmetric multiprocessing (SMP), and that's where EEMBC is starting, Levy said. The benchmark thus assumes that it's dispatching threads to identical processor cores.
"The easiest way is to take existing benchmarks and make them threadable," Levy said. "But to run a benchmark, you need some way of monitoring the system, which is a whole new layer of complexity. You're not only running an application, you're running an application, an operating system, and a non-intrusive monitor."
To accomplish this task, Levy said, EEMBC has developed a patent-pending "test harness" that provides an abstraction layer that watches and monitors what's going on during the benchmark. It provides a way to launch multiple benchmark configurations and "workloads" comprised of applications or threads.
EEMBC has found that performance does not necessarily scale linearly with the number of processors. "As you throw more threads, it's okay to keep it up until a certain point, but then you hit the wall when the processor gets saturated," Levy said.
Levy showed several benchmark case examples. One involved a device processing multiple channels of voice over IP data. The test here was to increase the number of channels, and see where performance starts to degrade. In an MPEG benchmark example, a single task was decomposed into multiple subtasks, and in a video example, different algorithms — such as video input and video decoding — were run on different cores.
"We want to come up with a method that the majority of the industry agrees is running on an equal basis," Levy said.
The Synopsys EDA Interoperability Developer's Conference also addressed low-power standards, EDA software piracy protection, and the need for analog constraints and interoperable parameterized cells (p-cells). Proceedings are available on line.
