Performance is the primary goal behind most parallel-programming effort. After all, if performance is not a concern, why not do yourself a favor, just write sequential code, and be happy? It will very likely be easier, and you will probably get done much more quickly.
Quick Quiz 3.7: Are there no cases where parallel programming is about something other than performance? End Quick Quiz
Note that ``performance'' is interpreted quite broadly here, including scalability (performance per CPU) and efficiency (for example, performance per watt).
That said, the focus of performance has shifted from hardware to
parallel software.
This change in focus is due to the fact that although Moore's Law
continues to deliver increases in transistor density, it has ceased to
provide the traditional single-threaded performance
increases, as can be seen in
Figure .3.1This means that writing single-threaded code and simply waiting
a year or two for the CPUs to catch up may no longer be an option.
Given the recent trends on the part of all major manufacturers towards
multicore/multithreaded systems, parallelism is the way to go for
those wanting the avail themselves of the full performance of their
systems.
Even so, the first goal is performance rather than scalability, especially given that the easiest way to attain linear scalability is to reduce the performance of each CPU [Tor01]. Given a four-CPU system, which would you prefer? A program that provides 100 transactions per second on a single CPU, but does not scale at all? Or a program that provides 10 transactions per second on a single CPU, but scales perfectly? The first program seems like a better bet, though the answer might change if you happened to be one of the lucky few with access to a 32-CPU system.
That said, just because you have multiple CPUs is not necessarily in and of itself a reason to use them all, especially given the recent decreases in price of multi-CPU systems. The key point to understand is that parallel programming is primarily a performance optimization, and, as such, it is one potential optimization of many. If your program is fast enough as currently written, there is no reason to optimize, either by parallelizing it or by applying any of a number of potential sequential optimizations.3.2By the same token, if you are looking to apply parallelism as an optimization to a sequential program, then you will need to compare parallel algorithms to the best sequential algorithms. This may require some care, as far too many publications ignore the sequential case when analyzing the performance of parallel algorithms.
Paul E. McKenney 2011-12-16