3.4 What Makes Parallel Programming Hard?

It is important to note that the difficulty of parallel programming is as much a human-factors issue as it is a set of technical properties of the parallel programming problem. This is the case because we need human beings to be able to tell parallel systems what to do, and this two-way communication between human and computer is as much a function of the human as it is of the computer. Therefore, appeals to abstractions or to mathematical analyses will necessarily be of severely limited utility.

In the Industrial Revolution, the interface between human and machine was evaluated by human-factor studies, then called time-and-motion studies. Although there have been a few human-factor studies examining parallel programming [ENS05,ES05,HCS+05,SS94], these studies have been extremely narrowly focused, and hence unable to demonstrate any general results. Furthermore, given that the normal range of programmer productivity spans more than an order of magnitude, it is unrealistic to expect an affordable study to be capable of detecting (say) a 10% difference in productivity. Although the multiple-order-of-magnitude differences that such studies can reliably detect are extremely valuable, the most impressive improvements tend to be based on a long series of 10% improvements.

We must therefore take a different approach.

Figure: Categories of Tasks Required of Parallel Programmers
\resizebox{3in}{!}{\includegraphics{intro/FourTaskCategories}}

One such approach is to carefully consider the tasks that parallel programmers must undertake that are not required of sequential programmers. We can then evaluate how well a given programming language or environment assists the developer with these tasks. These tasks fall into the four categories shown in Figure [*], each of which is covered in the following sections.



Subsections
Paul E. McKenney 2011-12-16