Most people have an intuitive understanding that passing messages between systems is considerably more expensive than performing simple calculations within the confines of a single system. However, it is not always so clear that communicating among threads within the confines of a single shared-memory system can also be quite expensive. This chapter therefore looks the cost of synchronization and communication within a shared-memory system. This chapter merely scratches the surface of shared-memory parallel hardware design; readers desiring more detail would do well to start with a recent edition of Hennessy and Patterson's classic text [HP95].
Quick Quiz 4.1: Why should parallel programmers bother learning low-level properties of the hardware? Wouldn't it be easier, better, and more general to remain at a higher level of abstraction? End Quick Quiz