17.1.12 The exec() System Call

One can execute an exec() system call while holding a lock, and also from within an RCU read-side critical section. The exact semantics depends on the type of primitive.

In the case of non-persistent primitives (including pthread_mutex_lock(), pthread_rwlock_rdlock(), and RCU), if the exec() succeeds, the whole address space vanishes, along with any locks being held. Of course, if the exec() fails, the address space still lives, so any associated locks would also still live. A bit strange perhaps, but reasonably well defined.

On the other hand, persistent primitives (including the flock family, lockf(), System V semaphores, and the O_CREAT flag to open()) would survive regardless of whether the exec() succeeded or failed, so that the exec()ed program might well release them.

Quick Quiz 17.1: What about non-persistent primitives represented by data structures in mmap() regions of memory? What happens when their is an exec() within a critical section of such a primitive? End Quick Quiz

What happens when you attempt to execute an exec() system call from within a transaction?

  1. Disallow exec() within transactions, so that the enclosing transactions abort upon encountering the exec(). This is well defined, but clearly requires non-TM synchronization primitives for use in conjunction with exec().
  2. Disallow exec() within transactions, with the compiler enforcing this prohibition. There is a draft specification for TM in C++ that takes this approach, allowing functions to be decorated with the transaction_safe and transaction_unsafe attributes.17.3 This approach has some advantages over aborting the transaction at runtime, but again requires non-TM synchronization primitives for use in conjunction with exec().
  3. Treat the transaction in a manner similar to non-persistent Locking primitives, so that the transaction survives if exec() fails, and silently commits if the exec() succeeds. The case were some of the variables affected by the transaction reside in mmap()ed memory (and thus could survive a successful exec() system call) is left as an exercise for the reader.
  4. Abort the transaction (and the exec() system call) if the exec() system call would have succeeded, but allow the transaction to continue if the exec() system call would fail. This is in some sense the ``correct'' approach, but it would require considerable work for a rather unsatisfying result.

The exec() system call is perhaps the strangest example of an obstacle to universal TM applicability, as it is not completely clear what approach makes sense, and some might argue that this is merely a reflection of the perils of interacting with execs in real life. That said, the two options prohibiting exec() within transactions are perhaps the most logical of the group.

Paul E. McKenney 2011-12-16