D.1.2.4 Cleaning Up Safely

Cleaning up SRCU safely can be a challenge, but fortunately many uses need not do so. For example, uses in operating-system kernels that are initialized at boot time need not be cleaned up. However, uses within loadable modules must clean up if the corresponding module is to be safely unloaded.

In some cases, such as the RCU torture module, only a small known set of threads are using the SRCU read-side primitives against a particular struct srcu_struct. In these cases, the module-exit code need only kill that set of threads, wait for them to exit, and then clean up.

In other cases, for example, for device drivers, any thread in the system might be using the SRCU read-side primitives. Although one could apply the method of the previous paragraph, this ends up being equivalent to a full reboot, which can be unattractive. Figure [*] shows one way that cleanup could be accomplished without a reboot.

Figure: SRCU Safe Cleanup
\begin{figure}{ \scriptsize
\begin{verbatim}1 int readside(void)
2 {
3 int ...
...ize_srcu(&ss);
22 cleanup_srcu_struct(&ss);
23 }\end{verbatim}
}\end{figure}

The readside() function overlaps an RCU and an SRCU read-side critical section, with the former running from lines 5-11 and the latter running from lines 10-13. The RCU read-side critical section uses Pure RCU [McK04] to guard the value of the nomoresrcu variable. If this variable is set, we are cleaning up, and therefore must not enter the SRCU read-side critical section, so we return -EINVAL instead. On the other hand, if we are not yet cleaning up, we proceed into the SRCU read-side critical section.

The cleanup() function first sets the nomoresrcu variable on line 19, but then must wait for all currently executing RCU read-side critical sections to complete via the synchronize_rcu() primitive on line 20. Once the cleanup() function reaches line 21, all calls to readside() that could possibly have seen nomorersrcu equal to zero must have already reached line 11, and therefore already must have entered their SRCU read-side critical section. All future calls to readside() will exit via line 8, and will thus refrain from entering the read-side critical section.

Therefore, once cleanup() completes its call to synchronize_srcu() on line 21, all SRCU read-side critical sections will have completed, and no new ones will be able to start. It is therefore safe on line 22 to call cleanup_srcu_struct() to clean up.

Paul E. McKenney 2011-12-16