Figure
shows the code for __rcu_offline_cpu() and its wrapper
function, rcu_offline_cpu().
The purpose of this wrapper function (shown in lines 43-47 of the figure)
is simply to invoke __rcu_offline_cpu() twice, once for ``rcu'' and
again for ``rcu_bh''.
The purpose of the __rcu_offline_cpu() function is to
prevent future grace periods from waiting on the CPU being offlined,
to note the extended quiescent state, and to find a new home for
any RCU callbacks in process on this CPU.
Turning to __rcu_offline_cpu(), shown on lines 1-41 of the figure, line 12 acquires the specified rcu_state structure's ->onofflock, excluding grace-period initialization for multi-rcu_node hierarchies.
Quick Quiz D.37:
But what if the rcu_node hierarchy has only a single
structure, as it would on a small system?
What prevents concurrent grace-period initialization in that
case, given the code in
Figure ?
End Quick Quiz
Line 13 picks up a pointer to the leaf rcu_node structure corresponding
to this CPU, using the ->mynode pointer in this CPU's rcu_data
structure
(see Figure ).
Line 14 picks up a mask with this CPU's bit set for use on
the leaf rcu_node structure's qsmask field.
The loop spanning lines 15-25 then clears this CPU's bits up the rcu_node hierarchy, starting with this CPU's leaf rcu_node structure. Line 16 acquires the current rcu_node structure's ->lock field, and line 17 clears the bit corresponding to this CPU (or group, higher up in the hierarchy) from the ->qsmaskinit field, so that future grace periods will not wait on quiescent states from this CPU. If the resulting ->qsmaskinit value is non-zero, as checked by line 18, then the current rcu_node structure has other online CPUs that it must track, so line 19 releases the current rcu_node structure's ->lock and line 20 exits the loop. Otherwise, we need to continue walking up the rcu_node hierarchy. In this case, line 22 picks up the mask to apply to the next level up, line 23 releases the current rcu_node structure's ->lock, and line 24 advances up to the next level of the hierarchy. Line 25 exits the loop should we exit out the top of the hierarchy.
Quick Quiz D.38:
But does line 25 of
Figure
ever really exit the loop?
Why or why not?
End Quick Quiz
Line 26 picks up the specified rcu_state structure's ->completed field into the local variable lastcomp, line 27 releases ->onofflock (but leaves irqs disabled), and line 28 invokes cpu_quiet() in order to note that the CPU being offlined is now in an extended quiescent state, passing in lastcomp to avoid reporting this quiescent state against a different grace period than it occurred in.
Quick Quiz D.39:
Suppose that line 26 got executed seriously out of order in
Figure ,
so that lastcomp is set to some prior grace period, but
so that the current grace period is still waiting on the
now-offline CPU?
In this case, won't the call to cpu_quiet() fail to
report the quiescent state, thus causing the grace period
to wait forever for this now-offline CPU?
End Quick Quiz
Quick Quiz D.40:
Given that an offline CPU is in an extended quiescent state,
why does line 28 of
Figure
need to care which grace period it is
dealing with?
End Quick Quiz
Lines 29-39 move any RCU callbacks from the CPU going offline to the currently running CPU. This operation must avoid reordering the callbacks being moved, as otherwise rcu_barrier() will not work correctly. Line 29 puts a pointer to the currently running CPU's rcu_data structure into local variable rdp_me. Line 30 then checks to see if the CPU going offline has any RCU callbacks. If so, lines 31-38 move them. Line 31 splices the list of callbacks onto the end of the running CPU's list. Lines 32-33 sets the running CPU's callback tail pointer to that of the CPU going offline, and then lines 34-36 initialize the going-offline CPU's list to be empty. Line 37 adds the length of the going-offline CPU's callback list to that of the currently running CPU, and, finally, line 38 zeroes the going-offline CPU's list length.
Quick Quiz D.41:
But this list movement in
Figure
makes all of the going-offline CPU's callbacks go through
another grace period, even if they were ready to invoke.
Isn't that inefficient?
Furthermore, couldn't an unfortunate pattern of CPUs going
offline then coming back online prevent a given callback from
ever being invoked?
End Quick Quiz
Finally, line 40 re-enables irqs.
Paul E. McKenney 2011-12-16