D.3.2.3 rcu_check_callbacks()

Figure: rcu_check_callbacks() Code
\begin{figure}{ \scriptsize
\begin{verbatim}1 static int __rcu_pending(struct...
...(cpu);
44 }
45 raise_softirq(RCU_SOFTIRQ);
46 }\end{verbatim}
}\end{figure}

Figure [*] shows the code that is called from the scheduling-clock interrupt handler once per jiffy from each CPU. The rcu_pending() function (which is a wrapper for __rcu_pending()) is invoked, and if it returns non-zero, then rcu_check_callbacks() is invoked. (Note that there is some thought being given to merging rcu_pending() into rcu_check_callbacks().)

Starting with __rcu_pending(), line 4 counts this call to rcu_pending() for use in deciding when to force quiescent states. Line 6 invokes check_cpu_stall() in order to report on CPUs that are spinning in the kernel, or perhaps that have hardware problems, if CONFIG_RCU_CPU_STALL_DETECTOR is selected. Lines 7-23 perform a series of checks, returning non-zero if RCU needs the current CPU to do something. Line 7 checks to see if the current CPU owes RCU a quiescent state for the current grace period, line 9 invokes cpu_has_callbacks_ready_to_invoke() to see if the current CPU has callbacks whose grace period has ended, thus being ready to invoke, line 11 invokes cpu_needs_another_gp() to see if the current CPU has callbacks that need another RCU grace period to elapse, line 13 checks to see if the current grace period has ended, line 15 checks to see if a new grace period has started, and, finally, lines 17-22 check to see if it is time to attempt to force holdout CPUs to pass through a quiescent state. This latter check breaks down as follows: (1) lines 17-18 check to see if there is a grace period in progress, and, if so, lines 19-22 check to see if sufficient jiffies (lines 19-20) or calls to rcu_pending() (lines 21-22) have elapsed that force_quiescent_state() should be invoked. If none of the checks in the series triggers, then line 24 returns zero, indicating that rcu_check_callbacks() need not be invoked.

Lines 27-33 show rcu_pending(), which simply invokes __rcu_pending() twice, once for ``rcu'' and again for ``rcu_bh''.

Quick Quiz D.29: Given that rcu_pending() is always called twice on lines 29-32 of Figure [*], shouldn't there be some way to combine the checks of the two structures? End Quick Quiz

Lines 35-48 show rcu_check_callbacks(), which checks to see if the scheduling-clock interrupt interrupted an extended quiescent state, and then initiates RCU's softirq processing (rcu_process_callbacks()). Lines 37-41 perform this check for ``rcu'', while lines 42-43 perform the check for ``rcu_bh''.

Lines 37-39 check to see if the scheduling clock interrupt came from user-mode execution (line 37) or directly from the idle loop (line 38's idle_cpu() invocation) with no intervening levels of interrupt (the remainder of line 38 and all of line 39). If this check succeeds, so that the scheduling clock interrupt did come from an extended quiescent state, then because any quiescent state for ``rcu'' is also a quiescent state for ``rcu_bh'', lines 40 and 41 report the quiescent state for both flavors of RCU.

Similarly for ``rcu_bh'', line 42 checks to see if the scheduling-clock interrupt came from a region of code with softirqs enabled, and, if so line 43 reports the quiescent state for ``rcu_bh'' only.

Quick Quiz D.30: Shouldn't line 42 of Figure [*] also check for in_hardirq()? End Quick Quiz

In either case, line 45 invokes an RCU softirq, which will result in rcu_process_callbacks() being called on this CPU at some future time (like when interrupts are re-enabled after exiting the scheduler-clock interrupt).

Paul E. McKenney 2011-12-16