D.3.7.4 Checking for Dyntick-Idle Mode

The dyntick_save_progress_counter() and rcu_implicit_dynticks_qs() functions are used to check whether a CPU is in dynticks-idle mode. The dyntick_save_progress_counter() function is invoked first, and returns non-zero if the CPU is currently in dynticks-idle mode. If the CPU was not in dynticks-idle mode, for example, because it is currently handling an interrupt or NMI, then the rcu_implicit_dynticks_qs() function is called some jiffies later. This function looks at the current state in conjunction with state stored away by the earlier call to dyntick_save_progress_counter(), again returning non-zero if the CPU either is in dynticks-idle mode or was in dynticks-idle mode during the intervening time. The rcu_implicit_dynticks_qs() function may be invoked repeatedly, if need be, until is returns true.

Figure: Code for dyntick_save_progress_counter()
\begin{figure}{ \scriptsize
\begin{verbatim}1 static int
2 dyntick_save_prog...
...et)
15 rdp->dynticks_fqs++;
16 return ret;
17 }\end{verbatim}
}\end{figure}

Figure [*] shows the code for dyntick_save_progress_counter(), which is passed a given CPU-rcu_state pair's rcu_data structure. Lines 8 and 9 take snapshots of the CPU's rcu_dynticks structure's ->dynticks and ->dynticks_nmi fields, and then line 10 executes a memory barrier to ensure that the snapshot is seen by all CPUs to have happened before any later processing depending on these values. This memory barrier pairs up with those in rcu_enter_nohz(), rcu_exit_nohz(), rcu_nmi_enter(), rcu_nmi_exit(), rcu_irq_enter(), and rcu_irq_exit(). Lines 11 and 12 store these two snapshots away so that they can be accessed by a later call to rcu_implicit_dynticks_qs(). Line 13 checks to see if both snapshots have even-numbered values, indicating that the CPU in question was in neither non-idle process state, an interrupt handler, nor an NMI handler. If so, lines 14 and 15 increment the statistical counter ->dynticks_fqs, which is used only for tracing. Either way, line 16 returns the indication of whether the CPU was in dynticks-idle mode.

Quick Quiz D.52: Why isn't there a memory barrier between lines 8 and 9 of Figure [*]? Couldn't this cause the code to fetch even-numbered values from both the ->dynticks and ->dynticks_nmi fields, even though these two fields never were zero at the same time? End Quick Quiz

Figure: Code for rcu_implicit_dynticks_qs()
\begin{figure}{ \scriptsize
\begin{verbatim}1 static int
2 rcu_implicit_dynt...
...8 }
19 return rcu_implicit_offline_qs(rdp);
20 }\end{verbatim}
}\end{figure}

Figure [*] shows the code for rcu_implicit_dynticks_qs(). Lines 9-12 pick up both new values for the CPU's rcu_dynticks structure's ->dynticks and ->dynticks_nmi fields, as well as the snapshots taken by the last call to dyntick_save_progress_counter(). Line 13 then executes a memory barrier to ensure that the values are seen by other CPUs to be gathered prior to subsequent RCU processing. As with dyntick_save_progress_counter(), this memory barrier pairs with those in rcu_enter_nohz(), rcu_exit_nohz(), rcu_nmi_enter(), rcu_nmi_exit(), rcu_irq_enter(), and rcu_irq_exit(). Lines 14-15 then check to make sure that this CPU is either currently in dynticks-idle mode ((curr & 0x1) == 0 and (curr_nmi & 0x1) == 0) or has passed through dynticks-idle mode since the last call to dyntick_save_progress_counter() (curr != snap and curr_nmi != snap_nmi). If so, line 16 increments the ->dynticks_fqs statistical counter (again, used only for tracing) and line 17 returns non-zero to indicate that the specified CPU has passed through a quiescent state. Otherwise, line 19 invokes rcu_implicit_offline_qs() (described in Section [*]) to check whether the specified CPU is currently offline.

Paul E. McKenney 2011-12-16