• Steven Rostedt's avatar
    ftrace: Optimize the function tracer list loop · 0a016409
    Steven Rostedt authored
    There is lots of places that perform:
    
           op = rcu_dereference_raw(ftrace_control_list);
           while (op != &ftrace_list_end) {
    
    Add a helper macro to do this, and also optimize for a single
    entity. That is, gcc will optimize a loop for either no iterations
    or more than one iteration. But usually only a single callback
    is registered to the function tracer, thus the optimized case
    should be a single pass. to do this we now do:
    
    	op = rcu_dereference_raw(list);
    	do {
    		[...]
    	} while (likely(op = rcu_dereference_raw((op)->next)) &&
    	       unlikely((op) != &ftrace_list_end));
    
    An op is always registered (ftrace_list_end when no callbacks is
    registered), thus when a single callback is registered, the link
    list looks like:
    
     top => callback => ftrace_list_end => NULL.
    
    The likely(op = op->next) still must be performed due to the race
    of removing the callback, where the first op assignment could
    equal ftrace_list_end. In that case, the op->next would be NULL.
    But this is unlikely (only happens in a race condition when
    removing the callback).
    
    But it is very likely that the next op would be ftrace_list_end,
    unless more than one callback has been registered. This tells
    gcc what the most common case is and makes the fast path with
    the least amount of branches.
    Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
    0a016409
ftrace.c 106 KB