The scheduler changes had another thing missing: the appreciation of
sync wakeups.  (I had this in one of the earlier sched-domains cleanup
patches before but it got lost in the shuffle.)

When a sync waker is waking, we should subtract its load from the
current load - it will schedule away for sure in the near future. 
That's what the "sync" bit means.

This change is necessary because with the sched-domains balancer we have
a much more sensitive cpu-load estimator, and in this particular context
of try_to_wake_up() the sync waker's effect will always be part of the
load.  Patch against your patch attached. 

In my testing there's an additional increase in bw_pipe numbers on a
dual P2 box, it went from 110-120 MB/sec to 120-130 MB/sec. 
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>