Commit e03c05ac authored by Andrii Nakryiko's avatar Andrii Nakryiko Committed by Masami Hiramatsu (Google)

rethook: honor CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING in rethook_try_get()

Take into account CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING when validating
that RCU is watching when trying to setup rethooko on a function entry.

One notable exception when we force rcu_is_watching() check is
CONFIG_KPROBE_EVENTS_ON_NOTRACE=y case, in which case kretprobes will use
old-style int3-based workflow instead of relying on ftrace, making RCU
watching check important to validate.

This further (in addition to improvements in the previous patch)
improves BPF multi-kretprobe (which rely on rethook) runtime throughput
by 2.3%, according to BPF benchmarks ([0]).

  [0] https://lore.kernel.org/bpf/CAEf4BzauQ2WKMjZdc9s0rBWa01BYbgwHN6aNDXQSHYia47pQ-w@mail.gmail.com/

Link: https://lore.kernel.org/all/20240418190909.704286-2-andrii@kernel.org/Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
Acked-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Acked-by: default avatarMasami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: default avatarMasami Hiramatsu (Google) <mhiramat@kernel.org>
parent b0e28a4b
...@@ -166,6 +166,7 @@ struct rethook_node *rethook_try_get(struct rethook *rh) ...@@ -166,6 +166,7 @@ struct rethook_node *rethook_try_get(struct rethook *rh)
if (unlikely(!handler)) if (unlikely(!handler))
return NULL; return NULL;
#if defined(CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING) || defined(CONFIG_KPROBE_EVENTS_ON_NOTRACE)
/* /*
* This expects the caller will set up a rethook on a function entry. * This expects the caller will set up a rethook on a function entry.
* When the function returns, the rethook will eventually be reclaimed * When the function returns, the rethook will eventually be reclaimed
...@@ -174,6 +175,7 @@ struct rethook_node *rethook_try_get(struct rethook *rh) ...@@ -174,6 +175,7 @@ struct rethook_node *rethook_try_get(struct rethook *rh)
*/ */
if (unlikely(!rcu_is_watching())) if (unlikely(!rcu_is_watching()))
return NULL; return NULL;
#endif
return (struct rethook_node *)objpool_pop(&rh->pool); return (struct rethook_node *)objpool_pop(&rh->pool);
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment