Commit 52a85468 authored by Jakub Kicinski's avatar Jakub Kicinski

Merge branch 'optimise-local-cpu-skb_attempt_defer_free'

Pavel Begunkov says:

====================
optimise local CPU skb_attempt_defer_free

Optimise the case when an skb comes to skb_attempt_defer_free()
on the same CPU it was allocated on. The patch 1 enables skb caches
and gives frags a chance to hit the page pool's fast path.
CPU bound benchmarking with perfect skb_attempt_defer_free()
gives around 1% of extra throughput.
====================

Link: https://lore.kernel.org/r/cover.1712711977.git.asml.silence@gmail.comSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
parents 9b9fd458 d8415a16
...@@ -6974,6 +6974,19 @@ void __skb_ext_put(struct skb_ext *ext) ...@@ -6974,6 +6974,19 @@ void __skb_ext_put(struct skb_ext *ext)
EXPORT_SYMBOL(__skb_ext_put); EXPORT_SYMBOL(__skb_ext_put);
#endif /* CONFIG_SKB_EXTENSIONS */ #endif /* CONFIG_SKB_EXTENSIONS */
static void kfree_skb_napi_cache(struct sk_buff *skb)
{
/* if SKB is a clone, don't handle this case */
if (skb->fclone != SKB_FCLONE_UNAVAILABLE) {
__kfree_skb(skb);
return;
}
local_bh_disable();
__napi_kfree_skb(skb, SKB_CONSUMED);
local_bh_enable();
}
/** /**
* skb_attempt_defer_free - queue skb for remote freeing * skb_attempt_defer_free - queue skb for remote freeing
* @skb: buffer * @skb: buffer
...@@ -6992,7 +7005,7 @@ void skb_attempt_defer_free(struct sk_buff *skb) ...@@ -6992,7 +7005,7 @@ void skb_attempt_defer_free(struct sk_buff *skb)
if (WARN_ON_ONCE(cpu >= nr_cpu_ids) || if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
!cpu_online(cpu) || !cpu_online(cpu) ||
cpu == raw_smp_processor_id()) { cpu == raw_smp_processor_id()) {
nodefer: __kfree_skb(skb); nodefer: kfree_skb_napi_cache(skb);
return; return;
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment