• Tariq Toukan's avatar
    net/mlx5e: Remove page_ref bulking in Striding RQ · 9f9e9cd5
    Tariq Toukan authored
    When many packets reside on the same page, the bulking of
    page_ref modifications reduces the total number of atomic
    operations executed.
    
    Besides the necessary 2 operations on page alloc/free, we
    have the following extra ops per page:
    - one on WQE allocation (bump refcnt to maximum possible),
    - zero ops for SKBs,
    - one on WQE free,
    a constant of two operations in total, no matter how many
    packets/SKBs actually populate the page.
    
    Without this bulking, we have:
    - no ops on WQE allocation or free,
    - one op per SKB,
    
    Comparing the two methods when PAGE_SIZE is 4K:
    - As mentioned above, bulking method always executes 2 operations,
      not more, but not less.
    - In the default MTU configuration (1500, stride size is 2K),
      the non-bulking method execute 2 ops as well.
    - For larger MTUs with stride size of 4K, non-bulking method
      executes only a single op.
    - For XDP (stride size of 4K, no SKBs), non-bulking method
      executes no ops at all!
    
    Hence, to optimize the flows with linear SKB and XDP over Striding RQ,
    we here remove the page_ref bulking method.
    
    Performance testing:
    ConnectX-5, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz.
    
    Single core packet rate (64 bytes).
    
    Early drop in TC: no degradation.
    
    XDP_DROP:
    before: 14,270,188 pps
    after:  20,503,603 pps, 43% improvement.
    Signed-off-by: default avatarTariq Toukan <tariqt@mellanox.com>
    Signed-off-by: default avatarSaeed Mahameed <saeedm@mellanox.com>
    9f9e9cd5
en.h 33.6 KB