• Gilad Naaman's avatar
    net-next: When a bond have a massive amount of VLANs with IPv6 addresses,... · 406f42fa
    Gilad Naaman authored
    net-next: When a bond have a massive amount of VLANs with IPv6 addresses, performance of changing link state, attaching a VRF, changing an IPv6 address, etc. go down dramtically.
    
    The source of most of the slow down is the `dev_addr_lists.c` module,
    which mainatins a linked list of HW addresses.
    When using IPv6, this list grows for each IPv6 address added on a
    VLAN, since each IPv6 address has a multicast HW address associated with
    it.
    
    When performing any modification to the involved links, this list is
    traversed many times, often for nothing, all while holding the RTNL
    lock.
    
    Instead, this patch adds an auxilliary rbtree which cuts down
    traversal time significantly.
    
    Performance can be seen with the following script:
    
    	#!/bin/bash
    	ip netns del test || true 2>/dev/null
    	ip netns add test
    
    	echo 1 | ip netns exec test tee /proc/sys/net/ipv6/conf/all/keep_addr_on_down > /dev/null
    
    	set -e
    
    	ip -n test link add foo type veth peer name bar
    	ip -n test link add b1 type bond
    	ip -n test link add florp type vrf table 10
    
    	ip -n test link set bar master b1
    	ip -n test link set foo up
    	ip -n test link set bar up
    	ip -n test link set b1 up
    	ip -n test link set florp up
    
    	VLAN_COUNT=1500
    	BASE_DEV=b1
    
    	echo Creating vlans
    	ip netns exec test time -p bash -c "for i in \$(seq 1 $VLAN_COUNT);
    	do ip -n test link add link $BASE_DEV name foo.\$i type vlan id \$i; done"
    
    	echo Bringing them up
    	ip netns exec test time -p bash -c "for i in \$(seq 1 $VLAN_COUNT);
    	do ip -n test link set foo.\$i up; done"
    
    	echo Assiging IPv6 Addresses
    	ip netns exec test time -p bash -c "for i in \$(seq 1 $VLAN_COUNT);
    	do ip -n test address add dev foo.\$i 2000::\$i/64; done"
    
    	echo Attaching to VRF
    	ip netns exec test time -p bash -c "for i in \$(seq 1 $VLAN_COUNT);
    	do ip -n test link set foo.\$i master florp; done"
    
    On an Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz machine, the performance
    before the patch is (truncated):
    
    	Creating vlans
    	real 108.35
    	Bringing them up
    	real 4.96
    	Assiging IPv6 Addresses
    	real 19.22
    	Attaching to VRF
    	real 458.84
    
    After the patch:
    
    	Creating vlans
    	real 5.59
    	Bringing them up
    	real 5.07
    	Assiging IPv6 Addresses
    	real 5.64
    	Attaching to VRF
    	real 25.37
    
    Cc: David S. Miller <davem@davemloft.net>
    Cc: Jakub Kicinski <kuba@kernel.org>
    Cc: Lu Wei <luwei32@huawei.com>
    Cc: Xiongfeng Wang <wangxiongfeng2@huawei.com>
    Cc: Taehee Yoo <ap420073@gmail.com>
    Signed-off-by: default avatarGilad Naaman <gnaaman@drivenets.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    406f42fa
dev_addr_lists.c 25.6 KB