• Hoang Le's avatar
    tipc: improve throughput between nodes in netns · f73b1281
    Hoang Le authored
    Currently, TIPC transports intra-node user data messages directly
    socket to socket, hence shortcutting all the lower layers of the
    communication stack. This gives TIPC very good intra node performance,
    both regarding throughput and latency.
    
    We now introduce a similar mechanism for TIPC data traffic across
    network namespaces located in the same kernel. On the send path, the
    call chain is as always accompanied by the sending node's network name
    space pointer. However, once we have reliably established that the
    receiving node is represented by a namespace on the same host, we just
    replace the namespace pointer with the receiving node/namespace's
    ditto, and follow the regular socket receive patch though the receiving
    node. This technique gives us a throughput similar to the node internal
    throughput, several times larger than if we let the traffic go though
    the full network stacks. As a comparison, max throughput for 64k
    messages is four times larger than TCP throughput for the same type of
    traffic.
    
    To meet any security concerns, the following should be noted.
    
    - All nodes joining a cluster are supposed to have been be certified
    and authenticated by mechanisms outside TIPC. This is no different for
    nodes/namespaces on the same host; they have to auto discover each
    other using the attached interfaces, and establish links which are
    supervised via the regular link monitoring mechanism. Hence, a kernel
    local node has no other way to join a cluster than any other node, and
    have to obey to policies set in the IP or device layers of the stack.
    
    - Only when a sender has established with 100% certainty that the peer
    node is located in a kernel local namespace does it choose to let user
    data messages, and only those, take the crossover path to the receiving
    node/namespace.
    
    - If the receiving node/namespace is removed, its namespace pointer
    is invalidated at all peer nodes, and their neighbor link monitoring
    will eventually note that this node is gone.
    
    - To ensure the "100% certainty" criteria, and prevent any possible
    spoofing, received discovery messages must contain a proof that the
    sender knows a common secret. We use the hash mix of the sending
    node/namespace for this purpose, since it can be accessed directly by
    all other namespaces in the kernel. Upon reception of a discovery
    message, the receiver checks this proof against all the local
    namespaces'hash_mix:es. If it finds a match, that, along with a
    matching node id and cluster id, this is deemed sufficient proof that
    the peer node in question is in a local namespace, and a wormhole can
    be opened.
    
    - We should also consider that TIPC is intended to be a cluster local
    IPC mechanism (just like e.g. UNIX sockets) rather than a network
    protocol, and hence we think it can justified to allow it to shortcut the
    lower protocol layers.
    
    Regarding traceability, we should notice that since commit 6c9081a3
    ("tipc: add loopback device tracking") it is possible to follow the node
    internal packet flow by just activating tcpdump on the loopback
    interface. This will be true even for this mechanism; by activating
    tcpdump on the involved nodes' loopback interfaces their inter-name
    space messaging can easily be tracked.
    
    v2:
    - update 'net' pointer when node left/rejoined
    v3:
    - grab read/write lock when using node ref obj
    v4:
    - clone traffics between netns to loopback
    Suggested-by: default avatarJon Maloy <jon.maloy@ericsson.com>
    Acked-by: default avatarJon Maloy <jon.maloy@ericsson.com>
    Signed-off-by: default avatarHoang Le <hoang.h.le@dektech.com.au>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    f73b1281
msg.h 26.6 KB