• Al Viro's avatar
    epoll: EPOLL_CTL_ADD: close the race in decision to take fast path · fe0a916c
    Al Viro authored
    Checking for the lack of epitems refering to the epoll we want to insert into
    is not enough; we might have an insertion of that epoll into another one that
    has already collected the set of files to recheck for excessive reverse paths,
    but hasn't gotten to creating/inserting the epitem for it.
    
    However, any such insertion in progress can be detected - it will update the
    generation count in our epoll when it's done looking through it for files
    to check.  That gets done under ->mtx of our epoll and that allows us to
    detect that safely.
    
    We are *not* holding epmutex here, so the generation count is not stable.
    However, since both the update of ep->gen by loop check and (later)
    insertion into ->f_ep_link are done with ep->mtx held, we are fine -
    the sequence is
    	grab epmutex
    	bump loop_check_gen
    	...
    	grab tep->mtx		// 1
    	tep->gen = loop_check_gen
    	...
    	drop tep->mtx		// 2
    	...
    	grab tep->mtx		// 3
    	...
    	insert into ->f_ep_link
    	...
    	drop tep->mtx		// 4
    	bump loop_check_gen
    	drop epmutex
    and if the fastpath check in another thread happens for that
    eventpoll, it can come
    	* before (1) - in that case fastpath is just fine
    	* after (4) - we'll see non-empty ->f_ep_link, slow path
    taken
    	* between (2) and (3) - loop_check_gen is stable,
    with ->mtx providing barriers and we end up taking slow path.
    
    Note that ->f_ep_link emptiness check is slightly racy - we are protected
    against insertions into that list, but removals can happen right under us.
    Not a problem - in the worst case we'll end up taking a slow path for
    no good reason.
    Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
    fe0a916c
eventpoll.c 65.7 KB