• Jason Madden's avatar
    Add benchmark for using subprocess.Popen to create /usr/bin/true [skip ci] · b166aaf8
    Jason Madden authored
    In response to #1172
    
    The following numbers are for my machine on macOS 10.13.3 with MAXFD
    of 50000.
    
    Python 2.7:
    
    .....................
    spawn native no close_fds: Mean +- std dev: 1.81 ms +- 0.04 ms
    .....................
    spawn gevent no close_fds: Mean +- std dev: 2.11 ms +- 0.08 ms
    .....................
    spawn native close_fds: Mean +- std dev: 31.0 ms +- 0.7 ms
    .....................
    spawn gevent close_fds: Mean +- std dev: 31.6 ms +- 0.6 ms
    
    Notice that the times when close_fd=True (not the default on 2.7) are
    about the same. 2.7 uses the same Python loop we do to close all the
    fds.
    
    Now 3.7:
    
    .....................
    spawn native no close_fds: Mean +- std dev: 1.34 ms +- 0.04 ms
    .....................
    spawn gevent no close_fds: Mean +- std dev: 117 ms +- 2 ms
    .....................
    spawn native close_fds: Mean +- std dev: 1.36 ms +- 0.03 ms
    .....................
    spawn gevent close_fds: Mean +- std dev: 32.5 ms +- 0.4 ms
    
    Notice that gevent is *much* slower when we *don't* close the fds.
    This is because, starting in Python 3.4, close_fds defaults to true,
    and when it's false we have to check os.get_inheritable() for each fd
    before we close it. gevent performs the same as it did on Python 2.7
    when closing fds, but the native implementation is much faster due to
    the C optimizations outlined in #1172---it turns out they apply to BSD
    and Apple platforms in addition to Linux, although they're not async
    safe.
    
    Now, the C code does the opposite for inheritable handles: it
    explicitly calls make_inheritable() for the ones it wants to keep and
    lets the OS close the others with CLOEXEC. We could probably do that
    too; the slow down for this case counts as a regression, I think.
    b166aaf8
bench_subprocess.py 1.54 KB