Add benchmark for using subprocess.Popen to create /usr/bin/true [skip ci]
In response to #1172 The following numbers are for my machine on macOS 10.13.3 with MAXFD of 50000. Python 2.7: ..................... spawn native no close_fds: Mean +- std dev: 1.81 ms +- 0.04 ms ..................... spawn gevent no close_fds: Mean +- std dev: 2.11 ms +- 0.08 ms ..................... spawn native close_fds: Mean +- std dev: 31.0 ms +- 0.7 ms ..................... spawn gevent close_fds: Mean +- std dev: 31.6 ms +- 0.6 ms Notice that the times when close_fd=True (not the default on 2.7) are about the same. 2.7 uses the same Python loop we do to close all the fds. Now 3.7: ..................... spawn native no close_fds: Mean +- std dev: 1.34 ms +- 0.04 ms ..................... spawn gevent no close_fds: Mean +- std dev: 117 ms +- 2 ms ..................... spawn native close_fds: Mean +- std dev: 1.36 ms +- 0.03 ms ..................... spawn gevent close_fds: Mean +- std dev: 32.5 ms +- 0.4 ms Notice that gevent is *much* slower when we *don't* close the fds. This is because, starting in Python 3.4, close_fds defaults to true, and when it's false we have to check os.get_inheritable() for each fd before we close it. gevent performs the same as it did on Python 2.7 when closing fds, but the native implementation is much faster due to the C optimizations outlined in #1172---it turns out they apply to BSD and Apple platforms in addition to Linux, although they're not async safe. Now, the C code does the opposite for inheritable handles: it explicitly calls make_inheritable() for the ones it wants to keep and lets the OS close the others with CLOEXEC. We could probably do that too; the slow down for this case counts as a regression, I think.
Showing
Please register or sign in to comment