• Kirill Smelkov's avatar
    component/python-2.7: Lib/subprocess: Speedup close_fds=True · ce75e16e
    Kirill Smelkov authored
    @romain reports that Popen(close_fds=True) is slow on py2. Let's semantically
    backport from py3 how to close only actually opened file descriptors instead of
    whole 3..`ulimit -n` range.
    
    Attached test benchmark shows the following results with `ulimit -n`=65K:
    
    Before this patch:
    
        $ ./bin/python2.7 ~/x.py
        close_fds=False:
        0.001251 s/call
        0.001337 s/call
        0.001486 s/call
    
        close_fds=True:
        0.017973 s/call
        0.018152 s/call
        0.018204 s/call
    
    After the patch:
    
        $ ./bin/python2.7 ~/x.py
        close_fds=False:
        0.001391 s/call
        0.001416 s/call
        0.001570 s/call
    
        close_fds=True:
        0.001469 s/call
        0.001479 s/call
        0.001491 s/call
    
    i.e. ~12x speedup.
    
    References on this subject are in the patch itself.
    
    The test benchmark is below:
    
    ---- 8< ----
    import timeit
    from subprocess import check_call
    
    def f():
        check_call(['true'], close_fds=False)
    def g():
        check_call(['true'], close_fds=True)
    
    N=3
    n=100
    print 'close_fds=False:'
    for i in range(N):
        print '%.6f s/call' % (timeit.timeit(f, number=n) / n)
    print
    print 'close_fds=True:'
    for i in range(N):
        print '%.6f s/call' % (timeit.timeit(g, number=n) / n)
    
    /helped-by @jm
    ce75e16e