Skip to content

C
cpython
Commit 47920566 authored by Reid Kleckner's avatar Reid Kleckner
Browse files
Options

Merge with default.

parents f62c0a24 6e3b793b
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 367 additions and 98 deletions
Doc/library/subprocess.rst
View file @ 47920566
......@@ -249,15 +249,21 @@ Convenience Functions
This module also defines four shortcut functions:
.. function:: call(*popenargs, **kwargs)
.. function:: call(*popenargs, timeout=None, **kwargs)
Run command with arguments. Wait for command to complete, then return the
:attr:`returncode` attribute.
The arguments are the same as for the :class:`Popen` constructor. Example::
The arguments are the same as for the :class:`Popen` constructor, with the
exception of the *timeout* argument, which is given to :meth:`Popen.wait`.
Example::
>>> retcode = subprocess.call(["ls", "-l"])
If the timeout expires, the child process will be killed and then waited for
again. The :exc:`TimeoutExpired` exception will be re-raised after the child
process has terminated.
.. warning::
Like :meth:`Popen.wait`, this will deadlock when using
......@@ -265,34 +271,43 @@ This module also defines four shortcut functions:
generates enough output to a pipe such that it blocks waiting
for the OS pipe buffer to accept more data.
.. versionchanged:: 3.2
*timeout* was added.
.. function:: check_call(*popenargs, **kwargs)
.. function:: check_call(*popenargs, timeout=None, **kwargs)
Run command with arguments. Wait for command to complete. If the exit code was
zero then return, otherwise raise :exc:`CalledProcessError`. The
:exc:`CalledProcessError` object will have the return code in the
:attr:`returncode` attribute.
The arguments are the same as for the :class:`Popen` constructor. Example::
The arguments are the same as for the :func:`call` function. Example::
>>> subprocess.check_call(["ls", "-l"])
0
As in the :func:`call` function, if the timeout expires, the child process
will be killed and the wait retried. The :exc:`TimeoutExpired` exception
will be re-raised after the child process has terminated.
.. warning::
See the warning for :func:`call`.
.. versionchanged:: 3.2
*timeout* was added.
.. function:: check_output(*popenargs, **kwargs)
.. function:: check_output(*popenargs, timeout=None, **kwargs)
Run command with arguments and return its output as a byte string.
If the exit code was non-zero it raises a :exc:`CalledProcessError`. The
:exc:`CalledProcessError` object will have the return code in the
:attr:`returncode`
attribute and output in the :attr:`output` attribute.
:attr:`returncode` attribute and output in the :attr:`output` attribute.
The arguments are the same as for the :class:`Popen` constructor. Example::
The arguments are the same as for the :func:`call` function. Example::
>>> subprocess.check_output(["ls", "-l", "/dev/null"])
b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
......@@ -305,8 +320,17 @@ This module also defines four shortcut functions:
... stderr=subprocess.STDOUT)
b'ls: non_existent_file: No such file or directory\n'
As in the :func:`call` function, if the timeout expires, the child process
will be killed and the wait retried. The :exc:`TimeoutExpired` exception
will be re-raised after the child process has terminated. The output from
the child process so far will be in the :attr:`output` attribute of the
exception.
.. versionadded:: 3.1
.. versionchanged:: 3.2
*timeout* was added.
.. function:: getstatusoutput(cmd)
......@@ -359,6 +383,10 @@ arguments.
check_call() will raise :exc:`CalledProcessError`, if the called process returns
a non-zero return code.
All of the functions and methods that accept a *timeout* parameter, such as
:func:`call` and :meth:`Popen.communicate` will raise :exc:`TimeoutExpired` if
the timeout expires before the process exits.
Security
^^^^^^^^
......@@ -380,11 +408,15 @@ Instances of the :class:`Popen` class have the following methods:
attribute.
.. method:: Popen.wait()
.. method:: Popen.wait(timeout=None)
Wait for child process to terminate. Set and return :attr:`returncode`
attribute.
If the process does not terminate after *timeout* seconds, raise a
:exc:`TimeoutExpired` exception. It is safe to catch this exception and
retry the wait.
.. warning::
This will deadlock when using ``stdout=PIPE`` and/or
......@@ -392,11 +424,14 @@ Instances of the :class:`Popen` class have the following methods:
a pipe such that it blocks waiting for the OS pipe buffer to
accept more data. Use :meth:`communicate` to avoid that.
.. versionchanged:: 3.2
*timeout* was added.
.. method:: Popen.communicate(input=None)
.. method:: Popen.communicate(input=None, timeout=None)
Interact with process: Send data to stdin. Read data from stdout and stderr,
until end-of-file is reached. Wait for process to terminate. The optional
until end-of-file is reached. Wait for process to terminate. The optional
*input* argument should be a byte string to be sent to the child process, or
``None``, if no data should be sent to the child.
......@@ -407,11 +442,29 @@ Instances of the :class:`Popen` class have the following methods:
``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
``stderr=PIPE`` too.
If the process does not terminate after *timeout* seconds, a
:exc:`TimeoutExpired` exception will be raised. Catching this exception and
retrying communication will not lose any output.
The child process is not killed if the timeout expires, so in order to
cleanup properly a well-behaved application should kill the child process and
finish communication::
proc = subprocess.Popen(...)
try:
outs, errs = proc.communicate(timeout=15)
except TimeoutExpired:
proc.kill()
outs, errs = proc.communicate()
.. note::
The data read is buffered in memory, so do not use this method if the data
size is large or unlimited.
.. versionchanged:: 3.2
*timeout* was added.
.. method:: Popen.send_signal(signal)
......
Lib/subprocess.py
View file @ 47920566
This diff is collapsed.
Lib/test/test_subprocess.py
View file @ 47920566
......@@ -56,6 +56,8 @@ class BaseTestCase(unittest.TestCase):
# shutdown time. That frustrates tests trying to check stderr produced
# from a spawned Python process.
actual = support.strip_python_stderr(stderr)
# strip_python_stderr also strips whitespace, so we do too.
expected = expected.strip()
self.assertEqual(actual, expected, msg)
......@@ -67,6 +69,15 @@ class ProcessTestCase(BaseTestCase):
"import sys; sys.exit(47)"])
self.assertEqual(rc, 47)
def test_call_timeout(self):
# call() function with timeout argument; we want to test that the child
# process gets killed when the timeout expires. If the child isn't
# killed, this call will deadlock since subprocess.call waits for the
# child.
self.assertRaises(subprocess.TimeoutExpired, subprocess.call,
[sys.executable, "-c", "while True: pass"],
timeout=0.1)
def test_check_call_zero(self):
# check_call() function with zero return code
rc = subprocess.check_call([sys.executable, "-c",
......@@ -109,6 +120,18 @@ class ProcessTestCase(BaseTestCase):
self.fail("Expected ValueError when stdout arg supplied.")
self.assertIn('stdout', c.exception.args[0])
def test_check_output_timeout(self):
# check_output() function with timeout arg
with self.assertRaises(subprocess.TimeoutExpired) as c:
output = subprocess.check_output(
[sys.executable, "-c",
"import sys; sys.stdout.write('BDFL')\n"
"sys.stdout.flush()\n"
"while True: pass"],
timeout=0.5)
self.fail("Expected TimeoutExpired.")
self.assertEqual(c.exception.output, b'BDFL')
def test_call_kwargs(self):
# call() function with keyword args
newenv = os.environ.copy()
......@@ -366,6 +389,41 @@ class ProcessTestCase(BaseTestCase):
self.assertEqual(stdout, b"banana")
self.assertStderrEqual(stderr, b"pineapple")
def test_communicate_timeout(self):
p = subprocess.Popen([sys.executable, "-c",
'import sys,os,time;'
'sys.stderr.write("pineapple\\n");'
'time.sleep(1);'
'sys.stderr.write("pear\\n");'
'sys.stdout.write(sys.stdin.read())'],
universal_newlines=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
self.assertRaises(subprocess.TimeoutExpired, p.communicate, "banana",
timeout=0.3)
# Make sure we can keep waiting for it, and that we get the whole output
# after it completes.
(stdout, stderr) = p.communicate()
self.assertEqual(stdout, "banana")
self.assertStderrEqual(stderr.encode(), b"pineapple\npear\n")
def test_communicate_timeout_large_ouput(self):
# Test a expring timeout while the child is outputting lots of data.
p = subprocess.Popen([sys.executable, "-c",
'import sys,os,time;'
'sys.stdout.write("a" * (64 * 1024));'
'time.sleep(0.2);'
'sys.stdout.write("a" * (64 * 1024));'
'time.sleep(0.2);'
'sys.stdout.write("a" * (64 * 1024));'
'time.sleep(0.2);'
'sys.stdout.write("a" * (64 * 1024));'],
stdout=subprocess.PIPE)
self.assertRaises(subprocess.TimeoutExpired, p.communicate, timeout=0.4)
(stdout, _) = p.communicate()
self.assertEqual(len(stdout), 4 * 64 * 1024)
# Test for the fd leak reported in http://bugs.python.org/issue2791.
def test_communicate_pipe_fd_leak(self):
for stdin_pipe in (False, True):
......@@ -561,6 +619,13 @@ class ProcessTestCase(BaseTestCase):
self.assertEqual(p.wait(), 0)
def test_wait_timeout(self):
p = subprocess.Popen([sys.executable,
"-c", "import time; time.sleep(1)"])
self.assertRaises(subprocess.TimeoutExpired, p.wait, timeout=0.1)
self.assertEqual(p.wait(timeout=2), 0)
def test_invalid_bufsize(self):
# an invalid type of the bufsize argument should raise
# TypeError.
......
PC/_subprocess.c
View file @ 47920566
......@@ -682,6 +682,7 @@ PyInit__subprocess()
defint(d, "SW_HIDE", SW_HIDE);
defint(d, "INFINITE", INFINITE);
defint(d, "WAIT_OBJECT_0", WAIT_OBJECT_0);
defint(d, "WAIT_TIMEOUT", WAIT_TIMEOUT);
defint(d, "CREATE_NEW_CONSOLE", CREATE_NEW_CONSOLE);
defint(d, "CREATE_NEW_PROCESS_GROUP", CREATE_NEW_PROCESS_GROUP);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7