Commit 4ee32532 authored by Jason Madden's avatar Jason Madden

Explicit test and documentation that fork-watchers and gevent.subprocess do...

Explicit test and documentation that fork-watchers and gevent.subprocess do not work in background threads. Fixes #688.
parent 949f0129
...@@ -103,6 +103,9 @@ possible in a monkey patched system, at least on POSIX platforms. ...@@ -103,6 +103,9 @@ possible in a monkey patched system, at least on POSIX platforms.
reason, :class:`concurrent.futures.ProcessPoolExecutor`, reason, :class:`concurrent.futures.ProcessPoolExecutor`,
which internally uses a ``Queue``, will hang. which internally uses a ``Queue``, will hang.
.. caution:: It is not possible to use :mod:`gevent.subprocess` from
native threads. See :mod:`gevent.subprocess` for details.
.. tip:: All of the above entail forking a child process. Forking .. tip:: All of the above entail forking a child process. Forking
a child process that uses gevent, greenlets, and libev a child process that uses gevent, greenlets, and libev
can have some unexpected consequences if the child can have some unexpected consequences if the child
......
""" """
Cooperative ``subprocess`` module. Cooperative ``subprocess`` module.
.. caution:: This module is not usable from native threads other than
the main thread; attempting to do so will raise a :exc:`TypeError`.
This module depends on libev's fork watchers. On POSIX systems,
fork watchers are implemented using signals, and the thread to
which process-directed signals are delivered `is not defined`_.
Because each native thread has its own gevent/libev loop, this
means that a fork watcher registered with one loop (thread) may
never see the signal about a child it spawned if the signal is sent
to a different thread.
.. note:: The interface of this module is intended to match that of .. note:: The interface of this module is intended to match that of
the standard library :mod:`subprocess` module. There are some small the standard library :mod:`subprocess` module. There are some small
differences between the Python 2 and Python 3 versions of that differences between the Python 2 and Python 3 versions of that
module and between the POSIX and Windows versions. The HTML module and between the POSIX and Windows versions. The HTML
documentation here can only describe one version; for definitive documentation here can only describe one version; for definitive
documentation, see the standard library or the source code. documentation, see the standard library or the source code.
.. _is not defined: http://www.linuxprogrammingblog.com/all-about-linux-signals?page=11
""" """
from __future__ import absolute_import, print_function from __future__ import absolute_import, print_function
import errno import errno
......
# mostly tests from test_subprocess.py that used to have problems
import sys import sys
import os import os
import errno import errno
...@@ -200,6 +199,30 @@ class Test(greentest.TestCase): ...@@ -200,6 +199,30 @@ class Test(greentest.TestCase):
r = p.stdout.readline() r = p.stdout.readline()
self.assertEqual(r, b'foobar\n') self.assertEqual(r, b'foobar\n')
def test_subprocess_in_native_thread(self):
# gevent.subprocess doesn't work from a background
# native thread. See #688
from gevent import monkey
# must be a native thread; defend against monkey-patching
ex = []
Thread = monkey.get_original('threading', 'Thread')
def fn():
try:
gevent.subprocess.Popen('echo 123', shell=True)
raise AssertionError("Should not be able to construct Popen")
except Exception as e:
ex.append(e)
thread = Thread(target=fn)
thread.start()
thread.join()
self.assertEqual(len(ex), 1)
self.assertTrue(isinstance(ex[0], TypeError), ex)
self.assertEqual(ex[0].args[0], 'child watchers are only available on the default loop')
if __name__ == '__main__': if __name__ == '__main__':
greentest.main() greentest.main()
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment