Document hang with multiprocessing.Queue and monkey-patched thread module. [skip ci]

doc/whatsnew_1_1.rst

@@ -90,14 +90,20 @@ downstream libraries, notably `gunicorn`_.
 In addition, simple use of :class:`multiprocessing.Process` is now
 possible in a monkey patched system, at least on POSIX platforms.
 
-.. note:: All of the above entail forking a child process. Forking
-		  a child process that uses gevent, greenlets, and libev
-		  can have some unexpected consequences if the child
-		  doesn't immediately ``exec`` a new binary. Be sure you
-		  understand these consequences before using this
-		  functionality, especially late in a program's lifecycle.
-		  For a more robust solution to certain uses of child
-		  process, consider `gipc`_.
+.. caution:: Use of :class:`multiprocessing.Queue` when :mod:`thread`
+             has been monkey-patched will lead to a hang due to
+             ``Queue``'s internal use of a blocking pipe and threads. For the same
+             reason, :class:`concurrent.futures.ProcessPoolExecutor`,
+             which internally uses a ``Queue``, will hang.
+
+.. tip:: All of the above entail forking a child process. Forking
+		 a child process that uses gevent, greenlets, and libev
+		 can have some unexpected consequences if the child
+		 doesn't immediately ``exec`` a new binary. Be sure you
+		 understand these consequences before using this
+		 functionality, especially late in a program's lifecycle.
+		 For a more robust solution to certain uses of child
+		 process, consider `gipc`_.
 
 .. _gunicorn: http://gunicorn.org
 .. _gipc: https://gehrcke.de/gipc/

gevent/monkey.py

@@ -46,6 +46,9 @@ Sometimes it is useful to run existing python scripts or modules that
 were not built to be gevent aware under gevent. To do so, this module
 can be run as the main module, passing the script and its arguments.
 For details, see the :func:`main` function.
+
+Functions
+=========
 """
 from __future__ import absolute_import
 from __future__ import print_function
@@ -250,7 +253,8 @@ def _patch_existing_locks(threading):
 
 def patch_thread(threading=True, _threading_local=True, Event=False, logging=True,
                  existing_locks=True):
-    """Replace the standard :mod:`thread` module to make it greenlet-based.
+    """
+    Replace the standard :mod:`thread` module to make it greenlet-based.
 
     - If *threading* is true (the default), also patch ``threading``.
     - If *_threading_local* is true (the default), also patch ``_threading_local.local``.
@@ -259,7 +263,33 @@ def patch_thread(threading=True, _threading_local=True, Event=False, logging=Tru
     - If *existing_locks* is True (the default), and the process is still single threaded,
       make sure than any :class:`threading.RLock` (and, under Python 3, :class:`importlib._bootstrap._ModuleLock`)
       instances that are currently locked can be properly unlocked.
+
+    .. caution::
+        Monkey-patching :mod:`thread` and using
+        :class:`multiprocessing.Queue` or
+        :class:`concurrent.futures.ProcessPoolExecutor` (which uses a
+        ``Queue``) will hang the process.
     """
+    # Description of the hang:
+    # There is an incompatibility with patching 'thread' and the 'multiprocessing' module:
+	# The problem is that multiprocessing.queues.Queue uses a half-duplex multiprocessing.Pipe,
+	# which is implemented with os.pipe() and _multiprocessing.Connection. os.pipe isn't patched
+	# by gevent, as it returns just a fileno. _multiprocessing.Connection is an internal implementation
+	# class implemented in C, which exposes a 'poll(timeout)' method; under the covers, this issues a
+	# (blocking) select() call: hence the need for a real thread. Except for that method, we could
+	# almost replace Connection with gevent.fileobject.SocketAdapter, plus a trivial
+	# patch to os.pipe (below). Sigh, so close. (With a little work, we could replicate that method)
+
+	# import os
+	# import fcntl
+	# os_pipe = os.pipe
+	# def _pipe():
+	# 	r, w = os_pipe()
+	# 	fcntl.fcntl(r, fcntl.F_SETFL, os.O_NONBLOCK)
+	# 	fcntl.fcntl(w, fcntl.F_SETFL, os.O_NONBLOCK)
+	# 	return r, w
+	# os.pipe = _pipe
+
     patch_module('thread')
     if threading:
         patch_module('threading')