Commit a280fd72 authored by Jesse Noller's avatar Jesse Noller

mp docs - fix issues 4012,3518,4193

parent aa674712
...@@ -37,7 +37,6 @@ def get_logger(): ...@@ -37,7 +37,6 @@ def get_logger():
_logger = logging.getLogger('distributing') _logger = logging.getLogger('distributing')
_logger.propogate = 0 _logger.propogate = 0
util.fix_up_logger(_logger)
_formatter = logging.Formatter(util.DEFAULT_LOGGING_FORMAT) _formatter = logging.Formatter(util.DEFAULT_LOGGING_FORMAT)
_handler = logging.StreamHandler() _handler = logging.StreamHandler()
_handler.setFormatter(_formatter) _handler.setFormatter(_formatter)
......
...@@ -26,6 +26,29 @@ Windows. ...@@ -26,6 +26,29 @@ Windows.
import it will result in an :exc:`ImportError`. See import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information. :issue:`3770` for additional information.
.. note::
Functionality within this package requires that the ``__main__`` method be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.Pool` examples will not work in the
interactive interpreter. For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
The :class:`Process` class The :class:`Process` class
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -34,17 +57,36 @@ object and then calling its :meth:`~Process.start` method. :class:`Process` ...@@ -34,17 +57,36 @@ object and then calling its :meth:`~Process.start` method. :class:`Process`
follows the API of :class:`threading.Thread`. A trivial example of a follows the API of :class:`threading.Thread`. A trivial example of a
multiprocess program is :: multiprocess program is ::
from multiprocessing import Process from multiprocessing import Process
def f(name): def f(name):
print 'hello', name print 'hello', name
if __name__ == '__main__': if __name__ == '__main__':
p = Process(target=f, args=('bob',)) p = Process(target=f, args=('bob',))
p.start() p.start()
p.join() p.join()
Here the function ``f`` is run in a child process. To show the individual process IDs involved, here is an expanded example::
from multiprocessing import Process
import os
def info(title):
print title
print 'module name:', __name__
print 'parent process:', os.getppid()
print 'process id:', os.getpid()
def f(name):
info('function f')
print 'hello', name
if __name__ == '__main__':
info('main line')
p = Process(target=f, args=('bob',))
p.start()
p.join()
For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
necessary, see :ref:`multiprocessing-programming`. necessary, see :ref:`multiprocessing-programming`.
...@@ -234,7 +276,7 @@ For example:: ...@@ -234,7 +276,7 @@ For example::
if __name__ == '__main__': if __name__ == '__main__':
pool = Pool(processes=4) # start 4 worker processes pool = Pool(processes=4) # start 4 worker processes
result = pool.applyAsync(f, [10]) # evaluate "f(10)" asynchronously result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously
print result.get(timeout=1) # prints "100" unless your computer is *very* slow print result.get(timeout=1) # prints "100" unless your computer is *very* slow
print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]"
...@@ -307,7 +349,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the ...@@ -307,7 +349,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the
semantics. Multiple processes may be given the same name. The initial semantics. Multiple processes may be given the same name. The initial
name is set by the constructor. name is set by the constructor.
.. method:: is_alive() .. method:: is_alive
Return whether the process is alive. Return whether the process is alive.
...@@ -816,6 +858,9 @@ object -- see :ref:`multiprocessing-managers`. ...@@ -816,6 +858,9 @@ object -- see :ref:`multiprocessing-managers`.
acceptable. If *block* is ``True`` and *timeout* is not ``None`` then it acceptable. If *block* is ``True`` and *timeout* is not ``None`` then it
specifies a timeout in seconds. If *block* is ``False`` then *timeout* is specifies a timeout in seconds. If *block* is ``False`` then *timeout* is
ignored. ignored.
Note that on OS/X ``sem_timedwait`` is unsupported, so timeout arguments
for these will be ignored.
.. note:: .. note::
...@@ -1089,6 +1134,27 @@ their parent process exits. The manager classes are defined in the ...@@ -1089,6 +1134,27 @@ their parent process exits. The manager classes are defined in the
A class method which creates a manager object referring to a pre-existing A class method which creates a manager object referring to a pre-existing
server process which is using the given address and authentication key. server process which is using the given address and authentication key.
.. method:: get_server()
Returns a :class:`Server` object which represents the actual server under
the control of the Manager. The :class:`Server` object supports the
:meth:`serve_forever` method::
>>> from multiprocessing.managers import BaseManager
>>> m = BaseManager(address=('', 50000), authkey='abc'))
>>> server = m.get_server()
>>> s.serve_forever()
:class:`Server` additionally have an :attr:`address` attribute.
.. method:: connect()
Connect a local manager object to a remote manager process::
>>> from multiprocessing.managers import BaseManager
>>> m = BaseManager(address='127.0.0.1', authkey='abc))
>>> m.connect()
.. method:: shutdown() .. method:: shutdown()
Stop the process used by the manager. This is only available if Stop the process used by the manager. This is only available if
...@@ -1267,19 +1333,20 @@ remote clients can access:: ...@@ -1267,19 +1333,20 @@ remote clients can access::
>>> queue = Queue.Queue() >>> queue = Queue.Queue()
>>> class QueueManager(BaseManager): pass >>> class QueueManager(BaseManager): pass
... ...
>>> QueueManager.register('getQueue', callable=lambda:queue) >>> QueueManager.register('get_queue', callable=lambda:queue)
>>> m = QueueManager(address=('', 50000), authkey='abracadabra') >>> m = QueueManager(address=('', 50000), authkey='abracadabra')
>>> m.serveForever() >>> s = m.get_server()
>>> s.serveForever()
One client can access the server as follows:: One client can access the server as follows::
>>> from multiprocessing.managers import BaseManager >>> from multiprocessing.managers import BaseManager
>>> class QueueManager(BaseManager): pass >>> class QueueManager(BaseManager): pass
... ...
>>> QueueManager.register('getQueue') >>> QueueManager.register('get_queue')
>>> m = QueueManager.from_address(address=('foo.bar.org', 50000), >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra')
>>> authkey='abracadabra') >>> m.connect()
>>> queue = m.getQueue() >>> queue = m.get_queue()
>>> queue.put('hello') >>> queue.put('hello')
Another client can also use it:: Another client can also use it::
...@@ -1293,6 +1360,27 @@ Another client can also use it:: ...@@ -1293,6 +1360,27 @@ Another client can also use it::
>>> queue.get() >>> queue.get()
'hello' 'hello'
Local processes can also access that queue, using the code from above on the
client to access it remotely::
>>> from multiprocessing import Process, Queue
>>> from multiprocessing.managers import BaseManager
>>> class Worker(Process):
... def __init__(self, q):
... self.q = q
... super(Worker, self).__init__()
... def run(self):
... self.q.put('local hello')
...
>>> queue = Queue()
>>> w = Worker(queue)
>>> w.start()
>>> class QueueManager(BaseManager): pass
...
>>> QueueManager.register('get_queue', callable=lambda: queue)
>>> m = QueueManager(address=('', 50000), authkey='abracadabra')
>>> s = m.get_server()
>>> s.serve_forever()
Proxy Objects Proxy Objects
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
...@@ -1507,7 +1595,7 @@ with the :class:`Pool` class. ...@@ -1507,7 +1595,7 @@ with the :class:`Pool` class.
The class of the result returned by :meth:`Pool.apply_async` and The class of the result returned by :meth:`Pool.apply_async` and
:meth:`Pool.map_async`. :meth:`Pool.map_async`.
.. method:: get([timeout) .. method:: get([timeout])
Return the result when it arrives. If *timeout* is not ``None`` and the Return the result when it arrives. If *timeout* is not ``None`` and the
result does not arrive within *timeout* seconds then result does not arrive within *timeout* seconds then
...@@ -1537,7 +1625,7 @@ The following example demonstrates the use of a pool:: ...@@ -1537,7 +1625,7 @@ The following example demonstrates the use of a pool::
if __name__ == '__main__': if __name__ == '__main__':
pool = Pool(processes=4) # start 4 worker processes pool = Pool(processes=4) # start 4 worker processes
result = pool.applyAsync(f, (10,)) # evaluate "f(10)" asynchronously result = pool.apply_async(f, (10,)) # evaluate "f(10)" asynchronously
print result.get(timeout=1) # prints "100" unless your computer is *very* slow print result.get(timeout=1) # prints "100" unless your computer is *very* slow
print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]"
...@@ -1548,7 +1636,7 @@ The following example demonstrates the use of a pool:: ...@@ -1548,7 +1636,7 @@ The following example demonstrates the use of a pool::
print it.next(timeout=1) # prints "4" unless your computer is *very* slow print it.next(timeout=1) # prints "4" unless your computer is *very* slow
import time import time
result = pool.applyAsync(time.sleep, (10,)) result = pool.apply_async(time.sleep, (10,))
print result.get(timeout=1) # raises TimeoutError print result.get(timeout=1) # raises TimeoutError
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment