Commit 22bcde63 authored by Antoine Pitrou's avatar Antoine Pitrou

Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.

Patch by Davin Potts.
parent 3a8677b7
...@@ -18,41 +18,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully ...@@ -18,41 +18,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
leverage multiple processors on a given machine. It runs on both Unix and leverage multiple processors on a given machine. It runs on both Unix and
Windows. Windows.
.. warning:: The :mod:`multiprocessing` module also introduces APIs which do not have
analogs in the :mod:`threading` module. A prime example of this is the
:class:`Pool` object which offers a convenient means of parallelizing the
execution of a function across multiple input values, distributing the
input data across processes (data parallelism). The following example
demonstrates the common practice of defining such functions in a module so
that child processes can successfully import that module. This basic example
of data parallelism using :class:`Pool`, ::
Some of this package's functionality requires a functioning shared semaphore from multiprocessing import Pool
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
.. note:: def f(x):
return x*x
if __name__ == '__main__':
p = Pool(5)
print(p.map(f, [1, 2, 3]))
will print to standard output ::
Functionality within this package requires that the ``__main__`` module be [1, 4, 9]
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.Pool` examples will not work in the
interactive interpreter. For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
The :class:`Process` class The :class:`Process` class
...@@ -99,7 +85,6 @@ For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is ...@@ -99,7 +85,6 @@ For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
necessary, see :ref:`multiprocessing-programming`. necessary, see :ref:`multiprocessing-programming`.
Exchanging objects between processes Exchanging objects between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -175,6 +160,14 @@ that only one process prints to standard output at a time:: ...@@ -175,6 +160,14 @@ that only one process prints to standard output at a time::
Without using the lock output from the different processes is liable to get all Without using the lock output from the different processes is liable to get all
mixed up. mixed up.
.. warning::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
Sharing state between processes Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -290,6 +283,34 @@ For example:: ...@@ -290,6 +283,34 @@ For example::
Note that the methods of a pool should only ever be used by the Note that the methods of a pool should only ever be used by the
process which created it. process which created it.
.. note::
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`Pool` examples will not work in the interactive interpreter.
For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
Reference Reference
--------- ---------
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment