Commit 73dd030c authored by Antoine Pitrou's avatar Antoine Pitrou

Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.

Patch by Davin Potts.
parent dfe0b232
...@@ -16,41 +16,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully ...@@ -16,41 +16,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
leverage multiple processors on a given machine. It runs on both Unix and leverage multiple processors on a given machine. It runs on both Unix and
Windows. Windows.
.. note:: The :mod:`multiprocessing` module also introduces APIs which do not have
analogs in the :mod:`threading` module. A prime example of this is the
:class:`~multiprocessing.pool.Pool` object which offers a convenient means of
parallelizing the execution of a function across multiple input values,
distributing the input data across processes (data parallelism). The following
example demonstrates the common practice of defining such functions in a module
so that child processes can successfully import that module. This basic example
of data parallelism using :class:`~multiprocessing.pool.Pool`, ::
Some of this package's functionality requires a functioning shared semaphore from multiprocessing import Pool
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
.. note:: def f(x):
return x*x
if __name__ == '__main__':
with Pool(5) as p:
print(p.map(f, [1, 2, 3]))
Functionality within this package requires that the ``__main__`` module be will print to standard output ::
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such [1, 4, 9]
as the :class:`multiprocessing.pool.Pool` examples will not work in the
interactive interpreter. For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
The :class:`Process` class The :class:`Process` class
...@@ -276,6 +262,14 @@ that only one process prints to standard output at a time:: ...@@ -276,6 +262,14 @@ that only one process prints to standard output at a time::
Without using the lock output from the different processes is liable to get all Without using the lock output from the different processes is liable to get all
mixed up. mixed up.
.. note::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
Sharing state between processes Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -406,6 +400,34 @@ For example:: ...@@ -406,6 +400,34 @@ For example::
Note that the methods of a pool should only ever be used by the Note that the methods of a pool should only ever be used by the
process which created it. process which created it.
.. note::
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.pool.Pool` examples will not work in the
interactive interpreter. For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
Reference Reference
--------- ---------
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment