Commit 3732ed24 authored by Larry Hastings's avatar Larry Hastings

Merge in all documentation changes since branching 3.4.0rc1.

parent b6b6a6d5
...@@ -45,6 +45,7 @@ in any early abort case). ...@@ -45,6 +45,7 @@ in any early abort case).
Unless otherwise stated, buffers are not NUL-terminated. Unless otherwise stated, buffers are not NUL-terminated.
.. note:: .. note::
For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of For all ``#`` variants of formats (``s#``, ``y#``, etc.), the type of
the length argument (int or :c:type:`Py_ssize_t`) is controlled by the length argument (int or :c:type:`Py_ssize_t`) is controlled by
defining the macro :c:macro:`PY_SSIZE_T_CLEAN` before including defining the macro :c:macro:`PY_SSIZE_T_CLEAN` before including
......
...@@ -525,11 +525,11 @@ Exception Objects ...@@ -525,11 +525,11 @@ Exception Objects
reference, as accessible from Python through :attr:`__cause__`. reference, as accessible from Python through :attr:`__cause__`.
.. c:function:: void PyException_SetCause(PyObject *ex, PyObject *ctx) .. c:function:: void PyException_SetCause(PyObject *ex, PyObject *cause)
Set the cause associated with the exception to *ctx*. Use *NULL* to clear Set the cause associated with the exception to *cause*. Use *NULL* to clear
it. There is no type check to make sure that *ctx* is either an exception it. There is no type check to make sure that *cause* is either an exception
instance or :const:`None`. This steals a reference to *ctx*. instance or :const:`None`. This steals a reference to *cause*.
:attr:`__suppress_context__` is implicitly set to ``True`` by this function. :attr:`__suppress_context__` is implicitly set to ``True`` by this function.
......
...@@ -582,6 +582,7 @@ code, or when embedding the Python interpreter: ...@@ -582,6 +582,7 @@ code, or when embedding the Python interpreter:
.. index:: module: _thread .. index:: module: _thread
.. note:: .. note::
When only the main thread exists, no GIL operations are needed. This is a When only the main thread exists, no GIL operations are needed. This is a
common situation (most Python programs do not use threads), and the lock common situation (most Python programs do not use threads), and the lock
operations slow the interpreter down a bit. Therefore, the lock is not operations slow the interpreter down a bit. Therefore, the lock is not
......
...@@ -120,7 +120,7 @@ There are only a few functions special to module objects. ...@@ -120,7 +120,7 @@ There are only a few functions special to module objects.
Return a pointer to the :c:type:`PyModuleDef` struct from which the module was Return a pointer to the :c:type:`PyModuleDef` struct from which the module was
created, or *NULL* if the module wasn't created with created, or *NULL* if the module wasn't created with
:c:func:`PyModule_Create`.i :c:func:`PyModule_Create`.
.. c:function:: PyObject* PyState_FindModule(PyModuleDef *def) .. c:function:: PyObject* PyState_FindModule(PyModuleDef *def)
......
...@@ -357,9 +357,9 @@ attribute is considered sufficient for this determination. ...@@ -357,9 +357,9 @@ attribute is considered sufficient for this determination.
.. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default) .. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default)
Return an estimated length for the object *o*. First trying to return its Return an estimated length for the object *o*. First try to return its
actual length, then an estimate using ``__length_hint__``, and finally actual length, then an estimate using :meth:`~object.__length_hint__`, and
returning the default value. On error ``-1`` is returned. This is the finally return the default value. On error return ``-1``. This is the
equivalent to the Python expression ``operator.length_hint(o, default)``. equivalent to the Python expression ``operator.length_hint(o, default)``.
.. versionadded:: 3.4 .. versionadded:: 3.4
......
...@@ -129,6 +129,14 @@ type. ...@@ -129,6 +129,14 @@ type.
Initializes a struct sequence type *type* from *desc* in place. Initializes a struct sequence type *type* from *desc* in place.
.. c:function:: int PyStructSequence_InitType2(PyTypeObject *type, PyStructSequence_Desc *desc)
The same as ``PyStructSequence_InitType``, but returns ``0`` on success and ``-1`` on
failure.
.. versionadded:: 3.4
.. c:type:: PyStructSequence_Desc .. c:type:: PyStructSequence_Desc
Contains the meta information of a struct sequence type to create. Contains the meta information of a struct sequence type to create.
......
...@@ -205,9 +205,8 @@ type objects) *must* have the :attr:`ob_size` field. ...@@ -205,9 +205,8 @@ type objects) *must* have the :attr:`ob_size` field.
bit currently defined is :const:`Py_PRINT_RAW`. When the :const:`Py_PRINT_RAW` bit currently defined is :const:`Py_PRINT_RAW`. When the :const:`Py_PRINT_RAW`
flag bit is set, the instance should be printed the same way as :c:member:`~PyTypeObject.tp_str` flag bit is set, the instance should be printed the same way as :c:member:`~PyTypeObject.tp_str`
would format it; when the :const:`Py_PRINT_RAW` flag bit is clear, the instance would format it; when the :const:`Py_PRINT_RAW` flag bit is clear, the instance
should be printed the same was as :c:member:`~PyTypeObject.tp_repr` would format it. It should should be printed the same way as :c:member:`~PyTypeObject.tp_repr` would format it. It should
return ``-1`` and set an exception condition when an error occurred during the return ``-1`` and set an exception condition when an error occurs.
comparison.
It is possible that the :c:member:`~PyTypeObject.tp_print` field will be deprecated. In any case, It is possible that the :c:member:`~PyTypeObject.tp_print` field will be deprecated. In any case,
it is recommended not to define :c:member:`~PyTypeObject.tp_print`, but instead to rely on it is recommended not to define :c:member:`~PyTypeObject.tp_print`, but instead to rely on
......
...@@ -61,6 +61,8 @@ add_module_names = True ...@@ -61,6 +61,8 @@ add_module_names = True
# By default, highlight as Python 3. # By default, highlight as Python 3.
highlight_language = 'python3' highlight_language = 'python3'
needs_sphinx = '1.1'
# Options for HTML output # Options for HTML output
# ----------------------- # -----------------------
...@@ -118,11 +120,11 @@ _stdauthor = r'Guido van Rossum\\Fred L. Drake, Jr., editor' ...@@ -118,11 +120,11 @@ _stdauthor = r'Guido van Rossum\\Fred L. Drake, Jr., editor'
latex_documents = [ latex_documents = [
('c-api/index', 'c-api.tex', ('c-api/index', 'c-api.tex',
'The Python/C API', _stdauthor, 'manual'), 'The Python/C API', _stdauthor, 'manual'),
('distutils/index', 'distutils.tex', ('distributing/index', 'distributing.tex',
'Distributing Python Modules', _stdauthor, 'manual'), 'Distributing Python Modules', _stdauthor, 'manual'),
('extending/index', 'extending.tex', ('extending/index', 'extending.tex',
'Extending and Embedding Python', _stdauthor, 'manual'), 'Extending and Embedding Python', _stdauthor, 'manual'),
('install/index', 'install.tex', ('installing/index', 'installing.tex',
'Installing Python Modules', _stdauthor, 'manual'), 'Installing Python Modules', _stdauthor, 'manual'),
('library/index', 'library.tex', ('library/index', 'library.tex',
'The Python Library Reference', _stdauthor, 'manual'), 'The Python Library Reference', _stdauthor, 'manual'),
......
...@@ -11,8 +11,8 @@ ...@@ -11,8 +11,8 @@
library/index.rst library/index.rst
extending/index.rst extending/index.rst
c-api/index.rst c-api/index.rst
distutils/index.rst distributing/index.rst
install/index.rst installing/index.rst
howto/index.rst howto/index.rst
faq/index.rst faq/index.rst
glossary.rst glossary.rst
...@@ -21,3 +21,11 @@ ...@@ -21,3 +21,11 @@
bugs.rst bugs.rst
copyright.rst copyright.rst
license.rst license.rst
.. include legacy packaging docs in build
.. toctree::
:hidden:
distutils/index.rst
install/index.rst
.. _distributing-index:
###############################
Distributing Python Modules
###############################
:Email: distutils-sig@python.org
As a popular open source development project, Python has an active
supporting community of contributors and users that also make their software
available for other Python developers to use under open source license terms.
This allows Python users to share and collaborate effectively, benefiting
from the solutions others have already created to common (and sometimes
even rare!) problems, as well as potentially contributing their own
solutions to the common pool.
This guide covers the distribution part of the process. For a guide to
installing other Python projects, refer to the
:ref:`installation guide <installing-index>`.
.. note::
For corporate and other institutional users, be aware that many
organisations have their own policies around using and contributing to
open source software. Please take such policies into account when making
use of the distribution and installation tools provided with Python.
Key terms
=========
* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
repository of open source licensed packages made available for use by
other Python users
* the `Python Packaging Authority
<http://packaging.python.org/en/latest/future.html>`__ are the group of
developers and documentation authors responsible for the maintenance and
evolution of the standard packaging tools and the associated metadata and
file format standards. They maintain a variety of tools, documentation
and issue trackers on both `GitHub <https://github.com/pypa>`__ and
`BitBucket <https://bitbucket.org/pypa/>`__.
* ``distutils`` is the original build and distribution system first added to
the Python standard library in 1998. While direct use of ``distutils`` is
being phased out, it still laid the foundation for the current packaging
and distribution infrastructure, and it not only remains part of the
standard library, but its name lives on in other ways (such as the name
of the mailing list used to coordinate Python packaging standards
development).
Open source licensing and collaboration
=======================================
In most parts of the world, software is automatically covered by copyright.
This means that other developers require explicit permission to copy, use,
modify and redistribute the software.
Open source licensing is a way of explicitly granting such permission in a
relatively consistent way, allowing developers to share and collaborate
efficiently by making common solutions to various problems freely available.
This leaves many developers free to spend more time focusing on the problems
that are relatively unique to their specific situation.
The distribution tools provided with Python are designed to make it
reasonably straightforward for developers to make their own contributions
back to that common pool of software if they choose to do so.
The same distribution tools can also be used to distribute software within
an organisation, regardless of whether that software is published as open
source software or not.
Installing the tools
====================
The standard library does not include build tools that support modern
Python packaging standards, as the core development team has found that it
is important to have standard tools that work consistently, even on older
versions of Python.
The currently recommended build and distribution tools can be installed
using ``pip``::
pip install setuptools wheel twine
Reading the guide
=================
The Python Packaging User Guide covers the various key steps and elements
involved in creating a project
* `Project structure`_
* `Building and packaging the project`_
* `Uploading the project to the Python Package Index`_
.. _Project structure: \
http://packaging.python.org/en/latest/tutorial.html#creating-your-own-project
.. _Building and packaging the project: \
http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project
.. _Uploading the project to the Python Package Index: \
http://packaging.python.org/en/latest/tutorial.html#uploading-your-project-to-pypi
How do I...?
============
These are quick answers or links for some common tasks.
... choose a name for my project?
---------------------------------
This isn't an easy topic, but here are a few tips:
* check the Python Package Index to see if the name is already in use
* check popular hosting sites like GitHub, BitBucket, etc to see if there
is already a project with that name
* check what comes up in a web search for the name you're considering
* avoid particularly common words, especially ones with multiple meanings,
as they can make it difficult for users to find your software when
searching for it
... create and distribute binary extensions?
--------------------------------------------
This is actually quite a complex topic, with a variety of alternatives
available depending on exactly what you're aiming to achieve. See the
Python Packaging User Guide for more information and recommendations.
.. seealso::
`Python Packaging User Guide: Binary Extensions
<http://packaging.python.org/en/latest/extensions.html>`__
.. other topics:
Once the Development & Deployment part of PPUG is fleshed out, some of
those sections should be linked from new questions here (most notably,
we should have a question about avoiding depending on PyPI that links to
http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
...@@ -853,17 +853,6 @@ Windows. It also contains the Mingw32CCompiler class which handles the mingw32 ...@@ -853,17 +853,6 @@ Windows. It also contains the Mingw32CCompiler class which handles the mingw32
port of GCC (same as cygwin in no-cygwin mode). port of GCC (same as cygwin in no-cygwin mode).
:mod:`distutils.emxccompiler` --- OS/2 EMX Compiler
===================================================
.. module:: distutils.emxccompiler
:synopsis: OS/2 EMX Compiler support
This module provides the EMXCCompiler class, a subclass of
:class:`UnixCCompiler` that handles the EMX port of the GNU C compiler to OS/2.
:mod:`distutils.archive_util` --- Archiving utilities :mod:`distutils.archive_util` --- Archiving utilities
====================================================== ======================================================
...@@ -1171,15 +1160,6 @@ other utility module. ...@@ -1171,15 +1160,6 @@ other utility module.
underscore. No { } or ( ) style quoting is available. underscore. No { } or ( ) style quoting is available.
.. function:: grok_environment_error(exc[, prefix='error: '])
Generate a useful error message from an :exc:`OSError` exception object.
Handles Python 1.5.1 and later styles, and does what it can to deal with
exception objects that don't have a filename (which happens when the error
is due to a two-file operation, such as :func:`~os.rename` or :func:`~os.link`).
Returns the error message as a string prefixed with *prefix*.
.. function:: split_quoted(s) .. function:: split_quoted(s)
Split a string up according to Unix shell-like rules for quotes and backslashes. Split a string up according to Unix shell-like rules for quotes and backslashes.
...@@ -1943,8 +1923,12 @@ Subclasses of :class:`Command` must define the following methods. ...@@ -1943,8 +1923,12 @@ Subclasses of :class:`Command` must define the following methods.
.. module:: distutils.command.clean .. module:: distutils.command.clean
:synopsis: Clean a package build area :synopsis: Clean a package build area
This command removes the temporary files created by :command:`build`
and its subcommands, like intermediary compiled object files. With
the ``--all`` option, the complete build directory will be removed.
.. % todo Extension modules built :ref:`in place <distutils-build-ext-inplace>`
will not be cleaned, as they are not in the build directory.
:mod:`distutils.command.config` --- Perform package configuration :mod:`distutils.command.config` --- Perform package configuration
......
...@@ -69,6 +69,8 @@ universal :option:`--help` option, e.g. :: ...@@ -69,6 +69,8 @@ universal :option:`--help` option, e.g. ::
Note that an option spelled :option:`--foo-bar` on the command-line is spelled Note that an option spelled :option:`--foo-bar` on the command-line is spelled
:option:`foo_bar` in configuration files. :option:`foo_bar` in configuration files.
.. _distutils-build-ext-inplace:
For example, say you want your extensions to be built "in-place"---that is, you For example, say you want your extensions to be built "in-place"---that is, you
have an extension :mod:`pkg.ext`, and you want the compiled extension file have an extension :mod:`pkg.ext`, and you want the compiled extension file
(:file:`ext.so` on Unix, say) to be put in the same source directory as your (:file:`ext.so` on Unix, say) to be put in the same source directory as your
......
.. _distutils-index: .. _distutils-index:
############################### ##############################################
Distributing Python Modules Distributing Python Modules (Legacy version)
############################### ##############################################
:Authors: Greg Ward, Anthony Baxter :Authors: Greg Ward, Anthony Baxter
:Email: distutils-sig@python.org :Email: distutils-sig@python.org
......
...@@ -685,6 +685,8 @@ include the following code fragment in your :file:`setup.py` before the ...@@ -685,6 +685,8 @@ include the following code fragment in your :file:`setup.py` before the
DistributionMetadata.download_url = None DistributionMetadata.download_url = None
.. _debug-setup-script:
Debugging the setup script Debugging the setup script
========================== ==========================
...@@ -700,7 +702,8 @@ installation is broken because they don't read all the way down to the bottom ...@@ -700,7 +702,8 @@ installation is broken because they don't read all the way down to the bottom
and see that it's a permission problem. and see that it's a permission problem.
On the other hand, this doesn't help the developer to find the cause of the On the other hand, this doesn't help the developer to find the cause of the
failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set
to anything except an empty string, and distutils will now print detailed to anything except an empty string, and distutils will now print detailed
information what it is doing, and prints the full traceback in case an exception information about what it is doing, dump the full traceback when an exception
occurs. occurs, and print the whole command line when an external program (like a C
compiler) fails.
...@@ -21,14 +21,31 @@ Python) that give the language its wide application range. ...@@ -21,14 +21,31 @@ Python) that give the language its wide application range.
For a detailed description of the whole Python/C API, see the separate For a detailed description of the whole Python/C API, see the separate
:ref:`c-api-index`. :ref:`c-api-index`.
.. note::
This guide only covers the basic tools for creating extensions provided Recommended third party tools
as part of this version of CPython. Third party tools may offer simpler =============================
alternatives. Refer to the `binary extensions section
<https://python-packaging-user-guide.readthedocs.org/en/latest/extensions.html>`__
in the Python Packaging User Guide for more information.
This guide only covers the basic tools for creating extensions provided
as part of this version of CPython. Third party tools like Cython,
``cffi``, SWIG and Numba offer both simpler and more sophisticated
approaches to creating C and C++ extensions for Python.
.. seealso::
`Python Packaging User Guide: Binary Extensions <https://packaging.python.org/en/latest/extensions.html>`_
The Python Packaging User Guide not only covers several available
tools that simplify the creation of binary extensions, but also
discusses the various reasons why creating an extension module may be
desirable in the first place.
Creating extensions without third party tools
=============================================
This section of the guide covers creating C and C++ extensions without
assistance from third party tools. It is intended primarily for creators
of those tools, rather than being a recommended way to create your own
C extensions.
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
...@@ -38,4 +55,17 @@ For a detailed description of the whole Python/C API, see the separate ...@@ -38,4 +55,17 @@ For a detailed description of the whole Python/C API, see the separate
newtypes.rst newtypes.rst
building.rst building.rst
windows.rst windows.rst
Embedding the CPython runtime in a larger application
=====================================================
Sometimes, rather than creating an extension that runs inside the Python
interpreter as the main application, it is desirable to instead embed
the CPython runtime inside a larger application. This section covers
some of the details involved in doing that successfully.
.. toctree::
:maxdepth: 2
:numbered:
embedding.rst embedding.rst
...@@ -471,7 +471,8 @@ that is written in Python using Tkinter. PythonWin is a Windows-specific IDE. ...@@ -471,7 +471,8 @@ that is written in Python using Tkinter. PythonWin is a Windows-specific IDE.
Emacs users will be happy to know that there is a very good Python mode for Emacs users will be happy to know that there is a very good Python mode for
Emacs. All of these programming environments provide syntax highlighting, Emacs. All of these programming environments provide syntax highlighting,
auto-indenting, and access to the interactive interpreter while coding. Consult auto-indenting, and access to the interactive interpreter while coding. Consult
http://www.python.org/editors/ for a full list of Python editing environments. `the Python wiki <https://wiki.python.org/moin/PythonEditors>`_ for a full list
of Python editing environments.
If you want to discuss Python's use in education, you may be interested in If you want to discuss Python's use in education, you may be interested in
joining `the edu-sig mailing list joining `the edu-sig mailing list
......
...@@ -513,6 +513,7 @@ For data that is more regular (e.g. a homogeneous list of ints or floats), ...@@ -513,6 +513,7 @@ For data that is more regular (e.g. a homogeneous list of ints or floats),
you can also use the :mod:`array` module. you can also use the :mod:`array` module.
.. note:: .. note::
To read and write binary data, it is mandatory to open the file in To read and write binary data, it is mandatory to open the file in
binary mode (here, passing ``"rb"`` to :func:`open`). If you use binary mode (here, passing ``"rb"`` to :func:`open`). If you use
``"r"`` instead (the default), the file will be open in text mode ``"r"`` instead (the default), the file will be open in text mode
......
...@@ -1103,6 +1103,7 @@ Use a list comprehension:: ...@@ -1103,6 +1103,7 @@ Use a list comprehension::
result = [obj.method() for obj in mylist] result = [obj.method() for obj in mylist]
.. _faq-augmented-assignment-tuple-error:
Why does a_tuple[i] += ['item'] raise an exception when the addition works? Why does a_tuple[i] += ['item'] raise an exception when the addition works?
--------------------------------------------------------------------------- ---------------------------------------------------------------------------
......
...@@ -783,6 +783,14 @@ Glossary ...@@ -783,6 +783,14 @@ Glossary
mapping rather than a sequence because the lookups use arbitrary mapping rather than a sequence because the lookups use arbitrary
:term:`immutable` keys rather than integers. :term:`immutable` keys rather than integers.
The :class:`collections.abc.Sequence` abstract base class
defines a much richer interface that goes beyond just
:meth:`__getitem__` and :meth:`__len__`, adding :meth:`count`,
:meth:`index`, :meth:`__contains__`, and
:meth:`__reversed__`. Types that implement this expanded
interface can be registered explicitly using
:func:`~abc.register`.
single dispatch single dispatch
A form of :term:`generic function` dispatch where the implementation is A form of :term:`generic function` dispatch where the implementation is
chosen based on the type of a single argument. chosen based on the type of a single argument.
......
...@@ -26,6 +26,32 @@ Porting Python 2 Code to Python 3 ...@@ -26,6 +26,32 @@ Porting Python 2 Code to Python 3
For help with porting, you can email the python-porting_ mailing list with For help with porting, you can email the python-porting_ mailing list with
questions. questions.
The Short Version
=================
* Decide what's the oldest version of Python 2 you want to support (if at all)
* Make sure you have a thorough test suite and use continuous integration
testing to make sure you stay compatible with the versions of Python you care
about
* If you have dependencies, check their Python 3 status using caniusepython3
(`command-line tool <https://pypi.python.org/pypi/caniusepython3>`__,
`web app <https://caniusepython3.com/>`__)
With that done, your options are:
* If you are dropping Python 2 support, use 2to3_ to port to Python 3
* If you are keeping Python 2 support, then start writing Python 2/3-compatible
code starting **TODAY**
+ If you have dependencies that have not been ported, reach out to them to port
their project while working to make your code compatible with Python 3 so
you're ready when your dependencies are all ported
+ If all your dependencies have been ported (or you have none), go ahead and
port to Python 3
* If you are creating a new project that wants to have 2/3 compatibility,
code in Python 3 and then backport to Python 2
Before You Begin Before You Begin
================ ================
...@@ -85,7 +111,7 @@ between Python 2 and 3 easier. ...@@ -85,7 +111,7 @@ between Python 2 and 3 easier.
Projects to Consider Projects to Consider
-------------------- --------------------
The lowest level library for suppoting Python 2 & 3 simultaneously is six_. The lowest level library for supporting Python 2 & 3 simultaneously is six_.
Reading through its documentation will give you an idea of where exactly the Reading through its documentation will give you an idea of where exactly the
Python language changed between versions 2 & 3 and thus what you will want the Python language changed between versions 2 & 3 and thus what you will want the
library to help you continue to support. library to help you continue to support.
...@@ -548,7 +574,10 @@ Backporting Python 3 code to Python 2 ...@@ -548,7 +574,10 @@ Backporting Python 3 code to Python 2
If you have Python 3 code and have little interest in supporting Python 2 you If you have Python 3 code and have little interest in supporting Python 2 you
can use 3to2_ to translate from Python 3 code to Python 2 code. This is only can use 3to2_ to translate from Python 3 code to Python 2 code. This is only
recommended if you don't plan to heavily support Python 2 users. recommended if you don't plan to heavily support Python 2 users. Otherwise
write your code for Python 3 and then backport as far back as you want. This
is typically easier than going from Python 2 to 3 as you will have worked out
any difficulties with e.g. bytes/strings, etc.
Other Resources Other Resources
......
...@@ -19,7 +19,7 @@ Cela ressemble à un excellent recipie[1] déjeuner. ...@@ -19,7 +19,7 @@ Cela ressemble à un excellent recipie[1] déjeuner.
[1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718 [1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718
--Éric --Pepé
""") """)
# Add the html version. This converts the message into a multipart/alternative # Add the html version. This converts the message into a multipart/alternative
......
...@@ -2,9 +2,9 @@ ...@@ -2,9 +2,9 @@
.. _install-index: .. _install-index:
***************************** ********************************************
Installing Python Modules Installing Python Modules (Legacy version)
***************************** ********************************************
:Author: Greg Ward :Author: Greg Ward
...@@ -58,7 +58,8 @@ new goodies to their toolbox. You don't need to know Python to read this ...@@ -58,7 +58,8 @@ new goodies to their toolbox. You don't need to know Python to read this
document; there will be some brief forays into using Python's interactive mode document; there will be some brief forays into using Python's interactive mode
to explore your installation, but that's it. If you're looking for information to explore your installation, but that's it. If you're looking for information
on how to distribute your own Python modules so that others may use them, see on how to distribute your own Python modules so that others may use them, see
the :ref:`distutils-index` manual. the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of
interest.
.. _inst-trivial-install: .. _inst-trivial-install:
......
.. highlightlang:: none
.. _installing-index:
*****************************
Installing Python Modules
*****************************
:Email: distutils-sig@python.org
As a popular open source development project, Python has an active
supporting community of contributors and users that also make their software
available for other Python developers to use under open source license terms.
This allows Python users to share and collaborate effectively, benefiting
from the solutions others have already created to common (and sometimes
even rare!) problems, as well as potentially contributing their own
solutions to the common pool.
This guide covers the installation part of the process. For a guide to
creating and sharing your own Python projects, refer to the
:ref:`distribution guide <distributing-index>`.
.. note::
For corporate and other institutional users, be aware that many
organisations have their own policies around using and contributing to
open source software. Please take such policies into account when making
use of the distribution and installation tools provided with Python.
Key terms
=========
* ``pip`` is the preferred installer program. Starting with Python 3.4, it
is included by default with the Python binary installers.
* a virtual environment is a semi-isolated Python environment that allows
packages to be installed for use by a particular application, rather than
being installed system wide
* ``pyvenv`` is the standard tool for creating virtual environments, and has
been part of Python since Python 3.3. Starting with Python 3.4, it
defaults to installing ``pip`` into all created virtual environments
* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
repository of open source licensed packages made available for use by
other Python users
* the `Python Packaging Authority
<http://packaging.python.org/en/latest/future.html>`__ are the group of
developers and documentation authors responsible for the maintenance and
evolution of the standard packaging tools and the associated metadata and
file format standards. They maintain a variety of tools, documentation
and issue trackers on both `GitHub <https://github.com/pypa>`__ and
`BitBucket <https://bitbucket.org/pypa/>`__.
* ``distutils`` is the original build and distribution system first added to
the Python standard library in 1998. While direct use of ``distutils`` is
being phased out, it still laid the foundation for the current packaging
and distribution infrastructure, and it not only remains part of the
standard library, but its name lives on in other ways (such as the name
of the mailing list used to coordinate Python packaging standards
development).
Basic usage
===========
The standard packaging tools are all designed to be used from the command
line. For Windows users, the examples below assume that the option to
adjust the system PATH environment variable was selected when installing
Python. For Linux users, the command to install into the system version of
Python 3 is likely to be ``pip3`` rather than ``pip``.
The following command will install the latest version of a module and its
dependencies from the Python Package Index::
pip install SomePackage
It's also possible to specify an exact or minimum version directly on the
command line::
pip install SomePackage==1.0.4 # specific version
pip install 'SomePackage>=1.0.4' # minimum version
Normally, if a suitable module is already installed, attempting to install
it again will have no effect. Upgrading existing modules must be requested
explicitly::
pip install --upgrade SomePackage
More information and resources regarding ``pip`` and its capabilities can be
found in the `Python Packaging User Guide <http://packaging.python.org>`__.
``pyvenv`` has its own documentation at :ref:`scripts-pyvenv`. Installing
into an active virtual environment uses the commands shown above.
.. seealso::
`Python Packaging User Guide: Installing Python packages
<http://packaging.python.org/en/latest/tutorial.html#installing-python-packages>`__
How do I ...?
=============
These are quick answers or links for some common tasks.
... install ``pip`` in versions of Python prior to Python 3.4?
--------------------------------------------------------------
Python only started bundling ``pip`` with Python 3.4. For earlier versions,
``pip`` needs to be "bootstrapped" as described in the Python Packaging
User Guide.
.. seealso::
`Python Packaging User Guide: Installing the Tools
<http://packaging.python.org/en/latest/tutorial.html#installing-the-tools>`__
.. installing-per-user-installation:
... install packages just for the current user?
-----------------------------------------------
Passing the ``--user`` option to ``pip install`` will install a package
just for the current user, rather than for all users of the system.
... install scientific Python packages?
---------------------------------------
A number of scientific Python packages have complex binary dependencies, and
aren't currently easy to install using ``pip`` directly. At this point in
time, it will often be easier for users to install these packages by
`other means
<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
rather than attempting to install them with ``pip``.
.. seealso::
`Python Packaging User Guide: Installing Scientific Packages
<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
... work with multiple versions of Python installed in parallel?
----------------------------------------------------------------
On Linux, Mac OS X and other POSIX systems, use the versioned Python commands
in combination with the ``-m`` switch to run the appropriate copy of
``pip``::
python2 -m pip install SomePackage # default Python 2
python2.7 -m pip install SomePackage # specifically Python 2.7
python3 -m pip install SomePackage # default Python 3
python3.4 -m pip install SomePackage # specifically Python 3.4
(appropriately versioned ``pip`` commands may also be available)
On Windows, use the ``py`` Python launcher in combination with the ``-m``
switch::
py -2 -m pip install SomePackage # default Python 2
py -2.7 -m pip install SomePackage # specifically Python 2.7
py -3 -m pip install SomePackage # default Python 3
py -3.4 -m pip install SomePackage # specifically Python 3.4
.. other questions:
Once the Development & Deployment part of PPUG is fleshed out, some of
those sections should be linked from new questions here (most notably,
we should have a question about avoiding depending on PyPI that links to
http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
Common installation issues
==========================
Installing into the system Python on Linux
------------------------------------------
On Linux systems, a Python installation will typically be included as part
of the distribution. Installing into this Python installation requires
root access to the system, and may interfere with the operation of the
system package manager and other components of the system if a component
is unexpectedly upgraded using ``pip``.
On such systems, it is often better to use a virtual environment or a
per-user installation when installing packages with ``pip``.
Installing binary extensions
----------------------------
Python has typically relied heavily on source based distribution, with end
users being expected to compile extension modules from source as part of
the installation process.
With the introduction of support for the binary ``wheel`` format, and the
ability to publish wheels for at least Windows and Mac OS X through the
Python Package Index, this problem is expected to diminish over time,
as users are more regularly able to install pre-built extensions rather
than needing to build them themselves.
Some of the solutions for installing `scientific software
<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
that is not yet available as pre-built ``wheel`` files may also help with
obtaining other binary extensions without needing to build them locally.
.. seealso::
`Python Packaging User Guide: Binary Extensions
<http://packaging.python.org/en/latest/extensions.html>`__
...@@ -226,7 +226,7 @@ number of frames must be filled in. ...@@ -226,7 +226,7 @@ number of frames must be filled in.
file parameters have been set. file parameters have been set.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
.. method:: aifc.writeframesraw(data) .. method:: aifc.writeframesraw(data)
...@@ -235,7 +235,7 @@ number of frames must be filled in. ...@@ -235,7 +235,7 @@ number of frames must be filled in.
updated. updated.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
.. method:: aifc.close() .. method:: aifc.close()
......
...@@ -488,7 +488,7 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. ...@@ -488,7 +488,7 @@ Run subprocesses asynchronously using the :mod:`subprocess` module.
* *stdout*: Either a file-like object representing the pipe to be connected * *stdout*: Either a file-like object representing the pipe to be connected
to the subprocess's standard output stream using to the subprocess's standard output stream using
:meth:`~BaseEventLoop.connect_write_pipe`, or the constant :meth:`~BaseEventLoop.connect_read_pipe`, or the constant
:const:`subprocess.PIPE` (the default). By default a new pipe will be :const:`subprocess.PIPE` (the default). By default a new pipe will be
created and connected. created and connected.
...@@ -589,8 +589,8 @@ pool of processes). By default, an event loop uses a thread pool executor ...@@ -589,8 +589,8 @@ pool of processes). By default, an event loop uses a thread pool executor
Arrange for a callback to be called in the specified executor. Arrange for a callback to be called in the specified executor.
*executor* is a :class:`~concurrent.futures.Executor` instance, The *executor* argument should be an :class:`~concurrent.futures.Executor`
the default executor is used if *executor* is ``None``. instance. The default executor is used if *executor* is ``None``.
This method is a :ref:`coroutine <coroutine>`. This method is a :ref:`coroutine <coroutine>`.
......
...@@ -376,8 +376,8 @@ The following callbacks are called on :class:`DatagramProtocol` instances. ...@@ -376,8 +376,8 @@ The following callbacks are called on :class:`DatagramProtocol` instances.
Flow control callbacks Flow control callbacks
---------------------- ----------------------
These callbacks may be called on :class:`Protocol` and These callbacks may be called on :class:`Protocol`,
:class:`SubprocessProtocol` instances: :class:`DatagramProtocol` and :class:`SubprocessProtocol` instances:
.. method:: BaseProtocol.pause_writing() .. method:: BaseProtocol.pause_writing()
...@@ -402,6 +402,15 @@ buffer size reaches the low-water mark. ...@@ -402,6 +402,15 @@ buffer size reaches the low-water mark.
are important to ensure that things go as expected when either are important to ensure that things go as expected when either
mark is zero. mark is zero.
.. note::
On BSD systems (OS X, FreeBSD, etc.) flow control is not supported
for :class:`DatagramProtocol`, because send failures caused by
writing too many packets cannot be detected easily. The socket
always appears 'ready' and excess packets are dropped; an
:class:`OSError` with errno set to :const:`errno.ENOBUFS` may or
may not be raised; if it is raised, it will be reported to
:meth:`DatagramProtocol.error_received` but otherwise ignored.
Coroutines and protocols Coroutines and protocols
------------------------ ------------------------
...@@ -488,6 +497,6 @@ TCP echo server example, send back received data and close the connection:: ...@@ -488,6 +497,6 @@ TCP echo server example, send back received data and close the connection::
:meth:`Transport.close` can be called immediately after :meth:`Transport.close` can be called immediately after
:meth:`WriteTransport.write` even if data are not sent yet on the socket: both :meth:`WriteTransport.write` even if data are not sent yet on the socket: both
methods are asynchronous. ``yield from`` is not needed because these transport methods are asynchronous. ``yield from`` is not needed because these transport
methods don't return coroutines. methods are not coroutines.
...@@ -8,16 +8,16 @@ Create a subprocess ...@@ -8,16 +8,16 @@ Create a subprocess
.. function:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) .. function:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
Run the shell command *cmd* given as a string. Return a :class:`Process` Run the shell command *cmd* given as a string. Return a :class:`~asyncio.subprocess.Process`
instance. instance.
This function returns a :ref:`coroutine object <coroutine>`. This function is a :ref:`coroutine <coroutine>`.
.. function:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) .. function:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
Create a subprocess. Return a :class:`Process` instance. Create a subprocess. Return a :class:`~asyncio.subprocess.Process` instance.
This function returns a :ref:`coroutine object <coroutine>`. This function is a :ref:`coroutine <coroutine>`.
Use the :meth:`BaseEventLoop.connect_read_pipe` and Use the :meth:`BaseEventLoop.connect_read_pipe` and
:meth:`BaseEventLoop.connect_write_pipe` methods to connect pipes. :meth:`BaseEventLoop.connect_write_pipe` methods to connect pipes.
......
...@@ -64,7 +64,7 @@ Lock ...@@ -64,7 +64,7 @@ Lock
.. method:: locked() .. method:: locked()
Return ``True`` if lock is acquired. Return ``True`` if the lock is acquired.
.. method:: acquire() .. method:: acquire()
...@@ -73,7 +73,7 @@ Lock ...@@ -73,7 +73,7 @@ Lock
This method blocks until the lock is unlocked, then sets it to locked and This method blocks until the lock is unlocked, then sets it to locked and
returns ``True``. returns ``True``.
This method returns a :ref:`coroutine object <coroutine>`. This method is a :ref:`coroutine <coroutine>`.
.. method:: release() .. method:: release()
...@@ -141,6 +141,15 @@ Condition ...@@ -141,6 +141,15 @@ Condition
A new :class:`Lock` object is created and used as the underlying lock. A new :class:`Lock` object is created and used as the underlying lock.
.. method:: acquire()
Acquire the underlying lock.
This method blocks until the lock is unlocked, then sets it to locked and
returns ``True``.
This method is a :ref:`coroutine <coroutine>`.
.. method:: notify(n=1) .. method:: notify(n=1)
By default, wake up one coroutine waiting on this condition, if any. By default, wake up one coroutine waiting on this condition, if any.
...@@ -156,6 +165,10 @@ Condition ...@@ -156,6 +165,10 @@ Condition
call until it can reacquire the lock. Since :meth:`notify` does not call until it can reacquire the lock. Since :meth:`notify` does not
release the lock, its caller should. release the lock, its caller should.
.. method:: locked()
Return ``True`` if the underlying lock is acquired.
.. method:: notify_all() .. method:: notify_all()
Wake up all threads waiting on this condition. This method acts like Wake up all threads waiting on this condition. This method acts like
...@@ -163,6 +176,18 @@ Condition ...@@ -163,6 +176,18 @@ Condition
calling thread has not acquired the lock when this method is called, a calling thread has not acquired the lock when this method is called, a
:exc:`RuntimeError` is raised. :exc:`RuntimeError` is raised.
.. method:: release()
Release the underlying lock.
When the lock is locked, reset it to unlocked, and return. If any other
coroutines are blocked waiting for the lock to become unlocked, allow
exactly one of them to proceed.
When invoked on an unlocked lock, a :exc:`RuntimeError` is raised.
There is no return value.
.. method:: wait() .. method:: wait()
Wait until notified. Wait until notified.
......
...@@ -12,10 +12,8 @@ integers, unless specified otherwise. ...@@ -12,10 +12,8 @@ integers, unless specified otherwise.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Support for 24-bit samples was added. Support for 24-bit samples was added.
All functions now accept any :term:`bytes-like object`.
.. versionchanged:: 3.4 String input now results in an immediate error.
Any :term:`bytes-like object`\ s are now accepted by all functions in this
module. Strings no more supported.
.. index:: .. index::
single: Intel/DVI ADPCM single: Intel/DVI ADPCM
...@@ -82,7 +80,7 @@ The module defines the following variables and functions: ...@@ -82,7 +80,7 @@ The module defines the following variables and functions:
"Byteswap" all samples in a fragment and returns the modified fragment. "Byteswap" all samples in a fragment and returns the modified fragment.
Converts big-endian samples to little-endian and vice versa. Converts big-endian samples to little-endian and vice versa.
.. versionadded: 3.4 .. versionadded:: 3.4
.. function:: cross(fragment, width) .. function:: cross(fragment, width)
......
:mod:`base64` --- RFC 3548: Base16, Base32, Base64 Data Encodings :mod:`base64` --- Base16, Base32, Base64, Base85 Data Encodings
================================================================= ===============================================================
.. module:: base64 .. module:: base64
:synopsis: RFC 3548: Base16, Base32, Base64 Data Encodings :synopsis: RFC 3548: Base16, Base32, Base64 Data Encodings;
Base85 and Ascii85
.. index:: .. index::
pair: base64; encoding pair: base64; encoding
single: MIME; base64 encoding single: MIME; base64 encoding
This module provides data encoding and decoding as specified in :rfc:`3548`. This module provides functions for encoding binary data to printable
This standard defines the Base16, Base32, and Base64 algorithms for encoding ASCII characters and decoding such encodings back to binary data.
and decoding arbitrary binary strings into ASCII-only byte strings that can be It provides encoding and decoding functions for the encodings specified in
in :rfc:`3548`, which defines the Base16, Base32, and Base64 algorithms,
and for the de-facto standard Ascii85 and Base85 encodings.
The :rfc:`3548` encodings are suitable for encoding binary data so that it can
safely sent by email, used as parts of URLs, or included as part of an HTTP safely sent by email, used as parts of URLs, or included as part of an HTTP
POST request. The encoding algorithm is not the same as the POST request. The encoding algorithm is not the same as the
:program:`uuencode` program. :program:`uuencode` program.
There are two interfaces provided by this module. The modern interface There are two :rfc:`3548` interfaces provided by this module. The modern
supports encoding and decoding ASCII byte string objects using all three interface supports encoding and decoding ASCII byte string objects using all
alphabets. Additionally, the decoding functions of the modern interface also three :rfc:`3548` defined alphabets (normal, URL-safe, and filesystem-safe).
accept Unicode strings containing only ASCII characters. The legacy interface Additionally, the decoding functions of the modern interface also accept
provides for encoding and decoding to and from file-like objects as well as Unicode strings containing only ASCII characters. The legacy interface provides
byte strings, but only using the Base64 standard alphabet. for encoding and decoding to and from file-like objects as well as byte
strings, but only using the Base64 standard alphabet.
.. versionchanged:: 3.3 .. versionchanged:: 3.3
ASCII-only Unicode strings are now accepted by the decoding functions of ASCII-only Unicode strings are now accepted by the decoding functions of
...@@ -29,7 +35,7 @@ byte strings, but only using the Base64 standard alphabet. ...@@ -29,7 +35,7 @@ byte strings, but only using the Base64 standard alphabet.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted by all Any :term:`bytes-like object`\ s are now accepted by all
encoding and decoding functions in this module. encoding and decoding functions in this module. Ascii85/Base85 support added.
The modern interface provides: The modern interface provides:
......
...@@ -142,36 +142,43 @@ The :mod:`csv` module defines the following functions: ...@@ -142,36 +142,43 @@ The :mod:`csv` module defines the following functions:
The :mod:`csv` module defines the following classes: The :mod:`csv` module defines the following classes:
.. class:: DictReader(csvfile, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds) .. class:: DictReader(csvfile, fieldnames=None, restkey=None, restval=None, \
dialect='excel', *args, **kwds)
Create an object which operates like a regular reader but maps the information
read into a dict whose keys are given by the optional *fieldnames* parameter. Create an object which operates like a regular reader but maps the
If the *fieldnames* parameter is omitted, the values in the first row of the information read into a dict whose keys are given by the optional
*csvfile* will be used as the fieldnames. If the row read has more fields *fieldnames* parameter. The *fieldnames* parameter is a :mod:`sequence
than the fieldnames sequence, the remaining data is added as a sequence <collections.abc>` whose elements are associated with the fields of the
keyed by the value of *restkey*. If the row read has fewer fields than the input data in order. These elements become the keys of the resulting
fieldnames sequence, the remaining keys take the value of the optional dictionary. If the *fieldnames* parameter is omitted, the values in the
*restval* parameter. Any other optional or keyword arguments are passed to first row of the *csvfile* will be used as the fieldnames. If the row read
the underlying :class:`reader` instance. has more fields than the fieldnames sequence, the remaining data is added as
a sequence keyed by the value of *restkey*. If the row read has fewer
fields than the fieldnames sequence, the remaining keys take the value of
.. class:: DictWriter(csvfile, fieldnames, restval='', extrasaction='raise', dialect='excel', *args, **kwds) the optional *restval* parameter. Any other optional or keyword arguments
are passed to the underlying :class:`reader` instance.
Create an object which operates like a regular writer but maps dictionaries onto
output rows. The *fieldnames* parameter identifies the order in which values in
the dictionary passed to the :meth:`writerow` method are written to the .. class:: DictWriter(csvfile, fieldnames, restval='', extrasaction='raise', \
*csvfile*. The optional *restval* parameter specifies the value to be written dialect='excel', *args, **kwds)
if the dictionary is missing a key in *fieldnames*. If the dictionary passed to
the :meth:`writerow` method contains a key not found in *fieldnames*, the Create an object which operates like a regular writer but maps dictionaries
optional *extrasaction* parameter indicates what action to take. If it is set onto output rows. The *fieldnames* parameter is a :mod:`sequence
to ``'raise'`` a :exc:`ValueError` is raised. If it is set to ``'ignore'``, <collections.abc>` of keys that identify the order in which values in the
extra values in the dictionary are ignored. Any other optional or keyword dictionary passed to the :meth:`writerow` method are written to the
arguments are passed to the underlying :class:`writer` instance. *csvfile*. The optional *restval* parameter specifies the value to be
written if the dictionary is missing a key in *fieldnames*. If the
Note that unlike the :class:`DictReader` class, the *fieldnames* parameter of dictionary passed to the :meth:`writerow` method contains a key not found in
the :class:`DictWriter` is not optional. Since Python's :class:`dict` objects *fieldnames*, the optional *extrasaction* parameter indicates what action to
are not ordered, there is not enough information available to deduce the order take. If it is set to ``'raise'`` a :exc:`ValueError` is raised. If it is
in which the row should be written to the *csvfile*. set to ``'ignore'``, extra values in the dictionary are ignored. Any other
optional or keyword arguments are passed to the underlying :class:`writer`
instance.
Note that unlike the :class:`DictReader` class, the *fieldnames* parameter
of the :class:`DictWriter` is not optional. Since Python's :class:`dict`
objects are not ordered, there is not enough information available to deduce
the order in which the row should be written to the *csvfile*.
.. class:: Dialect .. class:: Dialect
......
...@@ -12,7 +12,7 @@ The :mod:`curses` module provides an interface to the curses library, the ...@@ -12,7 +12,7 @@ The :mod:`curses` module provides an interface to the curses library, the
de-facto standard for portable advanced terminal handling. de-facto standard for portable advanced terminal handling.
While curses is most widely used in the Unix environment, versions are available While curses is most widely used in the Unix environment, versions are available
for DOS, OS/2, and possibly other systems as well. This extension module is for Windows, DOS, and possibly other systems as well. This extension module is
designed to match the API of ncurses, an open-source curses library hosted on designed to match the API of ncurses, an open-source curses library hosted on
Linux and the BSD variants of Unix. Linux and the BSD variants of Unix.
......
...@@ -12,14 +12,15 @@ additional modules into a Python installation. The new modules may be either ...@@ -12,14 +12,15 @@ additional modules into a Python installation. The new modules may be either
100%-pure Python, or may be extension modules written in C, or may be 100%-pure Python, or may be extension modules written in C, or may be
collections of Python packages which include modules coded in both Python and C. collections of Python packages which include modules coded in both Python and C.
Most Python users will *not* want to use this module directly, but instead
use the cross-version tools maintained by the Python Packaging Authority.
Refer to the `Python Packaging User Guide <http://packaging.python.org>`_
for more information.
User documentation and API reference are provided in another document: For the benefits of packaging tool authors and users seeking a deeper
understanding of the details of the current packaging and distribution
system, the legacy :mod:`distutils` based user documentation and API
reference remain available:
.. seealso:: * :ref:`install-index`
* :ref:`distutils-index`
:ref:`distutils-index`
The manual for developers and packagers of Python modules. This describes
how to prepare :mod:`distutils`\ -based packages so that they may be
easily installed into an existing Python installation. It also contains
instructions for end-users wanting to install a distutils-based package,
:ref:`install-index`.
...@@ -40,8 +40,10 @@ text version: [2]_ ...@@ -40,8 +40,10 @@ text version: [2]_
.. literalinclude:: ../includes/email-alternative.py .. literalinclude:: ../includes/email-alternative.py
Examples using the Provision API .. _email-contentmanager-api-examples:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Examples using the Provisional API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is a reworking of the last example using the provisional API. To make Here is a reworking of the last example using the provisional API. To make
things a bit more interesting, we include a related image in the html part, and things a bit more interesting, we include a related image in the html part, and
we save a copy of what we are going to send to disk, as well as sending it. we save a copy of what we are going to send to disk, as well as sending it.
......
...@@ -54,6 +54,7 @@ this module. ...@@ -54,6 +54,7 @@ this module.
documented in this module because of the provisional nature of the code, the documented in this module because of the provisional nature of the code, the
implementation lives in the :mod:`email.message` module. implementation lives in the :mod:`email.message` module.
.. currentmodule:: email.message
.. class:: EmailMessage(policy=default) .. class:: EmailMessage(policy=default)
...@@ -235,6 +236,16 @@ this module. ...@@ -235,6 +236,16 @@ this module.
all other headers intact and in their original order. all other headers intact and in their original order.
.. class:: MIMEPart(policy=default)
This class represents a subpart of a MIME message. It is identical to
:class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are
added when :meth:`~EmailMessage.set_content` is called, since sub-parts do
not need their own :mailheader:`MIME-Version` headers.
.. currentmodule:: email.contentmanager
.. class:: ContentManager() .. class:: ContentManager()
Base class for content managers. Provides the standard registry mechanisms Base class for content managers. Provides the standard registry mechanisms
...@@ -305,14 +316,6 @@ this module. ...@@ -305,14 +316,6 @@ this module.
values of *typekey*, see :meth:`set_content`. values of *typekey*, see :meth:`set_content`.
.. class:: MIMEPart(policy=default)
This class represents a subpart of a MIME message. It is identical to
:class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are
added when :meth:`~EmailMessage.set_content` is called, since sub-parts do
not need their own :mailheader:`MIME-Version` headers.
Content Manager Instances Content Manager Instances
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
......
...@@ -466,7 +466,7 @@ Here are the methods of the :class:`Message` class: ...@@ -466,7 +466,7 @@ Here are the methods of the :class:`Message` class:
to ``False``. to ``False``.
.. method:: set_param(param, value, header='Content-Type', requote=True, .. method:: set_param(param, value, header='Content-Type', requote=True, \
charset=None, language='', replace=False) charset=None, language='', replace=False)
Set a parameter in the :mailheader:`Content-Type` header. If the Set a parameter in the :mailheader:`Content-Type` header. If the
...@@ -488,7 +488,7 @@ Here are the methods of the :class:`Message` class: ...@@ -488,7 +488,7 @@ Here are the methods of the :class:`Message` class:
end of the list of headers. If *replace* is ``True``, the header end of the list of headers. If *replace* is ``True``, the header
will be updated in place. will be updated in place.
.. versionchanged: 3.4 ``replace`` keyword was added. .. versionchanged:: 3.4 ``replace`` keyword was added.
.. method:: del_param(param, header='content-type', requote=True) .. method:: del_param(param, header='content-type', requote=True)
......
...@@ -419,7 +419,7 @@ added matters. To illustrate:: ...@@ -419,7 +419,7 @@ added matters. To illustrate::
additional arguments. By default ``content_manager`` is set to additional arguments. By default ``content_manager`` is set to
:data:`~email.contentmanager.raw_data_manager`. :data:`~email.contentmanager.raw_data_manager`.
.. versionadded 3.4 .. versionadded:: 3.4
The class provides the following concrete implementations of the abstract The class provides the following concrete implementations of the abstract
......
...@@ -28,7 +28,7 @@ when creating a virtual environment) or after explicitly uninstalling ...@@ -28,7 +28,7 @@ when creating a virtual environment) or after explicitly uninstalling
.. seealso:: .. seealso::
:ref:`install-index` :ref:`installing-index`
The end user guide for installing Python packages The end user guide for installing Python packages
:pep:`453`: Explicit bootstrapping of pip in Python installations :pep:`453`: Explicit bootstrapping of pip in Python installations
......
...@@ -139,7 +139,7 @@ If you want to access enum members by *name*, use item access:: ...@@ -139,7 +139,7 @@ If you want to access enum members by *name*, use item access::
>>> Color['green'] >>> Color['green']
<Color.green: 2> <Color.green: 2>
If have an enum member and need its :attr:`name` or :attr:`value`:: If you have an enum member and need its :attr:`name` or :attr:`value`::
>>> member = Color.red >>> member = Color.red
>>> member.name >>> member.name
......
...@@ -265,7 +265,6 @@ The following exceptions are the exceptions that are usually raised. ...@@ -265,7 +265,6 @@ The following exceptions are the exceptions that are usually raised.
:exc:`mmap.error` have been merged into :exc:`OSError`. :exc:`mmap.error` have been merged into :exc:`OSError`.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
The :attr:`filename` attribute is now the original file name passed to The :attr:`filename` attribute is now the original file name passed to
the function, instead of the name encoded to or decoded from the the function, instead of the name encoded to or decoded from the
filesystem encoding. Also, the :attr:`filename2` attribute was added. filesystem encoding. Also, the :attr:`filename2` attribute was added.
......
...@@ -99,7 +99,9 @@ another rational number, or from a string. ...@@ -99,7 +99,9 @@ another rational number, or from a string.
value of *flt*, which must be a :class:`float`. Beware that value of *flt*, which must be a :class:`float`. Beware that
``Fraction.from_float(0.3)`` is not the same value as ``Fraction(3, 10)`` ``Fraction.from_float(0.3)`` is not the same value as ``Fraction(3, 10)``
.. note:: From Python 3.2 onwards, you can also construct a .. note::
From Python 3.2 onwards, you can also construct a
:class:`Fraction` instance directly from a :class:`float`. :class:`Fraction` instance directly from a :class:`float`.
...@@ -108,7 +110,9 @@ another rational number, or from a string. ...@@ -108,7 +110,9 @@ another rational number, or from a string.
This class method constructs a :class:`Fraction` representing the exact This class method constructs a :class:`Fraction` representing the exact
value of *dec*, which must be a :class:`decimal.Decimal` instance. value of *dec*, which must be a :class:`decimal.Decimal` instance.
.. note:: From Python 3.2 onwards, you can also construct a .. note::
From Python 3.2 onwards, you can also construct a
:class:`Fraction` instance directly from a :class:`decimal.Decimal` :class:`Fraction` instance directly from a :class:`decimal.Decimal`
instance. instance.
......
...@@ -540,12 +540,13 @@ are always available. They are listed here in alphabetical order. ...@@ -540,12 +540,13 @@ are always available. They are listed here in alphabetical order.
A call to ``format(value, format_spec)`` is translated to A call to ``format(value, format_spec)`` is translated to
``type(value).__format__(format_spec)`` which bypasses the instance ``type(value).__format__(format_spec)`` which bypasses the instance
dictionary when searching for the value's :meth:`__format__` method. A dictionary when searching for the value's :meth:`__format__` method. A
:exc:`TypeError` exception is raised if the method is not found or if either :exc:`TypeError` exception is raised if the method search reaches
the *format_spec* or the return value are not strings. :mod:`object` and the *format_spec* is non-empty, or if either the
*format_spec* or the return value are not strings.
.. versionadded:: 3.4 .. versionchanged:: 3.4
``object().__format__(format_spec)`` raises :exc:`TypeError` ``object().__format__(format_spec)`` raises :exc:`TypeError`
if *format_spec* is not empty string. if *format_spec* is not an empty string.
.. _func-frozenset: .. _func-frozenset:
...@@ -609,12 +610,26 @@ are always available. They are listed here in alphabetical order. ...@@ -609,12 +610,26 @@ are always available. They are listed here in alphabetical order.
This function is added to the built-in namespace by the :mod:`site` module. This function is added to the built-in namespace by the :mod:`site` module.
.. versionchanged:: 3.4
Changes to :mod:`pydoc` and :mod:`inspect` mean that the reported
signatures for callables are now more comprehensive and consistent.
.. function:: hex(x) .. function:: hex(x)
Convert an integer number to a hexadecimal string. The result is a valid Python Convert an integer number to a lowercase hexadecimal string
expression. If *x* is not a Python :class:`int` object, it has to define an prefixed with "0x", for example:
:meth:`__index__` method that returns an integer.
>>> hex(255)
'0xff'
>>> hex(-42)
'-0x2a'
If x is not a Python :class:`int` object, it has to define an __index__()
method that returns an integer.
See also :func:`int` for converting a hexadecimal string to an
integer using a base of 16.
.. note:: .. note::
...@@ -780,6 +795,9 @@ are always available. They are listed here in alphabetical order. ...@@ -780,6 +795,9 @@ are always available. They are listed here in alphabetical order.
such as ``sorted(iterable, key=keyfunc, reverse=True)[0]`` and such as ``sorted(iterable, key=keyfunc, reverse=True)[0]`` and
``heapq.nlargest(1, iterable, key=keyfunc)``. ``heapq.nlargest(1, iterable, key=keyfunc)``.
.. versionadded:: 3.4
The *default* keyword-only argument.
.. _func-memoryview: .. _func-memoryview:
.. function:: memoryview(obj) .. function:: memoryview(obj)
...@@ -811,6 +829,9 @@ are always available. They are listed here in alphabetical order. ...@@ -811,6 +829,9 @@ are always available. They are listed here in alphabetical order.
such as ``sorted(iterable, key=keyfunc)[0]`` and ``heapq.nsmallest(1, such as ``sorted(iterable, key=keyfunc)[0]`` and ``heapq.nsmallest(1,
iterable, key=keyfunc)``. iterable, key=keyfunc)``.
.. versionadded:: 3.4
The *default* keyword-only argument.
.. function:: next(iterator[, default]) .. function:: next(iterator[, default])
Retrieve the next item from the *iterator* by calling its Retrieve the next item from the *iterator* by calling its
......
...@@ -10,6 +10,7 @@ ...@@ -10,6 +10,7 @@
-------------- --------------
.. note:: .. note::
The :mod:`getopt` module is a parser for command line options whose API is The :mod:`getopt` module is a parser for command line options whose API is
designed to be familiar to users of the C :c:func:`getopt` function. Users who designed to be familiar to users of the C :c:func:`getopt` function. Users who
are unfamiliar with the C :c:func:`getopt` function or who would like to write are unfamiliar with the C :c:func:`getopt` function or who would like to write
......
...@@ -101,18 +101,18 @@ Hashlib provides the following constant attributes: ...@@ -101,18 +101,18 @@ Hashlib provides the following constant attributes:
.. data:: algorithms_guaranteed .. data:: algorithms_guaranteed
Contains the names of the hash algorithms guaranteed to be supported A set containing the names of the hash algorithms guaranteed to be supported
by this module on all platforms. by this module on all platforms.
.. versionadded:: 3.2 .. versionadded:: 3.2
.. data:: algorithms_available .. data:: algorithms_available
Contains the names of the hash algorithms that are available A set containing the names of the hash algorithms that are available in the
in the running Python interpreter. These names will be recognized running Python interpreter. These names will be recognized when passed to
when passed to :func:`new`. :attr:`algorithms_guaranteed` :func:`new`. :attr:`algorithms_guaranteed` will always be a subset. The
will always be a subset. Duplicate algorithms with different same algorithm may appear multiple times in this set under different names
name formats may appear in this set (thanks to OpenSSL). (thanks to OpenSSL).
.. versionadded:: 3.2 .. versionadded:: 3.2
......
...@@ -23,9 +23,8 @@ This module implements the HMAC algorithm as described by :rfc:`2104`. ...@@ -23,9 +23,8 @@ This module implements the HMAC algorithm as described by :rfc:`2104`.
defaults to the :data:`hashlib.md5` constructor. defaults to the :data:`hashlib.md5` constructor.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Parameter *key* can be a bytes or bytearray object. Parameter *msg* can Parameter *key* can be a bytes or bytearray object.
be of any type supported by :mod:`hashlib`. Parameter *msg* can be of any type supported by :mod:`hashlib`.
Paramter *digestmod* can be the name of a hash algorithm. Paramter *digestmod* can be the name of a hash algorithm.
.. deprecated:: 3.4 .. deprecated:: 3.4
......
...@@ -351,7 +351,7 @@ of which this module provides three different variants: ...@@ -351,7 +351,7 @@ of which this module provides three different variants:
The :class:`SimpleHTTPRequestHandler` class can be used in the following The :class:`SimpleHTTPRequestHandler` class can be used in the following
manner in order to create a very basic webserver serving files relative to manner in order to create a very basic webserver serving files relative to
the current directory. :: the current directory::
import http.server import http.server
import socketserver import socketserver
...@@ -365,15 +365,17 @@ the current directory. :: ...@@ -365,15 +365,17 @@ the current directory. ::
print("serving at port", PORT) print("serving at port", PORT)
httpd.serve_forever() httpd.serve_forever()
.. _http-server-cli:
:mod:`http.server` can also be invoked directly using the :option:`-m` :mod:`http.server` can also be invoked directly using the :option:`-m`
switch of the interpreter with a ``port number`` argument. Similar to switch of the interpreter with a ``port number`` argument. Similar to
the previous example, this serves files relative to the current directory. :: the previous example, this serves files relative to the current directory::
python -m http.server 8000 python -m http.server 8000
By default, server binds itself to all interfaces. To restrict it to bind to a By default, server binds itself to all interfaces. The option ``-b/--bind``
particular interface only, ``--bind ADDRESS`` argument can be used. For e.g, to specifies a specific address to which it should bind. For example, the
restrict the server to bind only to localhost. :: following command causes the server to bind to localhost only::
python -m http.server 8000 --bind 127.0.0.1 python -m http.server 8000 --bind 127.0.0.1
...@@ -422,7 +424,7 @@ restrict the server to bind only to localhost. :: ...@@ -422,7 +424,7 @@ restrict the server to bind only to localhost. ::
reasons. Problems with the CGI script will be translated to error 403. reasons. Problems with the CGI script will be translated to error 403.
:class:`CGIHTTPRequestHandler` can be enabled in the command line by passing :class:`CGIHTTPRequestHandler` can be enabled in the command line by passing
the ``--cgi`` option.:: the ``--cgi`` option::
python -m http.server --cgi 8000 python -m http.server --cgi 8000
...@@ -134,7 +134,7 @@ Functions ...@@ -134,7 +134,7 @@ Functions
When :func:`reload` is executed: When :func:`reload` is executed:
* Python modules' code is recompiled and the module-level code re-executed, * Python module's code is recompiled and the module-level code re-executed,
defining a new set of objects which are bound to names in the module's defining a new set of objects which are bound to names in the module's
dictionary by reusing the :term:`loader` which originally loaded the dictionary by reusing the :term:`loader` which originally loaded the
module. The ``init`` function of extension modules is not called a second module. The ``init`` function of extension modules is not called a second
......
...@@ -729,6 +729,11 @@ Classes and functions ...@@ -729,6 +729,11 @@ Classes and functions
Consider using the new :ref:`Signature Object <inspect-signature-object>` Consider using the new :ref:`Signature Object <inspect-signature-object>`
interface, which provides a better way of introspecting functions. interface, which provides a better way of introspecting functions.
.. versionchanged:: 3.4
This function is now based on :func:`signature`, but still ignores
``__wrapped__`` attributes and includes the already bound first
parameter in the signature output for bound methods.
.. function:: getargvalues(frame) .. function:: getargvalues(frame)
......
...@@ -686,6 +686,7 @@ than raw I/O does. ...@@ -686,6 +686,7 @@ than raw I/O does.
:exc:`UnsupportedOperation`. :exc:`UnsupportedOperation`.
.. warning:: .. warning::
:class:`BufferedRWPair` does not attempt to synchronize accesses to :class:`BufferedRWPair` does not attempt to synchronize accesses to
its underlying raw streams. You should not pass it the same object its underlying raw streams. You should not pass it the same object
as reader and writer; use :class:`BufferedRandom` instead. as reader and writer; use :class:`BufferedRandom` instead.
......
...@@ -154,20 +154,20 @@ write code that handles both IP versions correctly. ...@@ -154,20 +154,20 @@ write code that handles both IP versions correctly.
.. attribute:: is_private .. attribute:: is_private
``True`` if the address is allocated for private networks. See ``True`` if the address is allocated for private networks. See
iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_
(for IPv6). (for IPv6).
.. attribute:: is_global .. attribute:: is_global
``True`` if the address is allocated for public networks. See ``True`` if the address is allocated for public networks. See
iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_
(for IPv6). (for IPv6).
.. versionadded:: 3.4 .. versionadded:: 3.4
.. attribute:: is_unspecified .. attribute:: is_unspecified
``True`` if the address is unspecified. See :RFC:`5375` (for IPv4) ``True`` if the address is unspecified. See :RFC:`5735` (for IPv4)
or :RFC:`2373` (for IPv6). or :RFC:`2373` (for IPv6).
.. attribute:: is_reserved .. attribute:: is_reserved
...@@ -184,6 +184,9 @@ write code that handles both IP versions correctly. ...@@ -184,6 +184,9 @@ write code that handles both IP versions correctly.
``True`` if the address is reserved for link-local usage. See ``True`` if the address is reserved for link-local usage. See
:RFC:`3927`. :RFC:`3927`.
.. _iana-ipv4-special-registry: http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
.. _iana-ipv6-special-registry: http://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml
.. class:: IPv6Address(address) .. class:: IPv6Address(address)
...@@ -218,18 +221,23 @@ write code that handles both IP versions correctly. ...@@ -218,18 +221,23 @@ write code that handles both IP versions correctly.
The long form of the address representation, with all leading zeroes and The long form of the address representation, with all leading zeroes and
groups consisting entirely of zeroes included. groups consisting entirely of zeroes included.
For the following attributes, see the corresponding documention of the
:class:`IPv4Address` class:
.. attribute:: packed .. attribute:: packed
.. attribute:: version .. attribute:: version
.. attribute:: max_prefixlen .. attribute:: max_prefixlen
.. attribute:: is_multicast .. attribute:: is_multicast
.. attribute:: is_private .. attribute:: is_private
.. attribute:: is_global
.. attribute:: is_unspecified .. attribute:: is_unspecified
.. attribute:: is_reserved .. attribute:: is_reserved
.. attribute:: is_loopback .. attribute:: is_loopback
.. attribute:: is_link_local .. attribute:: is_link_local
Refer to the corresponding attribute documentation in .. versionadded:: 3.4
:class:`IPv4Address` is_global
.. attribute:: is_site_local .. attribute:: is_site_local
......
...@@ -148,7 +148,9 @@ in :mod:`logging` itself) and defining handlers which are declared either in ...@@ -148,7 +148,9 @@ in :mod:`logging` itself) and defining handlers which are declared either in
send it to the socket as a string of bytes preceded by a four-byte length send it to the socket as a string of bytes preceded by a four-byte length
string packed in binary using ``struct.pack('>L', n)``. string packed in binary using ``struct.pack('>L', n)``.
.. note:: Because portions of the configuration are passed through .. note::
Because portions of the configuration are passed through
:func:`eval`, use of this function may open its users to a security risk. :func:`eval`, use of this function may open its users to a security risk.
While the function only binds to a socket on ``localhost``, and so does While the function only binds to a socket on ``localhost``, and so does
not accept connections from remote machines, there are scenarios where not accept connections from remote machines, there are scenarios where
...@@ -752,7 +754,9 @@ The ``class`` entry is optional. It indicates the name of the formatter's class ...@@ -752,7 +754,9 @@ The ``class`` entry is optional. It indicates the name of the formatter's class
:class:`~logging.Formatter` can present exception tracebacks in an expanded or :class:`~logging.Formatter` can present exception tracebacks in an expanded or
condensed format. condensed format.
.. note:: Due to the use of :func:`eval` as described above, there are .. note::
Due to the use of :func:`eval` as described above, there are
potential security risks which result from using the :func:`listen` to send potential security risks which result from using the :func:`listen` to send
and receive configurations via sockets. The risks are limited to where and receive configurations via sockets. The risks are limited to where
multiple users with no mutual trust run code on the same machine; see the multiple users with no mutual trust run code on the same machine; see the
......
...@@ -36,9 +36,9 @@ Number-theoretic and representation functions ...@@ -36,9 +36,9 @@ Number-theoretic and representation functions
.. function:: copysign(x, y) .. function:: copysign(x, y)
Return *x* with the sign of *y*. On a platform that supports Return a float with the magnitude (absolute value) of *x* but the sign of
signed zeros, ``copysign(1.0, -0.0)`` returns *-1.0*. *y*. On platforms that support signed zeros, ``copysign(1.0, -0.0)``
returns *-1.0*.
.. function:: fabs(x) .. function:: fabs(x)
......
...@@ -136,9 +136,11 @@ to start a process. These *start methods* are ...@@ -136,9 +136,11 @@ to start a process. These *start methods* are
Available on Unix platforms which support passing file descriptors Available on Unix platforms which support passing file descriptors
over Unix pipes. over Unix pipes.
Before Python 3.4 *fork* was the only option available on Unix. Also, .. versionchanged:: 3.4
prior to Python 3.4, child processes would inherit all the parents *spawn* added on all unix platforms, and *forkserver* added for
inheritable handles on Windows. some unix platforms.
Child processes no longer inherit all of the parents inheritable
handles on Windows.
On Unix using the *spawn* or *forkserver* start methods will also On Unix using the *spawn* or *forkserver* start methods will also
start a *semaphore tracker* process which tracks the unlinked named start a *semaphore tracker* process which tracks the unlinked named
...@@ -1474,7 +1476,7 @@ their parent process exits. The manager classes are defined in the ...@@ -1474,7 +1476,7 @@ their parent process exits. The manager classes are defined in the
*exposed* is used to specify a sequence of method names which proxies for *exposed* is used to specify a sequence of method names which proxies for
this typeid should be allowed to access using this typeid should be allowed to access using
:meth:`BaseProxy._callMethod`. (If *exposed* is ``None`` then :meth:`BaseProxy._callmethod`. (If *exposed* is ``None`` then
:attr:`proxytype._exposed_` is used instead if it exists.) In the case :attr:`proxytype._exposed_` is used instead if it exists.) In the case
where no exposed list is specified, all "public methods" of the shared where no exposed list is specified, all "public methods" of the shared
object will be accessible. (Here a "public method" means any attribute object will be accessible. (Here a "public method" means any attribute
...@@ -1853,26 +1855,31 @@ with the :class:`Pool` class. ...@@ -1853,26 +1855,31 @@ with the :class:`Pool` class.
callbacks and has a parallel map implementation. callbacks and has a parallel map implementation.
*processes* is the number of worker processes to use. If *processes* is *processes* is the number of worker processes to use. If *processes* is
``None`` then the number returned by :func:`os.cpu_count` is used. If ``None`` then the number returned by :func:`os.cpu_count` is used.
*initializer* is not ``None`` then each worker process will call
``initializer(*initargs)`` when it starts.
Note that the methods of the pool object should only be called by If *initializer* is not ``None`` then each worker process will call
the process which created the pool. ``initializer(*initargs)`` when it starts.
.. versionadded:: 3.2
*maxtasksperchild* is the number of tasks a worker process can complete *maxtasksperchild* is the number of tasks a worker process can complete
before it will exit and be replaced with a fresh worker process, to enable before it will exit and be replaced with a fresh worker process, to enable
unused resources to be freed. The default *maxtasksperchild* is None, which unused resources to be freed. The default *maxtasksperchild* is None, which
means worker processes will live as long as the pool. means worker processes will live as long as the pool.
.. versionadded:: 3.4
*context* can be used to specify the context used for starting *context* can be used to specify the context used for starting
the worker processes. Usually a pool is created using the the worker processes. Usually a pool is created using the
function :func:`multiprocessing.Pool` or the :meth:`Pool` method function :func:`multiprocessing.Pool` or the :meth:`Pool` method
of a context object. In both cases *context* is set of a context object. In both cases *context* is set
appropriately. appropriately.
Note that the methods of the pool object should only be called by
the process which created the pool.
.. versionadded:: 3.2
*maxtasksperchild*
.. versionadded:: 3.4
*context*
.. note:: .. note::
Worker processes within a :class:`Pool` typically live for the complete Worker processes within a :class:`Pool` typically live for the complete
......
...@@ -240,9 +240,9 @@ their character equivalents. ...@@ -240,9 +240,9 @@ their character equivalents.
.. function:: length_hint(obj, default=0) .. function:: length_hint(obj, default=0)
Return an estimated length for the object *o*. First trying to return its Return an estimated length for the object *o*. First try to return its
actual length, then an estimate using :meth:`object.__length_hint__`, and actual length, then an estimate using :meth:`object.__length_hint__`, and
finally returning the default value. finally return the default value.
.. versionadded:: 3.4 .. versionadded:: 3.4
......
...@@ -193,11 +193,17 @@ the :mod:`glob` module.) ...@@ -193,11 +193,17 @@ the :mod:`glob` module.)
.. function:: ismount(path) .. function:: ismount(path)
Return ``True`` if pathname *path* is a :dfn:`mount point`: a point in a file Return ``True`` if pathname *path* is a :dfn:`mount point`: a point in a
system where a different file system has been mounted. The function checks file system where a different file system has been mounted. On POSIX, the
whether *path*'s parent, :file:`path/..`, is on a different device than *path*, function checks whether *path*'s parent, :file:`path/..`, is on a different
or whether :file:`path/..` and *path* point to the same i-node on the same device than *path*, or whether :file:`path/..` and *path* point to the same
device --- this should detect mount points for all Unix and POSIX variants. i-node on the same device --- this should detect mount points for all Unix
and POSIX variants. On Windows, a drive letter root and a share UNC are
always mount points, and for any other path ``GetVolumePathName`` is called
to see if it is different from the input path.
.. versionadded:: 3.4
Support for detecting non-root mount points on Windows.
.. function:: join(path1[, path2[, ...]]) .. function:: join(path1[, path2[, ...]])
...@@ -251,7 +257,7 @@ the :mod:`glob` module.) ...@@ -251,7 +257,7 @@ the :mod:`glob` module.)
.. function:: samefile(path1, path2) .. function:: samefile(path1, path2)
Return ``True`` if both pathname arguments refer to the same file or directory. Return ``True`` if both pathname arguments refer to the same file or directory.
On Unix, this is determined by the device number and i-node number and raises an This is determined by the device number and i-node number and raises an
exception if a :func:`os.stat` call on either pathname fails. exception if a :func:`os.stat` call on either pathname fails.
Availability: Unix, Windows. Availability: Unix, Windows.
......
...@@ -260,7 +260,9 @@ process and user. ...@@ -260,7 +260,9 @@ process and user.
Availability: Unix. Availability: Unix.
.. note:: On Mac OS X, :func:`getgroups` behavior differs somewhat from .. note::
On Mac OS X, :func:`getgroups` behavior differs somewhat from
other Unix platforms. If the Python interpreter was built with a other Unix platforms. If the Python interpreter was built with a
deployment target of :const:`10.5` or earlier, :func:`getgroups` returns deployment target of :const:`10.5` or earlier, :func:`getgroups` returns
the list of effective group ids associated with the current user process; the list of effective group ids associated with the current user process;
...@@ -940,7 +942,8 @@ or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Window ...@@ -940,7 +942,8 @@ or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Window
the C library. the C library.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Add :data:`O_TMPFILE` constant. It's only available on Linux Kernel 3.11 Add :data:`O_PATH` on systems that support it.
Add :data:`O_TMPFILE`, only available on Linux Kernel 3.11
or newer. or newer.
...@@ -2653,7 +2656,7 @@ written in Python, such as a mail server's external command delivery program. ...@@ -2653,7 +2656,7 @@ written in Python, such as a mail server's external command delivery program.
Fork a child process. Return ``0`` in the child and the child's process id in the Fork a child process. Return ``0`` in the child and the child's process id in the
parent. If an error occurs :exc:`OSError` is raised. parent. If an error occurs :exc:`OSError` is raised.
Note that some platforms including FreeBSD <= 6.3, Cygwin and OS/2 EMX have Note that some platforms including FreeBSD <= 6.3 and Cygwin have
known issues when using fork() from a thread. known issues when using fork() from a thread.
.. warning:: .. warning::
...@@ -2899,7 +2902,6 @@ written in Python, such as a mail server's external command delivery program. ...@@ -2899,7 +2902,6 @@ written in Python, such as a mail server's external command delivery program.
:manpage:`times(2)` or the corresponding Windows Platform API documentation. :manpage:`times(2)` or the corresponding Windows Platform API documentation.
On Windows, only :attr:`user` and :attr:`system` are known; the other On Windows, only :attr:`user` and :attr:`system` are known; the other
attributes are zero. attributes are zero.
On OS/2, only :attr:`elapsed` is known; the other attributes are zero.
Availability: Unix, Windows. Availability: Unix, Windows.
......
...@@ -522,6 +522,36 @@ Pure paths provide the following methods and properties: ...@@ -522,6 +522,36 @@ Pure paths provide the following methods and properties:
ValueError: '/etc/passwd' does not start with '/usr' ValueError: '/etc/passwd' does not start with '/usr'
.. method:: PurePath.with_name(name)
Return a new path with the :attr:`name` changed. If the original path
doesn't have a name, ValueError is raised::
>>> p = PureWindowsPath('c:/Downloads/pathlib.tar.gz')
>>> p.with_name('setup.py')
PureWindowsPath('c:/Downloads/setup.py')
>>> p = PureWindowsPath('c:/')
>>> p.with_name('setup.py')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/antoine/cpython/default/Lib/pathlib.py", line 751, in with_name
raise ValueError("%r has an empty name" % (self,))
ValueError: PureWindowsPath('c:/') has an empty name
.. method:: PurePath.with_suffix(suffix)
Return a new path with the :attr:`suffix` changed. If the original path
doesn't have a suffix, the new *suffix* is appended instead::
>>> p = PureWindowsPath('c:/Downloads/pathlib.tar.gz')
>>> p.with_suffix('.bz2')
PureWindowsPath('c:/Downloads/pathlib.tar.bz2')
>>> p = PureWindowsPath('README')
>>> p.with_suffix('.txt')
PureWindowsPath('README.txt')
.. _concrete-paths: .. _concrete-paths:
......
...@@ -147,6 +147,7 @@ support. ...@@ -147,6 +147,7 @@ support.
*prefix* is a string to output on the front of every module name on output. *prefix* is a string to output on the front of every module name on output.
.. note:: .. note::
Only works for a :term:`finder` which defines an ``iter_modules()`` Only works for a :term:`finder` which defines an ``iter_modules()``
method. This interface is non-standard, so the module also provides method. This interface is non-standard, so the module also provides
implementations for :class:`importlib.machinery.FileFinder` and implementations for :class:`importlib.machinery.FileFinder` and
...@@ -185,6 +186,7 @@ support. ...@@ -185,6 +186,7 @@ support.
walk_packages(ctypes.__path__, ctypes.__name__ + '.') walk_packages(ctypes.__path__, ctypes.__name__ + '.')
.. note:: .. note::
Only works for a :term:`finder` which defines an ``iter_modules()`` Only works for a :term:`finder` which defines an ``iter_modules()``
method. This interface is non-standard, so the module also provides method. This interface is non-standard, so the module also provides
implementations for :class:`importlib.machinery.FileFinder` and implementations for :class:`importlib.machinery.FileFinder` and
......
...@@ -32,6 +32,9 @@ Values can be strings, integers, floats, booleans, tuples, lists, dictionaries ...@@ -32,6 +32,9 @@ Values can be strings, integers, floats, booleans, tuples, lists, dictionaries
(but only with string keys), :class:`Data`, :class:`bytes`, :class:`bytesarray` (but only with string keys), :class:`Data`, :class:`bytes`, :class:`bytesarray`
or :class:`datetime.datetime` objects. or :class:`datetime.datetime` objects.
.. versionchanged:: 3.4
New API, old API deprecated. Support for binary format plists added.
.. seealso:: .. seealso::
`PList manual page <http://developer.apple.com/documentation/Darwin/Reference/ManPages/man5/plist.5.html>`_ `PList manual page <http://developer.apple.com/documentation/Darwin/Reference/ManPages/man5/plist.5.html>`_
...@@ -107,7 +110,7 @@ This module defines the following functions: ...@@ -107,7 +110,7 @@ This module defines the following functions:
An :exc:`OverflowError` will be raised for integer values that cannot An :exc:`OverflowError` will be raised for integer values that cannot
be represented in (binary) plist files. be represented in (binary) plist files.
.. versionadded: 3.4 .. versionadded:: 3.4
.. function:: dumps(value, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False) .. function:: dumps(value, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False)
...@@ -116,7 +119,7 @@ This module defines the following functions: ...@@ -116,7 +119,7 @@ This module defines the following functions:
the documentation for :func:`dump` for an explanation of the keyword the documentation for :func:`dump` for an explanation of the keyword
arguments of this function. arguments of this function.
.. versionadded: 3.4 .. versionadded:: 3.4
The following functions are deprecated: The following functions are deprecated:
...@@ -135,7 +138,7 @@ The following functions are deprecated: ...@@ -135,7 +138,7 @@ The following functions are deprecated:
to ``__getitem_``. This means that you can use attribute access to to ``__getitem_``. This means that you can use attribute access to
access items of these dictionaries. access items of these dictionaries.
.. deprecated: 3.4 Use :func:`load` instead. .. deprecated:: 3.4 Use :func:`load` instead.
.. function:: writePlist(rootObject, pathOrFile) .. function:: writePlist(rootObject, pathOrFile)
...@@ -143,7 +146,7 @@ The following functions are deprecated: ...@@ -143,7 +146,7 @@ The following functions are deprecated:
Write *rootObject* to an XML plist file. *pathOrFile* may be either a file name Write *rootObject* to an XML plist file. *pathOrFile* may be either a file name
or a (writable and binary) file object or a (writable and binary) file object
.. deprecated: 3.4 Use :func:`dump` instead. .. deprecated:: 3.4 Use :func:`dump` instead.
.. function:: readPlistFromBytes(data) .. function:: readPlistFromBytes(data)
...@@ -194,7 +197,7 @@ The following classes are available: ...@@ -194,7 +197,7 @@ The following classes are available:
.. deprecated:: 3.4 Use a :class:`bytes` object instead .. deprecated:: 3.4 Use a :class:`bytes` object instead
The following constants are avaiable: The following constants are available:
.. data:: FMT_XML .. data:: FMT_XML
......
...@@ -84,3 +84,8 @@ Reference Manual pages. ...@@ -84,3 +84,8 @@ Reference Manual pages.
.. versionchanged:: 3.2 .. versionchanged:: 3.2
Added the ``-b`` option, deprecated the ``-g`` option. Added the ``-b`` option, deprecated the ``-g`` option.
.. versionchanged:: 3.4
:mod:`pydoc` now uses :func:`inspect.signature` rather than
:func:`inspect.getfullargspec` to extract signature information from
callables.
...@@ -123,9 +123,13 @@ On systems that support :mod:`readline`, this module will also import and ...@@ -123,9 +123,13 @@ On systems that support :mod:`readline`, this module will also import and
configure the :mod:`rlcompleter` module, if Python is started in configure the :mod:`rlcompleter` module, if Python is started in
:ref:`interactive mode <tut-interactive>` and without the :option:`-S` option. :ref:`interactive mode <tut-interactive>` and without the :option:`-S` option.
The default behavior is enable tab-completion and to use The default behavior is enable tab-completion and to use
:file:`~/.python_history` as the history save file. To disable it, override :file:`~/.python_history` as the history save file. To disable it, delete (or
the :data:`sys.__interactivehook__` attribute in your :mod:`sitecustomize` override) the :data:`sys.__interactivehook__` attribute in your
or :mod:`usercustomize` module or your :envvar:`PYTHONSTARTUP` file. :mod:`sitecustomize` or :mod:`usercustomize` module or your
:envvar:`PYTHONSTARTUP` file.
.. versionchanged:: 3.4
Activation of rlcompleter and history was made automatic.
Module contents Module contents
......
...@@ -117,6 +117,9 @@ A nice selection of exceptions is defined as well: ...@@ -117,6 +117,9 @@ A nice selection of exceptions is defined as well:
Subclass of :exc:`OSError` that is the base exception class for all Subclass of :exc:`OSError` that is the base exception class for all
the other exceptions provided by this module. the other exceptions provided by this module.
.. versionchanged:: 3.4
SMTPException became subclass of :exc:`OSError`
.. exception:: SMTPServerDisconnected .. exception:: SMTPServerDisconnected
......
...@@ -6,8 +6,7 @@ ...@@ -6,8 +6,7 @@
This module provides access to the BSD *socket* interface. It is available on This module provides access to the BSD *socket* interface. It is available on
all modern Unix systems, Windows, MacOS, OS/2, and probably additional all modern Unix systems, Windows, MacOS, and probably additional platforms.
platforms.
.. note:: .. note::
...@@ -468,7 +467,7 @@ The :mod:`socket` module also offers various network-related services: ...@@ -468,7 +467,7 @@ The :mod:`socket` module also offers various network-related services:
(10, 1, 6, '', ('2001:888:2000:d::a2', 80, 0, 0))] (10, 1, 6, '', ('2001:888:2000:d::a2', 80, 0, 0))]
.. versionchanged:: 3.2 .. versionchanged:: 3.2
parameters can now be passed as single keyword arguments. parameters can now be passed using keyword arguments.
.. function:: getfqdn([name]) .. function:: getfqdn([name])
...@@ -634,6 +633,9 @@ The :mod:`socket` module also offers various network-related services: ...@@ -634,6 +633,9 @@ The :mod:`socket` module also offers various network-related services:
Availability: Unix (maybe not all platforms), Windows. Availability: Unix (maybe not all platforms), Windows.
.. versionchanged:: 3.4
Windows support added
.. function:: inet_ntop(address_family, packed_ip) .. function:: inet_ntop(address_family, packed_ip)
...@@ -650,6 +652,9 @@ The :mod:`socket` module also offers various network-related services: ...@@ -650,6 +652,9 @@ The :mod:`socket` module also offers various network-related services:
Availability: Unix (maybe not all platforms), Windows. Availability: Unix (maybe not all platforms), Windows.
.. versionchanged:: 3.4
Windows support added
.. ..
XXX: Are sendmsg(), recvmsg() and CMSG_*() available on any XXX: Are sendmsg(), recvmsg() and CMSG_*() available on any
...@@ -794,6 +799,7 @@ to sockets. ...@@ -794,6 +799,7 @@ to sockets.
:keyword:`with` statement around them. :keyword:`with` statement around them.
.. note:: .. note::
:meth:`close()` releases the resource associated with a connection but :meth:`close()` releases the resource associated with a connection but
does not necessarily close the connection immediately. If you want does not necessarily close the connection immediately. If you want
to close the connection in a timely fashion, call :meth:`shutdown()` to close the connection in a timely fashion, call :meth:`shutdown()`
...@@ -1512,4 +1518,3 @@ the :data:`SO_REUSEADDR` flag tells the kernel to reuse a local socket in ...@@ -1512,4 +1518,3 @@ the :data:`SO_REUSEADDR` flag tells the kernel to reuse a local socket in
details of socket semantics. For Unix, refer to the manual pages; for Windows, details of socket semantics. For Unix, refer to the manual pages; for Windows,
see the WinSock (or Winsock 2) specification. For IPv6-ready APIs, readers may see the WinSock (or Winsock 2) specification. For IPv6-ready APIs, readers may
want to refer to :rfc:`3493` titled Basic Socket Interface Extensions for IPv6. want to refer to :rfc:`3493` titled Basic Socket Interface Extensions for IPv6.
...@@ -818,6 +818,7 @@ SSL sockets also have the following additional methods and attributes: ...@@ -818,6 +818,7 @@ SSL sockets also have the following additional methods and attributes:
'version': 3} 'version': 3}
.. note:: .. note::
To validate a certificate for a particular service, you can use the To validate a certificate for a particular service, you can use the
:func:`match_hostname` function. :func:`match_hostname` function.
...@@ -841,8 +842,6 @@ SSL sockets also have the following additional methods and attributes: ...@@ -841,8 +842,6 @@ SSL sockets also have the following additional methods and attributes:
.. versionchanged:: 3.4 .. versionchanged:: 3.4
:exc:`ValueError` is raised when the handshake isn't done. :exc:`ValueError` is raised when the handshake isn't done.
.. versionchanged:: 3.4
The returned dictionary includes additional X509v3 extension items The returned dictionary includes additional X509v3 extension items
such as ``crlDistributionPoints``, ``caIssuers`` and ``OCSP`` URIs. such as ``crlDistributionPoints``, ``caIssuers`` and ``OCSP`` URIs.
......
...@@ -3,7 +3,6 @@ ...@@ -3,7 +3,6 @@
.. module:: stringprep .. module:: stringprep
:synopsis: String preparation, as per RFC 3453 :synopsis: String preparation, as per RFC 3453
:deprecated:
.. moduleauthor:: Martin v. Löwis <martin@v.loewis.de> .. moduleauthor:: Martin v. Löwis <martin@v.loewis.de>
.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de> .. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
......
...@@ -629,6 +629,12 @@ Instances of the :class:`Popen` class have the following methods: ...@@ -629,6 +629,12 @@ Instances of the :class:`Popen` class have the following methods:
:exc:`TimeoutExpired` exception. It is safe to catch this exception and :exc:`TimeoutExpired` exception. It is safe to catch this exception and
retry the wait. retry the wait.
.. note::
The function is implemented using a busy loop (non-blocking call and
short sleeps). Use the :mod:`asyncio` module for an asynchronous wait:
see :class:`asyncio.create_subprocess_exec`.
.. warning:: .. warning::
This will deadlock when using ``stdout=PIPE`` and/or This will deadlock when using ``stdout=PIPE`` and/or
...@@ -639,6 +645,11 @@ Instances of the :class:`Popen` class have the following methods: ...@@ -639,6 +645,11 @@ Instances of the :class:`Popen` class have the following methods:
.. versionchanged:: 3.3 .. versionchanged:: 3.3
*timeout* was added. *timeout* was added.
.. deprecated:: 3.4
Do not use the undocumented *endtime* parameter. It is was
unintentionally exposed in 3.3 but was intended to be private
for internal use. Use *timeout* instead.
.. method:: Popen.communicate(input=None, timeout=None) .. method:: Popen.communicate(input=None, timeout=None)
...@@ -1080,9 +1091,11 @@ handling consistency are valid for these functions. ...@@ -1080,9 +1091,11 @@ handling consistency are valid for these functions.
>>> subprocess.getstatusoutput('/bin/junk') >>> subprocess.getstatusoutput('/bin/junk')
(256, 'sh: /bin/junk: not found') (256, 'sh: /bin/junk: not found')
.. versionchanged:: 3.3
Availability: Unix & Windows Availability: Unix & Windows
.. versionchanged:: 3.3.4
Windows support added
.. function:: getoutput(cmd) .. function:: getoutput(cmd)
...@@ -1094,9 +1107,11 @@ handling consistency are valid for these functions. ...@@ -1094,9 +1107,11 @@ handling consistency are valid for these functions.
>>> subprocess.getoutput('ls /bin/ls') >>> subprocess.getoutput('ls /bin/ls')
'/bin/ls' '/bin/ls'
.. versionchanged:: 3.3
Availability: Unix & Windows Availability: Unix & Windows
.. versionchanged:: 3.3.4
Windows support added
Notes Notes
----- -----
......
...@@ -251,7 +251,7 @@ AU_write objects, as returned by :func:`.open` above, have the following methods ...@@ -251,7 +251,7 @@ AU_write objects, as returned by :func:`.open` above, have the following methods
Write audio frames, without correcting *nframes*. Write audio frames, without correcting *nframes*.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
.. method:: AU_write.writeframes(data) .. method:: AU_write.writeframes(data)
...@@ -259,7 +259,7 @@ AU_write objects, as returned by :func:`.open` above, have the following methods ...@@ -259,7 +259,7 @@ AU_write objects, as returned by :func:`.open` above, have the following methods
Write audio frames and make sure *nframes* is correct. Write audio frames and make sure *nframes* is correct.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
.. method:: AU_write.close() .. method:: AU_write.close()
......
...@@ -227,7 +227,9 @@ always available. ...@@ -227,7 +227,9 @@ always available.
installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y* installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y*
is the version number of Python, for example ``3.2``. is the version number of Python, for example ``3.2``.
.. note:: If a :ref:`virtual environment <venv-def>` is in effect, this .. note::
If a :ref:`virtual environment <venv-def>` is in effect, this
value will be changed in ``site.py`` to point to the virtual environment. value will be changed in ``site.py`` to point to the virtual environment.
The value for the Python installation will still be available, via The value for the Python installation will still be available, via
:data:`base_exec_prefix`. :data:`base_exec_prefix`.
...@@ -692,10 +694,11 @@ always available. ...@@ -692,10 +694,11 @@ always available.
.. data:: __interactivehook__ .. data:: __interactivehook__
When present, this function is automatically called (with no arguments) When this attribute exists, its value is automatically called (with no
when the interpreter is launched in :ref:`interactive mode <tut-interactive>`. arguments) when the interpreter is launched in :ref:`interactive mode
This is done after the :envvar:`PYTHONSTARTUP` file is read, so that you <tut-interactive>`. This is done after the :envvar:`PYTHONSTARTUP` file is
can set this hook there. read, so that you can set this hook there. The :mod:`site` module
:ref:`sets this <rlcompleter-config>`.
.. versionadded:: 3.4 .. versionadded:: 3.4
......
...@@ -199,6 +199,7 @@ The :mod:`test.support` module provides support for Python's regression ...@@ -199,6 +199,7 @@ The :mod:`test.support` module provides support for Python's regression
test suite. test suite.
.. note:: .. note::
:mod:`test.support` is not a public module. It is documented here to help :mod:`test.support` is not a public module. It is documented here to help
Python developers write tests. The API of this module is subject to change Python developers write tests. The API of this module is subject to change
without backwards compatibility concerns between releases. without backwards compatibility concerns between releases.
......
...@@ -40,13 +40,14 @@ functions should be good enough; otherwise, you should use an instance of ...@@ -40,13 +40,14 @@ functions should be good enough; otherwise, you should use an instance of
:func:`wrap`. :func:`wrap`.
.. function:: shorten(text, width=70, *, placeholder=" [...]") .. function:: shorten(text, width, **kwargs)
Collapse and truncate the given text to fit in the given width. Collapse and truncate the given *text* to fit in the given *width*.
The text first has its whitespace collapsed. If it then fits in First the whitespace in *text* is collapsed (all whitespace is replaced by
the *width*, it is returned unchanged. Otherwise, as many words single spaces). If the result fits in the *width*, it is returned.
as possible are joined and then the *placeholder* is appended:: Otherwise, enough words are dropped from the end so that the remaining words
plus the :attr:`placeholder` fit within :attr:`width`::
>>> textwrap.shorten("Hello world!", width=12) >>> textwrap.shorten("Hello world!", width=12)
'Hello world!' 'Hello world!'
...@@ -55,6 +56,12 @@ functions should be good enough; otherwise, you should use an instance of ...@@ -55,6 +56,12 @@ functions should be good enough; otherwise, you should use an instance of
>>> textwrap.shorten("Hello world", width=10, placeholder="...") >>> textwrap.shorten("Hello world", width=10, placeholder="...")
'Hello...' 'Hello...'
Optional keyword arguments correspond to the instance attributes of
:class:`TextWrapper`, documented below. Note that the whitespace is
collapsed before the text is passed to the :class:`TextWrapper` :meth:`fill`
function, so changing the value of :attr:`.tabsize`, :attr:`.expand_tabs`,
:attr:`.drop_whitespace`, and :attr:`.replace_whitespace` will have no effect.
.. versionadded:: 3.4 .. versionadded:: 3.4
...@@ -110,8 +117,8 @@ functions should be good enough; otherwise, you should use an instance of ...@@ -110,8 +117,8 @@ functions should be good enough; otherwise, you should use an instance of
:func:`wrap`, :func:`fill` and :func:`shorten` work by creating a :func:`wrap`, :func:`fill` and :func:`shorten` work by creating a
:class:`TextWrapper` instance and calling a single method on it. That :class:`TextWrapper` instance and calling a single method on it. That
instance is not reused, so for applications that process many text instance is not reused, so for applications that process many text
strings, it may be more efficient to create your own strings using :func:`wrap` and/or :func:`fill`, it may be more efficient to
:class:`TextWrapper` object. create your own :class:`TextWrapper` object.
Text is preferably wrapped on whitespaces and right after the hyphens in Text is preferably wrapped on whitespaces and right after the hyphens in
hyphenated words; only then will long words be broken if necessary, unless hyphenated words; only then will long words be broken if necessary, unless
...@@ -252,16 +259,16 @@ hyphenated words; only then will long words be broken if necessary, unless ...@@ -252,16 +259,16 @@ hyphenated words; only then will long words be broken if necessary, unless
.. attribute:: max_lines .. attribute:: max_lines
(default: ``None``) If not ``None``, then the text be will truncated to (default: ``None``) If not ``None``, then the output will contain at most
*max_lines* lines. *max_lines* lines, with *placeholder* appearing at the end of the output.
.. versionadded:: 3.4 .. versionadded:: 3.4
.. attribute:: placeholder .. attribute:: placeholder
(default: ``' [...]'``) String that will be appended to the last line of (default: ``' [...]'``) String that will appear at the end of the output
text if it will be truncated. text if it has been truncated.
.. versionadded:: 3.4 .. versionadded:: 3.4
......
...@@ -184,6 +184,7 @@ Pretty top ...@@ -184,6 +184,7 @@ Pretty top
Code to display the 10 lines allocating the most memory with a pretty output, Code to display the 10 lines allocating the most memory with a pretty output,
ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files:: ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files::
import linecache
import os import os
import tracemalloc import tracemalloc
...@@ -201,6 +202,9 @@ ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files:: ...@@ -201,6 +202,9 @@ ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files::
filename = os.sep.join(frame.filename.split(os.sep)[-2:]) filename = os.sep.join(frame.filename.split(os.sep)[-2:])
print("#%s: %s:%s: %.1f KiB" print("#%s: %s:%s: %.1f KiB"
% (index, filename, frame.lineno, stat.size / 1024)) % (index, filename, frame.lineno, stat.size / 1024))
line = linecache.getline(frame.filename, frame.lineno).strip()
if line:
print(' %s' % line)
other = top_stats[limit:] other = top_stats[limit:]
if other: if other:
...@@ -218,19 +222,28 @@ ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files:: ...@@ -218,19 +222,28 @@ ignoring ``<frozen importlib._bootstrap>`` and ``<unknown>`` files::
Example of output of the Python test suite:: Example of output of the Python test suite::
2013-11-08 14:16:58.149320: Top 10 lines Top 10 lines
#1: collections/__init__.py:368: 291.9 KiB #1: Lib/base64.py:414: 419.8 KiB
#2: Lib/doctest.py:1291: 200.2 KiB _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
#3: unittest/case.py:571: 160.3 KiB #2: Lib/base64.py:306: 419.8 KiB
#4: Lib/abc.py:133: 99.8 KiB _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
#5: urllib/parse.py:476: 71.8 KiB #3: collections/__init__.py:368: 293.6 KiB
#6: <string>:5: 62.7 KiB exec(class_definition, namespace)
#7: Lib/base64.py:140: 59.8 KiB #4: Lib/abc.py:133: 115.2 KiB
#8: Lib/_weakrefset.py:37: 51.8 KiB cls = super().__new__(mcls, name, bases, namespace)
#9: collections/__init__.py:362: 50.6 KiB #5: unittest/case.py:574: 103.1 KiB
#10: test/test_site.py:56: 48.0 KiB testMethod()
7496 other: 4161.9 KiB #6: Lib/linecache.py:127: 95.4 KiB
Total allocated size: 5258.8 KiB lines = fp.readlines()
#7: urllib/parse.py:476: 71.8 KiB
for a in _hexdig for b in _hexdig}
#8: <string>:5: 62.0 KiB
#9: Lib/_weakrefset.py:37: 60.0 KiB
self.data = set()
#10: Lib/base64.py:142: 59.8 KiB
_b32tab2 = [a + b for a in _b32tab for b in _b32tab]
6220 other: 3602.8 KiB
Total allocated size: 5303.1 KiB
See :meth:`Snapshot.statistics` for more options. See :meth:`Snapshot.statistics` for more options.
......
...@@ -15,6 +15,9 @@ It also defines names for some object types that are used by the standard ...@@ -15,6 +15,9 @@ It also defines names for some object types that are used by the standard
Python interpreter, but not exposed as builtins like :class:`int` or Python interpreter, but not exposed as builtins like :class:`int` or
:class:`str` are. :class:`str` are.
Finally, it provides some additional type-related utility classes and functions
that are not fundamental enough to be builtins.
Dynamic Type Creation Dynamic Type Creation
--------------------- ---------------------
...@@ -220,6 +223,9 @@ Standard names are defined for the following types: ...@@ -220,6 +223,9 @@ Standard names are defined for the following types:
Return a new view of the underlying mapping's values. Return a new view of the underlying mapping's values.
Additional Utility Classes and Functions
----------------------------------------
.. class:: SimpleNamespace .. class:: SimpleNamespace
A simple :class:`object` subclass that provides attribute access to its A simple :class:`object` subclass that provides attribute access to its
...@@ -246,3 +252,18 @@ Standard names are defined for the following types: ...@@ -246,3 +252,18 @@ Standard names are defined for the following types:
instead. instead.
.. versionadded:: 3.3 .. versionadded:: 3.3
.. function:: DynamicClassAttribute(fget=None, fset=None, fdel=None, doc=None)
Route attribute access on a class to __getattr__.
This is a descriptor, used to define attributes that act differently when
accessed through an instance and through a class. Instance access remains
normal, but access to an attribute through a class will be routed to the
class's __getattr__ method; this is done by raising AttributeError.
This allows one to have properties active on an instance, and have virtual
attributes on the class with the same name (see Enum for an example).
.. versionadded:: 3.4
...@@ -20,7 +20,7 @@ These modules are used to implement the :mod:`os.path` module, and are not ...@@ -20,7 +20,7 @@ These modules are used to implement the :mod:`os.path` module, and are not
documented beyond this mention. There's little need to document these. documented beyond this mention. There's little need to document these.
:mod:`ntpath` :mod:`ntpath`
--- Implementation of :mod:`os.path` on Win32, Win64, WinCE, and OS/2 platforms. --- Implementation of :mod:`os.path` on Win32, Win64, and WinCE platforms.
:mod:`posixpath` :mod:`posixpath`
--- Implementation of :mod:`os.path` on POSIX. --- Implementation of :mod:`os.path` on POSIX.
...@@ -426,7 +426,7 @@ mock using the "as" form of the with statement: ...@@ -426,7 +426,7 @@ mock using the "as" form of the with statement:
As an alternative `patch`, `patch.object` and `patch.dict` can be used as As an alternative `patch`, `patch.object` and `patch.dict` can be used as
class decorators. When used in this way it is the same as applying the class decorators. When used in this way it is the same as applying the
decorator indvidually to every method whose name starts with "test". decorator individually to every method whose name starts with "test".
.. _further-examples: .. _further-examples:
......
...@@ -958,7 +958,7 @@ method: ...@@ -958,7 +958,7 @@ method:
.. [#] The only exceptions are magic methods and attributes (those that have .. [#] The only exceptions are magic methods and attributes (those that have
leading and trailing double underscores). Mock doesn't create these but leading and trailing double underscores). Mock doesn't create these but
instead of raises an ``AttributeError``. This is because the interpreter instead raises an ``AttributeError``. This is because the interpreter
will often implicitly request these methods, and gets *very* confused to will often implicitly request these methods, and gets *very* confused to
get a new Mock object when it expects a magic method. If you need magic get a new Mock object when it expects a magic method. If you need magic
method support see :ref:`magic methods <magic-methods>`. method support see :ref:`magic methods <magic-methods>`.
...@@ -1509,7 +1509,7 @@ Patching Descriptors and Proxy Objects ...@@ -1509,7 +1509,7 @@ Patching Descriptors and Proxy Objects
Both patch_ and patch.object_ correctly patch and restore descriptors: class Both patch_ and patch.object_ correctly patch and restore descriptors: class
methods, static methods and properties. You should patch these on the *class* methods, static methods and properties. You should patch these on the *class*
rather than an instance. They also work with *some* objects rather than an instance. They also work with *some* objects
that proxy attribute access, like the `django setttings object that proxy attribute access, like the `django settings object
<http://www.voidspace.org.uk/python/weblog/arch_d7_2010_12_04.shtml#e1198>`_. <http://www.voidspace.org.uk/python/weblog/arch_d7_2010_12_04.shtml#e1198>`_.
......
...@@ -239,9 +239,10 @@ Test Discovery ...@@ -239,9 +239,10 @@ Test Discovery
Unittest supports simple test discovery. In order to be compatible with test Unittest supports simple test discovery. In order to be compatible with test
discovery, all of the test files must be :ref:`modules <tut-modules>` or discovery, all of the test files must be :ref:`modules <tut-modules>` or
:ref:`packages <tut-packages>` importable from the top-level directory of :ref:`packages <tut-packages>` (including :term:`namespace packages
the project (this means that their filenames must be valid <namespace package>`) importable from the top-level directory of
:ref:`identifiers <identifiers>`). the project (this means that their filenames must be valid :ref:`identifiers
<identifiers>`).
Test discovery is implemented in :meth:`TestLoader.discover`, but can also be Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
used from the command line. The basic command-line usage is:: used from the command line. The basic command-line usage is::
...@@ -306,6 +307,9 @@ as the start directory. ...@@ -306,6 +307,9 @@ as the start directory.
Test modules and packages can customize test loading and discovery by through Test modules and packages can customize test loading and discovery by through
the `load_tests protocol`_. the `load_tests protocol`_.
.. versionchanged:: 3.4
Test discovery supports :term:`namespace packages <namespace package>`.
.. _organizing-tests: .. _organizing-tests:
...@@ -1620,11 +1624,11 @@ Loading and running tests ...@@ -1620,11 +1624,11 @@ Loading and running tests
.. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
Find and return all test modules from the specified start directory, Find all the test modules by recursing into subdirectories from the
recursing into subdirectories to find them. Only test files that match specified start directory, and return a TestSuite object containing them.
*pattern* will be loaded. (Using shell style pattern matching.) Only Only test files that match *pattern* will be loaded. (Using shell style
module names that are importable (i.e. are valid Python identifiers) will pattern matching.) Only module names that are importable (i.e. are valid
be loaded. Python identifiers) will be loaded.
All test modules must be importable from the top level of the project. If All test modules must be importable from the top level of the project. If
the start directory is not the top level directory then the top level the start directory is not the top level directory then the top level
...@@ -1655,11 +1659,10 @@ Loading and running tests ...@@ -1655,11 +1659,10 @@ Loading and running tests
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Modules that raise :exc:`SkipTest` on import are recorded as skips, Modules that raise :exc:`SkipTest` on import are recorded as skips,
not errors. not errors.
Discovery works for :term:`namespace packages <namespace package>`.
.. versionchanged:: 3.4 Paths are sorted before being imported so that execution order is
Paths are sorted before being imported to ensure execution order for a the same even if the underlying file system's ordering is not
given test suite is the same even if the underlying file system's ordering dependent on file name.
is not dependent on file name like in ext3/4.
The following attributes of a :class:`TestLoader` can be configured either by The following attributes of a :class:`TestLoader` can be configured either by
......
...@@ -48,7 +48,7 @@ The following exceptions are raised by :mod:`urllib.error` as appropriate: ...@@ -48,7 +48,7 @@ The following exceptions are raised by :mod:`urllib.error` as appropriate:
.. attribute:: headers .. attribute:: headers
The HTTP response headers for the HTTP request that cause the The HTTP response headers for the HTTP request that caused the
:exc:`HTTPError`. :exc:`HTTPError`.
.. versionadded:: 3.4 .. versionadded:: 3.4
......
...@@ -218,7 +218,7 @@ The following classes are provided: ...@@ -218,7 +218,7 @@ The following classes are provided:
fetching of the image, this should be true. fetching of the image, this should be true.
*method* should be a string that indicates the HTTP request method that *method* should be a string that indicates the HTTP request method that
will be used (e.g. ``'HEAD'``). Its value is stored in the will be used (e.g. ``'HEAD'``). If provided, its value is stored in the
:attr:`~Request.method` attribute and is used by :meth:`get_method()`. :attr:`~Request.method` attribute and is used by :meth:`get_method()`.
Subclasses may indicate a default method by setting the Subclasses may indicate a default method by setting the
:attr:`~Request.method` attribute in the class itself. :attr:`~Request.method` attribute in the class itself.
...@@ -440,13 +440,20 @@ request. ...@@ -440,13 +440,20 @@ request.
.. attribute:: Request.method .. attribute:: Request.method
The HTTP request method to use. This value is used by The HTTP request method to use. By default its value is :const:`None`,
:meth:`~Request.get_method` to override the computed HTTP request which means that :meth:`~Request.get_method` will do its normal computation
method that would otherwise be returned. This attribute is initialized with of the method to be used. Its value can be set (thus overriding the default
the value of the *method* argument passed to the constructor. computation in :meth:`~Request.get_method`) either by providing a default
value by setting it at the class level in a :class:`Request` subclass, or by
passing a value in to the :class:`Request` constructor via the *method*
argument.
.. versionadded:: 3.3 .. versionadded:: 3.3
.. versionchanged:: 3.4
A default value can now be set in subclasses; previously it could only
be set via the constructor argument.
.. method:: Request.get_method() .. method:: Request.get_method()
......
...@@ -76,6 +76,8 @@ Creating virtual environments ...@@ -76,6 +76,8 @@ Creating virtual environments
without there needing to be any reference to its venv in ``PATH``. without there needing to be any reference to its venv in ``PATH``.
.. _venv-api:
API API
--- ---
......
...@@ -150,14 +150,30 @@ them, and is otherwise implementation dependent. ...@@ -150,14 +150,30 @@ them, and is otherwise implementation dependent.
Wave_write Objects Wave_write Objects
------------------ ------------------
For seekable output streams, the ``wave`` header will automatically be updated
to reflect the number of frames actually written. For unseekable streams, the
*nframes* value must be accurate when the first frame data is written. An
accurate *nframes* value can be achieved either by calling
:meth:`~Wave_write.setnframes` or :meth:`~Wave_write.setparams` with the number
of frames that will be written before :meth:`~Wave_write.close` is called and
then using :meth:`~Wave_write.writeframesraw` to write the frame data, or by
calling :meth:`~Wave_write.writeframes` with all of the frame data to be
written. In the latter case :meth:`~Wave_write.writeframes` will calculate
the number of frames in the data and set *nframes* accordingly before writing
the frame data.
Wave_write objects, as returned by :func:`.open`, have the following methods: Wave_write objects, as returned by :func:`.open`, have the following methods:
.. versionchanged:: 3.4
Added support for unseekable files.
.. method:: Wave_write.close() .. method:: Wave_write.close()
Make sure *nframes* is correct, and close the file if it was opened by Make sure *nframes* is correct, and close the file if it was opened by
:mod:`wave`. This method is called upon object collection. Can raise an :mod:`wave`. This method is called upon object collection. It will raise
exception if *nframes* is not correct and a file is not seekable. an exception if the output stream is not seekable and *nframes* does not
match the number of frames actually written.
.. method:: Wave_write.setnchannels(n) .. method:: Wave_write.setnchannels(n)
...@@ -181,8 +197,9 @@ Wave_write objects, as returned by :func:`.open`, have the following methods: ...@@ -181,8 +197,9 @@ Wave_write objects, as returned by :func:`.open`, have the following methods:
.. method:: Wave_write.setnframes(n) .. method:: Wave_write.setnframes(n)
Set the number of frames to *n*. This will be changed later if more frames are Set the number of frames to *n*. This will be changed later if the number
written. of frames actually written is different (this update attempt will
raise an error if the output stream is not seekable).
.. method:: Wave_write.setcomptype(type, name) .. method:: Wave_write.setcomptype(type, name)
...@@ -209,16 +226,18 @@ Wave_write objects, as returned by :func:`.open`, have the following methods: ...@@ -209,16 +226,18 @@ Wave_write objects, as returned by :func:`.open`, have the following methods:
Write audio frames, without correcting *nframes*. Write audio frames, without correcting *nframes*.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
.. method:: Wave_write.writeframes(data) .. method:: Wave_write.writeframes(data)
Write audio frames and make sure *nframes* is correct. Can raise an Write audio frames and make sure *nframes* is correct. It will raise an
exception if a file is not seekable. error if the output stream is not seekable and the total number of frames
that have been written after *data* has been written does not match the
previously set value for *nframes*.
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Any :term:`bytes-like object`\ s are now accepted. Any :term:`bytes-like object` is now accepted.
Note that it is invalid to set any parameters after calling :meth:`writeframes` Note that it is invalid to set any parameters after calling :meth:`writeframes`
......
...@@ -952,7 +952,8 @@ XMLParser Objects ...@@ -952,7 +952,8 @@ XMLParser Objects
specified in the XML file. specified in the XML file.
.. deprecated:: 3.4 .. deprecated:: 3.4
The *html* argument. The *html* argument. The remaining arguments should be passed via
keywword to prepare for the removal of the *html* argument.
.. method:: close() .. method:: close()
......
...@@ -14,9 +14,9 @@ Python's interfaces for processing XML are grouped in the ``xml`` package. ...@@ -14,9 +14,9 @@ Python's interfaces for processing XML are grouped in the ``xml`` package.
.. warning:: .. warning::
The XML modules are not secure against erroneous or maliciously The XML modules are not secure against erroneous or maliciously
constructed data. If you need to parse untrusted or unauthenticated data see constructed data. If you need to parse untrusted or
:ref:`xml-vulnerabilities`. unauthenticated data see the :ref:`xml-vulnerabilities` and
:ref:`defused-packages` sections.
It is important to note that modules in the :mod:`xml` package require that It is important to note that modules in the :mod:`xml` package require that
there be at least one SAX-compliant XML parser available. The Expat parser is there be at least one SAX-compliant XML parser available. The Expat parser is
...@@ -46,16 +46,15 @@ The XML handling submodules are: ...@@ -46,16 +46,15 @@ The XML handling submodules are:
.. _xml-vulnerabilities: .. _xml-vulnerabilities:
XML vulnerabilities XML vulnerabilities
=================== -------------------
The XML processing modules are not secure against maliciously constructed data. The XML processing modules are not secure against maliciously constructed data.
An attacker can abuse vulnerabilities for e.g. denial of service attacks, to An attacker can abuse XML features to carry out denial of service attacks,
access local files, to generate network connections to other machines, or access local files, generate network connections to other machines, or
to or circumvent firewalls. The attacks on XML abuse unfamiliar features circumvent firewalls.
like inline `DTD`_ (document type definition) with entities.
The following table gives an overview of the known attacks and if the various The following table gives an overview of the known attacks and whether
modules are vulnerable to them. the various modules are vulnerable to them.
========================= ======== ========= ========= ======== ========= ========================= ======== ========= ========= ======== =========
kind sax etree minidom pulldom xmlrpc kind sax etree minidom pulldom xmlrpc
...@@ -68,7 +67,7 @@ decompression bomb No No No No **Yes** ...@@ -68,7 +67,7 @@ decompression bomb No No No No **Yes**
========================= ======== ========= ========= ======== ========= ========================= ======== ========= ========= ======== =========
1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a 1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a
ParserError when an entity occurs. :exc:`ParserError` when an entity occurs.
2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns 2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
the unexpanded entity verbatim. the unexpanded entity verbatim.
3. :mod:`xmlrpclib` doesn't expand external entities and omits them. 3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
...@@ -77,23 +76,21 @@ decompression bomb No No No No **Yes** ...@@ -77,23 +76,21 @@ decompression bomb No No No No **Yes**
billion laughs / exponential entity expansion billion laughs / exponential entity expansion
The `Billion Laughs`_ attack -- also known as exponential entity expansion -- The `Billion Laughs`_ attack -- also known as exponential entity expansion --
uses multiple levels of nested entities. Each entity refers to another entity uses multiple levels of nested entities. Each entity refers to another entity
several times, the final entity definition contains a small string. Eventually several times, and the final entity definition contains a small string.
the small string is expanded to several gigabytes. The exponential expansion The exponential expansion results in several gigabytes of text and
consumes lots of CPU time, too. consumes lots of memory and CPU time.
quadratic blowup entity expansion quadratic blowup entity expansion
A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses
entity expansion, too. Instead of nested entities it repeats one large entity entity expansion, too. Instead of nested entities it repeats one large entity
with a couple of thousand chars over and over again. The attack isn't as with a couple of thousand chars over and over again. The attack isn't as
efficient as the exponential case but it avoids triggering countermeasures of efficient as the exponential case but it avoids triggering parser countermeasures
parsers against heavily nested entities. that forbid deeply-nested entities.
external entity expansion external entity expansion
Entity declarations can contain more than just text for replacement. They can Entity declarations can contain more than just text for replacement. They can
also point to external resources by public identifiers or system identifiers. also point to external resources or local files. The XML
System identifiers are standard URIs or can refer to local files. The XML parser accesses the resource and embeds the content into the XML document.
parser retrieves the resource with e.g. HTTP or FTP requests and embeds the
content into the XML document.
DTD retrieval DTD retrieval
Some XML libraries like Python's :mod:`xml.dom.pulldom` retrieve document type Some XML libraries like Python's :mod:`xml.dom.pulldom` retrieve document type
...@@ -101,31 +98,32 @@ DTD retrieval ...@@ -101,31 +98,32 @@ DTD retrieval
implications as the external entity expansion issue. implications as the external entity expansion issue.
decompression bomb decompression bomb
The issue of decompression bombs (aka `ZIP bomb`_) apply to all XML libraries Decompression bombs (aka `ZIP bomb`_) apply to all XML libraries
that can parse compressed XML stream like gzipped HTTP streams or LZMA-ed that can parse compressed XML streams such as gzipped HTTP streams or
LZMA-compressed
files. For an attacker it can reduce the amount of transmitted data by three files. For an attacker it can reduce the amount of transmitted data by three
magnitudes or more. magnitudes or more.
The documentation of `defusedxml`_ on PyPI has further information about The documentation for `defusedxml`_ on PyPI has further information about
all known attack vectors with examples and references. all known attack vectors with examples and references.
defused packages .. _defused-packages:
----------------
`defusedxml`_ is a pure Python package with modified subclasses of all stdlib The :mod:`defusedxml` and :mod:`defusedexpat` Packages
XML parsers that prevent any potentially malicious operation. The courses of ------------------------------------------------------
action are recommended for any server code that parses untrusted XML data. The
package also ships with example exploits and an extended documentation on more
XML exploits like xpath injection.
`defusedexpat`_ provides a modified libexpat and patched replacment `defusedxml`_ is a pure Python package with modified subclasses of all stdlib
:mod:`pyexpat` extension module with countermeasures against entity expansion XML parsers that prevent any potentially malicious operation. Use of this
DoS attacks. Defusedexpat still allows a sane and configurable amount of entity package is recommended for any server code that parses untrusted XML data. The
expansions. The modifications will be merged into future releases of Python. package also ships with example exploits and extended documentation on more
XML exploits such as XPath injection.
The workarounds and modifications are not included in patch releases as they `defusedexpat`_ provides a modified libexpat and a patched
break backward compatibility. After all inline DTD and entity expansion are :mod:`pyexpat` module that have countermeasures against entity expansion
well-definied XML features. DoS attacks. The :mod:`defusedexpat` module still allows a sane and configurable amount of entity
expansions. The modifications may be included in some future release of Python,
but will not be included in any bugfix releases of
Python because they break backward compatibility.
.. _defusedxml: https://pypi.python.org/pypi/defusedxml/ .. _defusedxml: https://pypi.python.org/pypi/defusedxml/
...@@ -133,4 +131,3 @@ well-definied XML features. ...@@ -133,4 +131,3 @@ well-definied XML features.
.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs .. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs
.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb .. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb
.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition .. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition
...@@ -401,18 +401,32 @@ The :class:`PyZipFile` constructor takes the same parameters as the ...@@ -401,18 +401,32 @@ The :class:`PyZipFile` constructor takes the same parameters as the
``2``, only files with that optimization level (see :func:`compile`) are ``2``, only files with that optimization level (see :func:`compile`) are
added to the archive, compiling if necessary. added to the archive, compiling if necessary.
If the pathname is a file, the filename must end with :file:`.py`, and If *pathname* is a file, the filename must end with :file:`.py`, and
just the (corresponding :file:`\*.py[co]`) file is added at the top level just the (corresponding :file:`\*.py[co]`) file is added at the top level
(no path information). If the pathname is a file that does not end with (no path information). If *pathname* is a file that does not end with
:file:`.py`, a :exc:`RuntimeError` will be raised. If it is a directory, :file:`.py`, a :exc:`RuntimeError` will be raised. If it is a directory,
and the directory is not a package directory, then all the files and the directory is not a package directory, then all the files
:file:`\*.py[co]` are added at the top level. If the directory is a :file:`\*.py[co]` are added at the top level. If the directory is a
package directory, then all :file:`\*.py[co]` are added under the package package directory, then all :file:`\*.py[co]` are added under the package
name as a file path, and if any subdirectories are package directories, name as a file path, and if any subdirectories are package directories,
all of these are added recursively. *basename* is intended for internal all of these are added recursively.
use only. When *filterfunc(pathname)* is given, it will be called for every
invocation. When it returns a false value, that path and its subpaths will *basename* is intended for internal use only.
be ignored.
*filterfunc*, if given, must be a function taking a single string
argument. It will be passed each path (including each individual full
file path) before it is added to the archive. If *filterfunc* returns a
false value, the path will not be added, and if it is a directory its
contents will be ignored. For example, if our test files are all either
in ``test`` directories or start with the string ``test_``, we can use a
*filterfunc* to exclude them::
>>> zf = PyZipFile('myprog.zip')
>>> def notests(s):
... fn = os.path.basename(s)
... return (not (fn == 'test' or fn.startswith('test_')))
>>> zf.writepy('myprog', filterfunc=notests)
The :meth:`writepy` method makes archives with file names like The :meth:`writepy` method makes archives with file names like
this:: this::
......
...@@ -1226,6 +1226,10 @@ Basic customization ...@@ -1226,6 +1226,10 @@ Basic customization
The return value must be a string object. The return value must be a string object.
.. versionchanged:: 3.4
The __format__ method of ``object`` itself raises a :exc:`TypeError`
if passed any non-empty string.
.. _richcmpfuncs: .. _richcmpfuncs:
.. method:: object.__lt__(self, other) .. method:: object.__lt__(self, other)
...@@ -1643,6 +1647,8 @@ of these candidate metaclasses. If none of the candidate metaclasses meets ...@@ -1643,6 +1647,8 @@ of these candidate metaclasses. If none of the candidate metaclasses meets
that criterion, then the class definition will fail with ``TypeError``. that criterion, then the class definition will fail with ``TypeError``.
.. _prepare:
Preparing the class namespace Preparing the class namespace
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...@@ -2044,11 +2050,13 @@ left undefined. ...@@ -2044,11 +2050,13 @@ left undefined.
``&=``, ``^=``, ``|=``). These methods should attempt to do the operation ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation
in-place (modifying *self*) and return the result (which could be, but does in-place (modifying *self*) and return the result (which could be, but does
not have to be, *self*). If a specific method is not defined, the augmented not have to be, *self*). If a specific method is not defined, the augmented
assignment falls back to the normal methods. For instance, to execute the assignment falls back to the normal methods. For instance, if *x* is an
statement ``x += y``, where *x* is an instance of a class that has an instance of a class with an :meth:`__iadd__` method, ``x += y`` is equivalent
:meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and ``y.__radd__(x)``
of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)`` are considered, as with the evaluation of ``x + y``. In certain situations,
and ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. augmented assignment can result in unexpected errors (see
:ref:`faq-augmented-assignment-tuple-error`), but this behavior is in
fact part of the data model.
.. method:: object.__neg__(self) .. method:: object.__neg__(self)
......
...@@ -16,14 +16,14 @@ ...@@ -16,14 +16,14 @@
<p class="biglink"><a class="biglink" href="{{ pathto("howto/index") }}">Python HOWTOs</a><br/> <p class="biglink"><a class="biglink" href="{{ pathto("howto/index") }}">Python HOWTOs</a><br/>
<span class="linkdescr">in-depth documents on specific topics</span></p> <span class="linkdescr">in-depth documents on specific topics</span></p>
</td><td width="50%"> </td><td width="50%">
<p class="biglink"><a class="biglink" href="{{ pathto("installing/index") }}">Installing Python Modules</a><br/>
<span class="linkdescr">installing from the Python Package Index &amp; other sources</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("distributing/index") }}">Distributing Python Modules</a><br/>
<span class="linkdescr">publishing modules for installation by others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("extending/index") }}">Extending and Embedding</a><br/> <p class="biglink"><a class="biglink" href="{{ pathto("extending/index") }}">Extending and Embedding</a><br/>
<span class="linkdescr">tutorial for C/C++ programmers</span></p> <span class="linkdescr">tutorial for C/C++ programmers</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("c-api/index") }}">Python/C API</a><br/> <p class="biglink"><a class="biglink" href="{{ pathto("c-api/index") }}">Python/C API</a><br/>
<span class="linkdescr">reference for C/C++ programmers</span></p> <span class="linkdescr">reference for C/C++ programmers</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("install/index") }}">Installing Python Modules</a><br/>
<span class="linkdescr">information for installers &amp; sys-admins</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("distutils/index") }}">Distributing Python Modules</a><br/>
<span class="linkdescr">sharing modules with others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">FAQs</a><br/> <p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">FAQs</a><br/>
<span class="linkdescr">frequently asked questions (with answers!)</span></p> <span class="linkdescr">frequently asked questions (with answers!)</span></p>
</td></tr> </td></tr>
......
...@@ -282,3 +282,4 @@ whatsnew/changelog,,:PythonCmd,"With Tk < 8.5 _tkinter.c:PythonCmd() raised Unic ...@@ -282,3 +282,4 @@ whatsnew/changelog,,:PythonCmd,"With Tk < 8.5 _tkinter.c:PythonCmd() raised Unic
whatsnew/changelog,,::,": Fix FTP tests for IPv6, bind to ""::1"" instead of ""localhost""." whatsnew/changelog,,::,": Fix FTP tests for IPv6, bind to ""::1"" instead of ""localhost""."
whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as"
whatsnew/changelog,,:password,user:password whatsnew/changelog,,:password,user:password
whatsnew/changelog,,:gz,w:gz
...@@ -371,9 +371,9 @@ values. The most versatile is the *list*, which can be written as a list of ...@@ -371,9 +371,9 @@ values. The most versatile is the *list*, which can be written as a list of
comma-separated values (items) between square brackets. Lists might contain comma-separated values (items) between square brackets. Lists might contain
items of different types, but usually the items all have the same type. :: items of different types, but usually the items all have the same type. ::
>>> squares = [1, 2, 4, 9, 16, 25] >>> squares = [1, 4, 9, 16, 25]
>>> squares >>> squares
[1, 2, 4, 9, 16, 25] [1, 4, 9, 16, 25]
Like strings (and all other built-in :term:`sequence` type), lists can be Like strings (and all other built-in :term:`sequence` type), lists can be
indexed and sliced:: indexed and sliced::
...@@ -389,12 +389,12 @@ All slice operations return a new list containing the requested elements. This ...@@ -389,12 +389,12 @@ All slice operations return a new list containing the requested elements. This
means that the following slice returns a new (shallow) copy of the list:: means that the following slice returns a new (shallow) copy of the list::
>>> squares[:] >>> squares[:]
[1, 2, 4, 9, 16, 25] [1, 4, 9, 16, 25]
Lists also supports operations like concatenation:: Lists also supports operations like concatenation::
>>> squares + [36, 49, 64, 81, 100] >>> squares + [36, 49, 64, 81, 100]
[1, 2, 4, 9, 16, 25, 36, 49, 64, 81, 100] [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
Unlike strings, which are :term:`immutable`, lists are a :term:`mutable` Unlike strings, which are :term:`immutable`, lists are a :term:`mutable`
type, i.e. it is possible to change their content:: type, i.e. it is possible to change their content::
......
...@@ -21,8 +21,8 @@ the set are: ...@@ -21,8 +21,8 @@ the set are:
and many other tasks. Skimming through the Library Reference will give you an and many other tasks. Skimming through the Library Reference will give you an
idea of what's available. idea of what's available.
* :ref:`install-index` explains how to install external modules written by other * :ref:`installing-index` explains how to install additional modules written
Python users. by other Python users.
* :ref:`reference-index`: A detailed explanation of Python's syntax and * :ref:`reference-index`: A detailed explanation of Python's syntax and
semantics. It's heavy reading, but is useful as a complete guide to the semantics. It's heavy reading, but is useful as a complete guide to the
......
...@@ -60,6 +60,8 @@ To install the newest Python versions on OpenSolaris, install `blastwave ...@@ -60,6 +60,8 @@ To install the newest Python versions on OpenSolaris, install `blastwave
prompt. prompt.
.. _building-python-on-unix:
Building Python Building Python
=============== ===============
......
...@@ -11,6 +11,11 @@ containing a copy of the ``python`` binary (or binaries, in the case of ...@@ -11,6 +11,11 @@ containing a copy of the ``python`` binary (or binaries, in the case of
Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages`` Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages``
subdirectory (on Windows, this is ``Lib\site-packages``). subdirectory (on Windows, this is ``Lib\site-packages``).
.. seealso::
`Python Packaging User Guide: Creating and using virtual environments
<http://packaging.python.org/en/latest/tutorial.html#creating-and-using-virtual-environments>`__
.. highlight:: none .. highlight:: none
On Windows, you may have to invoke the ``pyvenv`` script as follows, if you On Windows, you may have to invoke the ``pyvenv`` script as follows, if you
...@@ -38,6 +43,8 @@ The command, if run with ``-h``, will show the available options:: ...@@ -38,6 +43,8 @@ The command, if run with ``-h``, will show the available options::
virtual environment. virtual environment.
--symlinks Try to use symlinks rather than copies, when symlinks --symlinks Try to use symlinks rather than copies, when symlinks
are not the default for the platform. are not the default for the platform.
--copies Try to use copies rather than symlinks, even when
symlinks are the default for the platform.
--clear Delete the environment directory if it already exists. --clear Delete the environment directory if it already exists.
If not specified and the directory exists, an error is If not specified and the directory exists, an error is
raised. raised.
...@@ -47,7 +54,8 @@ The command, if run with ``-h``, will show the available options:: ...@@ -47,7 +54,8 @@ The command, if run with ``-h``, will show the available options::
environment (pip is bootstrapped by default) environment (pip is bootstrapped by default)
.. versionchanged:: 3.4 .. versionchanged:: 3.4
Installs pip by default, added the ``--without-pip`` option Installs pip by default, added the ``--without-pip`` and ``--copies``
options
If the target directory already exists an error will be raised, unless If the target directory already exists an error will be raised, unless
the ``--clear`` or ``--upgrade`` option was provided. the ``--clear`` or ``--upgrade`` option was provided.
......
...@@ -11,6 +11,10 @@ ...@@ -11,6 +11,10 @@
This document aims to give an overview of Windows-specific behaviour you should This document aims to give an overview of Windows-specific behaviour you should
know about when using Python on Microsoft Windows. know about when using Python on Microsoft Windows.
.. XXX (ncoghlan)
This looks rather stale to me...
Installing Python Installing Python
================= =================
......
This diff is collapsed.
...@@ -7,15 +7,17 @@ What's New in Python 3.4.0? ...@@ -7,15 +7,17 @@ What's New in Python 3.4.0?
Release date: 2014-03-16 Release date: 2014-03-16
Core and Builtins
-----------------
Library Library
------- -------
- Issue #20939: Fix test_geturl failure in test_urllibnet due to - Issue #20939: Fix test_geturl failure in test_urllibnet due to
new redirect of http://www.python.org/ to https://www.python.org. new redirect of http://www.python.org/ to https://www.python.org.
Documentation
-------------
- Merge in all documentation changes since branching 3.4.0rc1.
What's New in Python 3.4.0 release candidate 3? What's New in Python 3.4.0 release candidate 3?
=============================================== ===============================================
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment