Branch merge

Doc/glossary.rst

@@ -27,12 +27,14 @@ Glossary
       :ref:`2to3-reference`.
 
    abstract base class
-      :ref:`abstract-base-classes` complement :term:`duck-typing` by
+      Abstract base classes complement :term:`duck-typing` by
       providing a way to define interfaces when other techniques like
-      :func:`hasattr` would be clumsy. Python comes with many built-in ABCs for
-      data structures (in the :mod:`collections` module), numbers (in the
-      :mod:`numbers` module), and streams (in the :mod:`io` module). You can
-      create your own ABC with the :mod:`abc` module.
+      :func:`hasattr` would be clumsy or subtly wrong (for example with
+      :ref:`magic methods <special-lookup>`). Python comes with many built-in ABCs for
+      data structures (in the :mod:`collections.abc` module), numbers (in the
+      :mod:`numbers` module), streams (in the :mod:`io` module), import finders
+      and loaders (in the :mod:`importlib.abc` module).  You can create your own
+      ABCs with the :mod:`abc` module.
 
    argument
       A value passed to a function or method, assigned to a named local
@@ -430,8 +432,8 @@ Glossary
    mapping
       A container object that supports arbitrary key lookups and implements the
       methods specified in the :class:`Mapping` or :class:`MutableMapping`
-      :ref:`abstract base classes <abstract-base-classes>`. Examples include
-      :class:`dict`, :class:`collections.defaultdict`,
+      :ref:`abstract base classes <collections-abstract-base-classes>`.  Examples
+      include :class:`dict`, :class:`collections.defaultdict`,
       :class:`collections.OrderedDict` and :class:`collections.Counter`.
 
    metaclass

Doc/library/abc.rst

-.. _abstract-base-classes:
-
 :mod:`abc` --- Abstract Base Classes
 ====================================
 
@@ -20,7 +18,7 @@ regarding a type hierarchy for numbers based on ABCs.)
 
 The :mod:`collections` module has some concrete classes that derive from
 ABCs; these can, of course, be further derived. In addition the
-:mod:`collections` module has some ABCs that can be used to test whether
+:mod:`collections.abc` submodule has some ABCs that can be used to test whether
 a class or instance provides a particular interface, for example, is it
 hashable or a mapping.
 

Doc/library/collections.abc.rst

@@ -44,7 +44,7 @@ ABC                        Inherits from          Abstract Methods        Mixin
                            :class:`Iterable`,                             ``index``, and ``count``
                            :class:`Container`
 
-:class:`MutableSequence`   :class:`Sequence`      ``__setitem__``         Inherited :class:`Sequence` methods and
+:class:`MutableSequence`   :class:`Sequence`      ``__setitem__``,        Inherited :class:`Sequence` methods and
                                                   ``__delitem__``,        ``append``, ``reverse``, ``extend``, ``pop``,
                                                   ``insert``              ``remove``, ``clear``, and ``__iadd__``
 
@@ -175,7 +175,7 @@ Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
 
 .. seealso::
 
-  * `OrderedSet recipe <http://code.activestate.com/recipes/576694/>`_ that uses
-    :class:`MutableSet`.
+   * `OrderedSet recipe <http://code.activestate.com/recipes/576694/>`_ for an
+     example built on :class:`MutableSet`.
 
-  * For more about ABCs, see the :mod:`abc` module and :pep:`3119`.
+   * For more about ABCs, see the :mod:`abc` module and :pep:`3119`.

Doc/library/numbers.rst

@@ -5,9 +5,9 @@
    :synopsis: Numeric abstract base classes (Complex, Real, Integral, etc.).
 
 
-The :mod:`numbers` module (:pep:`3141`) defines a hierarchy of numeric abstract
-base classes which progressively define more operations.  None of the types
-defined in this module can be instantiated.
+The :mod:`numbers` module (:pep:`3141`) defines a hierarchy of numeric
+:term:`abstract base classes <abstract base class>` which progressively define
+more operations.  None of the types defined in this module can be instantiated.
 
 
 .. class:: Number

Doc/packaging/commandhooks.rst

+.. TODO integrate this in commandref and configfile
+
 =============
 Command hooks
 =============
@@ -9,6 +11,9 @@ The pre-hooks are run after the command is finalized (its options are
 processed), but before it is run. The post-hooks are run after the command
 itself. Both types of hooks receive an instance of the command object.
 
+See also global setup hooks in :ref:`setupcfg-spec`.
+
+
 Sample usage of hooks
 =====================
 

Doc/packaging/setupcfg.rst

 .. highlightlang:: cfg
 
+.. _setupcfg-spec:
+
 *******************************************
 Specification of the :file:`setup.cfg` file
 *******************************************
 
-.. :version: 1.0
+:version: 0.9
 
 This document describes the :file:`setup.cfg`, an ini-style configuration file
-(compatible with :class:`configparser.RawConfigParser`) configuration file used
-by Packaging to replace the :file:`setup.py` file.
+used by Packaging to replace the :file:`setup.py` file used by Distutils.
+This specification is language-agnostic, and will therefore repeat some
+information that's already documented for Python in the
+:class:`configparser.RawConfigParser` documentation.
+
+.. contents::
+   :depth: 3
+   :local:
+
+
+Syntax
+======
+
+The ini-style format used in the configuration file is a simple collection of
+sections that group sets of key-value fields separated by ``=`` or ``:`` and
+optional whitespace.  Lines starting with ``#`` or ``;`` are comments and will
+be ignored.  Empty lines are also ignored.  Example::
+
+   [section1]
+   # comment
+   name = value
+   name2 = "other value"
+
+   [section2]
+   foo = bar
+
+
+Parsing values
+---------------
+
+Here are a set of rules to parse values:
+
+- If a value is quoted with ``"`` chars, it's a string.  If a quote character is
+  present in the quoted value, it can be escaped as ``\"`` or left as-is.
+
+- If the value is ``true``, ``t``, ``yes``, ``y`` (case-insensitive) or ``1``,
+  it's converted to the language equivalent of a ``True`` value; if it's
+  ``false``, ``f``, ``no``, ``n`` (case-insensitive) or ``0``, it's converted to
+  the equivalent of ``False``.
+
+- A value can contain multiple lines.  When read, lines are converted into a
+  sequence of values.  Each line after the first must start with a least one
+  space or tab character; this leading indentation will be stripped.
+
+- All other values are considered strings.
+
+Examples::
+
+   [section]
+   foo = one
+         two
+         three
+
+   bar = false
+   baz = 1.3
+   boo = "ok"
+   beee = "wqdqw pojpj w\"ddq"
+
+
+Extending files
+---------------
+
+A configuration file can be extended (i.e. included) by other files.  For this,
+a ``DEFAULT`` section must contain an ``extends`` key which value points to one
+or more files which will be merged into the current files by adding new sections
+and fields.  If a file loaded by ``extends`` contains sections or keys that
+already exist in the original file, they will not override the previous values.
+
+Contents of :file:`one.cfg`::
+
+    [section1]
+    name = value
+
+    [section2]
+    foo = foo from one.cfg
+
+Contents of :file:`two.cfg`::
+
+    [DEFAULT]
+    extends = one.cfg
+
+    [section2]
+    foo = foo from two.cfg
+    baz = baz from two.cfg
+
+The result of parsing :file:`two.cfg` is equivalent to this file::
+
+    [section1]
+    name = value
+
+    [section2]
+    foo = foo from one.cfg
+    baz = baz from two.cfg
+
+Example use of multi-line notation to include more than one file::
+
+    [DEFAULT]
+    extends = one.cfg
+              two.cfg
+
+When several files are provided, they are processed sequentially, following the
+precedence rules explained above.  This means that the list of files should go
+from most specialized to most common.
+
+**Tools will need to provide a way to produce a merged version of the
+file**.  This will be useful to let users publish a single file.
+
+
+Description of sections and fields
+==================================
 
 Each section contains a description of its options.
 
@@ -108,10 +218,6 @@ description
    in Distutils1.) A file can be provided in the *description-file* field.
    *optional*
 
-description-file
-   path to a text file that will be used for the
-   **description** field. *optional*
-
 keywords
    A list of additional keywords to be used to assist searching
    for the distribution in a larger catalog. Comma or space-separated.
@@ -172,6 +278,13 @@ project-url
    A label, followed by a browsable URL for the project.
    "label, url". The label is limited to 32 signs. *optional*, *multi*
 
+One extra field not present in PEP 345 is supported:
+
+description-file
+   Path to a text file that will be used to fill the ``description`` field.
+   ``description-file`` and ``description`` are mutually exclusive.  *optional*
+
+
 
 Example::
 
@@ -278,7 +391,7 @@ The final paths where files will be placed are composed by : **source** +
 **destination**. In the previous example, **doc/doc.man** will be placed in
 **destination_doc/doc/doc.man** and **scripts/foo.sh** will be placed in
 **destination_scripts/scripts/foo.sh**. (If you want more control on the final
-path, take a look at base_prefix_).
+path, take a look at :ref:`setupcfg-resources-base-prefix`).
 
 The **destination** part of resources declaration are paths with categories.
 Indeed, it's generally a bad idea to give absolute path as it will be cross
@@ -402,13 +515,13 @@ Your **files** section will be::
 More control on destination part
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. _base_prefix:
+.. _setupcfg-resources-base-prefix:
 
 Defining a base prefix
 """"""""""""""""""""""
 
 When you define your resources, you can have more control of how the final path
-is compute.
+is computed.
 
 By default, the final path is::
 
@@ -435,7 +548,7 @@ The **prefix** is "docs/" and the **suffix** is "doc.html".
 
 .. note::
 
-   Glob syntax is working the same way with standard source and splitted source.
+   Glob syntax is working the same way with standard source and split source.
    So these rules::
 
       docs/*
@@ -444,7 +557,7 @@ The **prefix** is "docs/" and the **suffix** is "doc.html".
 
    Will match all the files in the docs directory.
 
-When you use splitted source, the final path is compute in this way::
+When you use split source, the final path is computed this way::
 
    destination + prefix
 
@@ -646,3 +759,64 @@ section named after the command.  Example::
 
 Option values given in the configuration file can be overriden on the command
 line.  See :ref:`packaging-setup-config` for more information.
+
+
+Extensibility
+=============
+
+Every section can have fields that are not part of this specification.  They are
+called **extensions**.
+
+An extension field starts with ``X-``.  Example::
+
+   [metadata]
+   name = Distribute
+   X-Debian-Name = python-distribute
+
+
+Changes in the specification
+============================
+
+The versioning scheme for this specification is **MAJOR.MINOR**.  Changes in the
+specification will cause the version number to be updated.
+
+Changes to the minor number reflect backwards-compatible changes:
+
+- New fields and sections (optional or mandatory) can be added.
+- Optional fields can be removed.
+
+The major number will be incremented for backwards-incompatible changes:
+
+- Mandatory fields or sections are removed.
+- Fields change their meaning.
+
+As a consequence, a tool written to consume 1.5 has these properties:
+
+- Can read 1.1, 1.2 and all versions < 1.5, since the tool knows what
+  optional fields weren't there.
+
+  .. XXX clarify
+
+- Can also read 1.6 and other 1.x versions: The tool will just ignore fields it
+  doesn't know about, even if they are mandatory in the new version.  If
+  optional fields were removed, the tool will just consider them absent.
+
+- Cannot read 2.x and should refuse to interpret such files.
+
+A tool written to produce 1.x should have these properties:
+
+- Writes all mandatory fields.
+- May write optional fields.
+
+
+Acknowledgments
+===============
+
+This specification includes work and feedback from these people:
+
+- Tarek Ziadé
+- Julien Jehannet
+- Boris Feld
+- Éric Araujo
+
+(If your name is missing, please :ref:`let us know <reporting-bugs>`.)

Lib/packaging/compiler/cygwinccompiler.py

@@ -48,7 +48,6 @@ of GCC (same as cygwin in no-cygwin mode).
 
 import os
 import sys
-import copy
 
 from packaging import logger
 from packaging.compiler.unixccompiler import UnixCCompiler
@@ -172,9 +171,9 @@ class CygwinCCompiler(UnixCCompiler):
              extra_postargs=None, build_temp=None, target_lang=None):
         """Link the objects."""
         # use separate copies, so we can modify the lists
-        extra_preargs = copy.copy(extra_preargs or [])
-        libraries = copy.copy(libraries or [])
-        objects = copy.copy(objects or [])
+        extra_preargs = list(extra_preargs or [])
+        libraries = list(libraries or [])
+        objects = list(objects or [])
 
         # Additional libraries
         libraries.extend(self.dll_libraries)

Lib/packaging/create.py

-#!/usr/bin/env python
 """Interactive helper used to create a setup.cfg file.
 
 This script will generate a packaging configuration file by looking at
 the current directory and asking the user questions.  It is intended to
-be called as
-
-  pysetup create
-
-or
-
-  python3.3 -m packaging.create
+be called as *pysetup create*.
 """
 
 #  Original code by Sean Reifschneider <jafo@tummy.com>
@@ -26,17 +19,17 @@ or
 #  Detect scripts (not sure how.  #! outside of package?)
 
 import os
+import re
 import imp
 import sys
 import glob
-import re
 import shutil
 import sysconfig
 import tokenize
-from configparser import RawConfigParser
-from textwrap import dedent
 from hashlib import md5
+from textwrap import dedent
 from functools import cmp_to_key
+from configparser import RawConfigParser
 # importing this with an underscore as it should be replaced by the
 # dict form or another structures for all purposes
 from packaging._trove import all_classifiers as _CLASSIFIERS_LIST
@@ -68,7 +61,7 @@ E-mail address of the project author (typically you).
     'do_classifier': '''
 Trove classifiers are optional identifiers that allow you to specify the
 intended audience by saying things like "Beta software with a text UI
-for Linux under the PSF license.  However, this can be a somewhat involved
+for Linux under the PSF license".  However, this can be a somewhat involved
 process.
 ''',
     'packages': '''
@@ -95,7 +88,7 @@ human language, programming language, user interface, etc...
 ''',
     'setup.py found': '''
 The setup.py script will be executed to retrieve the metadata.
-A wizard will be run if you answer "n",
+An interactive helper will be run if you answer "n",
 ''',
 }
 
@@ -230,7 +223,7 @@ class MainProgram:
         self._write_cfg()
 
     def has_setup_py(self):
-        """Test for the existance of a setup.py file."""
+        """Test for the existence of a setup.py file."""
         return os.path.exists('setup.py')
 
     def define_cfg_values(self):
@@ -281,9 +274,13 @@ class MainProgram:
 
         with open(_FILENAME, 'w', encoding='utf-8') as fp:
             fp.write('[metadata]\n')
+            # TODO use metadata module instead of hard-coding field-specific
+            # behavior here
+
             # simple string entries
             for name in ('name', 'version', 'summary', 'download_url'):
                 fp.write('%s = %s\n' % (name, self.data.get(name, 'UNKNOWN')))
+
             # optional string entries
             if 'keywords' in self.data and self.data['keywords']:
                 fp.write('keywords = %s\n' % ' '.join(self.data['keywords']))
@@ -295,6 +292,7 @@ class MainProgram:
                 fp.write(
                     'description = %s\n'
                     % '\n       |'.join(self.data['description'].split('\n')))
+
             # multiple use string entries
             for name in ('platform', 'supported-platform', 'classifier',
                          'requires-dist', 'provides-dist', 'obsoletes-dist',
@@ -329,8 +327,8 @@ class MainProgram:
 
         def setup_mock(**attrs):
             """Mock the setup(**attrs) in order to retrieve metadata."""
-            # use the distutils v1 processings to correctly parse metadata.
-            #XXX we could also use the setuptools distibution ???
+
+            # TODO use config and metadata instead of Distribution
             from distutils.dist import Distribution
             dist = Distribution(attrs)
             dist.parse_config_files()
@@ -362,13 +360,14 @@ class MainProgram:
             data['modules'].extend(dist.py_modules or [])
             # 2.1 data_files -> resources
             if dist.data_files:
-                if len(dist.data_files) < 2 or \
-                   isinstance(dist.data_files[1], str):
+                if (len(dist.data_files) < 2 or
+                    isinstance(dist.data_files[1], str)):
                     dist.data_files = [('', dist.data_files)]
                 # add tokens in the destination paths
                 vars = {'distribution.name': data['name']}
                 path_tokens = list(sysconfig.get_paths(vars=vars).items())
 
+                # TODO replace this with a key function
                 def length_comparison(x, y):
                     len_x = len(x[1])
                     len_y = len(y[1])
@@ -391,12 +390,12 @@ class MainProgram:
 
                         dest = ('{%s}' % tok) + dest[len(path):]
                         files = [('/ '.join(src.rsplit('/', 1)), dest)
-                                    for src in srcs]
+                                 for src in srcs]
                         data['resources'].extend(files)
 
             # 2.2 package_data -> extra_files
             package_dirs = dist.package_dir or {}
-            for package, extras in iter(dist.package_data.items()) or []:
+            for package, extras in dist.package_data.items() or []:
                 package_dir = package_dirs.get(package, package)
                 for file_ in extras:
                     if package_dir:
@@ -458,10 +457,10 @@ class MainProgram:
         if match:
             self.data['name'] = match.group(1)
             self.data['version'] = match.group(2)
-            # TODO Needs tested!
+            # TODO needs testing!
             if not is_valid_version(self.data['version']):
                 msg = "Invalid version discovered: %s" % self.data['version']
-                raise RuntimeError(msg)
+                raise ValueError(msg)
 
     def query_user(self):
         self.data['name'] = ask('Project name', self.data['name'],
@@ -476,25 +475,25 @@ class MainProgram:
               self.data.get('author'), _helptext['author'])
         self.data['author_email'] = ask('Author e-mail address',
               self.data.get('author_email'), _helptext['author_email'])
-        self.data['home_page'] = ask('Project Home Page',
+        self.data['home_page'] = ask('Project home page',
               self.data.get('home_page'), _helptext['home_page'],
               required=False)
 
         if ask_yn('Do you want me to automatically build the file list '
-              'with everything I can find in the current directory ? '
+              'with everything I can find in the current directory? '
               'If you say no, you will have to define them manually.') == 'y':
             self._find_files()
         else:
-            while ask_yn('Do you want to add a single module ?'
+            while ask_yn('Do you want to add a single module?'
                         ' (you will be able to add full packages next)',
                     helptext=_helptext['modules']) == 'y':
                 self._set_multi('Module name', 'modules')
 
-            while ask_yn('Do you want to add a package ?',
+            while ask_yn('Do you want to add a package?',
                     helptext=_helptext['packages']) == 'y':
                 self._set_multi('Package name', 'packages')
 
-            while ask_yn('Do you want to add an extra file ?',
+            while ask_yn('Do you want to add an extra file?',
                         helptext=_helptext['extra_files']) == 'y':
                 self._set_multi('Extra file/dir name', 'extra_files')
 
@@ -582,7 +581,7 @@ class MainProgram:
         self.set_other_classifier(self.classifiers)
 
     def set_other_classifier(self, classifiers):
-        if ask_yn('Do you want to set other trove identifiers', 'n',
+        if ask_yn('Do you want to set other trove identifiers?', 'n',
                   _helptext['trove_generic']) != 'y':
             return
         self.walk_classifiers(classifiers, [CLASSIFIERS], '')
@@ -599,7 +598,7 @@ class MainProgram:
                     classifiers.add(desc[4:] + ' :: ' + key)
                 continue
 
-            if ask_yn('Do you want to set items under\n   "%s" (%d sub-items)'
+            if ask_yn('Do you want to set items under\n   "%s" (%d sub-items)?'
                       % (key, len(trove[key])), 'n',
                       _helptext['trove_generic']) == 'y':
                 self.walk_classifiers(classifiers, trovepath + [trove[key]],
@@ -607,7 +606,7 @@ class MainProgram:
 
     def set_license(self, classifiers):
         while True:
-            license = ask('What license do you use',
+            license = ask('What license do you use?',
                           helptext=_helptext['trove_license'], required=False)
             if not license:
                 return

Lib/packaging/pypi/dist.py

@@ -135,7 +135,7 @@ class ReleaseInfo(IndexReference):
         not return one existing distribution.
         """
         if len(self.dists) == 0:
-            raise LookupError()
+            raise LookupError
         if dist_type:
             return self[dist_type]
         if prefer_source:

Lib/packaging/pypi/simple.py

@@ -189,7 +189,7 @@ class Crawler(BaseClient):
         self._process_index_page(predicate.name)
 
         if predicate.name.lower() not in self._projects:
-            raise ProjectNotFound()
+            raise ProjectNotFound
 
         releases = self._projects.get(predicate.name.lower())
         releases.sort_releases(prefer_final=prefer_final)

Lib/packaging/run.py

@@ -5,7 +5,6 @@ import re
 import sys
 import getopt
 import logging
-from copy import copy
 
 from packaging import logger
 from packaging.dist import Distribution
@@ -35,14 +34,14 @@ create_usage = """\
 Usage: pysetup create
    or: pysetup create --help
 
-Create a new Python package.
+Create a new Python project.
 """
 
 generate_usage = """\
 Usage: pysetup generate-setup
    or: pysetup generate-setup --help
 
-Generates a setup.py script for backward-compatibility purposes.
+Generate a setup.py script for backward-compatibility purposes.
 """
 
 
@@ -96,7 +95,7 @@ positional arguments:
    dist  installed distribution name
 
 optional arguments:
-   -y  auto confirm package removal
+   -y  auto confirm distribution removal
 """
 
 run_usage = """\
@@ -218,7 +217,6 @@ def _generate(distpatcher, args, **kw):
     print('The setup.py was generated')
 
 
-
 @action_help(graph_usage)
 def _graph(dispatcher, args, **kw):
     name = args[1]
@@ -384,7 +382,7 @@ def _search(dispatcher, args, **kw):
     """
     #opts = _parse_args(args[1:], '', ['simple', 'xmlrpc'])
     # 1. what kind of index is requested ? (xmlrpc / simple)
-    raise NotImplementedError()
+    raise NotImplementedError
 
 
 actions = [
@@ -393,10 +391,10 @@ actions = [
     ('install', 'Install a project', _install),
     ('remove', 'Remove a project', _remove),
     ('search', 'Search for a project in the indexes', _search),
-    ('list', 'Search for local projects', _list),
+    ('list', 'List installed releases', _list),
     ('graph', 'Display a graph', _graph),
-    ('create', 'Create a Project', _create),
-    ('generate-setup', 'Generates a backward-comptatible setup.py', _generate)
+    ('create', 'Create a project', _create),
+    ('generate-setup', 'Generate a backward-comptatible setup.py', _generate),
 ]
 
 
@@ -673,7 +671,7 @@ class Dispatcher:
 
 def main(args=None):
     old_level = logger.level
-    old_handlers = copy(logger.handlers)
+    old_handlers = list(logger.handlers)
     try:
         dispatcher = Dispatcher(args)
         if dispatcher.action is None:

Lib/packaging/tests/test_uninstall.py

@@ -111,7 +111,7 @@ class UninstallTestCase(support.TempdirManager,
         old = os.rename
 
         def _rename(source, target):
-            raise OSError()
+            raise OSError
 
         os.rename = _rename
         try:

Lib/packaging/util.py

-"""packaging.util
-Miscellaneous utility functions.
-"""
-import errno
-import csv
-import hashlib
+"""Miscellaneous utility functions."""
+
 import os
-import sys
 import re
+import csv
+import sys
+import errno
 import shutil
 import string
+import hashlib
 import tarfile
 import zipfile
 import posixpath
-import sysconfig
 import subprocess
-from copy import copy
+import sysconfig
 from glob import iglob as std_iglob
 from fnmatch import fnmatchcase
 from inspect import getsource
@@ -273,7 +271,7 @@ def execute(func, args, msg=None, verbose=0, dry_run=False):
 
 
 def strtobool(val):
-    """Convert a string representation of truth to true (1) or false (0).
+    """Convert a string representation of truth to a boolean.
 
     True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values
     are 'n', 'no', 'f', 'false', 'off', and '0'.  Raises ValueError if
@@ -384,7 +382,7 @@ byte_compile(files, optimize=%r, force=%r,
         elif optimize == 2:
             cmd.insert(1, "-OO")
 
-        env = copy(os.environ)
+        env = os.environ.copy()
         env['PYTHONPATH'] = os.path.pathsep.join(sys.path)
         try:
             spawn(cmd, env=env)