Branch merge

Doc/library/packaging.compiler.rst

@@ -569,10 +569,10 @@ extension modules.
 .. class:: Extension
 
    The Extension class describes a single C or C++ extension module.  It accepts
-   the following keyword arguments in its constructor
+   the following keyword arguments in its constructor:
 
    +------------------------+--------------------------------+---------------------------+
-   | argument name          | value                          | type                      |
+   | argument name          | value                          | type [#]_                 |
    +========================+================================+===========================+
    | *name*                 | the full name of the           | string                    |
    |                        | extension, including any       |                           |
@@ -670,3 +670,5 @@ extension modules.
    |                        | from the source extensions if  |                           |
    |                        | not provided.                  |                           |
    +------------------------+--------------------------------+---------------------------+
+
+.. [#] For values documented as lists, the given type is the type of each element.

Doc/packaging/setupcfg.rst

@@ -141,13 +141,16 @@ files
    Modules, scripts, data, documentation and other files to include in the
    distribution.
 
+extension sections
+   Options used to build extension modules.
+
 command sections
    Options given for specific commands, identical to those that can be given
    on the command line.
 
 
 Global options
-==============
+--------------
 
 Contains global options for Packaging. This section is shared with Distutils.
 
@@ -185,7 +188,7 @@ setup_hook
 
 
 Metadata
-========
+--------
 
 The metadata section contains the metadata for the project as described in
 :PEP:`345`.  Field names are case-insensitive.
@@ -308,7 +311,7 @@ from the fields present in the file.
 
 
 Files
-=====
+-----
 
 This section describes the files included in the project.
 
@@ -352,7 +355,7 @@ Example::
 
 
 Resources
----------
+^^^^^^^^^
 
 This section describes the files used by the project which must not be installed
 in the same place that python modules or libraries, they are called
@@ -448,10 +451,10 @@ Where {datafir} category will be platform-dependent.
 
 
 More control on source part
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""""
 
 Glob syntax
-"""""""""""
+'''''''''''
 
 When you declare source file, you can use a glob-like syntax to match multiples file, for example::
 
@@ -469,7 +472,7 @@ Glob tokens are:
 .. TODO Add examples
 
 Order of declaration
-""""""""""""""""""""
+''''''''''''''''''''
 
 The order of declaration is important if one file match multiple rules. The last
 rules matched by file is used, this is useful if you have this source tree::
@@ -492,7 +495,7 @@ one by one, you can declare them in this way::
        doc/README = {help}
 
 Exclude
-"""""""
+'''''''
 
 You can exclude some files of resources declaration by giving no destination, it
 can be useful if you have a non-resources file in the same directory of
@@ -513,12 +516,12 @@ Your **files** section will be::
        doc/RELEASES =
 
 More control on destination part
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+""""""""""""""""""""""""""""""""
 
 .. _setupcfg-resources-base-prefix:
 
 Defining a base prefix
-""""""""""""""""""""""
+''''''''''''''''''''''
 
 When you define your resources, you can have more control of how the final path
 is computed.
@@ -577,7 +580,7 @@ path will be::
 
 
 Overwriting paths for categories
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+""""""""""""""""""""""""""""""""
 
 This part is intended for system administrators or downstream OS packagers.
 
@@ -614,18 +617,18 @@ The platform-dependent categories are:
 
 
 Defining extra categories
-^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""
 
 .. TODO
 
 
 Examples
-^^^^^^^^
+""""""""
 
 These examples are incremental but work unitarily.
 
 Resources in root dir
-"""""""""""""""""""""
+'''''''''''''''''''''
 
 Source tree::
 
@@ -647,7 +650,7 @@ So babar.sh and launch.sh will be placed in {scripts} directory.
 Now let's move all the scripts into a scripts directory.
 
 Resources in sub-directory
-""""""""""""""""""""""""""
+''''''''''''''''''''''''''
 
 Source tree::
 
@@ -673,7 +676,7 @@ scripts into {scripts} instead of {scripts}/scripts.
 Now let's add some docs.
 
 Resources in multiple sub-directories
-"""""""""""""""""""""""""""""""""""""
+'''''''''''''''''''''''''''''''''''''
 
 Source tree::
 
@@ -706,7 +709,7 @@ file is used.
 Now let's add some scripts for windows users.
 
 Complete example
-""""""""""""""""
+''''''''''''''''
 
 Source tree::
 
@@ -736,8 +739,37 @@ We use brace expansion syntax to place all the shell and batch scripts into
 {scripts} category.
 
 
+Extension sections
+------------------
+
+If a project includes extension modules written in C or C++, each one of them
+needs to have its options defined in a dedicated section.  Here's an example::
+
+   [files]
+   packages = coconut
+
+   [extension=_fastcoconut]
+   name = coconut._fastcoconut
+   language = cxx
+   sources = cxx_src/cononut_utils.cxx
+             cxx_src/python_module.cxx
+   include_dirs = /usr/include/gecode
+                  /usr/include/blitz
+   extra_compile_args =
+       -fPIC -O2
+       -DGECODE_VERSION=$(./gecode_version) -- sys.platform != 'win32'
+       /DGECODE_VERSION='win32' -- sys.platform == 'win32'
+
+The section name must start with ``extension=``; the righ-hand part is currently
+discarded.  Valid fields and their values are listed in the documentation of the
+:class:`packaging.compiler.extension.Extension` class; values documented as
+Python lists translate to multi-line values in the configuration file.  In
+addition, multi-line values accept environment markers on each line, after a
+``--``.
+
+
 Command sections
-================
+----------------
 
 To pass options to commands without having to type them on the command line
 for each invocation, you can write them in the :file:`setup.cfg` file, in a

Lib/packaging/create.py

@@ -175,11 +175,11 @@ def convert_yn_to_bool(yn, yes=True, no=False):
 def _build_classifiers_dict(classifiers):
     d = {}
     for key in classifiers:
-        subDict = d
+        subdict = d
         for subkey in key.split(' :: '):
-            if not subkey in subDict:
-                subDict[subkey] = {}
-            subDict = subDict[subkey]
+            if subkey not in subdict:
+                subdict[subkey] = {}
+            subdict = subdict[subkey]
     return d
 
 CLASSIFIERS = _build_classifiers_dict(_CLASSIFIERS_LIST)

Lib/packaging/database.py

@@ -104,12 +104,12 @@ def _generate_cache(use_egg_info=False, paths=sys.path):
         for dist in _yield_distributions(gen_dist, gen_egg, paths):
             if isinstance(dist, Distribution):
                 _cache_path[dist.path] = dist
-                if not dist.name in _cache_name:
+                if dist.name not in _cache_name:
                     _cache_name[dist.name] = []
                 _cache_name[dist.name].append(dist)
             else:
                 _cache_path_egg[dist.path] = dist
-                if not dist.name in _cache_name_egg:
+                if dist.name not in _cache_name_egg:
                     _cache_name_egg[dist.name] = []
                 _cache_name_egg[dist.name].append(dist)
 
@@ -150,7 +150,7 @@ class Distribution:
         self.version = self.metadata['Version']
         self.path = path
 
-        if _cache_enabled and not path in _cache_path:
+        if _cache_enabled and path not in _cache_path:
             _cache_path[path] = self
 
     def __repr__(self):

Lib/packaging/depgraph.py

@@ -58,7 +58,7 @@ class DependencyGraph:
         """
         self.adjacency_list[x].append((y, label))
         # multiple edges are allowed, so be careful
-        if not x in self.reverse_list[y]:
+        if x not in self.reverse_list[y]:
             self.reverse_list[y].append(x)
 
     def add_missing(self, distribution, requirement):
@@ -72,7 +72,7 @@ class DependencyGraph:
         self.missing[distribution].append(requirement)
 
     def _repr_dist(self, dist):
-        return '%s %s' % (dist.name, dist.metadata['Version'])
+        return '%r %s' % (dist.name, dist.metadata['Version'])
 
     def repr_node(self, dist, level=1):
         """Prints only a subgraph"""
@@ -154,10 +154,10 @@ def generate_graph(dists):
             if len(comps) == 2:
                 version = comps[1]
                 if len(version) < 3 or version[0] != '(' or version[-1] != ')':
-                    raise PackagingError('Distribution %s has ill formed' \
-                                         'provides field: %s' % (dist.name, p))
+                    raise PackagingError('distribution %r has ill-formed'
+                                         'provides field: %r' % (dist.name, p))
                 version = version[1:-1]  # trim off parenthesis
-            if not name in provided:
+            if name not in provided:
                 provided[name] = []
             provided[name].append((version, dist))
 
@@ -174,7 +174,7 @@ def generate_graph(dists):
 
             name = predicate.name
 
-            if not name in provided:
+            if name not in provided:
                 graph.add_missing(dist, req)
             else:
                 matched = False
@@ -204,8 +204,9 @@ def dependent_dists(dists, dist):
     :param dists: a list of distributions
     :param dist: a distribution, member of *dists* for which we are interested
     """
-    if not dist in dists:
-        raise ValueError('The given distribution is not a member of the list')
+    if dist not in dists:
+        raise ValueError('given distribution %r is not a member of the list' %
+                         dist.name)
     graph = generate_graph(dists)
 
     dep = [dist]  # dependent distributions
@@ -215,7 +216,7 @@ def dependent_dists(dists, dist):
         node = fringe.pop()
         dep.append(node)
         for prev in graph.reverse_list[node]:
-            if not prev in dep:
+            if prev not in dep:
                 fringe.append(prev)
 
     dep.pop(0)  # remove dist from dep, was there to prevent infinite loops
@@ -236,17 +237,19 @@ def main():
     except Exception as e:
         tempout.seek(0)
         tempout = tempout.read()
-        print('Could not generate the graph\n%s\n%s\n' % (tempout, e))
+        print('Could not generate the graph')
+        print(tempout)
+        print(e)
         sys.exit(1)
 
     for dist, reqs in graph.missing.items():
         if len(reqs) > 0:
-            print("Warning: Missing dependencies for %s:" % dist.name,
+            print("Warning: Missing dependencies for %r:" % dist.name,
                   ", ".join(reqs))
     # XXX replace with argparse
     if len(sys.argv) == 1:
         print('Dependency graph:')
-        print('    ' + repr(graph).replace('\n', '\n    '))
+        print('   ', repr(graph).replace('\n', '\n    '))
         sys.exit(0)
     elif len(sys.argv) > 1 and sys.argv[1] in ('-d', '--dot'):
         if len(sys.argv) > 2:
@@ -259,7 +262,7 @@ def main():
         tempout.seek(0)
         tempout = tempout.read()
         print(tempout)
-        print('Dot file written at "%s"' % filename)
+        print('Dot file written at %r' % filename)
         sys.exit(0)
     else:
         print('Supported option: -d [filename]')

Lib/packaging/dist.py

@@ -509,14 +509,14 @@ Common commands: (see '--help-commands' for more)
                 options = self.global_options
             parser.set_option_table(options)
             parser.print_help(self.common_usage + "\nGlobal options:")
-            print('')
+            print()
 
         if display_options:
             parser.set_option_table(self.display_options)
             parser.print_help(
                 "Information display options (just display " +
                 "information, ignore any commands)")
-            print('')
+            print()
 
         for command in self.commands:
             if isinstance(command, type) and issubclass(command, Command):
@@ -529,7 +529,7 @@ Common commands: (see '--help-commands' for more)
             else:
                 parser.set_option_table(cls.user_options)
             parser.print_help("Options for %r command:" % cls.__name__)
-            print('')
+            print()
 
         print(gen_usage(self.script_name))
 
@@ -544,7 +544,7 @@ Common commands: (see '--help-commands' for more)
         # we ignore "foo bar").
         if self.help_commands:
             self.print_commands()
-            print('')
+            print()
             print(gen_usage(self.script_name))
             return 1
 

Lib/packaging/pypi/dist.py

@@ -256,7 +256,7 @@ class DistInfo(IndexReference):
                 hashlib.new(hashname)
             except ValueError:
                 raise UnsupportedHashName(hashname)
-        if not url in [u['url'] for u in self.urls]:
+        if url not in [u['url'] for u in self.urls]:
             self.urls.append({
                 'url': url,
                 'hashname': hashname,
@@ -329,7 +329,7 @@ class DistInfo(IndexReference):
         url param"""
         hashname = self.url['hashname']
         expected_hashval = self.url['hashval']
-        if not None in (expected_hashval, hashname):
+        if None not in (expected_hashval, hashname):
             with open(filename, 'rb') as f:
                 hashval = hashlib.new(hashname)
                 hashval.update(f.read())
@@ -409,7 +409,7 @@ class ReleasesList(IndexReference):
                                  (release.name, self.name))
             version = str(release.version)
 
-            if not version in self.get_versions():
+            if version not in self.get_versions():
                 # append only if not already exists
                 self.releases.append(release)
             for dist in release.dists.values():

Lib/packaging/pypi/simple.py

@@ -231,7 +231,8 @@ class Crawler(BaseClient):
         """
         self._mirrors_used.add(self.index_url)
         index_url = self._mirrors.pop()
-        if not ("http://" or "https://" or "file://") in index_url:
+        # XXX use urllib.parse for a real check of missing scheme part
+        if not index_url.startswith(("http://", "https://", "file://")):
             index_url = "http://%s" % index_url
 
         if not index_url.endswith("/simple"):
@@ -282,7 +283,7 @@ class Crawler(BaseClient):
             name = release.name
         else:
             name = release_info['name']
-        if not name.lower() in self._projects:
+        if name.lower() not in self._projects:
             self._projects[name.lower()] = ReleasesList(name, index=self._index)
 
         if release:
@@ -320,7 +321,7 @@ class Crawler(BaseClient):
                             # it's a distribution, so create a dist object
                             try:
                                 infos = get_infos_from_url(link, project_name,
-                                            is_external=not self.index_url in url)
+                                            is_external=self.index_url not in url)
                             except CantParseArchiveName as e:
                                 if self.verbose:
                                     logger.warning(

Lib/packaging/run.py

@@ -286,9 +286,9 @@ def _metadata(dispatcher, args, **kw):
             value = metadata[key]
             if isinstance(value, list):
                 for v in value:
-                    print('    ' + v)
+                    print('   ', v)
             else:
-                print('    ' + value.replace('\n', '\n    '))
+                print('   ', value.replace('\n', '\n    '))
 
 
 @action_help(remove_usage)
@@ -366,7 +366,7 @@ def _list(dispatcher, args, **kw):
         print('%s %s at %s' % (dist.name, dist.metadata['version'], dist.path))
         number += 1
 
-    print('')
+    print()
     if number == 0:
         print('Nothing seems to be installed.')
     else:
@@ -405,7 +405,6 @@ class Dispatcher:
         self.verbose = 1
         self.dry_run = False
         self.help = False
-        self.script_name = 'pysetup'
         self.cmdclass = {}
         self.commands = []
         self.command_options = {}
@@ -574,17 +573,17 @@ class Dispatcher:
         from packaging.command.cmd import Command
 
         print('Usage: pysetup [options] action [action_options]')
-        print('')
+        print()
         if global_options_:
             self.print_usage(self.parser)
-            print('')
+            print()
 
         if display_options_:
             parser.set_option_table(display_options)
             parser.print_help(
                 "Information display options (just display " +
                 "information, ignore any commands)")
-            print('')
+            print()
 
         for command in commands:
             if isinstance(command, type) and issubclass(command, Command):
@@ -598,15 +597,15 @@ class Dispatcher:
                 parser.set_option_table(cls.user_options)
 
             parser.print_help("Options for %r command:" % cls.__name__)
-            print('')
+            print()
 
     def _show_command_help(self, command):
         if isinstance(command, str):
             command = get_command_class(command)
 
         desc = getattr(command, 'description', '(no description available)')
-        print('Description: %s' % desc)
-        print('')
+        print('Description:', desc)
+        print()
 
         if (hasattr(command, 'help_options') and
             isinstance(command.help_options, list)):
@@ -616,7 +615,7 @@ class Dispatcher:
             self.parser.set_option_table(command.user_options)
 
         self.parser.print_help("Options:")
-        print('')
+        print()
 
     def _get_command_groups(self):
         """Helper function to retrieve all the command class names divided

Lib/packaging/tests/test_command_bdist_dumb.py

@@ -49,7 +49,6 @@ class BuildDumbTestCase(support.TempdirManager,
                              'py_modules': ['foo'],
                              'url': 'xxx', 'author': 'xxx',
                              'author_email': 'xxx'})
-        dist.script_name = 'setup.py'
         os.chdir(pkg_dir)
 
         sys.argv[:] = ['setup.py']

Lib/packaging/tests/test_command_build_py.py

@@ -33,9 +33,7 @@ class BuildPyTestCase(support.TempdirManager,
 
         dist = Distribution({"packages": ["pkg"],
                              "package_dir": sources})
-        # script_name need not exist, it just need to be initialized
 
-        dist.script_name = os.path.join(sources, "setup.py")
         dist.command_obj["build"] = support.DummyCommand(
             force=False,
             build_lib=destination,
@@ -89,8 +87,6 @@ class BuildPyTestCase(support.TempdirManager,
             dist = Distribution({"packages": ["pkg"],
                                  "package_dir": sources,
                                  "package_data": {"pkg": ["doc/*"]}})
-            # script_name need not exist, it just need to be initialized
-            dist.script_name = os.path.join(sources, "setup.py")
             dist.script_args = ["build"]
             dist.parse_command_line()
 

Lib/packaging/tests/test_command_install_dist.py

@@ -30,8 +30,6 @@ class InstallTestCase(support.TempdirManager,
         destination = os.path.join(builddir, "installation")
 
         dist = Distribution({"name": "foopkg"})
-        # script_name need not exist, it just need to be initialized
-        dist.script_name = os.path.join(builddir, "setup.py")
         dist.command_obj["build"] = support.DummyCommand(
             build_base=builddir,
             build_lib=os.path.join(builddir, "lib"),

Lib/packaging/tests/test_command_install_lib.py

@@ -65,7 +65,6 @@ class InstallLibTestCase(support.TempdirManager,
         self.write_file(f, '# python package')
         cmd.distribution.ext_modules = [Extension('foo', ['xxx'])]
         cmd.distribution.packages = [pkg_dir]
-        cmd.distribution.script_name = 'setup.py'
 
         # make sure the build_lib is set the temp dir
         build_dir = os.path.split(pkg_dir)[0]
@@ -86,7 +85,6 @@ class InstallLibTestCase(support.TempdirManager,
         self.write_file(f, '# python package')
         cmd.distribution.ext_modules = [Extension('foo', ['xxx'])]
         cmd.distribution.packages = [pkg_dir]
-        cmd.distribution.script_name = 'setup.py'
 
         # get_input should return 2 elements
         self.assertEqual(len(cmd.get_inputs()), 2)

Lib/packaging/tests/test_command_sdist.py

@@ -24,12 +24,6 @@ from packaging.util import find_executable
 from packaging.tests import support
 from shutil import get_archive_formats
 
-SETUP_PY = """
-from packaging.core import setup
-import somecode
-
-setup(name='fake')
-"""
 
 MANIFEST = """\
 # file GENERATED by packaging, do NOT edit
@@ -57,8 +51,6 @@ class SDistTestCase(support.TempdirManager,
     restore_environ = ['HOME']
 
     def setUp(self):
-        # PyPIRCCommandTestCase creates a temp dir already
-        # and put it in self.tmp_dir
         super(SDistTestCase, self).setUp()
         self.tmp_dir = self.mkdtemp()
         os.environ['HOME'] = self.tmp_dir
@@ -69,7 +61,6 @@ class SDistTestCase(support.TempdirManager,
         # a package, and a README
         self.write_file((self.tmp_dir, 'README'), 'xxx')
         self.write_file((self.tmp_dir, 'somecode', '__init__.py'), '#')
-        self.write_file((self.tmp_dir, 'setup.py'), SETUP_PY)
         os.chdir(self.tmp_dir)
 
     def tearDown(self):
@@ -84,7 +75,6 @@ class SDistTestCase(support.TempdirManager,
                         'url': 'xxx', 'author': 'xxx',
                         'author_email': 'xxx'}
         dist = Distribution(metadata)
-        dist.script_name = 'setup.py'
         dist.packages = ['somecode']
         dist.include_package_data = True
         cmd = sdist(dist)

Lib/packaging/tests/test_config.py

@@ -114,7 +114,7 @@ libraries = gecodeint gecodekernel -- sys.platform != 'win32'
     GecodeInt GecodeKernel -- sys.platform == 'win32'
 
 [extension=fast_taunt]
-name = three.fast_taunt
+name = two.fast_taunt
 sources = cxx_src/utils_taunt.cxx
           cxx_src/python_module.cxx
 include_dirs = /usr/include/gecode
@@ -305,7 +305,7 @@ class ConfigTestCase(support.TempdirManager,
         self.assertEqual(ext.extra_link_args,
             ['`gcc -print-file-name=libgcc.a`', '-shared'])
 
-        ext = ext_modules.get('three.fast_taunt')
+        ext = ext_modules.get('two.fast_taunt')
         self.assertEqual(ext.sources,
             ['cxx_src/utils_taunt.cxx', 'cxx_src/python_module.cxx'])
         self.assertEqual(ext.include_dirs,

Lib/packaging/util.py

@@ -1099,7 +1099,7 @@ def ask(message, options):
         response = input(message)
         response = response.strip().lower()
         if response not in options:
-            print('invalid response: %r' % response)
+            print('invalid response:', repr(response))
             print('choose one of', ', '.join(repr(o) for o in options))
         else:
             return response

setup.py

@@ -1780,6 +1780,13 @@ class PyBuildInstall(install):
         install.initialize_options(self)
         self.warn_dir=0
 
+    # Customize subcommands to not install an egg-info file for Python
+    sub_commands = [('install_lib', install.has_lib),
+                    ('install_headers', install.has_headers),
+                    ('install_scripts', install.has_scripts),
+                    ('install_data', install.has_data)]
+
+
 class PyBuildInstallLib(install_lib):
     # Do exactly what install_lib does but make sure correct access modes get
     # set on installed directories and files. All installed files with get