EasyInstall/setuptools 0.5a4: significant new features, including automatic

installation of dependencies, the ability to specify dependencies in a
setup script, and several new options to control EasyInstall's behavior.

--HG--
branch : setuptools
extra : convert_revision : svn%3A6015fed2-1504-0410-9fe1-9d1591cc4771/sandbox/trunk/setuptools%4041073

EasyInstall.txt

@@ -23,7 +23,7 @@ Installing "Easy Install"
 -------------------------
 
 Windows users can just download and run the `setuptools binary installer for
-Windows <http://peak.telecommunity.com/dist/setuptools-0.5a3.win32.exe>`_.
+Windows <http://peak.telecommunity.com/dist/setuptools-0.5a4.win32.exe>`_.
 All others should just download `ez_setup.py
 <http://peak.telecommunity.com/dist/ez_setup.py>`_, and run it; this will
 download and install the correct version of ``setuptools`` for your Python
@@ -62,7 +62,7 @@ version, and automatically downloading, building, and installing it::
 **Example 2**. Install or upgrade a package by name and version by finding
 links on a given "download page"::
 
-    easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a3"
+    easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a4"
 
 **Example 3**. Download a source distribution from a specified URL,
 automatically building and installing it::
@@ -73,6 +73,11 @@ automatically building and installing it::
 
     easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg
 
+**Example 5**.  Upgrade an already-installed package to the latest version
+listed on PyPI:
+
+    easy_install --upgrade PyProtocols
+
 Easy Install accepts URLs, filenames, PyPI package names (i.e., ``distutils``
 "distribution" names), and package+version specifiers.  In each case, it will
 attempt to locate the latest available version that meets your criteria.
@@ -118,23 +123,29 @@ a version greater than the one you have now::
 
     easy_install "SomePackage>2.0"
 
+using the upgrade flag, to find the latest available version on PyPI::
+
+    easy_install --upgrade SomePackage
+
 or by using a download page, direct download URL, or package filename::
 
-    easy_install -s http://example.com/downloads ExamplePackage
+    easy_install -f http://example.com/downloads ExamplePackage
 
     easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg
 
     easy_install my_downloads/ExamplePackage-2.0.tgz
 
 If you're using ``-m`` or ``--multi`` (or installing outside of
-``site-packages``), the ``require()`` function automatically selects the newest
-available version of a package that meets your version criteria at runtime, so
-installation is the only step needed.
+``site-packages``), using the ``require()`` function at runtime automatically
+selects the newest installed version of a package that meets your version
+criteria.  So, installing a newer version is the only step needed to upgrade
+such packages.
 
-If you're installing to ``site-packages`` and not using ``-m``, installing a
-package automatically replaces any previous version in the ``easy-install.pth``
-file, so that Python will import the most-recently installed version by
-default.
+If you're installing to Python's ``site-packages`` directory (and not
+using ``-m``), installing a package automatically replaces any previous version
+in the ``easy-install.pth`` file, so that Python will import the most-recently
+installed version by default.  So, again, installing the newer version is the
+only upgrade step needed.
 
 If you haven't suppressed script installation (using ``--exclude-scripts`` or
 ``-x``), then the upgraded version's scripts will be installed, and they will
@@ -339,6 +350,16 @@ Command-Line Options
     the ``--install-dir`` or ``-d`` option (or they are set via configuration
     file(s)) you must also use ``require()`` to enable packages at runtime.
 
+``--upgrade, -U``   (New in 0.5a4)
+    By default, EasyInstall only searches the Python Package Index if a
+    project/version requirement can't be met by distributions already installed
+    on sys.path or the installation directory.  However, if you supply the
+    ``--upgrade`` or ``-U`` flag, EasyInstall will always check the package
+    index before selecting a version to install.  In this way, you can force
+    EasyInstall to use the latest available version of any package it installs
+    (subject to any version requirements that might exclude such later
+    versions).
+
 ``--install-dir=DIR, -d DIR``
     Set the installation directory. It is up to you to ensure that this
     directory is on ``sys.path`` at runtime, and to use
@@ -366,6 +387,14 @@ Command-Line Options
     versions of a package, but do not want to reset the version that will be
     run by scripts that are already installed.
 
+``--always-copy, -a``   (New in 0.5a4)
+    Copy all needed distributions to the installation directory, even if they
+    are already present in a directory on sys.path.  In older versions of
+    EasyInstall, this was the default behavior, but now you must explicitly
+    request it.  By default, EasyInstall will no longer copy such distributions
+    from other sys.path directories to the installation directory, unless you
+    explicitly gave the distribution's filename on the command line.
+
 ``--find-links=URL, -f URL`` (Option renamed in 0.4a2)
     Scan the specified "download pages" for direct links to downloadable eggs
     or source distributions.  Any usable packages will be downloaded if they
@@ -434,6 +463,12 @@ Command-Line Options
     the default is 0 (unless it's set under ``install`` or ``install_lib`` in
     one of your distutils configuration files).
 
+``--record=FILENAME``  (New in 0.5a4)
+    Write a record of all installed files to FILENAME.  This is basically the
+    same as the same option for the standard distutils "install" command, and
+    is included for compatibility with tools that expect to pass this option
+    to "setup.py install".
+
 
 Release Notes/Change History
 ============================
@@ -442,6 +477,46 @@ Known Issues
  * There's no automatic retry for borked Sourceforge mirrors, which can easily
    time out or be missing a file.
 
+0.5a4
+ * Added ``--always-copy/-a`` option to always copy needed packages to the
+   installation directory, even if they're already present elsewhere on
+   sys.path. (In previous versions, this was the default behavior, but now
+   you must request it.)
+ 
+ * Added ``--upgrade/-U`` option to force checking PyPI for latest available
+   version(s) of all packages requested by name and version, even if a matching
+   version is available locally.
+
+ * Setup scripts using setuptools can now list their dependencies directly in
+   the setup.py file, without having to manually create a ``depends.txt`` file.
+   The ``install_requires`` and ``extras_require`` arguments to ``setup()``
+   are used to create a dependencies file automatically.  If you are manually
+   creating ``depends.txt`` right now, please switch to using these setup
+   arguments as soon as practical, because ``depends.txt`` support will be
+   removed in the 0.6 release cycle.  For documentation on the new arguments,
+   see the ``setuptools.dist.Distribution`` class.
+
+ * Added automatic installation of dependencies declared by a distribution
+   being installed.  These dependencies must be listed in the distribution's
+   ``EGG-INFO`` directory, so the distribution has to have declared its
+   dependencies by using setuptools.  If a package has requirements it didn't
+   declare, you'll still have to deal with them yourself.  (E.g., by asking
+   EasyInstall to find and install them.)
+
+ * Setup scripts using setuptools now always install using ``easy_install``
+   internally, for ease of uninstallation and upgrading.  Note: you *must*
+   remove any ``extra_path`` argument from your setup script, as it conflicts
+   with the proper functioning of the ``easy_install`` command.  (Also, added   
+   the ``--record`` option to ``easy_install`` for the benefit of tools that
+   run ``setup.py install --record=filename`` on behalf of another packaging
+   system.)
+
+ * ``pkg_resources.AvailableDistributions.resolve()`` and related methods now
+   accept an ``installer`` argument: a callable taking one argument, a
+   ``Requirement`` instance.  The callable must return a ``Distribution``
+   object, or ``None`` if no distribution is found.  This feature is used by
+   EasyInstall to resolve dependencies by recursively invoking itself.
+
 0.5a3
  * Fixed not setting script permissions to allow execution.
  
@@ -584,10 +659,8 @@ Known Issues
 Future Plans
 ============
 
-* Process the installed package's dependencies as well as the base package
 * Additional utilities to list/remove/verify packages
 * Signature checking?  SSL?  Ability to suppress PyPI search?
 * Display byte progress meter when downloading distributions and long pages?
 * Redirect stdout/stderr to log during run_setup?
 
-

ez_setup.py

@@ -14,7 +14,7 @@ the appropriate options to ``use_setuptools()``.
 This file can also be run as a script to install or upgrade setuptools.
 """
 
-DEFAULT_VERSION = "0.5a3"
+DEFAULT_VERSION = "0.5a4"
 DEFAULT_URL     = "http://peak.telecommunity.com/dist/"
 
 import sys, os

pkg_resources.py

@@ -296,7 +296,7 @@ class AvailableDistributions(object):
         """Remove `dist` from the distribution map"""
         self._distmap[dist.key].remove(dist)
 
-    def best_match(self,requirement,path=None):
+    def best_match(self, requirement, path=None, installer=None):
         """Find distribution best matching `requirement` and usable on `path`
 
         If a distribution that's already installed on `path` is unsuitable,
@@ -324,9 +324,9 @@ class AvailableDistributions(object):
         for dist in distros:
             if dist in requirement:
                 return dist
-        return self.obtain(requirement) # try and download
+        return self.obtain(requirement, installer) # try and download/install
 
-    def resolve(self, requirements, path=None):
+    def resolve(self, requirements, path=None, installer=None):
         """List all distributions needed to (recursively) meet requirements"""
 
         if path is None:
@@ -344,26 +344,26 @@ class AvailableDistributions(object):
                 continue
 
             dist = best.get(req.key)
-
             if dist is None:
                 # Find the best distribution and add it to the map
-                dist = best[req.key] = self.best_match(req,path)
+                dist = best[req.key] = self.best_match(req, path, installer)
                 if dist is None:
                     raise DistributionNotFound(req)  # XXX put more info here
                 to_install.append(dist)
 
             elif dist not in req:
                 # Oops, the "best" so far conflicts with a dependency
-                raise VersionConflict(req,dist) # XXX put more info here
+                raise VersionConflict(dist,req) # XXX put more info here
 
             requirements.extend(dist.depends(req.options)[::-1])
             processed[req] = True
 
         return to_install    # return list of distros to install
 
-    def obtain(self, requirement):
+    def obtain(self, requirement, installer=None):
         """Obtain a distro that matches requirement (e.g. via download)"""
-        return None     # override this in subclasses
+        if installer is not None:
+            return installer(requirement)
 
     def __len__(self): return len(self._distmap)
 
@@ -1316,10 +1316,9 @@ class Distribution(object):
             return self.__dep_map
         except AttributeError:
             dm = self.__dep_map = {None: []}
-            for section,contents in split_sections(
-                self._get_metadata('depends.txt')
-            ):
-                dm[section] = list(parse_requirements(contents))
+            for name in 'requires.txt', 'depends.txt':
+                for extra,reqs in split_sections(self._get_metadata(name)):
+                    dm.setdefault(extra,[]).extend(parse_requirements(reqs))
             return dm
 
     _dep_map = property(_dep_map)
@@ -1351,6 +1350,7 @@ class Distribution(object):
             fixup_namespace_packages(self.path)
             map(declare_namespace, self._get_metadata('namespace_packages.txt'))
 
+
     def egg_name(self):
         """Return what this distribution's standard .egg filename should be"""
         filename = "%s-%s-py%s" % (

setup.py

 #!/usr/bin/env python
 """Distutils setup file, used to install or test 'setuptools'"""
 
-VERSION = "0.5a3"
+VERSION = "0.5a4"
 from setuptools import setup, find_packages, Require
 
 setup(
@@ -42,7 +42,6 @@ setup(
     packages = find_packages(),
     py_modules = ['pkg_resources', 'easy_install'],
     scripts = ['easy_install.py'],
-    extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION),
 
     classifiers = [f.strip() for f in """
     Development Status :: 3 - Alpha
@@ -78,5 +77,6 @@ setup(
 
 
 
+
 
 

setuptools/__init__.py

@@ -8,7 +8,7 @@ from distutils.core import Command as _Command
 from distutils.util import convert_path
 import os.path
 
-__version__ = '0.5a3'
+__version__ = '0.5a4'
 
 __all__ = [
     'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require',

setuptools/command/bdist_egg.py

@@ -11,7 +11,7 @@ from distutils.sysconfig import get_python_version, get_python_lib
 from distutils.errors import *
 from distutils import log
 from pkg_resources import parse_requirements, get_platform, safe_name, \
-    safe_version, Distribution
+    safe_version, Distribution, yield_lines
 
 
 def write_stub(resource, pyfile):
@@ -78,7 +78,7 @@ class bdist_egg(Command):
         self.tag_build = None
         self.tag_svn_revision = 0
         self.tag_date = 0
-
+        self.egg_output = None
 
     def finalize_options (self):
         self.egg_name = safe_name(self.distribution.get_name())
@@ -105,19 +105,19 @@ class bdist_egg(Command):
             self.bdist_dir = os.path.join(bdist_base, 'egg')
         if self.plat_name is None:
             self.plat_name = get_platform()
-        self.set_undefined_options('bdist',('dist_dir', 'dist_dir'))
-
-
-
-
-
-
-
-
 
+        self.set_undefined_options('bdist',('dist_dir', 'dist_dir'))
 
+        if self.egg_output is None:
 
+            # Compute filename of the output egg
+            basename = Distribution(
+                None, None, self.egg_name, self.egg_version,
+                get_python_version(),
+                self.distribution.has_ext_modules() and self.plat_name
+            ).egg_name()
 
+            self.egg_output = os.path.join(self.dist_dir, basename+'.egg')
 
 
 
@@ -146,22 +146,22 @@ class bdist_egg(Command):
         finally:
             self.distribution.data_files = old
 
+    def get_outputs(self):
+        return [self.egg_output]
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+    def write_requirements(self):
+        dist = self.distribution
+        if not getattr(dist,'install_requires',None) and \
+           not getattr(dist,'extras_require',None): return
+        requires = os.path.join(self.egg_info,"requires.txt")
+        log.info("writing %s", requires)
+        if not self.dry_run:
+            f = open(requires, 'wt')
+            f.write('\n'.join(yield_lines(dist.install_requires)))
+            for extra,reqs in dist.extras_require.items():
+                f.write('\n\n[%s]\n%s' % (extra, '\n'.join(yield_lines(reqs))))
+            f.close()
+            
     def run(self):
         # We run install_lib before install_data, because some data hacks
         # pull their data path from the install_lib command.
@@ -189,24 +189,19 @@ class bdist_egg(Command):
         if self.distribution.data_files:
             self.do_install_data()
 
+        # Make the EGG-INFO directory
+        archive_root = self.bdist_dir
+        egg_info = os.path.join(archive_root,'EGG-INFO')
+        self.mkpath(egg_info)
+        self.mkpath(self.egg_info)
+
         if self.distribution.scripts:
-            script_dir = os.path.join(self.bdist_dir,'EGG-INFO','scripts')
+            script_dir = os.path.join(egg_info, 'scripts')
             log.info("installing scripts to %s" % script_dir)
             self.call_command('install_scripts', install_dir=script_dir)
 
-        # And make an archive relative to the root of the
-        # pseudo-installation tree.
-        archive_basename = Distribution(
-            None, None, self.egg_name, self.egg_version, get_python_version(),
-            ext_outputs and self.plat_name
-        ).egg_name()
-        pseudoinstall_root = os.path.join(self.dist_dir, archive_basename)
-        archive_root = self.bdist_dir
+        self.write_requirements()
 
-        # Make the EGG-INFO directory
-        egg_info = os.path.join(archive_root,'EGG-INFO')
-        self.mkpath(egg_info)
-        self.mkpath(self.egg_info)
 
         log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO'))
         if not self.dry_run:
@@ -231,15 +226,20 @@ class bdist_egg(Command):
             if not self.dry_run:
                 os.unlink(native_libs)
 
-        if self.egg_info and os.path.exists(self.egg_info):
-            for filename in os.listdir(self.egg_info):
-                path = os.path.join(self.egg_info,filename)
-                if os.path.isfile(path):
-                    self.copy_file(path,os.path.join(egg_info,filename))
+        for filename in os.listdir(self.egg_info):
+            path = os.path.join(self.egg_info,filename)
+            if os.path.isfile(path):
+                self.copy_file(path,os.path.join(egg_info,filename))
 
+        if os.path.exists(os.path.join(self.egg_info,'depends.txt')):
+            log.warn(
+                "WARNING: 'depends.txt' will not be used by setuptools 0.6!"
+            )
+            log.warn(
+                "Use the install_requires/extras_require setup() args instead."
+            )
         # Make the archive
-        make_zipfile(pseudoinstall_root+'.egg',
-                          archive_root, verbose=self.verbose,
+        make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
                           dry_run=self.dry_run)
         if not self.keep_temp:
             remove_tree(self.bdist_dir, dry_run=self.dry_run)

setuptools/dist.py

@@ -8,50 +8,44 @@ from setuptools.command.install import install
 from setuptools.command.install_lib import install_lib
 from distutils.errors import DistutilsOptionError, DistutilsPlatformError
 from distutils.errors import DistutilsSetupError
-import setuptools
+import setuptools, pkg_resources
 
 sequence = tuple, list
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
 class Distribution(_Distribution):
     """Distribution with support for features, tests, and package data
 
     This is an enhanced version of 'distutils.dist.Distribution' that
     effectively adds the following new optional keyword arguments to 'setup()':
 
+     'install_requires' -- a string or sequence of strings specifying project
+        versions that the distribution requires when installed, in the format
+        used by 'pkg_resources.require()'.  They will be installed
+        automatically when the package is installed.  If you wish to use
+        packages that are not available in PyPI, or want to give your users an
+        alternate download location, you can add a 'find_links' option to the
+        '[easy_install]' section of your project's 'setup.cfg' file, and then
+        setuptools will scan the listed web pages for links that satisfy the
+        requirements.
+
+     'extras_require' -- a dictionary mapping names of optional "extras" to the
+        additional requirement(s) that using those extras incurs. For example,
+        this::
+
+            extras_require = dict(reST = ["docutils>=0.3", "reSTedit"])
+
+        indicates that the distribution can optionally provide an extra
+        capability called "reST", but it can only be used if docutils and
+        reSTedit are installed.  If the user installs your package using
+        EasyInstall and requests one of your extras, the corresponding
+        additional requirements will be installed if needed.
+
      'features' -- a dictionary mapping option names to 'setuptools.Feature'
         objects.  Features are a portion of the distribution that can be
         included or excluded based on user options, inter-feature dependencies,
         and availability on the current system.  Excluded features are omitted
         from all setup commands, including source and binary distributions, so
         you can create multiple distributions from the same source tree.
-
         Feature names should be valid Python identifiers, except that they may
         contain the '-' (minus) sign.  Features can be included or excluded
         via the command line options '--with-X' and '--without-X', where 'X' is
@@ -84,6 +78,8 @@ class Distribution(_Distribution):
     the distribution.  They are used by the feature subsystem to configure the
     distribution for the included and excluded features.
     """
+
+
     def __init__ (self, attrs=None):
         have_package_data = hasattr(self, "package_data")
         if not have_package_data:
@@ -91,16 +87,68 @@ class Distribution(_Distribution):
         self.features = {}
         self.test_suite = None
         self.requires = []
+        self.install_requires = []
+        self.extras_require = {}
         _Distribution.__init__(self,attrs)
         if not have_package_data:
             from setuptools.command.build_py import build_py
             self.cmdclass.setdefault('build_py',build_py)
+
         self.cmdclass.setdefault('build_ext',build_ext)
         self.cmdclass.setdefault('install',install)
         self.cmdclass.setdefault('install_lib',install_lib)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+    def finalize_options(self):
+        _Distribution.finalize_options(self)
+
         if self.features:
             self._set_global_opts_from_features()
 
+        if self.extra_path:
+            raise DistutilsSetupError(
+                "The 'extra_path' parameter is not needed when using "
+                "setuptools.  Please remove it from your setup script."
+            )
+        try:
+            list(pkg_resources.parse_requirements(self.install_requires))
+        except (TypeError,ValueError):
+            raise DistutilsSetupError(
+                "'install_requires' must be a string or list of strings "
+                "containing valid project/version requirement specifiers"
+            )
+        try:
+            for k,v in self.extras_require.items():
+                list(pkg_resources.parse_requirements(v))
+        except (TypeError,ValueError,AttributeError):
+            raise DistutilsSetupError(
+                "'extras_require' must be a dictionary whose values are "
+                "strings or lists of strings containing valid project/version "
+                "requirement specifiers."
+            )
+
     def parse_command_line(self):
         """Process features after parsing command line options"""
         result = _Distribution.parse_command_line(self)
@@ -108,19 +156,12 @@ class Distribution(_Distribution):
             self._finalize_features()
         return result
 
+
     def _feature_attrname(self,name):
         """Convert feature name to corresponding option attribute name"""
         return 'with_'+name.replace('-','_')
 
 
-
-
-
-
-
-
-
-
     def _set_global_opts_from_features(self):
         """Add --with-X/--without-X options based on optional features"""
 
@@ -343,29 +384,29 @@ class Distribution(_Distribution):
 
         return nargs
 
-
     def has_dependencies(self):
         return not not self.requires
 
-
     def run_commands(self):
-        if setuptools.bootstrap_install_from and 'install' in self.commands:
+        for cmd in self.commands:
+            if cmd=='install' and not cmd in self.have_run:
+                self.install_eggs()
+            else:
+                self.run_command(cmd)
+
+    def install_eggs(self):
+        from easy_install import easy_install
+        cmd = easy_install(self, args="x")
+        cmd.ensure_finalized()  # finalize before bdist_egg munges install cmd
+        self.run_command('bdist_egg')
+        args = [self.get_command_obj('bdist_egg').egg_output]
+        if setuptools.bootstrap_install_from:
             # Bootstrap self-installation of setuptools
-            from easy_install import easy_install
-            cmd = easy_install(
-                self, args=[setuptools.bootstrap_install_from], zip_ok=1
-            )
-            cmd.ensure_finalized()
-            cmd.run()
-            setuptools.bootstrap_install_from = None
-
-        _Distribution.run_commands(self)
-
-
-
-
-
-
+            args.insert(0, setuptools.bootstrap_install_from)
+        cmd.args = args
+        cmd.run()
+        self.have_run['install'] = 1
+        setuptools.bootstrap_install_from = None
 
     def get_cmdline_options(self):
         """Return a '{cmd: {opt:val}}' map of all command-line options

setuptools/package_index.py

@@ -203,7 +203,7 @@ class PackageIndex(AvailableDistributions):
 
 
 
-    def find_packages(self,requirement):
+    def find_packages(self, requirement):
         self.scan_url(self.index_url + requirement.distname+'/')
         if not self.package_pages.get(requirement.key):
             # We couldn't find the target package, so search the index page too
@@ -221,13 +221,13 @@ class PackageIndex(AvailableDistributions):
             # scan each page that might be related to the desired package
             self.scan_url(url)
 
-    def obtain(self,requirement):
+    def obtain(self, requirement, installer=None):
         self.find_packages(requirement)
         for dist in self.get(requirement.key, ()):
             if dist in requirement:
                 return dist
             self.debug("%s does not match %s", requirement, dist)                
-
+        return super(PackageIndex, self).obtain(requirement,installer)
 
 
 
@@ -245,19 +245,20 @@ class PackageIndex(AvailableDistributions):
 
 
     def download(self, spec, tmpdir):
-        """Locate and/or download `spec`, returning a local filename
+        """Locate and/or download `spec` to `tmpdir`, returning a local path
 
         `spec` may be a ``Requirement`` object, or a string containing a URL,
-        an existing local filename, or a package/version requirement spec
+        an existing local filename, or a project/version requirement spec
         (i.e. the string form of a ``Requirement`` object).
 
-        If necessary, the requirement is searched for in the package index.
-        If the download is successful, the return value is a local file path,
-        and it is a subpath of `tmpdir` if the distribution had to be
-        downloaded.  If no matching distribution is found, return ``None``.
-        Various errors may be raised if a problem occurs during downloading.
+        If `spec` is a ``Requirement`` object or a string containing a
+        project/version requirement spec, this method is equivalent to
+        the ``fetch()`` method.  If `spec` is a local, existing file or
+        directory name, it is simply returned unchanged.  If `spec` is a URL,
+        it is downloaded to a subpath of `tmpdir`, and the local filename is
+        returned.  Various errors may be raised if a problem occurs during
+        downloading.
         """
-
         if not isinstance(spec,Requirement):
             scheme = URL_SCHEME(spec)
             if scheme:
@@ -275,16 +276,56 @@ class PackageIndex(AvailableDistributions):
                         "Not a URL, existing file, or requirement spec: %r" %
                         (spec,)
                     )
+
+        return self.fetch(spec, tmpdir, force_scan)
+
+
+
+
+
+
+
+    def fetch(self, requirement, tmpdir, force_scan=False):
+        """Obtain a file suitable for fulfilling `requirement`
+
+        `requirement` must be a ``pkg_resources.Requirement`` instance.
+        If necessary, or if the `force_scan` flag is set, the requirement is
+        searched for in the (online) package index as well as the locally
+        installed packages.  If a distribution matching `requirement` is found,
+        the return value is the same as if you had called the ``download()``
+        method with the matching distribution's URL.  If no matching
+        distribution is found, returns ``None``.
+        """
+
         # process a Requirement
-        self.info("Searching for %s", spec)
-        dist = self.best_match(spec,[])
+        self.info("Searching for %s", requirement)
+
+        if force_scan:
+            self.find_packages(requirement)
+
+        dist = self.best_match(requirement, [])     # XXX
+
         if dist is not None:
             self.info("Best match: %s", dist)
             return self.download(dist.path, tmpdir)
 
-        self.warn("No local packages or download links found for %s", spec)
+        self.warn(
+            "No local packages or download links found for %s", requirement
+        )
         return None
 
+
+
+
+
+
+
+
+
+
+
+
+
     dl_blocksize = 8192
 
     def _download_to(self, url, filename):