Implemented DWIM for PYTHONPATH. That is, ez_setup and easy_install

should now "just work" if you're using a PYTHONPATH target, and if it
can't "just work", you get helpful instructions and doc links.

--HG--
branch : setuptools
extra : convert_revision : svn%3A6015fed2-1504-0410-9fe1-9d1591cc4771/sandbox/trunk/setuptools%4042308

EasyInstall.txt

@@ -34,7 +34,9 @@ Download `ez_setup.py <http://peak.telecommunity.com/dist/ez_setup.py>`_, and
 run it; this will download and install the appropriate ``setuptools`` egg for
 your Python version.  (You will need at least Python 2.3.5, or if you are on a
 64-bit platform, Python 2.4.)  An ``easy_install`` script will be installed in
-the normal location for Python scripts on your platform.
+the normal location for Python scripts on your platform.  (Windows users, don't
+put ``ez_setup.py`` inside your Python installation; please put it in some
+other directory before running it.)
 
 You may receive a message telling you about an obsolete version of setuptools
 being present; if so, you must be sure to delete it entirely, along with the
@@ -185,11 +187,6 @@ file, so that Python will always use the most-recently-installed version of
 the package.  If you would like to be able to select which version to use at
 runtime, you should use the ``-m`` or ``--multi-version`` option.
 
-Note, however, that installing to a directory other than ``site-packages``
-already implies the ``-m`` option, so if you cannot install to
-``site-packages``, please see the `Command-Line Options`_ section below (under
-``--multi-version``) to find out how to select packages at runtime.
-
 
 Upgrading a Package
 -------------------
@@ -215,17 +212,16 @@ or by using a download page, direct download URL, or package filename::
 
     easy_install my_downloads/ExamplePackage-2.0.tgz
 
-If you're using ``-m`` or ``--multi`` (or installing outside of
-``site-packages``), using the ``require()`` function at runtime automatically
-selects the newest installed version of a package that meets your version
-criteria.  So, installing a newer version is the only step needed to upgrade
-such packages.
+If you're using ``-m`` or ``--multi-version`` , using the ``require()``
+function at runtime automatically selects the newest installed version of a
+package that meets your version criteria.  So, installing a newer version is
+the only step needed to upgrade such packages.
 
-If you're installing to Python's ``site-packages`` directory (and not
-using ``-m``), installing a package automatically replaces any previous version
-in the ``easy-install.pth`` file, so that Python will import the most-recently
-installed version by default.  So, again, installing the newer version is the
-only upgrade step needed.
+If you're installing to a directory on PYTHONPATH, or a configured "site"
+directory (and not using ``-m``), installing a package automatically replaces
+any previous version in the ``easy-install.pth`` file, so that Python will
+import the most-recently installed version by default.  So, again, installing
+the newer version is the only upgrade step needed.
 
 If you haven't suppressed script installation (using ``--exclude-scripts`` or
 ``-x``), then the upgraded version's scripts will be installed, and they will
@@ -412,17 +408,16 @@ Compressed Installation
 -----------------------
 
 EasyInstall tries to install packages in zipped form, if it can.  Zipping
-packages can significantly increase Python's overall import performance if
-you're installing to``site-packages`` and not using the ``--multi`` option,
-because Python processes zipfile entries on ``sys.path`` much faster than it
-does directories.
+packages can improve Python's overall import performance if you're not using
+the ``--multi-version`` option, because Python processes zipfile entries on
+``sys.path`` much faster than it does directories.
 
 As of version 0.5a9, EasyInstall analyzes packages to determine whether they
 can be safely installed as a zipfile, and then acts on its analysis.  (Previous
 versions would not install a package as a zipfile unless you used the
 ``--zip-ok`` option.)
 
-The current analysis approach is very conservative; it currenly looks for:
+The current analysis approach is fairly conservative; it currenly looks for:
 
  * Any use of the ``__file__`` or ``__path__`` variables (which should be
    replaced with ``pkg_resources`` API calls)
@@ -538,11 +533,9 @@ Command-Line Options
     versions and enabling optional dependencies, see the ``pkg_resources`` API
     doc.)
 
-    Note that if you install to a directory other than ``site-packages``,
-    this option is automatically in effect, because ``.pth`` files can only be
-    used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use
-    the ``--install-dir`` or ``-d`` option (or they are set via configuration
-    file(s)) you must also use ``require()`` to enable packages at runtime.
+    Changed in 0.6a10: this option is no longer silently enabled when
+    installing to a non-PYTHONPATH, non-"site" directory.  You must always
+    explicitly use this option if you want it to be active.
 
 ``--upgrade, -U``   (New in 0.5a4)
     By default, EasyInstall only searches online if a project/version
@@ -958,21 +951,16 @@ already have them::
     install_lib = ~/py-lib
     install_scripts = ~/bin
     
-    [easy_install]
-    site_dirs = ~/py_lib
-
 Be sure to do this *before* you try to run the ``ez_setup.py`` installation
-script.  Then, follow the standard `installation instructions`_, but take
-careful note of the full pathname of the ``.egg`` file that gets installed, so
-that you can add it to your ``PYTHONPATH``, along with ``~/py_lib``.
+script.  Then, follow the standard `installation instructions`_, but make
+sure that ``~/py-lib`` is listed in your ``PYTHONPATH`` environment variable.
+
+Your library installation directory *must* be in listed in ``PYTHONPATH``,
+not only when you install packages with EasyInstall, but also when you use
+any packages that are installed using EasyInstall.  You will probably want to
+edit your ``~/.profile`` or other configuration file(s) to ensure that it is
+set, if you haven't already got this set up on your machine.
 
-You *must* add the setuptools egg file *and* ``~/py_lib`` to your
-``PYTHONPATH`` environment variable manually, or it will not work, and neither
-will any other packages you install with EasyInstall.  You will not, however,
-have to manually add any other packages to the ``PYTHONPATH``; EasyInstall will
-take care of them for you by automatically editing
-``~/py-lib/easy-install.pth``, as long as the setuptools egg is explicitly
-listed in ``PYTHONPATH``.
 
 
 Release Notes/Change History
@@ -983,6 +971,11 @@ Known Issues
    time out or be missing a file.
 
 0.6a10
+ * Enhanced ``PYTHONPATH`` support so that you don't have to put any eggs on it
+   to make it work.  ``--multi-version`` is no longer a silent default; you
+   must explicitly use it if installing to a non-PYTHONPATH, non-"site"
+   directory.
+
  * Expand ``$variables`` used in the ``--site-dirs``, ``--build-directory``,
    ``--install-dir``, and ``--script-dir`` options, whether on the command line
    or in configuration files.

setuptools/command/develop.py

@@ -67,6 +67,8 @@ class develop(easy_install):
         self.reinitialize_command('build_ext', inplace=1)
         self.run_command('build_ext')
 
+        self.install_site_py()  # ensure that target dir is site-safe
+
         # create an .egg-link in the installation dir, pointing to our egg
         log.info("Creating %s (link to %s)", self.egg_link, self.egg_base)
         if not self.dry_run:
@@ -78,8 +80,6 @@ class develop(easy_install):
         # and handling requirements
         self.process_distribution(None, self.dist)
 
-
-
     def uninstall_link(self):
         if os.path.exists(self.egg_link):
             log.info("Removing %s (link to %s)", self.egg_link, self.egg_base)

setuptools/command/easy_install.py

@@ -99,7 +99,7 @@ class easy_install(Command):
         self.ignore_conflicts_at_my_risk = None
         self.site_dirs = None
         self.installed_projects = {}
-
+        self.sitepy_installed = False
         # Always read easy_install options, even if we are subclassed, or have
         # an independent instance created.  This ensures that defaults will
         # always come from the standard configuration file(s)' "easy_install"
@@ -155,21 +155,20 @@ class easy_install(Command):
                     )
                 else:
                     self.all_site_dirs.append(normalize_path(d))
-        instdir = normalize_path(self.install_dir or self.all_site_dirs[-1])
+        instdir = normalize_path(self.install_dir)
         if instdir in self.all_site_dirs:
             if self.pth_file is None:
                 self.pth_file = PthDistributions(
                     os.path.join(instdir,'easy-install.pth')
                 )
 
-        elif self.multi_version is None:
-            self.multi_version = True
-
         elif not self.multi_version:
-            # explicit false set from Python code; raise an error
-            raise DistutilsArgError(
-                "Can't do single-version installs outside 'site-package' dirs"
-            )
+            # Can't install non-multi to non-site dir
+            raise DistutilsError(self.no_default_version_msg())
+
+        if instdir in map(normalize_path, self.site_dirs or []):
+            # don't install site.py if install target is already a site dir
+            self.sitepy_installed = True
 
         self.install_dir = instdir
         self.index_url = self.index_url or "http://www.python.org/pypi"
@@ -194,6 +193,7 @@ class easy_install(Command):
                 self.find_links = self.find_links.split()
         else:
             self.find_links = []
+
         self.package_index.add_find_links(self.find_links)
         self.set_undefined_options('install_lib', ('optimize','optimize'))
         if not isinstance(self.optimize,int):
@@ -288,6 +288,7 @@ class easy_install(Command):
     def easy_install(self, spec, deps=False):
         tmpdir = tempfile.mkdtemp(prefix="easy_install-")
         download = None
+        self.install_site_py()
 
         try:
             if not isinstance(spec,Requirement):
@@ -325,7 +326,6 @@ class easy_install(Command):
             if os.path.exists(tmpdir):
                 rmtree(tmpdir)
 
-
     def install_item(self, spec, download, tmpdir, deps, install_needed=False):
 
         # Installation is also needed if file in tmpdir or is not an egg
@@ -687,7 +687,7 @@ class easy_install(Command):
                             if f: f.close()
                             if filename not in blockers:
                                 blockers.append(filename)
-                    elif ext in exts:
+                    elif ext in exts and base!='site':  # XXX ugh
                         blockers.append(os.path.join(path,filename))
 
         if blockers:
@@ -900,9 +900,92 @@ See the setuptools documentation for the "develop" command for more info.
 
 
 
+    def no_default_version_msg(self):
+        return """
+-----------------------------------------------------------------------
+CONFIGURATION PROBLEM:
+
+You are attempting to install a package to a directory that is not
+on PYTHONPATH and is not registered as supporting Python ".pth" files
+by default.  Here are some of your options for correcting this:
+
+* You can choose a different installation directory, i.e., one that is
+  on PYTHONPATH or supports .pth files
+
+* You can add the installation directory to the PYTHONPATH environment
+  variable.  (It must then also be on PYTHONPATH whenever you run
+  Python and want to use the package(s) you are installing.)
+
+* You can set up the installation directory to support ".pth" files,
+  and configure EasyInstall to recognize this, by using one of the
+  approaches described here:
+
+  http://peak.telecommunity.com/EasyInstall.html#custom-installation-locations
+
+Please make the appropriate changes for your system and try again.
+Thank you for your patience.
+-----------------------------------------------------------------------
+"""
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+    def install_site_py(self):
+        """Make sure there's a site.py in the target dir, if needed"""
+
+        if self.sitepy_installed:
+            return  # already did it, or don't need to
+
+        sitepy = os.path.join(self.install_dir, "site.py")
+        source = resource_string(Requirement.parse("setuptools"), "site.py")
+
+        if os.path.exists(sitepy):
+            log.debug("Checking existing site.py in %s", self.install_dir)
+            current = open(sitepy,'rb').read()
+            if current != source:
+                raise DistutilsError(
+                    "%s is not a setuptools-generated site.py; please"
+                    " remove it." % sitepy
+                )
+        else:
+            log.info("Creating %s", sitepy)
+            if not self.dry_run:
+                f = open(sitepy,'wb')
+                f.write(source)
+                f.close()
+            self.byte_compile([sitepy])
+
+        self.sitepy_installed = True
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 def get_site_dirs():
-    # return a list of 'site' dirs, based on 'site' module's code to do this
-    sitedirs = []
+    # return a list of 'site' dirs
+    sitedirs = filter(None,os.environ.get('PYTHONPATH','').split(os.pathsep))
     prefixes = [sys.prefix]
     if sys.exec_prefix != sys.prefix:
         prefixes.append(sys.exec_prefix)
@@ -939,7 +1022,7 @@ def get_site_dirs():
 
     sitedirs = filter(os.path.isdir, sitedirs)
     sitedirs = map(normalize_path, sitedirs)
-    return sitedirs     # ensure at least one
+    return sitedirs
 
 def expand_paths(inputs):
     """Yield sys.path directories that might contain "old-style" packages"""