Added exhaustive testing of the install directory, including a spawn test

for ``.pth`` file support, and directory writability/existence checks.  This
should virtually eliminate the need to set or configure ``--site-dirs``.

--HG--
branch : setuptools
extra : convert_revision : svn%3A6015fed2-1504-0410-9fe1-9d1591cc4771/sandbox/trunk/setuptools%4042346

EasyInstall.txt

@@ -182,9 +182,9 @@ installation directory, to ensure that the script will have access to the
 installed package.  You can override this using the ``-s`` or ``--script-dir``
 option.
 
-Packages installed to ``site-packages`` are added to an ``easy-install.pth``
-file, so that Python will always use the most-recently-installed version of
-the package.  If you would like to be able to select which version to use at
+Installed packages are added to an ``easy-install.pth`` file in the install
+directory, so that Python will always use the most-recently-installed version
+of the package.  If you would like to be able to select which version to use at
 runtime, you should use the ``-m`` or ``--multi-version`` option.
 
 
@@ -226,7 +226,8 @@ the newer version is the only upgrade step needed.
 If you haven't suppressed script installation (using ``--exclude-scripts`` or
 ``-x``), then the upgraded version's scripts will be installed, and they will
 be automatically patched to ``require()`` the corresponding version of the
-package, so that you can use them even if not installing to ``site-packages``.
+package, so that you can use them even if they are installed in multi-version
+mode.
 
 ``easy_install`` never actually deletes packages (unless you're installing a
 package with the same name and version number as an existing package), so if
@@ -234,8 +235,8 @@ you want to get rid of older versions of a package, please see `Uninstalling
 Packages`_, below.
 
 
-Changing the Active Version (``site-packages`` installs only)
--------------------------------------------------------------
+Changing the Active Version
+---------------------------
 
 If you've upgraded a package, but need to revert to a previously-installed
 version, you can do so like this::
@@ -645,7 +646,8 @@ Command-Line Options
     location is used.  Normally, this would be the ``site-packages`` directory,
     but if you are using distutils configuration files, setting things like
     ``prefix`` or ``install_lib``, then those settings are taken into
-    account when computing the default installation directory.
+    account when computing the default installation directory, as is the
+    ``--prefix`` option.
 
 ``--script-dir=DIR, -s DIR``
     Set the script installation directory.  If you don't supply this option
@@ -796,13 +798,15 @@ Command-Line Options
 ``--site-dirs=DIRLIST, -S DIRLIST``   (New in 0.6a1)
     Specify one or more custom "site" directories (separated by commas).
     "Site" directories are directories where ``.pth`` files are processed, such
-    as the main Python ``site-packages`` directory. By default, EasyInstall
-    only knows about Python-defined "site" directories, not those that may be
-    added by an OS distribution or site administrator using call(s) to
-    ``site.addsitedir()``.  You should not normally need to use this option
-    directly, as your system administrator should configure it in the
-    ``distutils.cfg`` file of the Python installation.  See the `Administrator
-    Installation`_ section below for details.
+    as the main Python ``site-packages`` directory.  As of 0.6a10, EasyInstall
+    automatically detects whether a given directory processes ``.pth`` files
+    (or can be made to do so), so you should not normally need to use this
+    option.  It is is now only necessary if you want to override EasyInstall's
+    judgment and force an installation directory to be treated as if it
+    supported ``.pth`` files.
+
+    (If you want to *make* a non-``PYTHONPATH`` directory support ``.pth``
+    files, please see the `Administrator Installation`_ section below.)
 
 ``--no-deps, -N``  (New in 0.6a6)
     Don't install any dependencies.  This is intended as a convenience for
@@ -829,6 +833,18 @@ Command-Line Options
     setting for this option in their `configuration files`_, and then manually
     override the setting on the command line as needed.
 
+``--prefix=DIR`` (New in 0.6a10)
+    Use the specified directory as a base for computing the default
+    installation and script directories.  On Windows, the resulting default
+    directories will be ``prefix\\Lib\\site-packages`` and ``prefix\\Scripts``,
+    while on other platforms the defaults will be
+    ``prefix/lib/python2.X/site-packages`` (with the appropriate version
+    substituted) for libraries and ``prefix/bin`` for scripts.
+
+    Note that the ``--prefix`` option only sets the *default* installation and
+    script directories, and does not override the ones set on the command line
+    or in a configuration file.
+
 
 .. _non-root installation:
 
@@ -883,10 +899,6 @@ free to choose which one best suits your system and needs.
     `Administrator Installation`_ to enable ``.pth`` processing in the custom
     location instead, as that is easier and more flexible than this approach.)
 
-    This is the least robust and least flexible of the approaches, however, so
-    you should probably at least take a look at the others and consider whether
-    one of them might be an improvement over your current setup.
-
 
 Administrator Installation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -922,12 +934,8 @@ to the file, substituting the correct Python version if necessary::
     #
     install_scripts = ~/bin
 
-    [easy_install]
-    site_dirs = ~/lib/python2.3
-
 This will configure the distutils and EasyInstall to install packages to the
-user's home directory by default, and will tell EasyInstall that Python has
-been configured to accept ``.pth`` files in that directory.
+user's home directory by default.
 
 Of course, you aren't limited to using a ``~/lib/python2.X`` directory with
 this approach.  You can substitute a specific systemwide directory if you like.
@@ -1058,6 +1066,10 @@ Known Issues
    time out or be missing a file.
 
 0.6a10
+ * Added exhaustive testing of the install directory, including a spawn test
+   for ``.pth`` file support, and directory writability/existence checks.  This
+   should virtually eliminate the need to set or configure ``--site-dirs``.
+
  * Added ``--prefix`` option for more do-what-I-mean-ishness in the absence of
    RTFM-ing.  :)
 

setuptools/command/easy_install.py

@@ -155,22 +155,7 @@ class easy_install(Command):
                     )
                 else:
                     self.all_site_dirs.append(normalize_path(d))
-        instdir = normalize_path(self.install_dir)
-        if instdir in self.all_site_dirs:
-            if self.pth_file is None:
-                self.pth_file = PthDistributions(
-                    os.path.join(instdir,'easy-install.pth')
-                )
-
-        elif not self.multi_version:
-            # Can't install non-multi to non-site dir
-            raise DistutilsError(self.no_default_version_msg())
-
-        if instdir in map(normalize_path, self.site_dirs or []):
-            # don't install site.py if install target is already a site dir
-            self.sitepy_installed = True
-
-        self.install_dir = instdir
+        self.check_site_dir()
         self.index_url = self.index_url or "http://www.python.org/pypi"
         self.shadow_path = self.all_site_dirs[:]
         for path_item in self.install_dir, normalize_path(self.script_dir):
@@ -212,14 +197,12 @@ class easy_install(Command):
             raise DistutilsArgError(
                 "Must specify a build directory (-b) when using --editable"
             )
-
         if not self.args:
             raise DistutilsArgError(
                 "No urls, filenames, or requirements specified (see --help)")
 
         self.outputs = []
 
-
     def run(self):
         if self.verbose<>self.distribution.verbose:
             log.set_verbosity(self.verbose)
@@ -242,8 +225,148 @@ class easy_install(Command):
             log.set_verbosity(self.distribution.verbose)
 
 
+    def pseudo_tempname(self):
+        """Return a pseudo-tempname base in the install directory.
+
+        This code is intentionally naive; if a malicious party can write to
+        the target directory you're already in deep doodoo.
+        """
+        try:
+            pid = os.getpid()
+        except:
+            import random
+            pid = random.randint(0,sys.maxint)
+        return os.path.join(self.install_dir, "test-easy-install-%s" % pid)
+
+
+
 
 
+
+
+    def check_site_dir(self):
+        """Verify that self.install_dir is .pth-capable dir, if needed"""
+
+        instdir = normalize_path(self.install_dir)
+        pth_file = os.path.join(instdir,'easy-install.pth')
+
+        # Is it a configured, PYTHONPATH, implicit, or explicit site dir?
+        is_site_dir = instdir in self.all_site_dirs
+
+        if not is_site_dir:
+            # No?  Then directly test whether it does .pth file processing
+            is_site_dir = self.check_pth_processing()
+        else:
+            # make sure we can write to target dir
+            testfile = self.pseudo_tempname()+'.write-test'
+            test_exists = os.path.exists(testfile)
+            try:
+                if test_exists: os.unlink(testfile)
+                open(testfile,'w').close()
+                os.unlink(testfile)
+            except (OSError,IOError):
+                self.cant_write_to_target()
+
+        if not is_site_dir and not self.multi_version:
+            # Can't install non-multi to non-site dir
+            raise DistutilsError(self.no_default_version_msg())
+
+        if is_site_dir:
+            if self.pth_file is None:
+                self.pth_file = PthDistributions(pth_file)
+        else:
+            self.pth_file = None
+
+        PYTHONPATH = os.environ.get('PYTHONPATH','').split(os.pathsep)
+        if instdir not in map(normalize_path, filter(None,PYTHONPATH)):
+            # only PYTHONPATH dirs need a site.py, so pretend it's there
+            self.sitepy_installed = True
+
+        self.install_dir = instdir
+
+
+    def cant_write_to_target(self):
+        msg = """can't create or remove files in install directory 
+
+The following error occurred while trying to add or remove files in the
+installation directory:
+
+    %s
+
+The installation directory you specified (via --install-dir, --prefix, or
+the distutils default setting) was:
+
+    %s
+"""     % (sys.exc_info()[1], self.install_dir,)
+
+        if not os.path.exists(self.install_dir):
+            msg += """
+This directory does not currently exist.  Please create it and try again, or
+choose a different installation directory (using the -d or --install-dir
+option).
+"""
+        else:
+            msg += """
+Perhaps your account does not have write access to this directory?  If the
+installation directory is a system-owned directory, you may need to sign in
+as the administrator or "root" account.  If you do not have administrative
+access to this machine, you may wish to choose a different installation
+directory, preferably one that is listed in your PYTHONPATH environment
+variable.
+
+For information on other options, you may wish to consult the
+documentation at:
+    
+  http://peak.telecommunity.com/EasyInstall.html
+
+Please make the appropriate changes for your system and try again.
+"""
+        raise DistutilsError(msg)
+
+
+
+
+    def check_pth_processing(self):
+        """Empirically verify whether .pth files are supported in inst. dir"""
+        instdir = self.install_dir
+        log.info("Checking .pth file support in %s", instdir)
+        pth_file = self.pseudo_tempname()+".pth"
+        ok_file = pth_file+'.ok'
+        ok_exists = os.path.exists(ok_file)
+        try:
+            if ok_exists: os.unlink(ok_file)
+            f = open(pth_file,'w')
+        except (OSError,IOError):
+            self.cant_write_to_target()
+        else:
+            try:
+                f.write("import os;open(%r,'w').write('OK')\n" % (ok_file,))
+                f.close(); f=None   
+                executable = sys.executable
+                if os.name=='nt':   
+                    dirname,basename = os.path.split(executable)
+                    alt = os.path.join(dirname,'pythonw.exe')
+                    if basename.lower()=='python.exe' and os.path.exists(alt):
+                        # use pythonw.exe to avoid opening a console window 
+                        executable = alt
+        
+                from distutils.spawn import spawn
+                spawn([executable,'-E','-c','pass'],0)
+
+                if os.path.exists(ok_file):
+                    log.info(
+                        "TEST PASSED: %s appears to support .pth files",
+                        instdir
+                    )
+                    return True
+            finally:
+                if f: f.close()
+                if os.path.exists(ok_file): os.unlink(ok_file) 
+                if os.path.exists(pth_file): os.unlink(pth_file)
+                
+        log.warn("TEST FAILED: %s does NOT support .pth files", instdir)
+        return False
+
     def install_egg_scripts(self, dist):
         """Write all the scripts for `dist`, unless scripts are excluded"""
 
@@ -904,9 +1027,9 @@ See the setuptools documentation for the "develop" command for more info.
         return """bad install directory or PYTHONPATH
 
 You are attempting to install a package to a directory that is not
-on PYTHONPATH and is not registered as supporting Python ".pth" files
-by default.  The installation directory you specified (via --install-dir,
---prefix, or the distutils default setting) was:
+on PYTHONPATH and which Python does not read ".pth" files from.  The
+installation directory you specified (via --install-dir, --prefix, or
+the distutils default setting) was:
 
     %s
 
@@ -923,9 +1046,8 @@ Here are some of your options for correcting the problem:
   variable.  (It must then also be on PYTHONPATH whenever you run
   Python and want to use the package(s) you are installing.)
 
-* You can set up the installation directory to support ".pth" files,
-  and configure EasyInstall to recognize this, by using one of the
-  approaches described here:
+* You can set up the installation directory to support ".pth" files by
+  using one of the approaches described here:
 
   http://peak.telecommunity.com/EasyInstall.html#custom-installation-locations
 
@@ -941,6 +1063,7 @@ Please make the appropriate changes for your system and try again.""" % (
 
 
 
+
     def install_site_py(self):
         """Make sure there's a site.py in the target dir, if needed"""