this is now the branch for my site-packages changes. It is merged with trunk....

this is now the branch for my site-packages changes.  It is merged with trunk.  The zc.recipe.egg recipe returns to being fully backward compatible; if you want to use a system Python, use the z3c.recipe.scripts recipe for scripts and interpreter.

CHANGES.txt

@@ -6,6 +6,25 @@ Change History
 
 New Features:
 
+- Buildout can be safely used with a system Python, as long as you use the
+  new z3c.recipe.scripts recipe to generate scripts and interpreters, rather
+  than zc.recipe.egg (which is still a fully supported, and simpler, way of
+  generating scripts and interpreters if you are using a "clean" Python).
+
+  A hopefully slight limitation: in no cases are distributions in your
+  site-packages used to satisfy buildout dependencies.  The
+  site-packages can be used in addition to the dependencies specified in
+  your buildout, and buildout dependencies can override code in your
+  site-packages, but even if your Python's site-packages has the same
+  exact version as specified in your buildout configuration, buildout
+  will still use its own copy.
+
+- Added new function, ``zc.buildout.easy_install.generate_scripts``, to
+  generate scripts and interpreter.  It produces a full-featured
+  interpreter (all command-line options supported) and the ability to
+  safely let scripts include site packages.  The ``z3c.recipe.scripts``
+  recipe uses this new function.
+
 - Improve bootstrap.
 
   * New options let you specify where to find ez_setup.py and where to find
@@ -23,6 +42,17 @@ Bugs fixed:
   This means, among other things, that ``bin/buildout -vv`` and
   ``bin/buildout annotate`` correctly list more of the options.
 
+- Installing a namespace package using a Python that already has a package
+  in the same namespace (e.g., in the Python's site-packages) failed in
+  some cases.
+
+- Another variation of this error showed itself when at least two
+  dependencies were in a shared location like site-packages, and the
+  first one met the "versions" setting.  The first dependency would be
+  added, but subsequent dependencies from the same location (e.g.,
+  site-packages) would use the version of the package found in the
+  shared location, ignoring the version setting.
+
 1.4.3 (2009-12-10)
 ==================
 

README.txt

@@ -37,6 +37,11 @@ Existing recipes include:
    dependencies.  It installs their console-script entry points with
    the needed eggs included in their paths.
 
+`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
+  This scripts recipe builds interpreter scripts and entry point scripts
+  based on eggs.  These scripts have more features and flexibility than the
+  ones offered by zc.recipe.egg.
+
 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
    The testrunner egg creates a test runner script for one or
    more eggs.

buildout.cfg

 [buildout]
-develop = zc.recipe.egg_ .
+develop = zc.recipe.egg_ z3c.recipe.scripts_ .
 parts = test oltest py
 
 [py]
@@ -13,6 +13,7 @@ recipe = zc.recipe.testrunner
 eggs = 
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 
 # Tests that can be run wo a network
 [oltest]
@@ -20,6 +21,7 @@ recipe = zc.recipe.testrunner
 eggs = 
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 defaults =
   [
   '-t',

src/zc/buildout/easy_install.py

@@ -137,9 +137,50 @@ if is_win32:
 else:
     _safe_arg = str
 
-_easy_install_cmd = _safe_arg(
-    'from setuptools.command.easy_install import main; main()'
-    )
+# The following string is used to run easy_install in
+# Installer._call_easy_install.  It is started with python -S (that is,
+# don't import site at start).  That flag, and all of the code in this
+# snippet above the last two lines, exist to work around a relatively rare
+# problem.  If
+#
+# - your buildout configuration is trying to install a package that is within
+#   a namespace package, and
+#
+# - you use a Python that has a different version of this package
+#   installed in in its site-packages using
+#   --single-version-externally-managed (that is, using the mechanism
+#   sometimes used by system packagers:
+#   http://peak.telecommunity.com/DevCenter/setuptools#install-command ), and
+#
+# - the new package tries to do sys.path tricks in the setup.py to get a
+#   __version__,
+#
+# then the older package will be loaded first, making the setup version
+# the wrong number. While very arguably packages simply shouldn't do
+# the sys.path tricks, some do, and we don't want buildout to fall over
+# when they do.
+#
+# The namespace packages installed in site-packages with
+# --single-version-externally-managed use a mechanism that cause them to
+# be processed when site.py is imported.  Simply starting Python with -S
+# addresses the problem in Python 2.4 and 2.5, but Python 2.6's distutils
+# imports a value from the site module, so we unfortunately have to do more
+# drastic surgery in the _easy_install_cmd code below.  The changes to
+# sys.modules specifically try to only remove namespace modules installed by
+# the --single-version-externally-managed code.
+
+_easy_install_cmd = _safe_arg('''\
+import sys; \
+p = sys.path[:]; \
+m = sys.modules.keys(); \
+import site; \
+sys.path[:] = p; \
+m_attrs = set(('__builtins__', '__file__', '__package__', '__path__')); \
+match = set(('__path__',)); \
+[sys.modules.pop(k) for k, v in sys.modules.items()\
+ if k not in m and v and m_attrs.intersection(dir(v)) == match]; \
+from setuptools.command.easy_install import main; \
+main()''')
 
 
 class Installer:
@@ -301,7 +342,7 @@ class Installer:
         try:
             path = setuptools_loc
 
-            args = ('-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
+            args = ('-Sc', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
             if self._always_unzip:
                 args += ('-Z', )
             level = logger.getEffectiveLevel()
@@ -904,6 +945,9 @@ def develop(setup, dest,
 def working_set(specs, executable, path):
     return install(specs, None, executable=executable, path=path)
 
+############################################################################
+# Script generation functions
+
 def scripts(reqs, working_set, executable, dest,
             scripts=None,
             extra_paths=(),
@@ -912,20 +956,85 @@ def scripts(reqs, working_set, executable, dest,
             initialization='',
             relative_paths=False,
             ):
+    """Generate scripts and/or an interpreter.
 
-    path = [dist.location for dist in working_set]
-    path.extend(extra_paths)
-    path = map(realpath, path)
+    See generate_scripts for a newer version with more options and a
+    different approach.
+    """
+    path = _get_path(working_set, extra_paths)
+    if initialization:
+        initialization = '\n'+initialization+'\n'
+    generated = _generate_scripts(
+        reqs, working_set, dest, path, scripts, relative_paths,
+        initialization, executable, arguments)
+    if interpreter:
+        sname = os.path.join(dest, interpreter)
+        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
+        generated.extend(
+            _pyscript(spath, sname, executable, rpsetup))
+    return generated
 
+def generate_scripts(
+    dest, working_set, executable, site_py_dest,
+    reqs=(), scripts=None, interpreter=None, extra_paths=(),
+    initialization='', add_site_packages=False, exec_sitecustomize=False,
+    relative_paths=False, script_arguments='', script_initialization=''):
+    """Generate scripts and/or an interpreter.
+
+    This accomplishes the same job as the ``scripts`` function, above,
+    but it does so in an alternative way that allows safely including
+    Python site packages, if desired, and  choosing to execute the Python's
+    sitecustomize.
+    """
     generated = []
+    generated.append(_generate_sitecustomize(
+        site_py_dest, executable, initialization, exec_sitecustomize))
+    generated.append(_generate_site(
+        site_py_dest, working_set, executable, extra_paths,
+        add_site_packages, relative_paths))
+    script_initialization = (
+        '\nimport site # imports custom buildbot-generated site.py\n%s' % (
+            script_initialization,))
+    if not script_initialization.endswith('\n'):
+        script_initialization += '\n'
+    generated.extend(_generate_scripts(
+        reqs, working_set, dest, [site_py_dest], scripts, relative_paths,
+        script_initialization, executable, script_arguments, block_site=True))
+    if interpreter:
+        generated.extend(_generate_interpreter(
+            interpreter, dest, executable, site_py_dest, relative_paths))
+    return generated
+
+# Utilities for the script generation functions.
+
+# These are shared by both ``scripts`` and ``generate_scripts``
 
+def _get_path(working_set, extra_paths=()):
+    """Given working set and extra paths, return a normalized path list."""
+    path = [dist.location for dist in working_set]
+    path.extend(extra_paths)
+    return map(realpath, path)
+
+def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
+                      initialization, executable, arguments,
+                      block_site=False):
+    """Generate scripts for the given requirements.
+
+    - reqs is an iterable of string requirements or entry points.
+    - The requirements must be findable in the given working_set.
+    - The dest is the directory in which the scripts should be created.
+    - The path is a list of paths that should be added to sys.path.
+    - The scripts is an optional dictionary.  If included, the keys should be
+      the names of the scripts that should be created, as identified in their
+      entry points; and the values should be the name the script should
+      actually be created with.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    """
     if isinstance(reqs, str):
         raise TypeError('Expected iterable of requirements or entry points,'
                         ' got string.')
-
-    if initialization:
-        initialization = '\n'+initialization+'\n'
-
+    generated = []
     entry_points = []
     for req in reqs:
         if isinstance(req, str):
@@ -939,7 +1048,6 @@ def scripts(reqs, working_set, executable, dest,
                     )
         else:
             entry_points.append(req)
-
     for name, module_name, attrs in entry_points:
         if scripts is not None:
             sname = scripts.get(name)
@@ -947,40 +1055,48 @@ def scripts(reqs, working_set, executable, dest,
                 continue
         else:
             sname = name
-
         sname = os.path.join(dest, sname)
         spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-
         generated.extend(
-            _script(module_name, attrs, spath, sname, executable, arguments,
-                    initialization, rpsetup)
-            )
-
-    if interpreter:
-        sname = os.path.join(dest, interpreter)
-        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-        generated.extend(_pyscript(spath, sname, executable, rpsetup))
-
+            _script(sname, executable, rpsetup, spath, initialization,
+                    module_name, attrs, arguments, block_site=block_site))
     return generated
 
-def _relative_path_and_setup(sname, path, relative_paths):
+def _relative_path_and_setup(sname, path,
+                             relative_paths=False, indent_level=1):
+    """Return a string of code of paths and of setup if appropriate.
+
+    - sname is the full path to the script name to be created.
+    - path is the list of paths to be added to sys.path.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    - indent_level is the number of four-space indents that the path should
+      insert before each element of the path.
+    """
     if relative_paths:
         relative_paths = os.path.normcase(relative_paths)
         sname = os.path.normcase(os.path.abspath(sname))
-        spath = ',\n  '.join(
+        spath = _format_paths(
             [_relativitize(os.path.normcase(path_item), sname, relative_paths)
-             for path_item in path]
-            )
+             for path_item in path], indent_level=indent_level)
         rpsetup = relative_paths_setup
         for i in range(_relative_depth(relative_paths, sname)):
             rpsetup += "base = os.path.dirname(base)\n"
     else:
-        spath = repr(path)[1:-1].replace(', ', ',\n  ')
+        spath = _format_paths((repr(p) for p in path),
+                              indent_level=indent_level)
         rpsetup = ''
     return spath, rpsetup
 
-
 def _relative_depth(common, path):
+    """Return number of dirs separating ``path`` from ancestor, ``common``.
+
+    For instance, if path is /foo/bar/baz/bing, and common is /foo, this will
+    return 2--in UNIX, the number of ".." to get from bing's directory
+    to foo.
+
+    This is a helper for _relative_path_and_setup.
+    """
     n = 0
     while 1:
         dirname = os.path.dirname(path)
@@ -993,6 +1109,11 @@ def _relative_depth(common, path):
     return n
 
 def _relative_path(common, path):
+    """Return the relative path from ``common`` to ``path``.
+
+    This is a helper for _relativitize, which is a helper to
+    _relative_path_and_setup.
+    """
     r = []
     while 1:
         dirname, basename = os.path.split(path)
@@ -1006,6 +1127,11 @@ def _relative_path(common, path):
     return os.path.join(*r)
 
 def _relativitize(path, script, relative_paths):
+    """Return a code string for the given path.
+
+    Path is relative to the base path ``relative_paths``if the common prefix
+    between ``path`` and ``script`` starts with ``relative_paths``.
+    """
     if path == script:
         raise AssertionError("path == script")
     common = os.path.dirname(os.path.commonprefix([path, script]))
@@ -1016,7 +1142,6 @@ def _relativitize(path, script, relative_paths):
     else:
         return repr(path)
 
-
 relative_paths_setup = """
 import os
 
@@ -1024,58 +1149,78 @@ join = os.path.join
 base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
 """
 
-def _script(module_name, attrs, path, dest, executable, arguments,
-            initialization, rsetup):
-    generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
-    contents = script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        module_name = module_name,
-        attrs = attrs,
-        arguments = arguments,
-        initialization = initialization,
-        relative_paths_setup = rsetup,
-        )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
+def _write_script(full_name, contents, logged_type):
+    """Write contents of script in full_name, logging the action.
 
+    The only tricky bit in this function is that it supports Windows by
+    creating exe files using a pkg_resources helper.
+    """
+    generated = []
+    script_name = full_name
     if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script+'.exe'
+        script_name += '-script.py'
+        # Generate exe file and give the script a magic name.
+        exe = full_name + '.exe'
         new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
         if not os.path.exists(exe) or (open(exe, 'rb').read() != new_data):
             # Only write it if it's different.
             open(exe, 'wb').write(new_data)
         generated.append(exe)
-
+    changed = not (os.path.exists(script_name) and
+                   open(script_name).read() == contents)
     if changed:
-        open(dest, 'w').write(contents)
-        logger.info("Generated script %r.", script)
-
+        open(script_name, 'w').write(contents)
         try:
-            os.chmod(dest, 0755)
+            os.chmod(script_name, 0755)
         except (AttributeError, os.error):
             pass
-
-    generated.append(dest)
+        logger.info("Generated %s %r.", logged_type, full_name)
+    generated.append(script_name)
     return generated
 
+def _format_paths(paths, indent_level=1):
+    """Format paths for inclusion in a script."""
+    separator = ',\n' + indent_level * '    '
+    return separator.join(paths)
+
+def _script(dest, executable, relative_paths_setup, path, initialization,
+            module_name, attrs, arguments, block_site=False):
+    if block_site:
+        dash_S = ' -S'
+    else:
+        dash_S = ''
+    contents = script_template % dict(
+        python=_safe_arg(executable),
+        dash_S=dash_S,
+        path=path,
+        module_name=module_name,
+        attrs=attrs,
+        arguments=arguments,
+        initialization=initialization,
+        relative_paths_setup=relative_paths_setup,
+        )
+    return _write_script(dest, contents, 'script')
+
 if is_jython and jython_os_name == 'linux':
-    script_header = '#!/usr/bin/env %(python)s'
+    script_header = '#!/usr/bin/env %(python)s%(dash_S)s'
 else:
-    script_header = '#!%(python)s'
+    script_header = '#!%(python)s%(dash_S)s'
 
+sys_path_template = '''\
+import sys
+sys.path[0:0] = [
+    %s,
+    ]
+'''
 
 script_template = script_header + '''\
 
 %(relative_paths_setup)s
 import sys
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
+
 %(initialization)s
 import %(module_name)s
 
@@ -1083,38 +1228,16 @@ if __name__ == '__main__':
     %(module_name)s.%(attrs)s(%(arguments)s)
 '''
 
+# These are used only by the older ``scripts`` function.
 
 def _pyscript(path, dest, executable, rsetup):
-    generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
     contents = py_script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        relative_paths_setup = rsetup,
+        python=_safe_arg(executable),
+        dash_S='',
+        path=path,
+        relative_paths_setup=rsetup,
         )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
-
-    if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script + '.exe'
-        open(exe, 'wb').write(
-            pkg_resources.resource_string('setuptools', 'cli.exe')
-            )
-        generated.append(exe)
-
-    if changed:
-        open(dest, 'w').write(contents)
-        try:
-            os.chmod(dest,0755)
-        except (AttributeError, os.error):
-            pass
-        logger.info("Generated interpreter %r.", script)
-
-    generated.append(dest)
-    return generated
+    return _write_script(dest, contents, 'interpreter')
 
 py_script_template = script_header + '''\
 
@@ -1122,8 +1245,8 @@ py_script_template = script_header + '''\
 import sys
 
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
 
 _interactive = True
 if len(sys.argv) > 1:
@@ -1151,6 +1274,313 @@ if _interactive:
     __import__("code").interact(banner="", local=globals())
 '''
 
+# These are used only by the newer ``generate_scripts`` function.
+
+def _get_system_paths(executable):
+    """return lists of standard lib and site paths for executable.
+    """
+    # We want to get a list of the site packages, which is not easy.
+    # The canonical way to do this is to use
+    # distutils.sysconfig.get_python_lib(), but that only returns a
+    # single path, which does not reflect reality for many system
+    # Pythons, which have multiple additions.  Instead, we start Python
+    # with -S, which does not import site.py and set up the extra paths
+    # like site-packages or (Ubuntu/Debian) dist-packages and
+    # python-support. We then compare that sys.path with the normal one
+    # (minus user packages if this is Python 2.6, because we don't
+    # support those (yet?).  The set of the normal one minus the set of
+    # the ones in ``python -S`` is the set of packages that are
+    # effectively site-packages.
+    #
+    # The given executable might not be the current executable, so it is
+    # appropriate to do another subprocess to figure out what the
+    # additional site-package paths are. Moreover, even if this
+    # executable *is* the current executable, this code might be run in
+    # the context of code that has manipulated the sys.path--for
+    # instance, to add local zc.buildout or setuptools eggs.
+    def get_sys_path(*args, **kwargs):
+        cmd = [executable]
+        cmd.extend(args)
+        cmd.extend([
+            "-c", "import sys, os;"
+            "print repr([os.path.normpath(p) for p in sys.path if p])"])
+        # Windows needs some (as yet to be determined) part of the real env.
+        env = os.environ.copy()
+        env.update(kwargs)
+        _proc = subprocess.Popen(
+            cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+        stdout, stderr = _proc.communicate();
+        if _proc.returncode:
+            raise RuntimeError(
+                'error trying to get system packages:\n%s' % (stderr,))
+        res = eval(stdout.strip())
+        try:
+            res.remove('.')
+        except ValueError:
+            pass
+        return res
+    stdlib = get_sys_path('-S') # stdlib only
+    no_user_paths = get_sys_path(PYTHONNOUSERSITE='x')
+    site_paths = [p for p in no_user_paths if p not in stdlib]
+    return (stdlib, site_paths)
+
+def _get_module_file(executable, name):
+    """Return a module's file path.
+
+    - executable is a path to the desired Python executable.
+    - name is the name of the (pure, not C) Python module.
+    """
+    cmd = [executable, "-c",
+           "import imp; "
+           "fp, path, desc = imp.find_module(%r); "
+           "fp.close; "
+           "print path" % (name,)]
+    _proc = subprocess.Popen(
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    stdout, stderr = _proc.communicate();
+    if _proc.returncode:
+        logger.info(
+            'Could not find file for module %s:\n%s', name, stderr)
+        return None
+    # else: ...
+    res = stdout.strip()
+    if res.endswith('.pyc') or res.endswith('.pyo'):
+        raise RuntimeError('Cannot find uncompiled version of %s' % (name,))
+    if not os.path.exists(res):
+        raise RuntimeError(
+            'File does not exist for module %s:\n%s' % (name, res))
+    return res
+
+def _generate_sitecustomize(dest, executable, initialization='',
+                            exec_sitecustomize=False):
+    """Write a sitecustomize file with optional custom initialization.
+
+    The created script will execute the underlying Python's
+    sitecustomize if exec_sitecustomize is True.
+    """
+    sitecustomize_path = os.path.join(dest, 'sitecustomize.py')
+    sitecustomize = open(sitecustomize_path, 'w')
+    if initialization:
+        sitecustomize.write(initialization + '\n')
+    if exec_sitecustomize:
+        real_sitecustomize_path = _get_module_file(
+            executable, 'sitecustomize')
+        if real_sitecustomize_path:
+            real_sitecustomize = open(real_sitecustomize_path, 'r')
+            sitecustomize.write(
+                '\n# The following is from\n# %s\n' %
+                (real_sitecustomize_path,))
+            sitecustomize.write(real_sitecustomize.read())
+            real_sitecustomize.close()
+    sitecustomize.close()
+    return sitecustomize_path
+
+def _generate_site(dest, working_set, executable, extra_paths=(),
+                   add_site_packages=False, relative_paths=False):
+    """Write a site.py file with eggs from working_set.
+
+    extra_paths will be added to the path.  If add_site_packages is True,
+    paths from the underlying Python will be added.
+    """
+    path = _get_path(working_set, extra_paths)
+    site_path = os.path.join(dest, 'site.py')
+    path_string, rpsetup = _relative_path_and_setup(
+        site_path, path, relative_paths, indent_level=2)
+    if rpsetup:
+        rpsetup = '\n'.join(
+            [(line and '    %s' % (line,) or line)
+             for line in rpsetup.split('\n')])
+    real_site_path = _get_module_file(executable, 'site')
+    real_site = open(real_site_path, 'r')
+    site = open(site_path, 'w')
+    extra_path_snippet = add_site_packages_snippet[add_site_packages]
+    extra_path_snippet_followup = add_site_packages_snippet_followup[
+        add_site_packages]
+    if add_site_packages:
+        stdlib, site_paths = _get_system_paths(executable)
+        extra_path_snippet = extra_path_snippet % _format_paths(
+            (repr(p) for p in site_paths), 2)
+    addsitepackages_marker = 'def addsitepackages('
+    enableusersite_marker = 'ENABLE_USER_SITE = '
+    successful_rewrite = False
+    for line in real_site.readlines():
+        if line.startswith(enableusersite_marker):
+            site.write(enableusersite_marker)
+            site.write('False # buildout does not support user sites.\n')
+        elif line.startswith(addsitepackages_marker):
+            site.write(addsitepackages_script % (
+                extra_path_snippet, rpsetup, path_string,
+                extra_path_snippet_followup))
+            site.write(line[len(addsitepackages_marker):])
+            successful_rewrite = True
+        else:
+            site.write(line)
+    if not successful_rewrite:
+        raise RuntimeError('Buildout did not successfully rewrite site.py')
+    return site_path
+
+add_site_packages_snippet = ['''
+    paths = []''', '''
+    paths = [ # These are the underlying Python's site-packages.
+        %s]
+    sys.path[0:0] = paths
+    known_paths.update([os.path.normcase(os.path.abspath(p)) for p in paths])
+    try:
+        import pkg_resources
+    except ImportError:
+        # No namespace packages in sys.path; no fixup needed.
+        pkg_resources = None''']
+
+add_site_packages_snippet_followup = ['', '''
+    if pkg_resources is not None:
+        # There may be namespace packages in sys.path.  This is much faster
+        # than importing pkg_resources after the sys.path has a large number
+        # of eggs.
+        for p in sys.path:
+            pkg_resources.fixup_namespace_packages(p)''']
+
+addsitepackages_script = '''\
+def addsitepackages(known_paths):%s
+%s    paths[0:0] = [ # eggs
+        %s
+        ]
+    # Process all dirs.  Look for .pth files.  If they exist, defer
+    # processing "import" varieties.
+    dotpth = os.extsep + "pth"
+    deferred = []
+    for path in reversed(paths):
+        # Duplicating addsitedir.
+        sitedir, sitedircase = makepath(path)
+        if not sitedircase in known_paths and os.path.exists(sitedir):
+            sys.path.insert(0, sitedir)
+            known_paths.add(sitedircase)
+        try:
+            names = os.listdir(sitedir)
+        except os.error:
+            continue
+        names = [name for name in names if name.endswith(dotpth)]
+        names.sort()
+        for name in names:
+            # Duplicating addpackage.
+            fullname = os.path.join(sitedir, name)
+            try:
+                f = open(fullname, "rU")
+            except IOError:
+                continue
+            try:
+                for line in f:
+                    if line.startswith("#"):
+                        continue
+                    if (line.startswith("import ") or
+                        line.startswith("import\t")):
+                        # This line is supposed to be executed.  It
+                        # might be a setuptools namespace package
+                        # installed with a system package manager.
+                        # Defer this so we can process egg namespace
+                        # packages first, or else the eggs with the same
+                        # namespace will be ignored.
+                        deferred.append((sitedir, name, fullname, line))
+                        continue
+                    line = line.rstrip()
+                    dir, dircase = makepath(sitedir, line)
+                    if not dircase in known_paths and os.path.exists(dir):
+                        sys.path.append(dir)
+                        known_paths.add(dircase)
+            finally:
+                f.close()%s
+    # Process "import ..." .pth lines.
+    for sitedir, name, fullname, line in deferred:
+        # Note that some lines--such as the one setuptools writes for
+        # namespace packages--expect some or all of sitedir, name, and
+        # fullname to be present in the frame locals, as it is in
+        # ``addpackage``.
+        try:
+            exec line
+        except:
+            print "Error in %%s" %% (fullname,)
+            raise
+    global addsitepackages
+    addsitepackages = original_addsitepackages
+    return known_paths
+
+buildout_addsitepackages = addsitepackages
+
+def original_addsitepackages('''
+
+def _generate_interpreter(name, dest, executable, site_py_dest,
+                          relative_paths=False):
+    """Write an interpreter script, using the site.py approach."""
+    full_name = os.path.join(dest, name)
+    site_py_dest_string, rpsetup = _relative_path_and_setup(
+        full_name, [site_py_dest], relative_paths)
+    if sys.platform == 'win32':
+        windows_import = '\nimport subprocess'
+        # os.exec* is a mess on Windows, particularly if the path
+        # to the executable has spaces and the Python is using MSVCRT.
+        # The standard fix is to surround the executable's path with quotes,
+        # but that has been unreliable in testing.
+        #
+        # Here's a demonstration of the problem.  Given a Python
+        # compiled with a MSVCRT-based compiler, such as the free Visual
+        # C++ 2008 Express Edition, and an executable path with spaces
+        # in it such as the below, we see the following.
+        #
+        # >>> import os
+        # >>> p0 = 'C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe'
+        # >>> os.path.exists(p0)
+        # True
+        # >>> os.execv(p0, [])
+        # Traceback (most recent call last):
+        #  File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # That seems like a standard problem.  The standard solution is
+        # to quote the path (see, for instance
+        # http://bugs.python.org/issue436259).  However, this solution,
+        # and other variations, fail:
+        #
+        # >>> p1 = '"C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe"'
+        # >>> os.execv(p1, [])
+        # Traceback (most recent call last):
+        #   File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # We simply use subprocess instead, since it handles everything
+        # nicely, and the transparency of exec* (that is, not running,
+        # perhaps unexpectedly, in a subprocess) is arguably not a
+        # necessity, at least for many use cases.
+        execute = 'subprocess.call(argv, env=environ)'
+    else:
+        windows_import = ''
+        execute = 'os.execve(sys.executable, argv, environ)'
+    contents = interpreter_template % dict(
+        python=_safe_arg(executable),
+        dash_S=' -S',
+        site_dest=site_py_dest_string,
+        relative_paths_setup=rpsetup,
+        windows_import=windows_import,
+        execute=execute,
+        )
+    return _write_script(full_name, contents, 'interpreter')
+
+interpreter_template = script_header + '''\
+
+%(relative_paths_setup)s
+import os
+import sys%(windows_import)s
+
+argv = [sys.executable] + sys.argv[1:]
+environ = os.environ.copy()
+path = %(site_dest)s
+if environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, environ['PYTHONPATH']])
+environ['PYTHONPATH'] = path
+%(execute)s
+'''
+
+# End of script generation code.
+############################################################################
+
 runsetup_template = """
 import sys
 sys.path.insert(0, %(setupdir)r)

src/zc/buildout/easy_install.txt

@@ -521,25 +521,38 @@ To return the dependency_links behavior to normal call the function again.
 Script generation
 -----------------
 
-The easy_install module provides support for creating scripts from
-eggs.  It provides a function similar to setuptools except that it
-provides facilities for baking a script's path into the script.  This
-has two advantages:
+The easy_install module provides support for creating scripts from eggs.
+It provides two competing functions.  One, ``scripts``, is a
+well-established approach to generating reliable scripts with a "clean"
+Python--e.g., one that does not have any packages in its site-packages.
+The other, ``generate_scripts``, is newer, a bit trickier, and is
+designed to work with a Python that has code in its site-packages, such
+as a system Python.
+
+Both are similar to setuptools except that they provides facilities for
+baking a script's path into the script.  This has two advantages:
 
 - The eggs to be used by a script are not chosen at run time, making
   startup faster and, more importantly, deterministic.
 
-- The script doesn't have to import pkg_resources because the logic
-  that pkg_resources would execute at run time is executed at
-  script-creation time.
+- The script doesn't have to import pkg_resources because the logic that
+  pkg_resources would execute at run time is executed at script-creation
+  time.  (There is an exception in ``generate_scripts`` if you want to
+  have your Python's site packages available, as discussed below, but
+  even in that case pkg_resources is only partially activated, which can
+  be a significant time savings.)
+
 
-The scripts method can be used to generate scripts. Let's create a
-destination directory for it to place them in:
+The ``scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``scripts`` function is the first way to generate scripts that we'll
+examine. It is the earlier approach that the package offered.  Let's
+create a destination directory for it to place them in:
 
-    >>> import tempfile
     >>> bin = tmpdir('bin')
 
-Now, we'll use the scripts method to generate scripts in this directory
+Now, we'll use the scripts function to generate scripts in this directory
 from the demo egg:
 
     >>> import sys
@@ -736,8 +749,8 @@ original script names to new script names.
     >>> print system(os.path.join(bin, 'run')),
     3 1
 
-Including extra paths in scripts
---------------------------------
+The ``scripts`` function: Including extra paths in scripts
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 We can pass a keyword argument, extra paths, to cause additional paths
 to be included in the a generated script:
@@ -762,8 +775,8 @@ to be included in the a generated script:
     if __name__ == '__main__':
         eggrecipedemo.main()
 
-Providing script arguments
---------------------------
+The ``scripts`` function: Providing script arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 An "argument" keyword argument can be used to pass arguments to an
 entry point.  The value passed is a source string to be placed between the
@@ -786,8 +799,8 @@ parentheses in the call:
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Passing initialization code
----------------------------
+The ``scripts`` function: Passing initialization code
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 You can also pass script initialization code:
 
@@ -812,8 +825,8 @@ You can also pass script initialization code:
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Relative paths
---------------
+The ``scripts`` function: Relative paths
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes, you want to be able to move a buildout directory around and
 have scripts still work without having to rebuild them.  We can
@@ -836,7 +849,7 @@ to pass a common base directory of the scripts and eggs:
     ...    interpreter='py',
     ...    relative_paths=bo)
 
-    >>> cat(bo, 'bin', 'run')
+    >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -868,7 +881,7 @@ Of course, running the script works:
 
 We specified an interpreter and its paths are adjusted too:
 
-    >>> cat(bo, 'bin', 'py')
+    >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -911,6 +924,555 @@ We specified an interpreter and its paths are adjusted too:
         del _interactive
         __import__("code").interact(banner="", local=globals())
 
+The ``generate_scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The newer function for creating scripts is ``generate_scripts``.  It has the
+same basic functionality as the ``scripts`` function: it can create scripts
+to run arbitrary entry points, and to run a Python interpreter.  The
+following are the differences from a user's perspective.
+
+- It can be used safely with a Python that has packages installed itself,
+  such as a system-installed Python.
+
+- In contrast to the interpreter generated by the ``scripts`` method, which
+  supports only a small subset of the usual Python executable's options,
+  the interpreter generated by ``generate_scripts`` supports all of them.
+  This makes it possible to use as full Python replacement for scripts that
+  need the distributions specified in your buildout.
+
+- Both the interpreter and the entry point scripts allow you to include the
+  site packages, and/or the sitecustomize, of the Python executable, if
+  desired.
+
+It works by creating site.py and sitecustomize.py files that set up the
+desired paths and initialization.  These must be placed within an otherwise
+empty directory.  Typically this is in a recipe's parts directory.
+
+Here's the simplest example, building an interpreter script.
+
+    >>> interpreter_dir = tmpdir('interpreter')
+    >>> interpreter_parts_dir = os.path.join(
+    ...     interpreter_dir, 'parts', 'interpreter')
+    >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
+    >>> mkdir(interpreter_bin_dir)
+    >>> mkdir(interpreter_dir, 'eggs')
+    >>> mkdir(interpreter_dir, 'parts')
+    >>> mkdir(interpreter_parts_dir)
+
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py')
+
+Depending on whether the machine being used is running Windows or not, this
+produces either three or four files.  In both cases, we have site.py and
+sitecustomize.py generated in the parts/interpreter directory.  For Windows,
+we have py.exe and py-script.py; for other operating systems, we have py.
+
+    >>> sitecustomize_path = os.path.join(
+    ...     interpreter_parts_dir, 'sitecustomize.py')
+    >>> site_path = os.path.join(interpreter_parts_dir, 'site.py')
+    >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')
+    >>> if sys.platform == 'win32':
+    ...     py_path = os.path.join(interpreter_bin_dir, 'py-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'py.exe'),
+    ...                 py_path]
+    ... else:
+    ...     py_path = interpreter_path
+    ...     expected = [sitecustomize_path, site_path, py_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+We didn't ask for any initialization, and we didn't ask to use the underlying
+sitecustomization, so sitecustomize.py is empty.
+
+    >>> cat(sitecustomize_path)
+
+The interpreter script is simple.  It puts the directory with the
+site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.
+
+    >>> cat(py_path)
+    #!/usr/bin/python2.4 -S
+    <BLANKLINE>
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/interpreter/parts/interpreter'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The site.py file is a modified version of the underlying Python's site.py.
+The most important modification is that it has a different version of the
+addsitepackages function.  It has all of the trickier bits, and sets up the
+Python path, similarly to the behavior of the function it replaces.  The
+following shows the part that buildout inserts.
+
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        paths = []
+        paths[0:0] = [ # eggs
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        # Process all dirs.  Look for .pth files.  If they exist, defer
+        # processing "import" varieties.
+        dotpth = os.extsep + "pth"
+        deferred = []
+        for path in reversed(paths):
+            # Duplicating addsitedir.
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.insert(0, sitedir)
+                known_paths.add(sitedircase)
+            try:
+                names = os.listdir(sitedir)
+            except os.error:
+                continue
+            names = [name for name in names if name.endswith(dotpth)]
+            names.sort()
+            for name in names:
+                # Duplicating addpackage.
+                fullname = os.path.join(sitedir, name)
+                try:
+                    f = open(fullname, "rU")
+                except IOError:
+                    continue
+                try:
+                    for line in f:
+                        if line.startswith("#"):
+                            continue
+                        if (line.startswith("import ") or
+                            line.startswith("import ")):
+                            # This line is supposed to be executed.  It
+                            # might be a setuptools namespace package
+                            # installed with a system package manager.
+                            # Defer this so we can process egg namespace
+                            # packages first, or else the eggs with the same
+                            # namespace will be ignored.
+                            deferred.append((sitedir, name, fullname, line))
+                            continue
+                        line = line.rstrip()
+                        dir, dircase = makepath(sitedir, line)
+                        if not dircase in known_paths and os.path.exists(dir):
+                            sys.path.append(dir)
+                            known_paths.add(dircase)
+                finally:
+                    f.close()
+        # Process "import ..." .pth lines.
+        for sitedir, name, fullname, line in deferred:
+            # Note that some lines--such as the one setuptools writes for
+            # namespace packages--expect some or all of sitedir, name, and
+            # fullname to be present in the frame locals, as it is in
+            # ``addpackage``.
+            try:
+                exec line
+            except:
+                print "Error in %s" % (fullname,)
+                raise
+        global addsitepackages
+        addsitepackages = original_addsitepackages
+        return known_paths
+    <BLANKLINE>
+    buildout_addsitepackages = addsitepackages
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+As you can see, it manipulates the path to insert the eggs and then processes
+any .pth files.  The lines in the .pth files that use the "import" feature
+are deferred because it is a pattern we will need in a later example, when we
+show how we can add site packages, and handle competing namespace packages
+in both site packages and eggs.
+
+Here are some examples of the interpreter in use.
+
+    >>> print call_py(interpreter_path, "print 16+26")
+    42
+    <BLANKLINE>
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     '/interpreter/parts/interpreter',
+     ...]
+    <BLANKLINE>
+    >>> clean_paths = eval(res.strip()) # This is used later for comparison.
+
+If you provide initialization, it goes in sitecustomize.py.
+
+    >>> def reset_interpreter():
+    ...     # This is necessary because, in our tests, the timestamps of the
+    ...     # .pyc files are not outdated when we want them to be.
+    ...     rmdir(interpreter_bin_dir)
+    ...     mkdir(interpreter_bin_dir)
+    ...     rmdir(interpreter_parts_dir)
+    ...     mkdir(interpreter_parts_dir)
+    ...
+    >>> reset_interpreter()
+
+    >>> initialization_string = """\
+    ... import os
+    ... os.environ['FOO'] = 'bar baz bing shazam'"""
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', initialization=initialization_string)
+    >>> cat(sitecustomize_path)
+    import os
+    os.environ['FOO'] = 'bar baz bing shazam'
+    >>> print call_py(interpreter_path, "import os; print os.environ['FOO']")
+    bar baz bing shazam
+    <BLANKLINE>
+
+If you use relative paths, this affects the interpreter and site.py.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', relative_paths=interpreter_dir)
+    >>> cat(py_path)
+    #!/usr/bin/python2.4 -S
+    <BLANKLINE>
+    import os
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = join(base, 'parts/interpreter')
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+For site.py, we again show only the pertinent parts.  Notice that the egg
+paths join a base to a path, as with the use of this argument in the
+``scripts`` function.
+
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        paths = []
+    <BLANKLINE>
+        import os
+    <BLANKLINE>
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        paths[0:0] = [ # eggs
+            join(base, 'eggs/demo-0.3-pyN.N.egg'),
+            join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
+            ]...
+
+The paths resolve in practice as you would expect.
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/eggs/demo-0.3-py2.4.egg',
+     '/interpreter/eggs/demoneeded-1.1-py2.4.egg',
+     '/interpreter/parts/interpreter',
+     ...]
+    <BLANKLINE>
+
+The ``extra_paths`` argument affects the path in site.py.  Notice that
+/interpreter/other is added after the eggs.
+
+    >>> reset_interpreter()
+    >>> mkdir(interpreter_dir, 'other')
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', extra_paths=[join(interpreter_dir, 'other')])
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        paths = []
+        paths[0:0] = [ # eggs
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+            '/interpreter/other'
+            ]...
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     '/interpreter/other',
+     '/interpreter/parts/interpreter',
+     ...]
+    <BLANKLINE>
+
+The ``generate_scripts`` function: using site-packages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``generate_scripts`` function supports including site packages.  This has
+some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildbot.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+That explained, let's see how it works.  Unfortunately, because of how
+setuptools namespace packages are implemented differently for operating
+system packages (debs or rpms) and normal installation, there's a tricky
+dance.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', add_site_packages=True)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        paths = [ # These are the underlying Python's site-packages.
+            '...']
+        sys.path[0:0] = paths
+        known_paths.update([os.path.normcase(os.path.abspath(p)) for p in paths])
+        try:
+            import pkg_resources
+        except ImportError:
+            # No namespace packages in sys.path; no fixup needed.
+            pkg_resources = None
+        paths[0:0] = [ # eggs
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        # Process all dirs.  Look for .pth files.  If they exist, defer
+        # processing "import" varieties.
+        dotpth = os.extsep + "pth"
+        deferred = []
+        for path in reversed(paths):
+            # Duplicating addsitedir.
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.insert(0, sitedir)
+                known_paths.add(sitedircase)
+            try:
+                names = os.listdir(sitedir)
+            except os.error:
+                continue
+            names = [name for name in names if name.endswith(dotpth)]
+            names.sort()
+            for name in names:
+                # Duplicating addpackage.
+                fullname = os.path.join(sitedir, name)
+                try:
+                    f = open(fullname, "rU")
+                except IOError:
+                    continue
+                try:
+                    for line in f:
+                        if line.startswith("#"):
+                            continue
+                        if (line.startswith("import ") or
+                            line.startswith("import	")):
+                            # This line is supposed to be executed.  It
+                            # might be a setuptools namespace package
+                            # installed with a system package manager.
+                            # Defer this so we can process egg namespace
+                            # packages first, or else the eggs with the same
+                            # namespace will be ignored.
+                            deferred.append((sitedir, name, fullname, line))
+                            continue
+                        line = line.rstrip()
+                        dir, dircase = makepath(sitedir, line)
+                        if not dircase in known_paths and os.path.exists(dir):
+                            sys.path.append(dir)
+                            known_paths.add(dircase)
+                finally:
+                    f.close()
+        if pkg_resources is not None:
+            # There may be namespace packages in sys.path.  This is much faster
+            # than importing pkg_resources after the sys.path has a large number
+            # of eggs.
+            for p in sys.path:
+                pkg_resources.fixup_namespace_packages(p)
+        # Process "import ..." .pth lines.
+        for sitedir, name, fullname, line in deferred:
+            # Note that some lines--such as the one setuptools writes for
+            # namespace packages--expect some or all of sitedir, name, and
+            # fullname to be present in the frame locals, as it is in
+            # ``addpackage``.
+            try:
+                exec line
+            except:
+                print "Error in %s" % (fullname,)
+                raise
+        global addsitepackages
+        addsitepackages = original_addsitepackages
+        return known_paths
+    <BLANKLINE>
+    buildout_addsitepackages = addsitepackages
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+As you can see, the script now first tries to import pkg_resources.  If it
+exists, then we need to process egg files specially to look for namespace
+packages there *before* we process process lines in .pth files that use the
+"import" feature--lines that might be part of the setuptools namespace
+package implementation for system packages, as mentioned above, and that
+must come after processing egg namespaces.
+
+Here's an example of the new script in use.  Other documents and tests in
+this package give the feature a more thorough workout, but this should
+give you an idea of the feature.
+
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/eggs/demo-0.3-py2.4.egg',
+     '/interpreter/eggs/demoneeded-1.1-py2.4.egg',
+     '...',
+     '/interpreter/parts/interpreter',
+     ...]
+    <BLANKLINE>
+
+The clean_paths gathered earlier is a subset of this full list of paths.
+
+    >>> full_paths = eval(res.strip())
+    >>> len(clean_paths) < len(full_paths)
+    True
+    >>> set(os.path.normpath(p) for p in clean_paths).issubset(
+    ...     os.path.normpath(p) for p in full_paths)
+    True
+
+The ``exec_sitecustomize`` argument does the same thing for the
+sitecustomize module--it allows you to include the code from the
+sitecustomize module in the underlying Python if you set the argument to
+True.  The z3c.recipe.scripts package sets up the full environment necessary
+to demonstrate this piece.
+
+The ``generate_scripts`` function: writing scripts for entry points
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All of the examples so far for this function have been creating
+interpreters.  The function can also write scripts for entry
+points.  They are almost identical to the scripts that we saw for the
+``scripts`` function except that they ``import site`` after setting the
+sys.path to include our custom site.py and sitecustomize.py files.  These
+files then initialize the Python environment as we have already seen.  Let's
+see a simple example.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'])
+
+As before, in Windows, 2 files are generated for each script.  A script
+file, ending in '-script.py', and an exe file that allows the script
+to be invoked directly without having to specify the Python
+interpreter and without having to provide a '.py' suffix.  This is in addition
+to the site.py and sitecustomize.py files that are generated as with our
+interpreter examples above.
+
+    >>> if sys.platform == 'win32':
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'demo.exe'),
+    ...                 demo_path]
+    ... else:
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo')
+    ...     expected = [sitecustomize_path, site_path, demo_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+The demo script runs the entry point defined in the demo egg:
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        '/interpreter/parts/interpreter',
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import site # imports custom buildbot-generated site.py
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main()
+
+    >>> demo_call = join(interpreter_bin_dir, 'demo')
+    >>> if sys.platform == 'win32':
+    ...     demo_call = '"%s"' % demo_call
+    >>> print system(demo_call)
+    3 1
+    <BLANKLINE>
+
+There are a few differences from the ``scripts`` function.  First, the
+``reqs`` argument (an iterable of string requirements or entry point
+tuples) is a keyword argument here.  We see that in the example above.
+Second, the ``arguments`` argument is now named ``script_arguments`` to
+try and clarify that it does not affect interpreters. While the
+``initialization`` argument continues to affect both the interpreters
+and the entry point scripts, if you have initialization that is only
+pertinent to the entry point scripts, you can use the
+``script_initialization`` argument.
+
+Let's see ``script_arguments`` and ``script_initialization`` in action.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.generate_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], script_arguments='1, 2',
+    ...    script_initialization='import os\nos.chdir("foo")')
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    import sys
+    sys.path[0:0] = [
+      '/interpreter/parts/interpreter',
+      ]
+    <BLANKLINE>
+    import site # imports custom buildbot-generated site.py
+    import os
+    os.chdir("foo")
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main(1, 2)
 
 Handling custom build options for extensions provided in source distributions
 -----------------------------------------------------------------------------

src/zc/buildout/testing.py

@@ -28,6 +28,7 @@ import socket
 import subprocess
 import sys
 import tempfile
+import textwrap
 import threading
 import time
 import urllib2
@@ -105,6 +106,16 @@ def system(command, input=''):
     e.close()
     return result
 
+def call_py(interpreter, cmd, flags=None):
+    if sys.platform == 'win32':
+        args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
+        args.insert(-1, '"-c"')
+        return system('"%s"' % ' '.join(args))
+    else:
+        cmd = repr(cmd)
+        return system(
+            ' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
+
 def get(url):
     return urllib2.urlopen(url).read()
 
@@ -116,7 +127,11 @@ def _runsetup(setup, executable, *args):
     args = [zc.buildout.easy_install._safe_arg(arg)
             for arg in args]
     args.insert(0, '-q')
-    args.append(dict(os.environ, PYTHONPATH=setuptools_location))
+    env = dict(os.environ)
+    if executable == sys.executable:
+        env['PYTHONPATH'] = setuptools_location
+    # else pass an executable that has setuptools! See testselectingpython.py.
+    args.append(env)
 
     here = os.getcwd()
     try:
@@ -135,6 +150,11 @@ def sdist(setup, dest):
 def bdist_egg(setup, executable, dest):
     _runsetup(setup, executable, 'bdist_egg', '-d', dest)
 
+def sys_install(setup, dest):
+    _runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
+              '--record', os.path.join(dest, '__added_files__'),
+              '--single-version-externally-managed')
+
 def find_python(version):
     e = os.environ.get('PYTHON%s' % version)
     if e is not None:
@@ -202,6 +222,24 @@ def wait_until(label, func, *args, **kw):
         time.sleep(0.01)
     raise ValueError('Timed out waiting for: '+label)
 
+def make_buildout():
+    # Create a basic buildout.cfg to avoid a warning from buildout:
+    open('buildout.cfg', 'w').write(
+        "[buildout]\nparts =\n"
+        )
+    # Use the buildout bootstrap command to create a buildout
+    zc.buildout.buildout.Buildout(
+        'buildout.cfg',
+        [('buildout', 'log-level', 'WARNING'),
+         # trick bootstrap into putting the buildout develop egg
+         # in the eggs dir.
+         ('buildout', 'develop-eggs-directory', 'eggs'),
+         ]
+        ).bootstrap([])
+    # Create the develop-eggs dir, which didn't get created the usual
+    # way due to the trick above:
+    os.mkdir('develop-eggs')
+
 def buildoutSetUp(test):
 
     test.globs['__tear_downs'] = __tear_downs = []
@@ -255,27 +293,7 @@ def buildoutSetUp(test):
     sample = tmpdir('sample-buildout')
 
     os.chdir(sample)
-
-    # Create a basic buildout.cfg to avoid a warning from buildout:
-    open('buildout.cfg', 'w').write(
-        "[buildout]\nparts =\n"
-        )
-
-    # Use the buildout bootstrap command to create a buildout
-    zc.buildout.buildout.Buildout(
-        'buildout.cfg',
-        [('buildout', 'log-level', 'WARNING'),
-         # trick bootstrap into putting the buildout develop egg
-         # in the eggs dir.
-         ('buildout', 'develop-eggs-directory', 'eggs'),
-         ]
-        ).bootstrap([])
-
-
-
-    # Create the develop-eggs dir, which didn't get created the usual
-    # way due to the trick above:
-    os.mkdir('develop-eggs')
+    make_buildout()
 
     def start_server(path):
         port, thread = _start_server(path, name=path)
@@ -283,6 +301,40 @@ def buildoutSetUp(test):
         register_teardown(lambda: stop_server(url, thread))
         return url
 
+    def make_py(initialization=''):
+        """Returns paths to new executable and to its site-packages.
+        """
+        buildout = tmpdir('executable_buildout')
+        site_packages_dir = os.path.join(buildout, 'site-packages')
+        mkdir(site_packages_dir)
+        old_wd = os.getcwd()
+        os.chdir(buildout)
+        make_buildout()
+        initialization = '\n'.join(
+            '  ' + line for line in initialization.split('\n'))
+        install_develop(
+            'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
+        install_develop(
+            'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
+        write('buildout.cfg', textwrap.dedent('''\
+            [buildout]
+            parts = py
+
+            [py]
+            recipe = z3c.recipe.scripts
+            interpreter = py
+            initialization =
+            %(initialization)s
+            extra-paths = %(site-packages)s
+            eggs = setuptools
+            ''') % {
+                'initialization': initialization,
+                'site-packages': site_packages_dir})
+        system(os.path.join(buildout, 'bin', 'buildout'))
+        os.chdir(old_wd)
+        return (
+            os.path.join(buildout, 'bin', 'py'), site_packages_dir)
+
     test.globs.update(dict(
         sample_buildout = sample,
         ls = ls,
@@ -293,6 +345,7 @@ def buildoutSetUp(test):
         tmpdir = tmpdir,
         write = write,
         system = system,
+        call_py = call_py,
         get = get,
         cd = (lambda *path: os.chdir(os.path.join(*path))),
         join = os.path.join,
@@ -301,6 +354,7 @@ def buildoutSetUp(test):
         start_server = start_server,
         buildout = os.path.join(sample, 'bin', 'buildout'),
         wait_until = wait_until,
+        make_py = make_py
         ))
 
     zc.buildout.easy_install.prefer_final(prefer_final)

src/zc/buildout/tests.py

@@ -53,6 +53,7 @@ We should be able to deal with setup scripts that aren't setuptools based.
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -84,6 +85,7 @@ We should be able to deal with setup scripts that aren't setuptools based.
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     >>> print system(join('bin', 'buildout')+' -vvv'), # doctest: +ELLIPSIS
@@ -668,6 +670,7 @@ Create a develop egg:
 
     >>> ls('develop-eggs')
     -  foox.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Create another:
@@ -692,6 +695,7 @@ Create another:
     >>> ls('develop-eggs')
     -  foox.egg-link
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove one:
@@ -709,6 +713,7 @@ It is gone
 
     >>> ls('develop-eggs')
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove the other:
@@ -723,6 +728,7 @@ Remove the other:
 All gone
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
     '''
 
@@ -797,6 +803,7 @@ On the other hand, if we have a regular egg, rather than a develop egg:
     ...            + join(sample_buildout, 'eggs'))
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     >>> ls('eggs') # doctest: +ELLIPSIS
@@ -1769,6 +1776,233 @@ def bug_105081_Specific_egg_versions_are_ignored_when_newer_eggs_are_around():
     1 2
     """
 
+def versions_section_ignored_for_dependency_in_favor_of_site_packages():
+    r"""
+This is a test for a bugfix.
+
+The error showed itself when at least two dependencies were in a shared
+location like site-packages, and the first one met the "versions" setting.  The
+first dependency would be added, but subsequent dependencies from the same
+location (e.g., site-packages) would use the version of the package found in
+the shared location, ignoring the version setting.
+
+We begin with a Python that has demoneeded version 1.1 installed and a
+demo version 0.3, all in a site-packages-like shared directory.  We need
+to create this.  ``eggrecipedemo.main()`` shows the number after the dot
+(that is, ``X`` in ``1.X``), for the demo package and the demoneeded
+package, so this demonstrates that our Python does in fact have demo
+version 0.3 and demoneeded version 1.1.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__"),
+    1.1
+
+Now here's a setup that would expose the bug, using the
+zc.buildout.easy_install API.
+
+    >>> example_dest = tmpdir('example_dest')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['tellmy.version'], example_dest, links=[sample_eggs],
+    ...     executable=py_path,
+    ...     index=None,
+    ...     versions={'tellmy.version': '1.0'})
+    >>> for dist in workingset:
+    ...     res = str(dist)
+    ...     if res.startswith('tellmy.version'):
+    ...         print res
+    ...         break
+    tellmy.version 1.0
+
+Before the bugfix, the desired tellmy.version distribution would have
+been blocked the one in site-packages.
+"""
+
+def handle_namespace_package_in_both_site_packages_and_buildout_eggs():
+    r"""
+If you have the same namespace package in both site-packages and in
+buildout, we need to be very careful that faux-Python-executables and
+scripts generated by easy_install.generate_scripts correctly combine the two.
+We show this with the local recipe that uses the function, z3c.recipe.scripts.
+
+To demonstrate this, we will create three packages: tellmy.version 1.0,
+tellmy.version 1.1, and tellmy.fortune 1.0.  tellmy.version 1.1 is installed.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.1
+    <BLANKLINE>
+
+Now we will create a buildout that creates a script and a faux-Python script.
+We want to see that both can successfully import the specified versions of
+tellmy.version and tellmy.fortune.
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(link_server)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... python = primed_python
+    ... interpreter = py
+    ... add-site-packages = true
+    ... eggs = tellmy.version == 1.0
+    ...        tellmy.fortune == 1.0
+    ...        demo
+    ... script-initialization =
+    ...     import tellmy.version
+    ...     print tellmy.version.__version__
+    ...     import tellmy.fortune
+    ...     print tellmy.fortune.__version__
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.0'.
+    Got tellmy.version 1.0.
+    Getting distribution for 'tellmy.fortune==1.0'.
+    Got tellmy.fortune 1.0.
+    Getting distribution for 'demo'.
+    Got demo 0.4c1.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+    <BLANKLINE>
+
+Finally, we are ready for the actual test.  Prior to the bug fix that
+this tests, the results of both calls below was the following::
+
+    1.1
+    Traceback (most recent call last):
+      ...
+    ImportError: No module named fortune
+    <BLANKLINE>
+
+In other words, we got the site-packages version of tellmy.version, and
+we could not import tellmy.fortune at all.  The following are the correct
+results for the interpreter and for the script.
+
+    >>> print call_py(
+    ...     join('bin', 'py'),
+    ...     "import tellmy.version; " +
+    ...     "print tellmy.version.__version__; " +
+    ...     "import tellmy.fortune; " +
+    ...     "print tellmy.fortune.__version__") # doctest: +ELLIPSIS
+    1.0
+    1.0...
+
+    >>> print system(join('bin', 'demo'))
+    1.0
+    1.0
+    4 2
+    <BLANKLINE>
+    """
+
+def handle_sys_path_version_hack():
+    r"""
+This is a test for a bugfix.
+
+If you use a Python that has a different version of one of your
+dependencies, and the new package tries to do sys.path tricks in the
+setup.py to get a __version__, and it uses namespace packages, the older
+package will be loaded first, making the setup version the wrong number.
+While very arguably packages simply shouldn't do this, some do, and we
+don't want buildout to fall over when they do.
+
+To demonstrate this, we will need to create a distribution that has one of
+these unpleasant tricks, and a Python that has an older version installed.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> for version in ('1.0', '1.1'):
+    ...     tmp = tempfile.mkdtemp()
+    ...     try:
+    ...         write(tmp, 'README.txt', '')
+    ...         mkdir(tmp, 'src')
+    ...         mkdir(tmp, 'src', 'tellmy')
+    ...         write(tmp, 'src', 'tellmy', '__init__.py',
+    ...             "__import__("
+    ...             "'pkg_resources').declare_namespace(__name__)\n")
+    ...         mkdir(tmp, 'src', 'tellmy', 'version')
+    ...         write(tmp, 'src', 'tellmy', 'version',
+    ...               '__init__.py', '__version__=%r\n' % version)
+    ...         write(
+    ...             tmp, 'setup.py',
+    ...             "from setuptools import setup\n"
+    ...             "import sys\n"
+    ...             "sys.path.insert(0, 'src')\n"
+    ...             "from tellmy.version import __version__\n"
+    ...             "setup(\n"
+    ...             " name='tellmy.version',\n"
+    ...             " package_dir = {'': 'src'},\n"
+    ...             " packages = ['tellmy', 'tellmy.version'],\n"
+    ...             " install_requires = ['setuptools'],\n"
+    ...             " namespace_packages=['tellmy'],\n"
+    ...             " zip_safe=True, version=__version__,\n"
+    ...             " author='bob', url='bob', author_email='bob')\n"
+    ...             )
+    ...         zc.buildout.testing.sdist(tmp, sample_eggs)
+    ...         if version == '1.0':
+    ...             # We install the 1.0 version in site packages the way a
+    ...             # system packaging system (debs, rpms) would do it.
+    ...             zc.buildout.testing.sys_install(tmp, site_packages_path)
+    ...     finally:
+    ...         shutil.rmtree(tmp)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.0
+    <BLANKLINE>
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(sample_eggs)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = zc.recipe.egg:eggs
+    ... python = primed_python
+    ... eggs = tellmy.version == 1.1
+    ... ''' % globals())
+
+Before the bugfix, running this buildout would generate this error:
+
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Installing tellmy.version 1.1
+    Caused installation of a distribution:
+    tellmy.version 1.0
+    with a different version.
+    Got None.
+    While:
+      Installing eggs.
+    Error: There is a version conflict.
+    We already have: tellmy.version 1.0
+    <BLANKLINE>
+
+The bugfix was simply to add Python's "-S" option when calling
+easyinstall (see zc.buildout.easy_install.Installer._call_easy_install).
+Now the install works correctly, as seen here.
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Got tellmy.version 1.1.
+    <BLANKLINE>
+
+    """
+
 if sys.version_info > (2, 4):
     def test_exit_codes():
         """
@@ -2367,6 +2601,7 @@ Distribution setup scripts can import modules in the distribution directory:
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -2654,6 +2889,44 @@ def increment_on_command_line():
 
 ######################################################################
 
+def make_py_with_system_install(make_py, sample_eggs):
+    from zc.buildout.testing import write, mkdir
+    py_path, site_packages_path = make_py()
+    for pkg, version in (('version', '1.0'), ('version', '1.1'),
+                         ('fortune', '1.0')):
+        tmp = tempfile.mkdtemp()
+        try:
+            write(tmp, 'README.txt', '')
+            mkdir(tmp, 'src')
+            mkdir(tmp, 'src', 'tellmy')
+            write(tmp, 'src', 'tellmy', '__init__.py',
+                "__import__("
+                "'pkg_resources').declare_namespace(__name__)\n")
+            mkdir(tmp, 'src', 'tellmy', pkg)
+            write(tmp, 'src', 'tellmy', pkg,
+                  '__init__.py', '__version__=%r\n' % version)
+            write(
+                tmp, 'setup.py',
+                "from setuptools import setup\n"
+                "setup(\n"
+                " name='tellmy.%(pkg)s',\n"
+                " package_dir = {'': 'src'},\n"
+                " packages = ['tellmy', 'tellmy.%(pkg)s'],\n"
+                " install_requires = ['setuptools'],\n"
+                " namespace_packages=['tellmy'],\n"
+                " zip_safe=True, version=%(version)r,\n"
+                " author='bob', url='bob', author_email='bob')\n"
+                % locals()
+                )
+            zc.buildout.testing.sdist(tmp, sample_eggs)
+            if pkg == 'version' and version == '1.1':
+                # We install the 1.1 version in site packages the way a
+                # system packaging system (debs, rpms) would do it.
+                zc.buildout.testing.sys_install(tmp, site_packages_path)
+        finally:
+            shutil.rmtree(tmp)
+    return py_path
+
 def create_sample_eggs(test, executable=sys.executable):
     write = test.globs['write']
     dest = test.globs['sample_eggs']
@@ -2776,6 +3049,7 @@ def easy_install_SetUp(test):
         test.globs['sample_eggs'])
     test.globs['update_extdemo'] = lambda : add_source_dist(test, 1.5)
     zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
 
 egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
                        ).match
@@ -2934,6 +3208,10 @@ def test_suite():
                (re.compile('[-d]  setuptools-\S+[.]egg'), 'setuptools.egg'),
                (re.compile(r'\\[\\]?'), '/'),
                (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
                ]+(sys.version_info < (2, 5) and [
                   (re.compile('.*No module named runpy.*', re.S), ''),
                   (re.compile('.*usage: pdb.py scriptfile .*', re.S), ''),

src/zc/buildout/testselectingpython.py

@@ -11,7 +11,7 @@
 # FOR A PARTICULAR PURPOSE.
 #
 ##############################################################################
-import os, re, sys, unittest
+import os, re, subprocess, sys, textwrap, unittest
 from zope.testing import doctest, renormalizing
 import zc.buildout.tests
 import zc.buildout.testing
@@ -42,6 +42,33 @@ We can specify a specific Python executable.
 
 def multi_python(test):
     other_executable = zc.buildout.testing.find_python(other_version)
+    command = textwrap.dedent('''\
+        try:
+            import setuptools
+        except ImportError:
+            import sys
+            sys.exit(1)
+        ''')
+    if subprocess.call([other_executable, '-c', command],
+                       env=os.environ):
+        # the other executable does not have setuptools.  Get setuptools.
+        # We will do this using the same tools we are testing, for better or
+        # worse.  Alternatively, we could try using bootstrap.
+        executable_dir = test.globs['tmpdir']('executable_dir')
+        executable_parts = os.path.join(executable_dir, 'parts')
+        test.globs['mkdir'](executable_parts)
+        ws = zc.buildout.easy_install.install(
+            ['setuptools'], executable_dir,
+            index='http://www.python.org/pypi/',
+            always_unzip=True, executable=other_executable)
+        zc.buildout.easy_install.generate_scripts(
+            executable_dir, ws, other_executable, executable_parts,
+            reqs=['setuptools'], interpreter='py')
+        original_executable = other_executable
+        other_executable = os.path.join(executable_dir, 'py')
+        assert not subprocess.call(
+            [other_executable, '-c', command], env=os.environ), (
+            'test set up failed')
     sample_eggs = test.globs['tmpdir']('sample_eggs')
     os.mkdir(os.path.join(sample_eggs, 'index'))
     test.globs['sample_eggs'] = sample_eggs

src/zc/buildout/update.txt

@@ -81,6 +81,7 @@ new versions found in new releases:
 Our buildout script has been updated to use the new eggs:
 
     >>> cat(sample_buildout, 'bin', 'buildout')
+    ... # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys

z3c.recipe.scripts_/CHANGES.txt

+Change History
+**************
+
+1.0.0
+=====
+
+Initial public version.

z3c.recipe.scripts_/README.txt

+********************************
+Buildout Script Recipe
+********************************
+
+.. contents::
+
+The script recipe installs eggs into a buildout eggs directory, exactly
+like zc.recipe.egg, and then generates scripts in a buildout bin
+directory with egg paths baked into them.
+

z3c.recipe.scripts_/setup.py

+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for z3c.recipe.scripts package
+
+$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
+"""
+
+version = '1.0'
+
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+    return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
+
+name = "z3c.recipe.scripts"
+setup(
+    name = name,
+    version = version,
+    author = "Gary Poster",
+    author_email = "gary.poster@canonical.com",
+    description = "Recipe for installing Python scripts",
+    long_description = (
+        read('README.txt')
+        + '\n' +
+        read('CHANGES.txt')
+        + '\n' +
+        'Detailed Documentation\n'
+        '**********************\n'
+        + '\n' +
+        read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
+        + '\n' +
+        'Download\n'
+        '*********\n'
+        ),
+    keywords = "development build",
+    classifiers = [
+       'Development Status :: 5 - Production/Stable',
+       'Framework :: Buildout',
+       'Intended Audience :: Developers',
+       'License :: OSI Approved :: Zope Public License',
+       'Topic :: Software Development :: Build Tools',
+       'Topic :: Software Development :: Libraries :: Python Modules',
+       ],
+    url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
+    license = "ZPL 2.1",
+
+    packages = find_packages('src'),
+    package_dir = {'':'src'},
+    namespace_packages = ['z3c', 'z3c.recipe'],
+    install_requires = [
+        'zc.buildout >=1.2.0',
+        'zc.recipe.egg',
+        'setuptools'],
+    tests_require = ['zope.testing'],
+    test_suite = name+'.tests.test_suite',
+    entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
+                                    'script = %s:Scripts' % name,
+                                    'scripts = %s:Scripts' % name,
+                                    'interpreter = %s:Interpreter' % name,
+                                    ]
+                    },
+    include_package_data = True,
+    zip_safe=False,
+    )

z3c.recipe.scripts_/src/z3c/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt

+Script and interpreter generation
+=================================
+
+This recipe is very similar to zc.recipe.egg, and if you are familiar with its
+options, you will be able to use this one easily.
+
+The script and interpreter generation in this recipe are improved from
+those provided by zc.recipe.egg in two basic ways.
+
+- The interpreter generated by the script supports all interpreter
+  options, as opposed to the subset provided by zc.recipe.egg.
+
+- Both scripts and interpreters from this recipe can optionally choose
+  to include site-packages, and even sitecustomize.
+
+The recipe takes several options.  First, here's the list of the options
+that overlap from the standard zc.recipe.eggs scripts recipe.  After
+this, we'll list the new options and describe them.
+
+* eggs
+* find-links
+* index
+* python
+* extra-paths
+* entry-points
+* scripts
+* dependent-scripts
+* interpreter
+* arguments
+* initialization
+* relative-paths
+
+In addition to these, the recipe offers these new options.  They are
+introduced here, and described more in depth below.
+
+add-site-packages
+    You can choose to have the site-packages of the underlying Python
+    available to your script or interpreter, in addition to the packages
+    from your eggs.  See the section on this option for motivations and
+    warnings.
+
+extends
+    You can extend another section using this value.  It is intended to be
+    used by extending a section that uses this package's scripts recipe.
+    In this manner, you can avoid repeating yourself.
+
+exec-sitecustomize
+    Normally the Python's real sitecustomize module is not processed.
+    If you want it to be processed, set this value to 'true'.  This will
+    be honored irrespective of the setting for include-site-paths.
+
+script-initialization
+    The standard initialization code affects both an interpreter and scripts.
+    The code in script-initialization is used only for the generated scripts.
+
+Finally, the "interpreter" entry point ignores ``script-initialization``,
+``scripts``, and ``arguments``, and provides yet another additional option.
+
+name
+    While, by default, the interpreter recipe takes the name of the
+    section to be the desired interpreter name, you can specify the
+    interpreter name here instead.
+
+Script generation
+-----------------
+
+Generating a basic script looks virtually identical to using zc.recipe.egg.
+
+(Note that the find-links and index values are typically not needed; they
+are included to help make this document run as a test successfully.)
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing demo.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    2 2
+
+Interpreter generation
+----------------------
+
+As with zc.recipe.egg, you can generate an interpreter with the default
+script recipe shown above by supplying the "interpreter" option.
+This example will create both an entry point script and an interpreter.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+You can also generate an interpreter alone with the ``interpreter`` recipe.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+In both cases, the bin/py script works by restarting Python after
+specifying a special path in PYTHONPATH.  This example shows the UNIX version;
+the Windows version actually uses subprocess instead.
+
+    >>> cat(sample_buildout, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/bin/python2.4 -S
+    <BLANKLINE>
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/sample-buildout/parts/py'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The path is a directory that contains two files: our own site.py and
+sitecustomize.py.  The site.py is modified from the underlying Python's
+site.py, and is responsible for setting up our paths. The
+sitecustomize.py is responsible for running the initialization code
+provided.
+
+    >>> ls(sample_buildout, 'parts', 'py')
+    -  site.py
+    -  sitecustomize.py
+
+Here's an example of using the generated interpreter.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[:3])"')
+    ['',
+     '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+
+Including site-packages and sitecustomize
+-----------------------------------------
+
+As introduced above, this recipe supports including site packages.  This has
+some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildbot.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+To show off these features, we need to use buildout with a Python
+executable with some extra paths to show ``add-site-packages``; and one
+guaranteed to have a sitecustomize module to show
+``exec-sitecustomize``.  We'll make one using a test fixture called
+``make_py``. The os.environ change below will go into the sitecustomize,
+and the site_packages_path will be in the Python's path.
+
+    >>> py_path, site_packages_path = make_py(initialization='''\
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+    >>> print site_packages_path
+    /executable_buildout/site-packages
+
+Now let's take a look at add-site-packages.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... add-site-packages = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys, pprint; pprint.pprint(sys.path)"''')
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
+     '/executable_buildout/eggs/setuptools-0.6c11-pyN.N.egg',
+     '/executable_buildout/site-packages',
+     '/sample-buildout/parts/py',
+     '/executable_buildout/parts/py',
+     ...]
+    <BLANKLINE>
+
+Next we will use the exec-sitecustomize option.  It simply copies
+Python's underlying sitecustomize module, if it exists, to the local
+version.  The os.environ change shown above in the make_py call will go
+into the sitecustomize.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    <BLANKLINE>
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"''')
+    foo bar baz shazam
+    <BLANKLINE>
+
+Options
+-------
+
+We'll focus now on the options that are different than zc.recipe.egg.
+
+Let's look at the ``extends`` option first.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo python
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ...
+    ... [python]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... extends = demo
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... """ % dict(server=link_server))
+
+That makes it easier to specify some initialization for the interpreter
+that is different than a script, while duplicating other configuration.
+
+Now let's put it in action.
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Installing python.
+    Generated interpreter '/sample-buildout/bin/python'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[:3])"')
+    ['',
+     '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+Note that the parts/py directory has been cleaned up, and parts/python has
+been created.
+
+    >>> ls(sample_buildout, 'parts')
+    d  demo
+    d  python
+
+If you want to have initialization that only affects scripts, not the
+interpreter, you can use script-initialization.  Here's a demonstration.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... script-initialization =
+    ...     print "Hi from the script"
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling python.
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "print 'Hi from the interpreter'"'''),
+    Hi from the interpreter
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    Hi from the script
+    2 2
+
+The last new option is ``name``.  This simply changes the name of the
+interpreter, so that you are not forced to use the name of the section.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = interpreter
+    ...
+    ... [interpreter]
+    ... name = python2
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing interpreter.
+    Generated interpreter '/sample-buildout/bin/python2'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python2') +
+    ...              ' -c "print 42"')
+    42
+    <BLANKLINE>
+
+The other options all identical to the zc.recipe.egg script.  Here are some
+quick demos and discussions.

z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py

+from z3c.recipe.scripts.scripts import Scripts, Interpreter

z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py

+##############################################################################
+#
+# Copyright (c) 2009-2010 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Install scripts from eggs.
+"""
+import os
+import zc.buildout
+import zc.buildout.easy_install
+from zc.recipe.egg.egg import ScriptBase
+
+
+class Base(ScriptBase):
+
+    def __init__(self, buildout, name, options):
+        if 'extends' in options:
+            options.update(buildout[options['extends']])
+        super(Base, self).__init__(buildout, name, options)
+        self.default_eggs = '' # Disables feature from zc.recipe.egg.
+        b_options = buildout['buildout']
+        options['parts-directory'] = os.path.join(
+            b_options['parts-directory'], self.name)
+
+        value = options.setdefault(
+            'add-site-packages',
+            b_options.get('add-site-packages', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for add-site-packages option: %s" %
+                (value,))
+        self.add_site_packages = (value == 'true')
+
+        value = options.setdefault(
+            'exec-sitecustomize',
+            b_options.get('exec-sitecustomize', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for exec-sitecustomize option: %s" %
+                (value,))
+        self.exec_sitecustomize = (value == 'true')
+
+
+class Interpreter(Base):
+
+    def __init__(self, buildout, name, options):
+        super(Interpreter, self).__init__(buildout, name, options)
+
+        options.setdefault('name', name)
+
+    def install(self):
+        reqs, ws = self.working_set()
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.generate_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'],
+            interpreter=options['name'],
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            add_site_packages=self.add_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            ))
+        return generated
+
+    update = install
+
+
+class Scripts(Base):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.generate_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'], reqs=reqs, scripts=scripts,
+            interpreter=options.get('interpreter'),
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            add_site_packages=self.add_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            script_arguments=options.get('arguments', ''),
+            script_initialization=options.get('script-initialization', '')
+            ))
+        return generated

z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py

+##############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+import os, re, shutil, sys
+import zc.buildout.tests
+import zc.buildout.testselectingpython
+import zc.buildout.testing
+
+import unittest
+from zope.testing import doctest, renormalizing
+
+# We do not explicitly test the recipe support for the ``eggs``,
+# ``find-links``, and ``index`` options because they are used for most or
+# all of the examples.  The README tests ``extends``,
+# ``include-site-customization`` and ``name``.  That leaves ``python``,
+# ``extra-paths``, ``initialization``, ``relative-paths``, and
+# ``include-site-packages``.
+
+def supports_python_option():
+    """
+This simply shows that the ``python`` option can specify another section to
+find the ``executable``.  (The ``python`` option defaults to looking in the
+``buildout`` section.)  We do this by creating a custom Python that will have
+some initialization that we can look for.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+"""
+
+def interpreter_recipe_supports_extra_paths_option():
+    """
+This shows that specifying extra-paths will affect sys.path.
+
+This recipe will not add paths that do not exist, so we create them.
+
+    >>> mkdir(sample_buildout, 'foo')
+    >>> mkdir(sample_buildout, 'foo', 'bar')
+    >>> mkdir(sample_buildout, 'spam')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... extra-paths =
+    ...    ${buildout:directory}/foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys;print 'path' + ' '.join(sys.path)"''')
+    ... # doctest:+ELLIPSIS
+    path.../foo/bar /sample-buildout/spam...
+
+"""
+
+def interpreter_recipe_supports_initialization_option():
+    """
+This simply shows that the ``initialization`` option can specify code to
+run on initialization.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE
+    <BLANKLINE>
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+This also works with the exec-sitecustomize option, processing local
+initialization, and then the Python's initialization.  We show this with a
+custom Python.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.recipe.egg'] = 'baLOOba'
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE
+    <BLANKLINE>
+    import os
+    os.environ['zc.recipe.egg'] = 'baLOOba'
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    <BLANKLINE>
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') + ' -c ' +
+    ...              '''"import os; print os.environ['zc.recipe.egg']"'''),
+    baLOOba
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+"""
+
+def interpreter_recipe_supports_relative_paths_option():
+    """
+This shows that the relative-paths option affects the code for inserting
+paths into sys.path.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... relative-paths = true
+    ... extra-paths =
+    ...    /foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+Let's look at the site.py that was generated:
+
+    >>> import sys
+    >>> sys.stdout.write('#'); cat(sample_buildout, 'parts', 'py', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        paths = []
+    <BLANKLINE>
+        import os
+    <BLANKLINE>
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        paths[0:0] = [ # eggs
+            '/foo/bar',
+            join(base, 'spam')
+            ]...
+
+
+"""
+
+def setUp(test):
+    zc.buildout.tests.easy_install_SetUp(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def setUpSelecting(test):
+    zc.buildout.testselectingpython.setup(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def test_suite():
+    suite = unittest.TestSuite((
+        doctest.DocFileSuite(
+            'README.txt',
+            setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+               zc.buildout.testing.normalize_path,
+               zc.buildout.testing.normalize_endings,
+               zc.buildout.testing.normalize_script,
+               zc.buildout.testing.normalize_egg_py,
+               zc.buildout.tests.normalize_bang,
+               (re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'),
+                'zc.buildout.egg'),
+               (re.compile('[-d]  setuptools-[^-]+-'), 'setuptools-X-'),
+               (re.compile(r'setuptools-[\w.]+-py'), 'setuptools-X-py'),
+               (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
+               (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+               (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
+               ])
+            ),
+        doctest.DocTestSuite(
+            setUp=setUp,
+            tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+                zc.buildout.testing.normalize_path,
+                zc.buildout.testing.normalize_endings,
+                zc.buildout.testing.normalize_egg_py,
+                (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+                ]),
+            ),
+
+        ))
+
+    return suite
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')
+

zc.recipe.egg_/src/zc/recipe/egg/README.txt

@@ -154,6 +154,8 @@ dependent-scripts
 interpreter
    The name of a script to generate that allows access to a Python
    interpreter that has the path set based on the eggs installed.
+   (See the ``z3c.recipe.scripts`` recipe for a more full-featured
+   interpreter.)
 
 extra-paths
    Extra paths to include in a generated script.
@@ -577,7 +579,7 @@ declare entry points using the entry-points option:
     -  demo
     -  other
 
-    >>> cat(sample_buildout, 'bin', 'other')
+    >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys
@@ -640,3 +642,4 @@ be made to contact an index server:
     Uninstalling bigdemo.
     Installing demo.
     Generated script '/sample-buildout/bin/foo'.
+

zc.recipe.egg_/src/zc/recipe/egg/api.txt

@@ -117,6 +117,7 @@ computed by the egg recipe by looking at .installed.cfg:
     extras = other
     find-links = http://localhost:27071/
     index = http://localhost:27071/index
+    python = buildout
     recipe = sample
 
 If we use the extra-paths option:

zc.recipe.egg_/src/zc/recipe/egg/custom.txt

@@ -150,6 +150,7 @@ eggs directory can be shared across multiple buildouts.
 
     >>> ls(sample_buildout, 'develop-eggs')
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Note that no scripts or dependencies are installed.  To install
@@ -231,6 +232,7 @@ We won't get an update.
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 But if we run the buildout in the default on-line and newest modes, we
@@ -248,6 +250,7 @@ version is imported:
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
     d  extdemo-1.5-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Controlling the version used
@@ -287,6 +290,7 @@ We can specify a specific version using the egg option:
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 
@@ -553,6 +557,7 @@ Our develop-eggs now includes an egg link for extdemo:
     >>> ls('develop-eggs')
     -  demo.egg-link
     -  extdemo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 and the extdemo now has a built extension:

zc.recipe.egg_/src/zc/recipe/egg/egg.py

@@ -19,11 +19,12 @@ $Id$
 import logging, os, re, zipfile
 import zc.buildout.easy_install
 
+
 class Eggs(object):
 
     def __init__(self, buildout, name, options):
         self.buildout = buildout
-        self.name = name
+        self.name = self.default_eggs = name
         self.options = options
         b_options = buildout['buildout']
         links = options.get('find-links', b_options['find-links'])
@@ -52,7 +53,7 @@ class Eggs(object):
         # verify that this is None, 'true' or 'false'
         get_bool(options, 'unzip')
 
-        python = options.get('python', b_options['python'])
+        python = options.setdefault('python', b_options['python'])
         options['executable'] = buildout[python]['executable']
 
     def working_set(self, extra=()):
@@ -65,15 +66,16 @@ class Eggs(object):
 
         distributions = [
             r.strip()
-            for r in options.get('eggs', self.name).split('\n')
+            for r in options.get('eggs', self.default_eggs).split('\n')
             if r.strip()]
         orig_distributions = distributions[:]
         distributions.extend(extra)
 
-        if self.buildout['buildout'].get('offline') == 'true':
+        if b_options.get('offline') == 'true':
             ws = zc.buildout.easy_install.working_set(
                 distributions, options['executable'],
-                [options['develop-eggs-directory'], options['eggs-directory']]
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
                 )
         else:
             kw = {}
@@ -85,7 +87,7 @@ class Eggs(object):
                 index=self.index,
                 executable=options['executable'],
                 path=[options['develop-eggs-directory']],
-                newest=self.buildout['buildout'].get('newest') == 'true',
+                newest=b_options.get('newest') == 'true',
                 allow_hosts=self.allow_hosts,
                 **kw)
 
@@ -97,16 +99,19 @@ class Eggs(object):
 
     update = install
 
-class Scripts(Eggs):
+
+class ScriptBase(Eggs):
 
     def __init__(self, buildout, name, options):
-        super(Scripts, self).__init__(buildout, name, options)
+        super(ScriptBase, self).__init__(buildout, name, options)
+
+        b_options = buildout['buildout']
 
-        options['bin-directory'] = buildout['buildout']['bin-directory']
+        options['bin-directory'] = b_options['bin-directory']
         options['_b'] = options['bin-directory'] # backward compat.
 
         self.extra_paths = [
-            os.path.join(buildout['buildout']['directory'], p.strip())
+            os.path.join(b_options['directory'], p.strip())
             for p in options.get('extra-paths', '').split('\n')
             if p.strip()
             ]
@@ -115,11 +120,9 @@ class Scripts(Eggs):
 
 
         relative_paths = options.get(
-            'relative-paths',
-            buildout['buildout'].get('relative-paths', 'false')
-            )
+            'relative-paths', b_options.get('relative-paths', 'false'))
         if relative_paths == 'true':
-            options['buildout-directory'] = buildout['buildout']['directory']
+            options['buildout-directory'] = b_options['directory']
             self._relative_paths = options['buildout-directory']
         else:
             self._relative_paths = ''
@@ -128,12 +131,13 @@ class Scripts(Eggs):
     parse_entry_point = re.compile(
         '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
         ).match
+
     def install(self):
         reqs, ws = self.working_set()
         options = self.options
 
         scripts = options.get('scripts')
-        if scripts or scripts is None:
+        if scripts or scripts is None or options.get('interpreter'):
             if scripts is not None:
                 scripts = scripts.split()
                 scripts = dict([
@@ -157,22 +161,32 @@ class Scripts(Eggs):
                     name = dist.project_name
                     if name != 'setuptools' and name not in reqs:
                         reqs.append(name)
-
-            return zc.buildout.easy_install.scripts(
-                reqs, ws, options['executable'],
-                options['bin-directory'],
-                scripts=scripts,
-                extra_paths=self.extra_paths,
-                interpreter=options.get('interpreter'),
-                initialization=options.get('initialization', ''),
-                arguments=options.get('arguments', ''),
-                relative_paths=self._relative_paths,
-                )
-
+            return self._install(reqs, ws, scripts)
         return ()
 
     update = install
 
+    def _install(self, reqs, ws, scripts):
+        # Subclasses implement this.
+        raise NotImplementedError()
+
+
+class Scripts(ScriptBase):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        return zc.buildout.easy_install.scripts(
+            reqs, ws, options['executable'],
+            options['bin-directory'],
+            scripts=scripts,
+            extra_paths=self.extra_paths,
+            interpreter=options.get('interpreter'),
+            initialization=options.get('initialization', ''),
+            arguments=options.get('arguments', ''),
+            relative_paths=self._relative_paths
+            )
+
+
 def get_bool(options, name, default=False):
     value = options.get(name)
     if not value:

zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt

@@ -35,7 +35,7 @@ install the demo package using Python 2.4.
     ... index = http://www.python.org/pypi/
     ...
     ... [python2.4]
-    ... executable = %(python23)s
+    ... executable = %(python24)s
     ...
     ... [demo]
     ... recipe = zc.recipe.egg
@@ -43,7 +43,7 @@ install the demo package using Python 2.4.
     ... find-links = %(server)s
     ... python = python2.4
     ... interpreter = py-demo
-    ... """ % dict(server=link_server, python23=other_executable))
+    ... """ % dict(server=link_server, python24=other_executable))
 
 Now, if we run the buildout: