Rewrite for zc.buildout >= 2.0.0.

 * Same parts directory as the normal buildout is used so that we can
   build faster and also make the code simpler.
 * Python part is now specified by `python` parameter in `buildout`
   section.
 * Restart automatically with the original python when running python
   part is removed.

CHANGES.txt

@@ -4,7 +4,13 @@ Changes
 3.2 (unreleased)
 ----------------
 
- * No changes yet.
+ * Support zc.buildout >= 2.0.0.
+ * Same parts directory as the normal buildout is used so that we can
+   build faster and also make the code simpler.
+ * Python part is now specified by `python` parameter in `buildout`
+   section.
+ * Restart automatically with the original python when running python
+   part is removed.
 
 3.1 (2011-06-24)
 ----------------

README.txt

-Chicken and egg python zc.buildout extension
-============================================
+Rerun buildout with a python provided by buildout
+===============================================
 
-This extensions for zc.buildout is created to solve chicken and egg problem
-while working with buildout and when some exact version of python, which is
-provided by buildout shall be used to *execute* this buildout itself.
+Buildout 2.0 no-longer supports using multiple versions of Python in a
+single buildout. This extension makes it possible to use 'yet another
+single' version of python, by building a specified python part and its
+dependencies first then reinvoking buildout with the executable.
 
 Usage
 -----
 
-Part to build python is required. By convention slapos.rebootstrap will try to
-find python executable in:
+A part to build python is required. You need to specify it by `python`
+parameter in `buildout` section, just same as buildout 1.x. The python
+section must provide executable option that gives the path to a Python
+executable.
 
-  special.parts.directory/partname/bin/partname
+And add slapos.rebootstrap to `extensions` parameter in `buildout` section.
 
-But when needed python-path parameter can be used to point rebootstrap to find
-python in:
-
-  special.parts.directory/partname/python-path
-
-Add slapos.rebootstrap to extensions and set rebooter-section to above section.
-
-Use whatever python to bootstrap and run buildout. If reboot will detect that
-sys.executable used to run buildout is different then executable provided in
-python section it will try to find this executable. If it does not exists it
-will install this section and then reinstall buildout using new python
-executable. Later buildout run will continue using new python.
-
-Because external buildout is used to provide buildout version parameter is
-introduced to being able to upgrade not in place python part. This parameter
-is required and becomes part of suffix.
-
-Whenever developer-mode is set to true no cleanup will be done in case of
-failure. Then user is responsible to cleanup directories.
+Use whatever python to bootstrap and run buildout. If this extension
+detects that sys.executable used to run buildout is different than
+executable provided in python section, it will try to find this
+executable. If it does not exists, it will install this section and
+then reinstall buildout using new python executable. Later buildout
+run will continue using new python.
 
 Example profile and invocation
 ------------------------------
@@ -39,14 +29,11 @@ Example profile and invocation
 
   [buildout]
   extensions = slapos.rebootstrap
+  python = slapospython
   
   parts =
     realrun
   
-  [rebootstrap]
-  section = slapospython
-  version = 1
-
   [slapospython]
   recipe = plone.recipe.command
   stop-on-error = true
@@ -61,7 +48,7 @@ Example profile and invocation
 
 After bootstrapping and running this buildout it will print:
 
-Running with python /path/to/buildout/parts.rebootstrap.1/slapospython/bin/slapospython
+Running with python /path/to/buildout/parts/slapospython/bin/slapospython
 
 Running tests
 -------------

setup.py

 from setuptools import setup, find_packages
-version = '3.2-dev'
+version = '3.2'
 name = "slapos.rebootstrap"
 
 long_description = open("README.txt").read() + '\n\n'
@@ -38,7 +38,7 @@ setup(
   zip_safe=True,
   install_requires=[
     'setuptools',
-    'zc.buildout'
+    'zc.buildout >=2.0.0'
     ],
   tests_require=[
       'zope.testing',

slapos/rebootstrap/__init__.py

@@ -12,7 +12,6 @@
 #
 ##############################################################################
 import os
-import shutil
 import zc.buildout
 import sys
 import logging
@@ -25,53 +24,22 @@ class Rebootstrap:
   def __init__(self, buildout):
     self.logger = logging.getLogger(__name__)
     self.buildout = buildout
-    # fetch section of rebootstrap, obligatory
-    rebootstrap_section = buildout['rebootstrap']
-    # leave garbage? only in case of developer mode
-    self.developer_mode = rebootstrap_section.get('developer-mode',
-      'false') == 'true'
     # fetch section to build python, obligatory
-    self.python_section = rebootstrap_section['section'].strip()
-    # fetch version of python build, obligatory
-    version = rebootstrap_section['version'].strip()
+    self.python_section = (
+      buildout['buildout'].get('python') or \
+      buildout['rebootstrap']['section']
+    ).strip()
+    self.wanted_python = buildout[self.python_section]['executable']
 
-    # usualy python will be try to found by convention in:
-    # prefixed-parts/part-name/bin/part-name
-    # but in case if part is non controlable it will be possible to
-    # find it as
-    # prefixed-parts/part-name/python-path
-    # where python-path is relative path to python interpreter
-    relative_python = rebootstrap_section.get('python-path', ''
-      ).strip() or os.path.join('bin', self.python_section)
     # query for currently running python
     self.running_python = sys.executable
 
-    # generate rebootstrap.realname.version
-    prefix = 'rebootstrap.' + version + '.'
-    self.installed = '.' + prefix + 'installed.cfg'
-    self.parts_directory = os.path.join(self.buildout['buildout'][
-      'directory'], prefix + 'parts')
-
-    # query for wanted python, which is combination of suffixed parts
-    # and working recipe
-    self.wanted_python = os.path.join(self.parts_directory,
-      self.python_section, relative_python)
-
-    # support additional eggs
-    self.eggs = []
-    eggs = rebootstrap_section.get('eggs')
-    if eggs:
-      eggs = [q.strip() for q in eggs.split() if q.strip()]
-      self.eggs = eggs
-
   def __call__(self):
-    self.install_section()
-    self.reboot()
+    if self.running_python != self.wanted_python:
+      self.install_section()
+      self.reboot()
 
   def reboot(self):
-    if self.running_python == self.wanted_python:
-      return
-
     message = """
 ************ REBOOTSTRAP: IMPORTANT NOTICE ************
 bin/buildout is being reinstalled right now, as new python:
@@ -83,114 +51,61 @@ Buildout will be restarted automatically to have this change applied.
 """ % dict(wanted_python=self.wanted_python,
     running_python=self.running_python)
     self.logger.info(message)
-    options = self.buildout['buildout']
-    self.eggs.append('zc.buildout')
-    # XXX: A lot of below code is took from zc.buildout.buildout.py:_maybe_upgrade
-    # which is code duplication issue, but even if newer buildout with needed
-    # hooks will be released, this extension shall work on older ones
-    if zc.buildout.easy_install.is_distribute:
-      self.eggs.append('distribute')
-    else:
-      self.eggs.append('setuptools')
-    ws = zc.buildout.easy_install.install(
-      [
-      (spec + ' ' + options.get(spec+'-version', '')).strip()
-      for spec in self.eggs
-      ],
-      options['eggs-directory'],
-      links = options.get('find-links', '').split(),
-      index = options.get('index'),
-      path = [options['develop-eggs-directory']],
-      prefer_final=False,
-      executable=self.wanted_python
-      )
-    # the new dist is different, so we've upgraded.
-    # Update the scripts and return True
-    # Ideally the new version of buildout would get a chance to write the
-    # script.  Not sure how to do that.
-    partsdir = os.path.join(options['parts-directory'], 'buildout')
-    if os.path.exists(partsdir):
-      # This is primarily for unit tests, in which .py files change too
-      # fast for Python to know to regenerate the .pyc/.pyo files.
-      shutil.rmtree(partsdir)
-    os.mkdir(partsdir)
-    script_initialization = ''
-    # (Honor the relative-paths option.)
-    relative_paths = options.get('relative-paths', 'false')
-    if relative_paths == 'true':
-      relative_paths = options['directory']
-    else:
-      assert relative_paths == 'false'
-      relative_paths = ''
-    zc.buildout.easy_install.sitepackage_safe_scripts(
-      options['bin-directory'], ws, self.wanted_python, partsdir,
-      reqs=['zc.buildout'], relative_paths=relative_paths,
-      include_site_packages=False,
-      script_initialization=script_initialization,
-      )
-
-    # Restart
     args = map(zc.buildout.easy_install._safe_arg, sys.argv)
-    # We want to make sure that our new site.py is used for rerunning
-    # buildout, so we put the partsdir in PYTHONPATH for our restart.
-    # This overrides any set PYTHONPATH, but since we generally are
-    # trying to run with a completely "clean" python (only the standard
-    # library) then that should be fine.
-    env = os.environ.copy()
-    env['PYTHONPATH'] = partsdir
-    sys.stdout.flush()
+    env = os.environ
+    env['ORIG_PYTHON'] = sys.executable
     os.execve(self.wanted_python, [self.wanted_python] + args, env)
 
   def install_section(self):
-    try_install = False
-    try:
-      if not os.path.exists(self.parts_directory) \
-          or not os.path.exists(self.wanted_python):
-        try_install = True
-        self.logger.info('Installing section %r to provide %r in %r' % (
-          self.python_section, self.wanted_python, self.parts_directory))
-        args = map(zc.buildout.easy_install._safe_arg, sys.argv)
-        if 'install' in args:
-          args =  args[:args.index('install')]
+    env = os.environ
+    if not os.path.exists(self.wanted_python):
+      self.logger.info('Installing section %r to provide %r' % (
+        self.python_section, self.wanted_python))
+      args = map(zc.buildout.easy_install._safe_arg, sys.argv)
+      if 'install' in args:
+        args =  args[:args.index('install')]
 
-        # remove rebootstrap extension, which is not needed in rebootstrap part
-        extension_list = self.buildout['buildout']['extensions'].split()
-        extension_list = [q.strip() for q in extension_list if q.strip() != \
-            __name__]
-        # rerun buildout with only neeeded section to reuse buildout
-        # ability to calcuate all dependency
-        args.extend([
-          # install only required section with dependencies
-          'buildout:parts=%s' % self.python_section,
-          # do not load extensions (even other ones)
-          'buildout:extensions=%s' % ' '.join(extension_list),
-          # create virutal python environment
-          'buildout:parts-directory=%s' % self.parts_directory,
-          'buildout:installed=%s' % self.installed,
-        ])
-        self.logger.info('Rerunning buildout to install section %s with '
-          'arguments %r.'% (self.python_section, args))
-        process = subprocess.Popen(args)
-        process.wait()
-        if process.returncode != 0:
-          raise zc.buildout.UserError('Error during installing python '
-            'provision section.')
-      if not os.path.exists(self.wanted_python):
-        raise zc.buildout.UserError('Section %r directed to python executable:\n'
-            '%r\nUnfortunately even after installing this section executable was'
-            ' not found.\nThis is section responsibility to provide python (eg. '
-            'by compiling it).' % (self.python_section, self.wanted_python))
-    except zc.buildout.UserError:
-      if try_install:
-        if not self.developer_mode:
-          if os.path.exists(self.parts_directory):
-            shutil.rmtree(self.parts_directory)
-          if os.path.exists(self.installed):
-            os.unlink(self.installed)
-        else:
-          self.logger.warning('Developer mode active.')
-          self.logger.warning('Directory %r and file %r are left for further '
-            'analysis.' % (self.parts_directory, self.installed))
-          self.logger.warning('Developer is responsible for removing those '
-            'directories.')
-      raise
+      # remove rebootstrap extension, which is not needed in rebootstrap part
+      extension_list = self.buildout['buildout']['extensions'].split()
+      extension_list = [q.strip() for q in extension_list if q.strip() != \
+          __name__]
+      # rerun buildout with only neeeded section to reuse buildout
+      # ability to calcuate all dependency
+      args.extend([
+        # install only required section with dependencies
+        'buildout:parts=%s' % self.python_section,
+        # do not load this extension
+        'buildout:extensions=%s' % ' '.join(extension_list),
+      ])
+      self.logger.info('Rerunning buildout to install section %s with '
+        'arguments %r.'% (self.python_section, args))
+      process = subprocess.Popen(args)
+      process.wait()
+      if process.returncode != 0:
+        raise zc.buildout.UserError('Error during installing python '
+          'provision section.')
+    if not os.path.exists(self.wanted_python):
+      raise zc.buildout.UserError('Section %r directed to python executable:\n'
+          '%r\nUnfortunately even after installing this section executable was'
+          ' not found.\nThis is section responsibility to provide python (eg. '
+          'by compiling it).' % (self.python_section, self.wanted_python))
+
+_uninstall_part_orig = zc.buildout.buildout.Buildout._uninstall_part
+def _uninstall_part(self, part, installed_part_options):
+  _uninstall_part_orig(self, part, installed_part_options)
+  location = self[part].get('location')
+  if location and sys.executable.startswith(location):
+    message = """
+************ REBOOTSTRAP: IMPORTANT NOTICE ************
+%r part that provides the running Python is uninstalled.
+Buildout will be restarted automatically with the original Python.
+************ REBOOTSTRAP: IMPORTANT NOTICE ************
+""" % part
+    self._logger.info(message)
+    if getattr(self, 'dry_run', False):
+      sys.exit()
+    args = map(zc.buildout.easy_install._safe_arg, sys.argv)
+    env = os.environ
+    orig_python = env['ORIG_PYTHON']
+    os.execve(orig_python, [orig_python] + args, env)
+zc.buildout.buildout.Buildout._uninstall_part = _uninstall_part