GitLab

Currently, bootstrapBuildout is called unconditionally, and since Software Releases use slapos.reboostrap, we end up with the following behaviour in the best case:

Processing software releases...
Installing software release /srv/slapgrid/slappart9/srv/testnode/aai/software.cfg...
Generated script '/srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/bin/buildout'.
slapos.rebootstrap: Make sure that the section 'python2.7' won't be reinstalled after rebootstrap.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
Updating patch.
...
Updating gcc.
Updating python2.7.
Stripping binaries ...
Done.
slapos.rebootstrap: 
************ REBOOTSTRAP: IMPORTANT NOTICE ************
bin/buildout is being reinstalled right now, as new python:
  /srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/parts/python2.7/bin/python2.7
is available, and buildout is using another python:
  /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
Buildout will be restarted automatically to have this change applied.
************ REBOOTSTRAP: IMPORTANT NOTICE ************

Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
Updating patch.
...
Updating gcc.
Updating python2.7.
Updating autoconf.
...

Which means that bin/buildout always runs twice: updating parts is usually fast but loading extends and checking that binaries are stripped can take a while.

The idea of this commit is minimize the amount of work when bin/buildout already exists, in particular if it is already changed by slapos.reboostrap to use the built Python. We also hope this will avoid complete rebuilds when building a different version of Python:

Installing software release /srv/slapgrid/slappart1/srv/testnode/bsu/software.cfg...
Generated script '/srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/bin/buildout'.
slapos.rebootstrap: Make sure that the section 'python3.5' won't be reinstalled after rebootstrap.
Uninstalling python3.5.
Uninstalling file.
Uninstalling file-msooxml.
Uninstalling gettext.
Uninstalling lunzip.
Uninstalling libxml2.
Uninstalling sqlite3.
Uninstalling openssl.
Uninstalling ca-certificates.
Uninstalling perl.
Uninstalling gdbm.
Uninstalling bzip2.
Uninstalling libffi.
Uninstalling libexpat.
Uninstalling readline.
Uninstalling ncurses.
Uninstalling zlib.
Uninstalling patch.
Uninstalling xz-utils.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Installing xz-utils.
...
Stripping binaries ...
Done.
slapos.rebootstrap: 
************ REBOOTSTRAP: IMPORTANT NOTICE ************
bin/buildout is being reinstalled right now, as new python:
  /srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/parts/python3.5/bin/python3.5
is available, and buildout is using another python:
  /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
Buildout will be restarted automatically to have this change applied.
************ REBOOTSTRAP: IMPORTANT NOTICE ************

Uninstalling template.
...
Uninstalling m4.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
...
Updating python3.5.
Installing m4.
...

About the implementation, we could merely add a if not os.path.exists(...):. I fear cases where a previous run may have left bin/buildout in a non-working state, so I suggest the use of a marker to force bootstrap if a previous run failed whereas the existing bin/buildout was reused.