GitLab

(see also slapos repository's shared-egg branch)

This merge request contains changes for both zc.buildout and zc.recipe.egg.

what can be shared ?

• buildout extension eggs
• eggs installed for recipes
• eggs installed with zc.recipe.egg and zc.recipe.egg:custom

what cannot be shared ?

• buildout bootstrap eggs, and their possible updates
• eggs installed with zc.recipe.egg:develop

performance impact ?

With a 'hot' enough shared directory, I could build EPR5 software release in another directory in 9 minutes, instead of 1 hour.

directories / files

# normal eggs, directly used from the shared directory
/(shared)/__eggs__/AccessControl-4.4-py2.7-linux-x86_64.egg
/(shared)/__eggs__/Acquisition-4.13+slapospatched001-py2.7-linux-x86_64.egg
/(shared)/__eggs__/aiohappyeyeballs-2.4.4-py3.9.egg
/(shared)/__eggs__/aiohttp-3.11.11-py3.9-linux-x86_64.egg
...
# shared eggs script, directly used from the shared directory
/(shared)/cython/(hash)/bin/cython
/(shared)/cython/(hash)/bin/cythonize
...
# shared develop-eggs, symlinks to them are put in /(SR)/develop-eggs directory
/(shared)/python-cffi/(hash)/cffi-1.15.0-py2.7-linux-x86_64.egg
/(shared)/python-cffi/(hash)/cffi-1.15.0-py3.9-linux-x86_64.egg

normal eggs are used directly from shared __eggs__ directory and we have symlinks of develop-eggs under (SR)/develop-eggs.

how they are protected from slapos node prune ?

Inside (SR)/.installed.cfg,

• [buildout] part contais the information of eggs for recipes like __eggs__ = slapos.recipe.cmmi-0.22-py3.9.egg plone.recipe.command-1.1-py3.9.egg.
• the part where egg is installed with zc.recipe.egg:custom contains like location = /(shared)/python-cffi/(hash) like slapos.recipe.cmmi shared ones.
• the part where eggs are installed with zc.recipe.egg contains the information of used eggs in the part like __eggs__ = six-1.16.0-py3.9.egg pycparser-2.20-py3.9.egg.