slapos-testing: rework to use buildout to install eggs and dependencies

Instead of letting `python setup.py test` install the depencies, use
buildout way of installing the eggs.

This software use `interpreter` recipe of `zc.recipe.egg` to install a
python with all eggs pre-installed. This is a way to get all the
dependencies.
`erp5.util.testsuite` has been extended to support a parameter to
specify which python to use.

This cause chicken and egg problem because cloning repository containing
`slapos.recipe.cmmi` needs git, and to compiling git needs
`slapos.recipe.cmmi`. The consequence of this is that re-running software
will install too many parts again. One solution for this would be to
clone `slapos.recipe.cmmi` with a `git` command provided by testnode or
system package. For now this is open issue. Another solution would be to
not install `slapos.recipe.cmmi` develop egg, simply install the egg
from it's current pypi version while installing the software (running
tests will be from the git checkout anyway).

Another point of attention is that `python setup.py test` install the
requirements listed in `test_requires`, but `zc.recipe.egg` does not
provide a way of installing these. Some of our packages have `[test]`
entrypoints, in this case, the software installs the test entrypoints.
For others, we install the eggs.

Other improvements:
 * use a simple `slapos.recipe:wrapper` instead of `slapos.cookbook:egg_test`
 * fix the typo in repository name erp5-util-repository ->
   erp5.util-repository ( this mean we will have to fix the test suites in
   nexedi ERP5 )
 * document "what is this software" and a scenario of how this software
   can be used to develop slapos eggs.
 * switch to buildout-hash.cfg for easier template hash management.

software/slapos-testing/README.md

+# Slapos egg tests
+
+This software release is used to run unit test of slapos eggs.
+
+The approach is to use setuptools' integrated test runner, `python setup.py test`, to run tests.
+
+The `python` used in this command will be a `zc.recipe.egg` interpreter with
+all eggs pre-installed by this software release.
+
+Nexedi staff can see the results of this test from the test suite
+`SLAPOS-EGG-TEST` in test result module.
+
+
+Here's an example session of how a developer could use this software release in
+slaprunner to develop a slapos egg, in the example `slapos.core`, to make
+changes to the code, run tests and publish changes.
+
+```bash
+# install this software release
+SR=https://lab.nexedi.com/nexedi/slapos/raw/master/software/slapos-testing/software.cfg 
+COMP=slaprunner
+slapos supply $SR $COMP
+slapos node software
+slapos request --node=node=$COMP $SR $COMP
+slapos node instance
+
+# The source code is a git clone working copy on the instance
+cd ~/srv/runner/instance/slappart0/parts/slapos.core/
+
+# make some changes to the code
+vim slapos/tests/client.py
+
+# run tests, using bundled python intepreter with pre-installed eggs dependencies
+~/srv/runner/instance/slappart0/software_release/bin/python_for_test setup.py build
+
+# when satified, commit changes
+git add -p && git commit
+
+# add developer's fork remote (this is only needed the first time)
+git remote add my_remote https://lab.nexedi.com/your_username/slapos.core.git/
+
+# push the changes
+git push my_remote HEAD:feature_branch_name
+
+# then submit merge request
+```

software/slapos-testing/buildout.hash.cfg

+# THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
+# The only allowed lines here are (regexes):
+# - "^#" comments, copied verbatim
+# - "^[" section beginings, copied verbatim
+# - lines containing an "=" sign which must fit in the following categorie.
+#   - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
+#     But avoid directories, they are not portable.
+#     Copied verbatim.
+#   - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
+#     by the re-generation script.
+#     Re-generated.
+# - other lines are copied verbatim
+# Substitution (${...:...}), extension ([buildout] extends = ...) and
+# section inheritance (< = ...) are NOT supported (but you should really
+# not need these here).
+
+[template]
+filename = instance.cfg
+md5sum = 9dece9d12dc94bf5c35d307cc8aa4d6b
+

software/slapos-testing/instance.cfg

 [buildout]
 parts =
-  slapos.core-setup
-  erp5.util-setup
   phantomjs-wrapper
   slapos-test-runner
-  sh-environment
 
 eggs-directory = ${buildout:eggs-directory}
 develop-eggs-directory = ${buildout:develop-eggs-directory}
@@ -24,33 +21,26 @@ bin = $${buildout:directory}/bin
 etc = $${buildout:directory}/etc
 services = $${:etc}/run
 srv = $${buildout:directory}/srv
-source-code = $${:srv}/eggs-source-code
 
 [download-source]
 recipe = slapos.recipe.build:gitclone
 git-executable = ${git:location}/bin/git
 
-# Local development
-[slapos.core]
-<= download-source
-repository = ${slapos.core-repository:location}
-
-[slapos.core-setup]
-recipe = plone.recipe.command
-command = echo "Updating setup...";cd $${slapos.core:location}; export PATH="$${slapos-test-runner:prepend-path}:$PATH"; export CPPFLAGS="$${environment:CPPFLAGS}"; export LDFLAGS="$${environment:LDFLAGS}"; export PYTHONPATH="$${environment:PYTHONPATH}"; export LOCAL_IPV4="$${environment:LOCAL_IPV4}"; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n
-update-command = $${:command}
-
 [caucase]
 <= download-source
 repository = ${caucase-repository:location}
 
+[erp5.util]
+<= download-source
+repository = ${erp5.util-repository:location}
+
 [slapos.cookbook]
 <= download-source
 repository = ${slapos.cookbook-repository:location}
 
-[slapos.recipe.template]
+[slapos.core]
 <= download-source
-repository = ${slapos.recipe.template-repository:location}
+repository = ${slapos.core-repository:location}
 
 [slapos.recipe.build]
 <= download-source
@@ -60,57 +50,33 @@ repository = ${slapos.recipe.build-repository:location}
 <= download-source
 repository = ${slapos.recipe.cmmi-repository:location}
 
-[slapos.toolbox]
+[slapos.recipe.template]
 <= download-source
-repository = ${slapos.toolbox-repository:location}
+repository = ${slapos.recipe.template-repository:location}
 
-[erp5-util]
+[slapos.toolbox]
 <= download-source
-repository = ${erp5-util-repository:location}
+repository = ${slapos.toolbox-repository:location}
 
-[erp5.util-setup]
-recipe = plone.recipe.command
-command = echo "Updating setup...";cd $${erp5-util:location}; export PATH="$${slapos-test-runner:prepend-path}:$PATH"; export CPPFLAGS="$${environment:CPPFLAGS}"; export LDFLAGS="$${environment:LDFLAGS}"; export PYTHONPATH="$${environment:PYTHONPATH}"; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n
-update-command = $${:command}
 
 [slapos-test-runner]
-recipe = slapos.cookbook:egg_test
-run-test-suite = $${create-directory:bin}/runTestSuite
-run-test-suite-binary = ${buildout:bin-directory}/runTestSuite
-# The list of executables should be defined here and a combination
-# of tests should dynamically generated.
-#python-list = $${}
-test-list =
-  $${slapos.cookbook:location}
-  $${slapos.core:location}
-  $${slapos.recipe.template:location}
-  $${slapos.recipe.build:location}
-  $${slapos.recipe.cmmi:location}
-  $${slapos.toolbox:location}
-  $${erp5-util:location}
-  $${caucase:location}
-prepend-path = ${curl:location}/bin:${openssl:location}/bin:${git:location}/bin:${libxslt:location}/bin:${python2.7:location}/bin
-environment = environment
-
-[environment]
-CPPFLAGS = -I${python2.7:location}/include/python2.7 -I${libxml2:location}/include -I${libxslt:location}/include
-LDFLAGS = -L${python2.7:location}/lib -L${libxml2:location}/lib -L${libxslt:location}/lib -L${libxslt:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${python2.7:location}/lib -Wl,-rpath=${libxml2:location}/lib -Wl,-rpath=${libxslt:location}/lib -Wl,-rpath=${zlib:location}/lib
-LD_LIBRARY_PATH = ${python2.7:location}/lib:${libxml2:location}/lib:${libxslt:location}/lib:${libxslt:location}/lib:${zlib:location}/lib
-PYTHONPATH = ${python-setuptools:pythonpath}:${buildout:eggs-directory}:${buildout:develop-eggs-directory}
-LOCAL_IPV4 = $${slap-configuration:ipv4-random}
-
-[sh-environment]
-# Section exposes testing default environment as sh file. It is thus easy
-# to directly develop and test the egg inside of this instance.
-recipe = collective.recipe.template
-input = inline:
-  export PATH="$${slapos-test-runner:prepend-path}:$PATH"
-  export CPPFLAGS="$${environment:CPPFLAGS}"
-  export LDFLAGS="$${environment:LDFLAGS}"
-  export PYTHONPATH="$${environment:PYTHONPATH}"
-  export PS1="[slapos-testing env Active] $PS1"
-output = $${create-directory:bin}/environment.sh
-mode = 755
+recipe = slapos.cookbook:wrapper
+wrapper-path = $${create-directory:bin}/runTestSuite
+command-line =
+  ${buildout:bin-directory}/runTestSuite
+  --python_interpreter=${buildout:bin-directory}/${eggs:interpreter}
+  --source_code_path_list=$${caucase:location},$${erp5.util:location},$${slapos.cookbook:location},$${slapos.core:location},$${slapos.recipe.build:location},$${slapos.recipe.cmmi:location},$${slapos.recipe.template:location},$${slapos.toolbox:location}
+
+# Notes about environment:
+# * slapos.cookbook:wrapper does not seem to allow "extending" PATH. Tests
+#   needs ping, which is a setuid binary that cannot be installed via slapos
+#   way of building software without root access, so we keep "standard"
+#   /usr/bin and /bin in $PATH
+# * LOCAL_IPV4 is needed for some slapos.core tests
+environment =
+  PATH=${coreutils:location}/bin:${curl:location}/bin:${openssl:location}/bin:${git:location}/bin:${libxslt:location}/bin:/usr/bin/:/bin/
+  LOCAL_IPV4=$${slap-configuration:ipv4-random}
+
 
 [phantomjs-wrapper]
 recipe = slapos.cookbook:wrapper

software/slapos-testing/software.cfg

@@ -6,26 +6,78 @@ extends =
   ../../component/libxml2/buildout.cfg
   ../../component/libxslt/buildout.cfg
   ../../component/bcrypt/buildout.cfg
-  ../../component/python-2.7/buildout.cfg
-  ../../component/python-setuptools/buildout.cfg
   ../../component/zlib/buildout.cfg
   ../../component/phantomjs/buildout.cfg
   ../../component/pycurl/buildout.cfg
+  ../../component/coreutils/buildout.cfg
   ../../stack/slapos.cfg
+  ./buildout.hash.cfg
 
 parts =
-  caucase-repository
-  slapos.cookbook-repository
-  slapos.core-repository
-  slapos.recipe.template-repository
-  slapos.recipe.build-repository
-  slapos.recipe.cmmi-repository
-  slapos.toolbox-repository
-  erp5-util-repository
+  bootstrap-slapos.recipe.cmmi
   eggs
   phantomjs
   template
 
+[bootstrap-slapos.recipe.cmmi]
+# install our develop version of slapos.recipe.cmmi before anything else,
+# otherwise it will be installed from pypi by dependencies.
+recipe = zc.recipe.egg
+eggs = ${slapos.recipe.cmmi-setup:egg}
+
+[setup-develop-egg]
+recipe = zc.recipe.egg:develop
+
+[caucase-setup]
+<= setup-develop-egg
+egg = caucase
+setup = ${caucase-repository:location}
+
+[erp5.util-setup]
+<= setup-develop-egg
+# XXX erp5.util does not have `test` extra require, but has a `testnode` extra require with same dependencies
+egg = erp5.util[testnode]
+setup = ${erp5.util-repository:location}
+depends = ${slapos.core-setup:egg}
+
+[slapos.cookbook-setup]
+<= setup-develop-egg
+# XXX slapos.cookbook does not have `test` extra require, `mock` is only listed in `tests_require` and is listed explicitly
+egg = slapos.cookbook
+setup = ${slapos.cookbook-repository:location}
+depends = ${slapos.core-setup:egg}
+
+[slapos.core-setup]
+<= setup-develop-egg
+# XXX slapos.cookbook does not have `test` extra require, `mock`, `pyflakes` and `httmock` are only listed in `tests_require` and are listed explicitly
+egg = slapos.core
+setup = ${slapos.core-repository:location}
+
+[slapos.recipe.build-setup]
+<= setup-develop-egg
+egg = slapos.recipe.build[test]
+setup = ${slapos.recipe.build-repository:location}
+
+[slapos.recipe.cmmi-setup]
+<= setup-develop-egg
+egg = slapos.recipe.cmmi[test]
+setup = ${slapos.recipe.cmmi-repository:location}
+depends = ${slapos.recipe.build-setup:egg}
+
+[slapos.recipe.template-setup]
+<= setup-develop-egg
+# XXX slapos.recipe.template does not have `test` extra require, `zope.testing` is only listed in `tests_require` and is listed explicitly
+egg = slapos.recipe.template
+setup = ${slapos.recipe.template-repository:location}
+
+[slapos.toolbox-setup]
+<= setup-develop-egg
+# XXX slapos.toolbox does not have `test` extra require, `mock` and `pycurl` are only listed in `tests_require` and are listed explicitly
+egg = slapos.toolbox
+setup = ${slapos.toolbox-repository:location}
+depends = ${slapos.core-setup:egg}
+
+
 [eggs]
 recipe = zc.recipe.egg
 eggs =
@@ -35,18 +87,24 @@ eggs =
   ${bcrypt:egg}
   dnspython
   Jinja2
-  caucase
-  erp5.util
-  slapos.cookbook
-  collective.recipe.template
-  plone.recipe.command
-  slapos.recipe.template
-  slapos.recipe.cmmi
-  slapos.toolbox
+  ${caucase-setup:egg}
+  ${erp5.util-setup:egg}
+  ${slapos.cookbook-setup:egg}
+  ${slapos.core-setup:egg}
+  ${slapos.recipe.build-setup:egg}
+  ${slapos.recipe.cmmi-setup:egg}
+  ${slapos.recipe.template-setup:egg}
+  ${slapos.toolbox-setup:egg}
+  mock
+  zope.testing
+  httmock
+  pyflakes
 entry-points =
   runTestSuite=erp5.util.testsuite:runTestSuite
 scripts =
   runTestSuite
+interpreter=
+  python_for_test
 
 [git-clone-repository]
 recipe = slapos.recipe.build:gitclone
@@ -58,6 +116,10 @@ branch = master
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/caucase.git
 
+[erp5.util-repository]
+<= git-clone-repository
+repository = https://lab.nexedi.com/nexedi/erp5.git
+
 [slapos.cookbook-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.git
@@ -73,28 +135,38 @@ repository = https://lab.nexedi.com/nexedi/slapos.recipe.template.git
 [slapos.recipe.build-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
+# We use the system git and not slapos provided one, because
+# slapos.recipe.build is a dependency of slapos.recipe.cmmi
+#git-executable = git
 
 [slapos.recipe.cmmi-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.recipe.cmmi.git
+# We use the system git and not slapos provided one, because slapos git needs
+# slapos.recipe.cmmi to be installed. This circular dependency cause parts to
+# be reinstalled everytime buildout is run because signatures are not stable.
+#git-executable = git
 
 [slapos.toolbox-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.toolbox.git
 
-[erp5-util-repository]
-<= git-clone-repository
-repository = https://lab.nexedi.com/nexedi/erp5.git
 
 [template]
 recipe = slapos.recipe.template
-url = ${:_profile_base_location_}/instance.cfg
-md5sum = 6626794c9dbb2530bb8ba3d331e27542
+url = ${:_profile_base_location_}/${:filename}
 output = ${buildout:directory}/template.cfg
 mode = 640
 
 [versions]
 Pygments = 2.1.3
-collective.recipe.template = 1.10
-plone.recipe.command = 1.1
-slapos.recipe.template = 4.3
+
+# clear the version of tested eggs, to make sure we installed the developped ones
+caucase =
+erp5.util =
+slapos.cookbook =
+slapos.core =
+slapos.recipe.build =
+slapos.recipe.cmmi =
+slapos.recipe.template =
+slapos.toolbox =