slapos-testing: rework to use buildout to install eggs and dependencies

Instead of letting `python setup.py test` install the depencies, use
buildout way of installing the eggs.

This software use `interpreter` recipe of `zc.recipe.egg` to install a
python with all eggs pre-installed. This is a way to get all the
dependencies at install time instead of getting them at run time from
pypi when running `python setup.py test`.
`erp5.util.testsuite` has been extended to support a parameter to
specify which python interpreter to use.

One issue is that this way of installing eggs by buildout cause chicken
and egg problem: cloning repository containing `slapos.recipe.cmmi`
needs git, and to compiling git needs `slapos.recipe.cmmi`.
The consequence of this is that re-running software will install too
many parts again.
One solution for this would be to clone `slapos.recipe.cmmi` with a
`git` command provided by testnode or system package.
Another solution would be to not install `slapos.recipe.cmmi` develop
egg, simply install the egg from it's current pypi version while
installing the software (running tests will be from the git checkout
anyway).
For now this is open issue.

Another point of attention is that `python setup.py test` install the
requirements listed in `test_requires`, but `zc.recipe.egg` does not
provide a way of installing these. Some of our packages have `[test]`
entrypoints, in this case, the software installs the test entrypoints.
For others, we install the eggs.

Other improvements:
 * use a simple `slapos.recipe:wrapper` instead of `slapos.cookbook:egg_test`
 * fix the typo in repository name erp5-util-repository ->
   erp5.util-repository ( this mean we will have to fix the test suites in
   nexedi ERP5 )
 * document "what is this software" and a scenario of how this software
   can be used to develop slapos eggs.
 * switch to buildout-hash.cfg for easier template hash management.

software/slapos-testing/README.md

+# Slapos egg tests
+
+This software release is used to run unit test of slapos eggs.
+
+The approach is to use setuptools' integrated test runner, `python setup.py test`, to run tests.
+
+The `python` used in this command will be a `zc.recipe.egg` interpreter with
+all eggs pre-installed by this software release.
+
+Nexedi staff can see the results of this test from the test suite
+`SLAPOS-EGG-TEST` in test result module.
+
+
+Here's an example session of how a developer could use this software release in
+slaprunner to develop a slapos egg, in the example `slapos.core`, to make
+changes to the code, run tests and publish changes.
+
+```bash
+# install this software release
+SR=https://lab.nexedi.com/nexedi/slapos/raw/master/software/slapos-testing/software.cfg 
+COMP=slaprunner
+INSTANCE_NAME=$COMP
+
+slapos supply $SR $COMP
+slapos node software
+slapos request --node=node=$COMP $INSTANCE_NAME $SR
+slapos node instance
+
+# The source code is a git clone working copy on the instance
+cd ~/srv/runner/instance/slappart0/parts/slapos.core/
+
+# make some changes to the code
+vim slapos/tests/client.py
+
+# run tests, using bundled python intepreter with pre-installed eggs dependencies
+~/srv/runner/instance/slappart0/software_release/bin/python_for_test setup.py build
+
+# when satified, commit changes
+git add -p && git commit
+
+# add developer's fork remote (this is only needed the first time)
+git remote add my_remote https://lab.nexedi.com/your_username/slapos.core.git/
+
+# push the changes
+git push my_remote HEAD:feature_branch_name
+
+# then submit merge request
+```

software/slapos-testing/buildout.hash.cfg

+# THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
+# The only allowed lines here are (regexes):
+# - "^#" comments, copied verbatim
+# - "^[" section beginings, copied verbatim
+# - lines containing an "=" sign which must fit in the following categorie.
+#   - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
+#     But avoid directories, they are not portable.
+#     Copied verbatim.
+#   - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
+#     by the re-generation script.
+#     Re-generated.
+# - other lines are copied verbatim
+# Substitution (${...:...}), extension ([buildout] extends = ...) and
+# section inheritance (< = ...) are NOT supported (but you should really
+# not need these here).
+
+[template]
+filename = instance.cfg
+md5sum = 9dece9d12dc94bf5c35d307cc8aa4d6b
+

software/slapos-testing/instance.cfg

 [buildout]
 parts =
-  slapos.core-setup
-  erp5.util-setup
   phantomjs-wrapper
   slapos-test-runner
-  sh-environment
 
 eggs-directory = ${buildout:eggs-directory}
 develop-eggs-directory = ${buildout:develop-eggs-directory}
@@ -24,33 +21,26 @@ bin = $${buildout:directory}/bin
 etc = $${buildout:directory}/etc
 services = $${:etc}/run
 srv = $${buildout:directory}/srv
-source-code = $${:srv}/eggs-source-code
 
 [download-source]
 recipe = slapos.recipe.build:gitclone
 git-executable = ${git:location}/bin/git
 
-# Local development
-[slapos.core]
-<= download-source
-repository = ${slapos.core-repository:location}
-
-[slapos.core-setup]
-recipe = plone.recipe.command
-command = echo "Updating setup...";cd $${slapos.core:location}; export PATH="$${slapos-test-runner:prepend-path}:$PATH"; export CPPFLAGS="$${environment:CPPFLAGS}"; export LDFLAGS="$${environment:LDFLAGS}"; export PYTHONPATH="$${environment:PYTHONPATH}"; export LOCAL_IPV4="$${environment:LOCAL_IPV4}"; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n
-update-command = $${:command}
-
 [caucase]
 <= download-source
 repository = ${caucase-repository:location}
 
+[erp5.util]
+<= download-source
+repository = ${erp5.util-repository:location}
+
 [slapos.cookbook]
 <= download-source
 repository = ${slapos.cookbook-repository:location}
 
-[slapos.recipe.template]
+[slapos.core]
 <= download-source
-repository = ${slapos.recipe.template-repository:location}
+repository = ${slapos.core-repository:location}
 
 [slapos.recipe.build]
 <= download-source
@@ -60,57 +50,33 @@ repository = ${slapos.recipe.build-repository:location}
 <= download-source
 repository = ${slapos.recipe.cmmi-repository:location}
 
-[slapos.toolbox]
+[slapos.recipe.template]
 <= download-source
-repository = ${slapos.toolbox-repository:location}
+repository = ${slapos.recipe.template-repository:location}
 
-[erp5-util]
+[slapos.toolbox]
 <= download-source
-repository = ${erp5-util-repository:location}
+repository = ${slapos.toolbox-repository:location}
 
-[erp5.util-setup]
-recipe = plone.recipe.command
-command = echo "Updating setup...";cd $${erp5-util:location}; export PATH="$${slapos-test-runner:prepend-path}:$PATH"; export CPPFLAGS="$${environment:CPPFLAGS}"; export LDFLAGS="$${environment:LDFLAGS}"; export PYTHONPATH="$${environment:PYTHONPATH}"; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n; ${python2.7:location}/bin/python setup.py test -n
-update-command = $${:command}
 
 [slapos-test-runner]
-recipe = slapos.cookbook:egg_test
-run-test-suite = $${create-directory:bin}/runTestSuite
-run-test-suite-binary = ${buildout:bin-directory}/runTestSuite
-# The list of executables should be defined here and a combination
-# of tests should dynamically generated.
-#python-list = $${}
-test-list =
-  $${slapos.cookbook:location}
-  $${slapos.core:location}
-  $${slapos.recipe.template:location}
-  $${slapos.recipe.build:location}
-  $${slapos.recipe.cmmi:location}
-  $${slapos.toolbox:location}
-  $${erp5-util:location}
-  $${caucase:location}
-prepend-path = ${curl:location}/bin:${openssl:location}/bin:${git:location}/bin:${libxslt:location}/bin:${python2.7:location}/bin
-environment = environment
-
-[environment]
-CPPFLAGS = -I${python2.7:location}/include/python2.7 -I${libxml2:location}/include -I${libxslt:location}/include
-LDFLAGS = -L${python2.7:location}/lib -L${libxml2:location}/lib -L${libxslt:location}/lib -L${libxslt:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${python2.7:location}/lib -Wl,-rpath=${libxml2:location}/lib -Wl,-rpath=${libxslt:location}/lib -Wl,-rpath=${zlib:location}/lib
-LD_LIBRARY_PATH = ${python2.7:location}/lib:${libxml2:location}/lib:${libxslt:location}/lib:${libxslt:location}/lib:${zlib:location}/lib
-PYTHONPATH = ${python-setuptools:pythonpath}:${buildout:eggs-directory}:${buildout:develop-eggs-directory}
-LOCAL_IPV4 = $${slap-configuration:ipv4-random}
-
-[sh-environment]
-# Section exposes testing default environment as sh file. It is thus easy
-# to directly develop and test the egg inside of this instance.
-recipe = collective.recipe.template
-input = inline:
-  export PATH="$${slapos-test-runner:prepend-path}:$PATH"
-  export CPPFLAGS="$${environment:CPPFLAGS}"
-  export LDFLAGS="$${environment:LDFLAGS}"
-  export PYTHONPATH="$${environment:PYTHONPATH}"
-  export PS1="[slapos-testing env Active] $PS1"
-output = $${create-directory:bin}/environment.sh
-mode = 755
+recipe = slapos.cookbook:wrapper
+wrapper-path = $${create-directory:bin}/runTestSuite
+command-line =
+  ${buildout:bin-directory}/runTestSuite
+  --python_interpreter=${buildout:bin-directory}/${eggs:interpreter}
+  --source_code_path_list=$${caucase:location},$${erp5.util:location},$${slapos.cookbook:location},$${slapos.core:location},$${slapos.recipe.build:location},$${slapos.recipe.cmmi:location},$${slapos.recipe.template:location},$${slapos.toolbox:location}
+
+# Notes about environment:
+# * slapos.cookbook:wrapper does not seem to allow "extending" PATH. Tests
+#   needs ping, which is a setuid binary that cannot be installed via slapos
+#   way of building software without root access, so we keep "standard"
+#   /usr/bin and /bin in $PATH
+# * LOCAL_IPV4 is needed for some slapos.core tests
+environment =
+  PATH=${coreutils:location}/bin:${curl:location}/bin:${openssl:location}/bin:${git:location}/bin:${libxslt:location}/bin:/usr/bin/:/bin/
+  LOCAL_IPV4=$${slap-configuration:ipv4-random}
+
 
 [phantomjs-wrapper]
 recipe = slapos.cookbook:wrapper

software/slapos-testing/software.cfg

@@ -6,26 +6,78 @@ extends =
   ../../component/libxml2/buildout.cfg
   ../../component/libxslt/buildout.cfg
   ../../component/bcrypt/buildout.cfg
-  ../../component/python-2.7/buildout.cfg
-  ../../component/python-setuptools/buildout.cfg
   ../../component/zlib/buildout.cfg
   ../../component/phantomjs/buildout.cfg
   ../../component/pycurl/buildout.cfg
+  ../../component/coreutils/buildout.cfg
   ../../stack/slapos.cfg
+  ./buildout.hash.cfg
 
 parts =
-  caucase-repository
-  slapos.cookbook-repository
-  slapos.core-repository
-  slapos.recipe.template-repository
-  slapos.recipe.build-repository
-  slapos.recipe.cmmi-repository
-  slapos.toolbox-repository
-  erp5-util-repository
+  bootstrap-slapos.recipe.cmmi
   eggs
   phantomjs
   template
 
+[bootstrap-slapos.recipe.cmmi]
+# install our develop version of slapos.recipe.cmmi before anything else,
+# otherwise it will be installed from pypi by dependencies.
+recipe = zc.recipe.egg
+eggs = ${slapos.recipe.cmmi-setup:egg}
+
+[setup-develop-egg]
+recipe = zc.recipe.egg:develop
+
+[caucase-setup]
+<= setup-develop-egg
+egg = caucase
+setup = ${caucase-repository:location}
+
+[erp5.util-setup]
+<= setup-develop-egg
+# XXX erp5.util does not have `test` extra require, but has a `testnode` extra require with same dependencies
+egg = erp5.util[testnode]
+setup = ${erp5.util-repository:location}
+depends = ${slapos.core-setup:egg}
+
+[slapos.cookbook-setup]
+<= setup-develop-egg
+# XXX slapos.cookbook does not have `test` extra require, `mock` is only listed in `tests_require` and is listed explicitly
+egg = slapos.cookbook
+setup = ${slapos.cookbook-repository:location}
+depends = ${slapos.core-setup:egg}
+
+[slapos.core-setup]
+<= setup-develop-egg
+# XXX slapos.cookbook does not have `test` extra require, `mock`, `pyflakes` and `httmock` are only listed in `tests_require` and are listed explicitly
+egg = slapos.core
+setup = ${slapos.core-repository:location}
+
+[slapos.recipe.build-setup]
+<= setup-develop-egg
+egg = slapos.recipe.build[test]
+setup = ${slapos.recipe.build-repository:location}
+
+[slapos.recipe.cmmi-setup]
+<= setup-develop-egg
+egg = slapos.recipe.cmmi[test]
+setup = ${slapos.recipe.cmmi-repository:location}
+depends = ${slapos.recipe.build-setup:egg}
+
+[slapos.recipe.template-setup]
+<= setup-develop-egg
+# XXX slapos.recipe.template does not have `test` extra require, `zope.testing` is only listed in `tests_require` and is listed explicitly
+egg = slapos.recipe.template
+setup = ${slapos.recipe.template-repository:location}
+
+[slapos.toolbox-setup]
+<= setup-develop-egg
+# XXX slapos.toolbox does not have `test` extra require, `mock` and `pycurl` are only listed in `tests_require` and are listed explicitly
+egg = slapos.toolbox
+setup = ${slapos.toolbox-repository:location}
+depends = ${slapos.core-setup:egg}
+
+
 [eggs]
 recipe = zc.recipe.egg
 eggs =
@@ -35,18 +87,24 @@ eggs =
   ${bcrypt:egg}
   dnspython
   Jinja2
-  caucase
-  erp5.util
-  slapos.cookbook
-  collective.recipe.template
-  plone.recipe.command
-  slapos.recipe.template
-  slapos.recipe.cmmi
-  slapos.toolbox
+  ${caucase-setup:egg}
+  ${erp5.util-setup:egg}
+  ${slapos.cookbook-setup:egg}
+  ${slapos.core-setup:egg}
+  ${slapos.recipe.build-setup:egg}
+  ${slapos.recipe.cmmi-setup:egg}
+  ${slapos.recipe.template-setup:egg}
+  ${slapos.toolbox-setup:egg}
+  mock
+  zope.testing
+  httmock
+  pyflakes
 entry-points =
   runTestSuite=erp5.util.testsuite:runTestSuite
 scripts =
   runTestSuite
+interpreter=
+  python_for_test
 
 [git-clone-repository]
 recipe = slapos.recipe.build:gitclone
@@ -58,6 +116,10 @@ branch = master
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/caucase.git
 
+[erp5.util-repository]
+<= git-clone-repository
+repository = https://lab.nexedi.com/nexedi/erp5.git
+
 [slapos.cookbook-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.git
@@ -73,28 +135,38 @@ repository = https://lab.nexedi.com/nexedi/slapos.recipe.template.git
 [slapos.recipe.build-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.recipe.build.git
+# We use the system git and not slapos provided one, because
+# slapos.recipe.build is a dependency of slapos.recipe.cmmi
+#git-executable = git
 
 [slapos.recipe.cmmi-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.recipe.cmmi.git
+# We use the system git and not slapos provided one, because slapos git needs
+# slapos.recipe.cmmi to be installed. This circular dependency cause parts to
+# be reinstalled everytime buildout is run because signatures are not stable.
+#git-executable = git
 
 [slapos.toolbox-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.toolbox.git
 
-[erp5-util-repository]
-<= git-clone-repository
-repository = https://lab.nexedi.com/nexedi/erp5.git
 
 [template]
 recipe = slapos.recipe.template
-url = ${:_profile_base_location_}/instance.cfg
-md5sum = 6626794c9dbb2530bb8ba3d331e27542
+url = ${:_profile_base_location_}/${:filename}
 output = ${buildout:directory}/template.cfg
 mode = 640
 
 [versions]
 Pygments = 2.1.3
-collective.recipe.template = 1.10
-plone.recipe.command = 1.1
-slapos.recipe.template = 4.3
+
+# clear the version of tested eggs, to make sure we installed the developped ones
+caucase =
+erp5.util =
+slapos.cookbook =
+slapos.core =
+slapos.recipe.build =
+slapos.recipe.cmmi =
+slapos.recipe.template =
+slapos.toolbox =