If not set, the commands in the middle of custom configure-command,
pre-build, post-build etc can fail without resulting in SystemError
for user and without stopping cmmi process, which thus can formally
succeed despite being misconfigured or in some cases with the code
not even built.
This is because when executing a multiline command, only the exit
status of the last one actually gets caught by check_call().

Setting -e makes debugging easier by allowing shell to terminate
mid-way if errors (non-zero exit status) occur.

-e only covers 'simple commands', and does not affect 'if' tests
and more complex cases where non-zero exit status is expected to
occur normally.
Detailed description of -e effects can be seen in the POSIX
standard:

https://pubs.opengroup.org/onlinepubs/009695399/utilities/set.html

When build process is interrupted (like when the buildout process
terminated by SIGKILL), building with shared leave the system in a state
that looks like installation succeeded, so next execution will assume
the part was properly installed.


The behavior was something like this:


    if isSharedPartSignatureOK():
      useThisAlreadyInstalledPart()
    else:
      writeSignatureInSharedPart()
      try:
        build()
      except:
        removeSharedPart()
        raise


and when program is terminated during build step, the signature is
there.

The fix is to write signature later, algorithm becomes:


    if isSharedPartSignatureOK():
      useThisAlreadyInstalledPart()
    else:
      try:
        build()
      except:
        removeSharedPart()
        raise
      writeSignatureInSharedPart()

/reviewed-on !10

Use a new option `buildout:shared-path-list` that is now a new line separated list of path, only the last part is used to install new parts, other are read-only.

Also some cleanups in the test suite.

/reviewed-on !9

on python3.6, subprocess error message is different, use a
zope.testing.renormalizing rule to normalize the message.

This will allow nested slapos where the inner slapos can re-use shared
parts from outer slapos, but not write into the outer shared parts.

For compatibility reasons, the syntax of the buildout option changed,
instead of the old ${buildout:shared-parts} that use to be one path, we
now use ${buildout:shared-part-list} as a \n separated list of paths.
All paths can be used to lookup existing parts, only the last one can be
used to install new parts.
There is no compatibility with the old option as it was not used much,
only in profiles which were not really compatible with shared options.

https://pypi.org/project/zope.testing/ features "renormalizing" that
this test was already using to make this test independent from actual
paths that can be different but use some paths that looks always the
same (like /sample_buildout/).
Also use this in our patches for shared support so that the test is a
bit more readable and also so that we can assert the actual hash being
used.

This is restructured text.

This is not the default with Python 2.

This reverts daa833ea ( nexedi/slapos.recipe.cmmi!3 ) and extend it to not set `$TMP` either.

De facto, setting only `$TMP` was never effective ( see also [#20120530-132221B](https://nexedi.erp5.net/bug_module/20120530-132221B) ) and as we saw in nexedi/slapos.recipe.cmmi!3 (comment 67059) it causes more harm than good.

The conclusion so far is that environment variables related to temporary files should be handled by the parent process. The parent process is responsible for providing a directory short enough so that we can create unix sockets in `$TMPDIR`  and to clean up this directory afterwards.


In a pure buildout it could be:

```bash
tmpdir=$(mktemp -d)
trap 'rm -rf $tmpdir' SIGINT SIGTERM
TEMP=$tmpdir TMPDIR=$tmpdir TMP=$tmpdir buildout
rm -rf $tmpdir
```

/reviewed-on nexedi/slapos.recipe.cmmi!8

It was inconsistent to only set TMP and not TEMP / TEMPDIR. In the case
of slapos, it should be set by slapos node software

Most installation seem to cleanup tmp properly, so the use case cleaning
up seem not critical.

This TMP was also used to know what new files have been created, by
considering files newer that the creation date of this directory.
Compare the file dates with the creation of the source directory instead
( xxx__compile__ )

This reverts commit daa833ea.

To make it easyier to diagnose build errors.

/reviewed-on nexedi/slapos.recipe.cmmi!7

We were already setting $TMP, but many software uses $TMPDIR or $TEMP.

Fix a doctest failure. Since shared part is read-only, it is necessary to change permission before removing.

To check shared directory is not enough. Shared directory may be created before completing installation. Check signature file too.

Revert "To check shared directory is not enough. Shared directory may be created before completing installation. Check signature file too."

This reverts commit ea8ac2d2.

To check shared directory is not enough. Shared directory may be created before completing installation. Check signature file too.

Rename 'shared-part' to 'shared-parts'. Since the original option name is 'parts', 'shared-parts' is consistent.

Disable shared feature by default since it is stil an experimental one. Enable it only if shared-part is defined.

1.use shared-part in [buildout] section to define shared path
2.shared option is True or Flase
3.a magic substitution @@LOCATION@@ is used to for dynamic path

```
======================================================================
ERROR: /srv/slapgrid/slappart0/parts/slapos.recipe.cmmi/slapos/recipe/cmmi/README.txt
Doctest: README.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/slapgrid/298b8e2a56fa3803aa68aa4cfe6a3b96/parts/python2.7/lib/python2.7/doctest.py", line 2194, in setUp
    self._dt_setUp(test)
  File "/srv/slapgrid/slappart0/parts/slapos.recipe.cmmi/slapos/recipe/cmmi/tests.py", line 26, in setUp
    os.environ.pop('MAKEFLAGS')
  File "/opt/slapgrid/298b8e2a56fa3803aa68aa4cfe6a3b96/parts/python2.7/lib/python2.7/os.py", line 505, in pop
    return self.data.pop(key, *args)
KeyError: 'MAKEFLAGS'
```

When running under testnode, there was sometimes this failure:

```
File "/srv/slapgrid/slappart8/srv/testnode/bee/inst/test0-0/parts/slapos.recipe.cmmi/slapos/recipe/cmmi/README.txt", line 521, in README.txt
Failed example:
    print(system(buildout))
Expected:
    Uninstalling package.
    Installing package.
    package: [ENV] CFLAGS = -I/sw/include
    package: [ENV] LDFLAGS = -L/sw/lib -L/some/extra/lib
    package: [ENV] TMP = /sample_buildout/parts/package/tmp
    package: Applying patches
    patching file configure
    patching file Makefile.dist
    patched-configure --prefix=/somewhere/else --with-threads --without-foobar
    building patched package
    installing patched package
    installing patched package-lib
    <BLANKLINE>
Got:
    Uninstalling package.
    Installing package.
    package: [ENV] CFLAGS = -I/sw/include
    package: [ENV] LDFLAGS = -L/sw/lib -L/some/extra/lib
    package: [ENV] TMP = /sample_buildout/parts/package/tmp
    package: Applying patches
    patching file configure
    patching file Makefile.dist
    patched-configure --prefix=/somewhere/else --with-threads --without-foobar
    building patched package
    installing patched package-lib
    installing patched package
    <BLANKLINE>
```

The order of in which `installing patched package-lib` and ` installing patched package` are printed is not always same.

This test does `make install install-lib` with a Makefile containing:

```
all:
        @echo building package

install:
        @echo installing package

install-lib:
        @echo installing package-lib
```

@vpelletier  and I investigated this and he pointed out that order of parts depends is not stable when make is invoked with `-j` option.

Turned out that [on test node](https://nexedi.erp5.net/test_result_module/20180308-55459479/7) `MAKEFLAGS` is  set to `-j12`.


/reviewed-on nexedi/slapos.recipe.cmmi!5