/reviewed-on nexedi/slapos!480

See sqlite-mixin.cfg to skip the building of MariaDB.

Use tap information from partition resource (coming from slapconfiguration recipe) instead of old (and deprecated softwaretype recipe).

Using
https://python-jsonschema.readthedocs.io/en/latest/validate/#versioned-validators
instead of a local copy of json schema

we are using trafficserver nowadays, varnish recipe is deprecated

As now the signature is parameterized (in the case of the webrunner),
computing the signature may require not-webrunner-based tools (like
a binary belonging to the SR. Then it can only be done on the
export and the import.

as always, using wrapper is easier and more flexible

We use jstestnode

Using slapos.cookbook:wrappper is easier

was dead code

This release should be now stable enough for release

/reviewed-on nexedi/slapos!391

It is combination of requestoptional and serialised.

In order to avoid needless inheritance create JSONCodec and use it, also
with renaming Serialised to RequestJSONEncoded.

Also provide a test asserting that important log is seen in case if no expected
return is set.

Add testfixtures module to catch logging.

This reverts commit 2c79bee1 because it depends on rdiff-backup version 1.3.4nxd4 which is not stable yet.

With new rdiff-backup version, it is possible to speed up push process by ignoring UID/GID changes (as it will always be different because of different partitions)

/reviewed-on nexedi/slapos!378

Strip earlier, as stripping so late leads to correct self.passwd
but incorrect passwd, thus 3rd party applications storing passwords
with additional newlines are still resulting in bad passwords.

/reviewed-on nexedi/slapos!366

The goal is to drop slapos.cookbook:erp5.test

No need for a recipe to do a bit of url parsing and generating 2 wrappers.
Also, drop unused & empty "host-common" section.

- stack/erp5: Add path to all binary links in PATH
- stack/erp5: Add single quotation marks to not break parameters in many lines
- stack/erp5: Fix path to avoid one section to overwrite the file from another section
- stack/erp5: pass cloudooo host instead of the IP address. The IP address and cloudooo hostname are well in $SLAPPART/etc/hosts
- stack/erp5: Use use-ipv6 instance parameter to control where caucased listens when locally deployed.
  Drop incomplete 6tunnel usage on balancer toward zopes.

Tests are somehow happy test_result_module/20180626-62AD32BB test_result_module/20180626-70AC16FA

If no objections I would like to merge tomorrow but feel free to press merge for me :)

The goal is rebase nexedi/slapos!318 to fix the conflict and merge after tomorrow.

/cc @rafael @vpelletier 

/reviewed-on nexedi/slapos!347

This prevent outdated instances to fail resilience, so it makes this change backward compatible.

The main idea is to rename the service wrapper using a hash of the corresponding configuration files. In that way, if the config files are updated, the corresponding script file, section in supervisor.conf and service process will be updated accordingly.
- the file name in wrapper_path contains a hash of the corresponding config files
- when config files change, and therefore the hash, the wrapper will be re-created and the corresponding service restarted
- the config files paths will be a parameter in the corresponding buildout section, if it isn't set there won't be hash-check

/reviewed-on nexedi/slapos!326

/reviewed-on nexedi/slapos!329

To be compatible with buildout (and not only `slapos.recipe.template:jinja2`)

See [#20180416-1138FFA](https://nexedi.erp5.net/bug_module/20180416-1138FFA/)

---

Running this buildout:

```
[buildout]
parts=x

[userinfo]
recipe = slapos.cookbook:userinfo

[x]
recipe = plone.recipe.command
command = echo "Hello user id ${userinfo:pw-uid}"
```


In a slapos patched buildout it outputs: `"Hello user id !py!991"` (which means it's not usable for references directly in other buildout parts)

In a "non patched" buildout fail with error:

```
While:
  Installing.
  Getting section x.
  Initializing section x.
  Getting option x:command.
  Getting section userinfo.
  Initializing section userinfo.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
    getattr(buildout, command)(args)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 681, in install
    [self[part]['recipe'] for part in install_parts]
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1376, in _initialize
    self._dosub(k, v)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1431, in _dosub
    v = '$$'.join([self._sub(s, seen) for s in v.split('$$')])
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1495, in _sub
    v = self.buildout[section].get(option, None, seen)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1386, in _initialize
    self.initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1395, in initialize
    self.recipe = recipe_class(buildout, name, self)
  File "/tmp/x/eggs/slapos.cookbook-1.0.62-py2.7.egg/slapos/recipe/userinfo.py", line 13, in __init__
    options['pw-uid'] = pinfo.pw_uid
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1520, in __setitem__
    raise TypeError('Option values must be strings', value)
TypeError: ('Option values must be strings', 991)
```

/cc @Nicolas @alain.takoudjou  @vpelletier 

/reviewed-on nexedi/slapos!321

This recipe will be used to generate promise in etc/plugin dir. It solve the problem of promise eggs
dependencies and allow to set custom parameter to use in promise.

The generated script will looks like:

    import sys
    sys.path[0:0] = [
      ...
    ]

    extra_config_dict = {
      'KEY': 'VALUE'
    }

    CONTENT

CONTENT is a python code, the expected content looks like:

    from namespace.module import RunPromise

then the promise section in buildout will be something like:

    [my-promise]
    recipe = slapos.cookbook:promise.plugin
    eggs =
      NAME
      ...
    output = OUTPUT
    content =
      from namespace.module import RunPromise

    config-KEY = VALUE

This function factorize the code to instanciate a recipe in a fake
buildout.
The new feature is that if running in a buildout directory, the recipe
will reuse the eggs from this buildout instead of trying to install eggs
again.

we now use slapos.cookbook:wrapper instead

We are supposed to have valid certificates and if really needed,
slapos.recipe.build:gitclone can ignore it.

reviewed on nexedi/slapos!300

thx @jerome

Current version of wendelin.core is limited in that there's no cooperation
in memory allocation between several processes sharing the same /dev/shm.
Over time, the distribution of memory among processes becomes more and more
inequal until some of them get killed due to allocation failure.

This is a temporary solution that relies on user namespaces.

This moves most of the wrapper recipe to GenericBaseRecipe so that
createWrapper automatically switches to Python scripts for complex cases.

Recipes should not use slapos.recipe.librecipe.execute.generic_exec directly
anymore.

It was used in only 2 places and createPythonScript has no equivalent
(maybe reusing 'initialization' from easy_install would produce something
readable enough). For the moment, let's drop this parameter, so that
createWrapper can switch to Python wrappers for complex cases.

There was an inconsistency between the 2 ways of creating executable wrappers:
- createWrapper: parameters-extra defaults to False
- slapos.recipe.librecipe.execute.generic_exec: no option, enabled

An option for that is not worth the effort.
Let's always enable even if it's meaningless.

/software/ and /stack/ will be cleaned up once a new version of this egg
is released.

The workaround for shebang size limitation is removed because it's redundant
with what is done by zc.buildout.easy_install and slapos.recipe.cmmi. This also
fix the issue that a process has a bad name even when the workaround is useless.

kumofs is even abandonned software.