Based on Jerome's experiment from
https://lab.nexedi.com/nexedi/slapos/blob/master/software/proftpd/test/utils.py

Main changes:

 * tearDownClass is called in case of errors in setUpClass (mimic addCleanup
   instance method)
 * the setUpClass is split to smaller methods for ease of further development
 * full instance path is exposed for test introspection

In order to follow proper way, move caddy to the component and reuse it.

- destination server gets its own server id
- this removes complexity at the cost of respecting consistency in names
- http-send-name-header is not well supported by haproxy and that it may leads to random errors with not well formated request

/reviewed-on nexedi/slapos!324

Currently cloudooo does not compile with gcc 6.3 ( default gcc from current debian stable ) because of ffmpeg component:

```
    [CC] nestegg/halloc/src/halloc.c.o
In file included from nestegg/halloc/src/halloc.c:19:0:
nestegg/halloc/src/align.h:33:25: error: conflicting types for 'max_align_t'
 typedef union max_align max_align_t;
                         ^~~~~~~~~~~
In file included from nestegg/halloc/src/../halloc.h:18:0,
                 from nestegg/halloc/src/halloc.c:18:
/usr/lib/gcc/x86_64-linux-gnu/6/include/stddef.h:429:3: note: previous declaration of 'max_align_t' was here
 } max_align_t;
   ^~~~~~~~~~~
Makefile:104: recipe for target 'nestegg/halloc/src/halloc.c.o' failed
```

Because we don't use ffmpeg in cloudooo, I'm just temporarily removing it .

/cc @rafael @kazuhiko @gabriel @vpelletier

/reviewed-on nexedi/slapos!264

To be compatible with buildout (and not only `slapos.recipe.template:jinja2`)

See [#20180416-1138FFA](https://nexedi.erp5.net/bug_module/20180416-1138FFA/)

---

Running this buildout:

```
[buildout]
parts=x

[userinfo]
recipe = slapos.cookbook:userinfo

[x]
recipe = plone.recipe.command
command = echo "Hello user id ${userinfo:pw-uid}"
```


In a slapos patched buildout it outputs: `"Hello user id !py!991"` (which means it's not usable for references directly in other buildout parts)

In a "non patched" buildout fail with error:

```
While:
  Installing.
  Getting section x.
  Initializing section x.
  Getting option x:command.
  Getting section userinfo.
  Initializing section userinfo.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
    getattr(buildout, command)(args)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 681, in install
    [self[part]['recipe'] for part in install_parts]
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1376, in _initialize
    self._dosub(k, v)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1431, in _dosub
    v = '$$'.join([self._sub(s, seen) for s in v.split('$$')])
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1495, in _sub
    v = self.buildout[section].get(option, None, seen)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1386, in _initialize
    self.initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1395, in initialize
    self.recipe = recipe_class(buildout, name, self)
  File "/tmp/x/eggs/slapos.cookbook-1.0.62-py2.7.egg/slapos/recipe/userinfo.py", line 13, in __init__
    options['pw-uid'] = pinfo.pw_uid
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1520, in __setitem__
    raise TypeError('Option values must be strings', value)
TypeError: ('Option values must be strings', 991)
```

/cc @Nicolas @alain.takoudjou  @vpelletier 

/reviewed-on nexedi/slapos!321

Proftpd with sftp and virtual users

http://www.proftpd.org/docs/

# Features

 * sftp only is enabled
 * partially uploadloaded are not visible thanks to [`HiddenStores`](http://proftpd.org/docs/directives/linked/config_ref_HiddenStores.html) ( in fact they are, but name starts with `.` )
 * 5 failed login attempts will cause the host to be temporary banned


# TODO

 * only password login is enabled. enabling [`SFTPAuthorizedUserKeys`](http://www.proftpd.org/docs/contrib/mod_sftp.html#SFTPAuthorizedUserKeys) seems to break password only login
 * log rotation
 * make sure SFTPLog is useful (seems very verbose and does not contain more than stdout)
 * make it easier to manage users ( using `mod_auth_web` against an ERP5 endpoint or accepting a list of user/password as instance parameter )
 * allow configuring webhooks when new file is uploaded



----

depends on nexedi/slapos.core!41

/reviewed-on nexedi/slapos!311

_ (underscore) is not supported by apache as domain name
(https://bugzilla.redhat.com/show_bug.cgi?id=1410130), so avoid using it.

/reviewed-on nexedi/slapos!320

The build of nodejs uses python, and python leaking from the system can lead
to unexpected results, thus use provided from SlapOS.

/reviewed-on nexedi/slapos!319

The correct is use require and not required.

fix getInstanceParameterDict typo

/reviewed-by: TrustMe

To run tests from software/proftpd/test

To run tests locally by `python setup.py test`

This is a new kind of test suite that compile software, request instance
with specific parameters and check that the instance works as expected.

@alain, @rafael 

Test is here: https://nexedi.erp5.net/test_result_module/20180412-3688F368/2/view

Its failures which are different from  master one (https://nexedi.erp5.net/test_result_module/20180412-3501D1A/1) seems unrelated to my changes thus I propose to emrge it in and include in next webrunner release.

/reviewed-on nexedi/slapos!317

This reverts commit 1bdac5da.

This commit breaks webrunner resilience importer. This is not the proper fix

Because of this: https://lab.nexedi.com/nexedi/slapos.core/blob/master/slapos/grid/promise/generic.py#L44

When slapgrid run promises, it now save promise logs. Thus we need to logrotate them.
Thanks @Alain.