an utility method to control and query supervisor instance used by
slapos using http://supervisord.org/api.html

Unless compiled with --enable-devel=coredump, proftpd's mod_rlimit will
use a prctl(PR_SET_DUMPABLE, 0) syscall[1] which prevents us from
inspecting files open by this process[2]

We don't want to enable debug build and I did not find a way to disable
only this PR_SET_DUMPABLE flag setting.

[1] https://github.com/proftpd/proftpd/blob/v1.3.6/modules/mod_rlimit.c#L580-L583
[2] http://man7.org/linux/man-pages/man5/proc.5.html

Some files and sockets where created in their default location, which
was in the software.
As a result, this software was working only on webrunner or testnodes,
which uses same unix user for software and instance.

To be usable in a "system level" slapos with multiple users,
executables scripts must be 0755

!311 was merged a bit too fast. After test suite was repaired ( the test node was stuck updating the git repository ), [test](https://nexedi.erp5.net/test_result_module/20180507-19395E1F/2) was not passing on testnode, because path was too deep.

While debugging this, I realized this check was wrong, because supervisor [also append the pid](https://github.com/Supervisor/supervisor/blob/70720a0311d868fbf58a31cd26a7147921dcff29/supervisor/http.py#L564-L574) to this path, so the path is even longer. I just checked that we have room for 7 digit pids.

Because path was too long on test node, test now run with `SLAPOS_TEST_WORKING_DIR` set to a slightly shorter path that's enough for this case.

There are some other minor fixes, see individual commits.

/cc @Nicolas @luke @rafael 

/reviewed-on nexedi/slapos!325

For supervisord AF_UNIX socket

This is a convention from
http://community.slapos.org/slapos_community/wiki/slapos-Wiki.Home/developer-SlapOS.Conventions.Guidelines

fix all getInstanceParameterDict typos

supervisord creates a socket with pid in the name, so we have to account
for the space of pid.

Because it's not so easy to know the max size of pids, we just assume 7
places.

otherwise we leak zombie slapproxy processes

 Pin eggsa re mandatory.

  Rename --config to --configuration on apache-dex among others changes.
  This is just to reduce the diff between erp5 and slapos-master

It was missing double slash (//) after schema.

/reviewed-on nexedi/slapos!332

- List of configuration files that are used to generate the hash for the service wrapper's name.

/reviewed-on nexedi/slapos!333

The main idea is to rename the service wrapper using a hash of the corresponding configuration files. In that way, if the config files are updated, the corresponding script file, section in supervisor.conf and service process will be updated accordingly.
- the file name in wrapper_path contains a hash of the corresponding config files
- when config files change, and therefore the hash, the wrapper will be re-created and the corresponding service restarted
- the config files paths will be a parameter in the corresponding buildout section, if it isn't set there won't be hash-check

/reviewed-on nexedi/slapos!326

The exposed apache_custom_http option for custom slave was
incorrectly missing, and apache_custom_http*s* was used instead.

/reviewed-on nexedi/slapos!330

/reviewed-on nexedi/slapos!329

HTTP2 in curl client will allow to test HTTP2 servers.

See: https://curl.haxx.se/docs/http2.html

/reviewed-on nexedi/slapos!328

- destination server gets its own server id
- this removes complexity at the cost of respecting consistency in names
- http-send-name-header is not well supported by haproxy and that it may leads to random errors with not well formated request

/reviewed-on nexedi/slapos!324

Currently cloudooo does not compile with gcc 6.3 ( default gcc from current debian stable ) because of ffmpeg component:

```
    [CC] nestegg/halloc/src/halloc.c.o
In file included from nestegg/halloc/src/halloc.c:19:0:
nestegg/halloc/src/align.h:33:25: error: conflicting types for 'max_align_t'
 typedef union max_align max_align_t;
                         ^~~~~~~~~~~
In file included from nestegg/halloc/src/../halloc.h:18:0,
                 from nestegg/halloc/src/halloc.c:18:
/usr/lib/gcc/x86_64-linux-gnu/6/include/stddef.h:429:3: note: previous declaration of 'max_align_t' was here
 } max_align_t;
   ^~~~~~~~~~~
Makefile:104: recipe for target 'nestegg/halloc/src/halloc.c.o' failed
```

Because we don't use ffmpeg in cloudooo, I'm just temporarily removing it .

/cc @rafael @kazuhiko @gabriel @vpelletier

/reviewed-on nexedi/slapos!264

To be compatible with buildout (and not only `slapos.recipe.template:jinja2`)

See [#20180416-1138FFA](https://nexedi.erp5.net/bug_module/20180416-1138FFA/)

---

Running this buildout:

```
[buildout]
parts=x

[userinfo]
recipe = slapos.cookbook:userinfo

[x]
recipe = plone.recipe.command
command = echo "Hello user id ${userinfo:pw-uid}"
```


In a slapos patched buildout it outputs: `"Hello user id !py!991"` (which means it's not usable for references directly in other buildout parts)

In a "non patched" buildout fail with error:

```
While:
  Installing.
  Getting section x.
  Initializing section x.
  Getting option x:command.
  Getting section userinfo.
  Initializing section userinfo.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
    getattr(buildout, command)(args)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 681, in install
    [self[part]['recipe'] for part in install_parts]
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1376, in _initialize
    self._dosub(k, v)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1431, in _dosub
    v = '$$'.join([self._sub(s, seen) for s in v.split('$$')])
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1495, in _sub
    v = self.buildout[section].get(option, None, seen)
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1278, in __getitem__
    options._initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1386, in _initialize
    self.initialize()
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1395, in initialize
    self.recipe = recipe_class(buildout, name, self)
  File "/tmp/x/eggs/slapos.cookbook-1.0.62-py2.7.egg/slapos/recipe/userinfo.py", line 13, in __init__
    options['pw-uid'] = pinfo.pw_uid
  File "/tmp/x/env/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1520, in __setitem__
    raise TypeError('Option values must be strings', value)
TypeError: ('Option values must be strings', 991)
```

/cc @Nicolas @alain.takoudjou  @vpelletier 

/reviewed-on nexedi/slapos!321

Proftpd with sftp and virtual users

http://www.proftpd.org/docs/

# Features

 * sftp only is enabled
 * partially uploadloaded are not visible thanks to [`HiddenStores`](http://proftpd.org/docs/directives/linked/config_ref_HiddenStores.html) ( in fact they are, but name starts with `.` )
 * 5 failed login attempts will cause the host to be temporary banned


# TODO

 * only password login is enabled. enabling [`SFTPAuthorizedUserKeys`](http://www.proftpd.org/docs/contrib/mod_sftp.html#SFTPAuthorizedUserKeys) seems to break password only login
 * log rotation
 * make sure SFTPLog is useful (seems very verbose and does not contain more than stdout)
 * make it easier to manage users ( using `mod_auth_web` against an ERP5 endpoint or accepting a list of user/password as instance parameter )
 * allow configuring webhooks when new file is uploaded



----

depends on nexedi/slapos.core!41

/reviewed-on nexedi/slapos!311

_ (underscore) is not supported by apache as domain name
(https://bugzilla.redhat.com/show_bug.cgi?id=1410130), so avoid using it.

/reviewed-on !320

The build of nodejs uses python, and python leaking from the system can lead
to unexpected results, thus use provided from SlapOS.

/reviewed-on nexedi/slapos!319

The correct is use require and not required.