Update Release Candidate

CHANGES.rst

-Changes
-=======
-
-1.0.138 (2020-03-03)
---------------------
-
-- Update postgresql recipe for postgres version 10 and later
-
-1.0.124 (2020-01-30)
---------------------
-
-- cookbook:erp5testnode: new shared_part_list option
-
-
-1.0.123 (2019-10-03)
---------------------
-
-- wrapper: accept hash-files already existing inside the partition directory
-
-
-1.0.122 (2019-09-24)
---------------------
-
-- wrapper: add hash-existing-files option
-
-
-1.0.121 (2019-09-12)
---------------------
-
-- generic.mysql.wrap_update_mysql: prepare for MariaDB 10.4
-- publish-early: process -init entries in specified order
-- Partial support of Python 3
-- Remove unused generic.mysql recipe
-
-
-1.0.119 (2019-08-14)
---------------------
-
- * publish_early: rework API
-
-
-1.0.118 (2019-08-13)
---------------------
-
-* NEO: new recipe to fix/optimize propagation of the 'masters' parameter
-* publish_early: new '-update' option, keep published values out of buildout installed file
-* publish: new -publish option to list explicitly options to publish
-* re6stnet: Fix typo
-* librecipe: Try to reuse existing file to avoid excessive IO on update and other minor optimisations
-* certificate_authority: unique_subject = no
-* wrapper: handle "=" in environment variables' content
- 
-
-1.0.92 (2019-02-21)
--------------------
-
-* plugin recipe: improve recipe to correctly generate promise with parameters which contain control characters
-
-1.0.85 (2018-12-28)
------------------------
-
-* Drop ``slapos.recipe:xvfb``, use simple ``slapos.recipe:wrapper`` instead.
-* Drop ``slapos.recipe:seleniumrunner`` and ``slapos.recipe:firefox``, they
-  were not used.
-* Encode unicode to UTF-8 on ``slapos.recipe:request`` and 
-  ``slapos.recipe:slapconfiguration`` 
-
-1.0.75 (2018-09-04)
--------------------
-
-* erp5_test: stop using erp5_test recipe
-* random: fix password generation with newlines
-* erp5testnode: enable password authentication for scalability test system
-* pbs: Ignore numerical IDs (UID/GID) when push
-* request: add requestoptional.serialised
-
-1.0.65 (2018-06-22)
--------------------
-
-* Automatic restart of services when configuration changes
-* erp5_test: define cloudooo-retry-count value in test
-* userinfo: expose values as string
-
-1.0.62 (2018-04-10)
--------------------
-
-* promise.plugin: new recipe for python promises plugin script generation
-
-1.0.59 (2018-03-15)
--------------------
-* librecipe.execute: fix convert process arguments to string formatting.
-
-1.0.58 (2018-03-14)
--------------------
-
-* generic.mysql: unregister UDFs before (re)adding UDFs
-* Remove obsolete/unused recipes.
-* neoppod: add support for new --dedup storage option.
-* Use inotify-simple instead of inotifyx.
-* erp5.test: remove duplicated code.
-* librecipe: bugfixes found by pylint, performance improvements, and major
-  refactoring of executable wrappers.
-* GenericBaseRecipe.createWrapper: remove 'comments' parameter.
-* Drop the 'parameters-extra' option and always forward extra parameters.
-* wrapper: new 'private-dev-shm' option (useful for wendelin.core).
-* generic.cloudooo: OnlyOffice converter support odf.
-* erp5testnode: don't tell git to ignore SSL errors.
-
-1.0.53 (2017-09-13)
--------------------
-
-* check_port_listening: workaround for shebang limitation, reduce to a single file
-* erp5.test: pass new --conversion_server_url option to runUnitTest
-
-1.0.52 (2017-07-04)
--------------------
-
-* wrapper: Add option to reserve CPU core
-* slapconfiguration: Recipe reads partitions resource file
-* neoppod: add support for new --disable-drop-partitions storage option
-* random: Fix the monkeypatch in random.py to incorporate the recent changes in buildout 'get' function
-* random: Add Integer recipe.
-* librecipe.execute: Notify on file moved
-* zero_knowledge: allow to set destination folder of configuration file
-
-
-1.0.50 (2017-04-18)
--------------------
-
-* pbs: Do not parallelize calculus when the heaviest task is IO
-* re6st-registry: Refactor integration with re6st registry
-* erp5testnode: make shellinabox reusing password file of pwgen
-
-1.0.48 (2017-01-31)
--------------------
-
-* random-recipe: add option create-once to prevent storage file deletion by buildout
-
-1.0.45 (2017-01-09)
--------------------
-
-* recipe: set default timeout of check url promise to 20 seconds
-
-1.0.44 (2016-12-30)
--------------------
-
-* pbs: handles the fact that some parameters are not present when slaves are down
-* recipe: allow usage of pidfile in wrapper recipe
-* sshd: fix generation of authorized_keys
-
-1.0.43 (2016-11-24)
--------------------
-
-* pbs: fixes trap command for dash intepreter
-* pbs: remove infinite loops from pbs scripts.
-* random.py: new file containing recipes generating random values.
-* testnode: disallow frontend access to all folders, avoiding publishing private repositories
-
-1.0.41 (2016-10-26)
--------------------
-
-* dcron: new parameter to get a random time, with a frequency of once a day
-* softwaretype: fix parse error on '+ =' when using buildout 2
-* pbs: General Improvement and fixes.
-
-1.0.35 (2016-09-19)
--------------------
-
-* pbs: fix/accelerates deployment of resilient instances
-* recipe: new recipe to get a free network port
-* Remove url-list parameter to download fonts from fontconfig instance
-
-1.0.31 (2016-05-30)
--------------------
-
-* Implement cross recipe cache for registerComputerPartition
-* Fix workaround for long shebang (place script on bin)
-
-1.0.30 (2016-05-23)
--------------------
-
-* Implement a workarround for long shebang
-* Implement Validation for user inputs ssl certificates
-
-1.0.25 (2016-04-15)
--------------------
-
-* fixup slap configuration: provide instance and root instance title
-
-1.0.22 (2016-04-01)
--------------------
-
-* slap configuration: provide instance and root instance title
-
-1.0.16 (2015-10.27)
--------------------
-
-* kvm recipe: fix bugs dowload image and disk creation
-
-1.0.14 (2015-10.26)
--------------------
-
-* kvm recipe: Allow to set keyboard layout language used by qemu and VNC
-* simplehttpserver-recipe: fix encoding error
-
-For older entries, see https://lab.nexedi.com/nexedi/slapos/blob/a662db75cc840df9d4664a9d048ef28ebfff4d50/CHANGES.rst

component/git/buildout.cfg

@@ -18,8 +18,8 @@ parts =
 [git]
 recipe = slapos.recipe.cmmi
 shared = true
-url = https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.23.0.tar.xz
-md5sum = 93ee0f867f81a39e0ef29eabfb1d2c5b
+url = https://www.kernel.org/pub/software/scm/git/git-2.25.1.tar.xz 
+md5sum = 92bf65673b4fc08b64108d807f36f4d9
 configure-options =
   --with-curl=${curl:location}
   --with-openssl=${openssl:location}

component/nodejs/buildout.cfg

@@ -31,6 +31,10 @@ md5sum = 4ddc1daff327d7e6f63da57fdfc24f55
 version = v8.6.0
 md5sum = 0c95e08220667d8a18b97ecec8218ac6
 
+[nodejs-8.12.0]
+<= nodejs-base
+version = v8.12.0
+md5sum = 5690333b77964edf81945fc724f6ea85
 
 [nodejs-base]
 # Server-side Javascript.

component/ruby/buildout.cfg

@@ -36,6 +36,10 @@ md5sum = f18ed96bd1d5890f97a17d0d17aaefdd
 url = http://ftp.ruby-lang.org/pub/ruby/2.2/ruby-2.2.2.tar.xz
 md5sum = dbce9b9d79d90f213ba8d448b0b6ed86
 
+[ruby2.3]
+<= ruby-common
+url = http://ftp.ruby-lang.org/pub/ruby/2.3/ruby-2.3.8.tar.xz
+md5sum = 927e1857f3dd5a1bdec26892dbae2a05
 
 [ruby]
 <= ruby2.2

setup.py

@@ -28,10 +28,9 @@ from setuptools import setup, find_packages
 import glob
 import os
 
-version = '1.0.138'
+version = '1.0.139'
 name = 'slapos.cookbook'
-long_description = open("README.rst").read() + "\n" + \
-    open("CHANGES.rst").read() + "\n"
+long_description = open("README.rst").read()
 
 for f in sorted(glob.glob(os.path.join('slapos', 'recipe', 'README.*.rst'))):
   long_description += '\n' + open(f).read() + '\n'

slapos/recipe/librecipe/execute.py

 from __future__ import print_function
 
+import errno
 import sys
 import os
 import signal
@@ -50,7 +51,7 @@ def _libc():
   return mount, unshare
 
 def generic_exec(args, extra_environ=None, wait_list=None,
-                 pidfile=None, reserve_cpu=False, private_dev_shm=None,
+                 pidfile=None, reserve_cpu=False, private_tmpfs=(),
                  #shebang_workaround=False, # XXX: still needed ?
                  ):
   args = list(args)
@@ -83,7 +84,7 @@ def generic_exec(args, extra_environ=None, wait_list=None,
   if wait_list:
     _wait_files_creation(wait_list)
 
-  if private_dev_shm:
+  if private_tmpfs:
     mount, unshare = _libc()
     CLONE_NEWNS   = 0x00020000
     CLONE_NEWUSER = 0x10000000
@@ -93,7 +94,13 @@ def generic_exec(args, extra_environ=None, wait_list=None,
     with open('/proc/self/setgroups', 'wb') as f: f.write('deny')
     with open('/proc/self/uid_map',   'wb') as f: f.write('%s %s 1' % (uid, uid))
     with open('/proc/self/gid_map',   'wb') as f: f.write('%s %s 1' % (gid, gid))
-    mount('tmpfs', '/dev/shm', 'tmpfs', 0, 'size=' + private_dev_shm)
+    for size, path in private_tmpfs:
+      try:
+        os.mkdir(path)
+      except OSError as e:
+        if e.errno != errno.EEXIST:
+          raise
+      mount('tmpfs', path, 'tmpfs', 0, 'size=' + size)
 
   if extra_environ:
     env = os.environ.copy()

slapos/recipe/librecipe/generic.py

@@ -43,7 +43,8 @@ from six.moves.urllib.parse import urlunparse
 
 
 import pkg_resources
-import zc.buildout
+from zc.buildout import easy_install, UserError
+from zc.recipe.egg import Egg
 
 from slapos.recipe.librecipe import shlex
 
@@ -85,8 +86,7 @@ class GenericBaseRecipe(object):
 
   def getWorkingSet(self):
     """If you want do override the default working set"""
-    egg = zc.recipe.egg.Egg(self.buildout, 'slapos.cookbook',
-                                  self.options.copy())
+    egg = Egg(self.buildout, 'slapos.cookbook', self.options.copy())
     requirements, ws = egg.working_set()
     return ws
 
@@ -156,10 +156,20 @@ class GenericBaseRecipe(object):
     args = itertools.chain(map(repr, args),
                            map('%s=%r'.__mod__, six.iteritems(kw)))
 
-    return zc.buildout.easy_install.scripts(
+    return easy_install.scripts(
       [(filename, module, function)], self._ws, sys.executable,
       path, arguments=', '.join(args))[0]
 
+  def parsePrivateTmpfs(self):
+    private_tmpfs = []
+    for line in (self.options.get('private-tmpfs') or '').splitlines():
+      if line:
+        x = line.split(None, 1)
+        if len(x) != 2:
+          raise UserError("failed to split %r into size and path" % line)
+        private_tmpfs.append(tuple(x))
+    return private_tmpfs
+
   def createWrapper(self, path, args, env=None, **kw):
     """Create a wrapper script for process replacement"""
     assert args

slapos/recipe/neoppod.py

@@ -88,7 +88,9 @@ class NeoBaseRecipe(GenericBaseRecipe):
         )
     args += self._getOptionList()
     args += shlex.split(options.get('extra-options', ''))
-    return self.createWrapper(options['wrapper'], args)
+    private_tmpfs = self.parsePrivateTmpfs()
+    kw = {'private_tmpfs': private_tmpfs} if private_tmpfs else {}
+    return self.createWrapper(options['wrapper'], args, **kw)
 
   def _getBindingAddress(self):
     options = self.options

slapos/recipe/postgres/__init__.py

@@ -91,16 +91,13 @@ class Recipe(GenericBaseRecipe):
                 # run we won't update it.
                 shutil.rmtree(pgdata)
                 raise
-
-
-
-        # install() methods usually return the pathnames of managed files.
-        # If they are missing, they will be rebuilt.
-        # In this case, we already check for the existence of pgdata,
-        # so we don't need to return anything here.
+        else:
+            self.createConfig()
+            self.createRunScript()
 
         return []
 
+    update = install
 
     def check_exists(self, path):
         if not os.path.isfile(path):

slapos/recipe/wrapper.py

@@ -38,7 +38,7 @@ class Recipe(GenericBaseRecipe):
     :param lines hash-files: list of buildout-generated files to be checked by hash
     :param lines hash-existing-files: list of existing files to be checked by hash
     :param str pidfile: path to pidfile ensure exclusivity for the process
-    :param str private-dev-shm: size of private /dev/shm, using user namespaces
+    :param lines private-tmpfs: list of "<size> <path>" private tmpfs, using user namespaces
     :param bool reserve-cpu: command will ask for an exclusive CPU core
     """
 
@@ -72,13 +72,14 @@ class Recipe(GenericBaseRecipe):
           raise UserError(
             "hash-files must only list files that are generated by buildout:"
             "\n  " + "\n  ".join(self._existing))
-        args = shlex.split(self.options['command-line'])
-        wait_files = self.options.get('wait-for-files')
-        pidfile = self.options.get('pidfile')
-        private_dev_shm = self.options.get('private-dev-shm')
+        options = self.options
+        args = shlex.split(options['command-line'])
+        wait_files = options.get('wait-for-files')
+        pidfile = options.get('pidfile')
+        private_tmpfs = self.parsePrivateTmpfs()
 
         environment = {}
-        for line in (self.options.get('environment') or '').splitlines():
+        for line in (options.get('environment') or '').splitlines():
           line = line.strip()
           if line:
             k, v = line.split('=', 1)
@@ -89,9 +90,9 @@ class Recipe(GenericBaseRecipe):
           kw['wait_list'] = wait_files.split()
         if pidfile:
           kw['pidfile'] = pidfile
-        if private_dev_shm:
-          kw['private_dev_shm'] = private_dev_shm
-        if self.isTrueValue(self.options.get('reserve-cpu')):
+        if private_tmpfs:
+          kw['private_tmpfs'] = private_tmpfs
+        if self.isTrueValue(options.get('reserve-cpu')):
           kw['reserve_cpu'] = True
         return self.createWrapper(self.getWrapperPath(),
                                   args, environment, **kw)

software/gitlab/buildout.hash.cfg

@@ -14,7 +14,7 @@
 # not need these here).
 [instance.cfg]
 filename = instance.cfg.in
-md5sum = 36252abb4d857da08d62bf3eb26faae1
+md5sum = dc3f318e8a3aa7a59f9394118543e9e3
 
 [watcher]
 _update_hash_filename_ = watcher.in
@@ -34,27 +34,31 @@ md5sum = 7782f5c5d75663c2586e28d029c51e49
 
 [gitlab-parameters.cfg]
 _update_hash_filename_ = gitlab-parameters.cfg
-md5sum = 8f4537cb8a0c9a8e0058c30cb687681c
+md5sum = c2e23c0f7baa1633df0436ca4e728424
 
 [gitlab-shell-config.yml.in]
 _update_hash_filename_ = template/gitlab-shell-config.yml.in
-md5sum = 58c09b1e609f903e483a76fe9e57366c
+md5sum = 52d18b521b8cd16352fc88b1e1d79d53
 
 [gitlab-unicorn-startup.in]
 _update_hash_filename_ = gitlab-unicorn-startup.in
-md5sum = a9cb347f60aad3465932fd36cd4fe25d
+md5sum = aff91edaf9786c213db8ea703ab3571e
 
 [gitlab.yml.in]
 _update_hash_filename_ = template/gitlab.yml.in
-md5sum = 0ddf4093dcf4427e5a160707e6017950
+md5sum = f4cc0bc898b8d59010d61473e2adc53b
+
+[gitaly-config.toml.in]
+_update_hash_filename_ = template/gitaly-config.toml.in
+md5sum = 056d7ed09e1bf20d022d3ef6b9363e00
 
 [instance-gitlab.cfg.in]
 _update_hash_filename_ = instance-gitlab.cfg.in
-md5sum = d794631233626d03b04894ca6b6d8496
+md5sum = f5e7f9717eaa999fbf11ce4b6c1abb1c
 
 [instance-gitlab-export.cfg.in]
 _update_hash_filename_ = instance-gitlab-export.cfg.in
-md5sum = 319d7dbe3ad9b260c1e292cfc0d13b11
+md5sum = 2af7dcf63f74e5edc53a3ff11fa4989b
 
 [instance-gitlab-test.cfg.in]
 _update_hash_filename_ = instance-gitlab-test.cfg.in
@@ -66,11 +70,11 @@ md5sum = a56a44e96f65f5ed20211bb6a54279f4
 
 [nginx-gitlab-http.conf.in]
 _update_hash_filename_ = template/nginx-gitlab-http.conf.in
-md5sum  = e74695aa1be60f0ffac64ddbe1c8eaf1
+md5sum = 79d2b4e8a32abf7a74a3d4528844c593
 
 [nginx.conf.in]
 _update_hash_filename_ = template/nginx.conf.in
-md5sum = 1374f38ab6f295b850d45ea0019ec05d
+md5sum = 8c904510eb39dc212204f68f2b81b068
 
 [rack_attack.rb.in]
 _update_hash_filename_ = template/rack_attack.rb.in
@@ -82,7 +86,7 @@ md5sum = 7c89a730889e3224548d9abe51a2d719
 
 [smtp_settings.rb.in]
 _update_hash_filename_ = template/smtp_settings.rb.in
-md5sum = 4e1ced687a86e4cfff2dde91237e3942
+md5sum = e2144b03f7247636143c65dc81550d75
 
 [template-gitlab-resiliency-restore.sh.in]
 _update_hash_filename_ = template/template-gitlab-resiliency-restore.sh.in
@@ -90,4 +94,4 @@ md5sum = 590fcadf26085fdd17487175bc0a469d
 
 [unicorn.rb.in]
 _update_hash_filename_ = template/unicorn.rb.in
-md5sum = 83921db1835d9e81cbbe808631cc40a9
+md5sum = 67728235a2c4c9425c80f0c856749885

software/gitlab/gitlab-parameters.cfg

@@ -45,7 +45,7 @@ configuration.default_projects_features.issues          = true
 configuration.default_projects_features.merge_requests  = true
 configuration.default_projects_features.wiki            = true
 configuration.default_projects_features.snippets        = true
-#configuration.default_projects_features.builds          = false
+configuration.default_projects_features.builds          = true
 
 configuration.webhook_timeout           = 10
 
@@ -102,6 +102,10 @@ configuration.nginx_gzip_proxied        = any
 configuration.nginx_gzip_types          = text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript application/json
 configuration.nginx_keepalive_timeout   = 65
 configuration.nginx_header_allow_origin = $http_origin
+configuration.nginx_hsts_max_age        = 31536000
+configuration.nginx_hsts_include_subdomains = false
+configuration.nginx_gzip_enabled        = true
+
 
 # configuring trusted proxies
 # GitLab is behind a reverse proxy, so we don't want the IP address of the proxy

software/gitlab/gitlab-unicorn-startup.in

@@ -27,7 +27,7 @@ psql() {
 # ( first quering PG several times waiting a bit till postgresql is started and ready )
 tpgwait=5
 while true; do
-    pgtables="$(psql -c '\d')" && break
+    pgtables="$(psql -c '\d' 2>&1)" && break
     tpgwait=$(( $tpgwait - 1 ))
     test $tpgwait = 0 && die "pg query problem"
     echo "I: PostgreSQL is not ready (yet ?); will retry $tpgwait times..." 1>&2
@@ -38,10 +38,11 @@ echo "I: PostgreSQL ready." 1>&2
 # make sure pg_trgm extension is enabled for gitlab db
 psql -c 'CREATE EXTENSION IF NOT EXISTS pg_trgm;'   || die "pg_trgm setup failed"
 
-if echo "$pgtables" | grep -q '^No relations found' ; then
+if echo "$pgtables" | grep -q '^Did not find any relations' ; then
     $RAKE db:schema:load db:seed_fu  || die "initial db setup failed"
 fi
 
+
 # re-build ssh keys
 # (we do not use them - just for cleannes)
 force=yes $RAKE gitlab:shell:setup   || die "gitlab:shell:setup failed"

software/gitlab/gowork.cfg

@@ -6,7 +6,6 @@ depends_gitfetch  =
     ${go_github.com_pkg_errors:recipe}
     ${go_lab.nexedi.com_kirr_git-backup:recipe}
     ${go_lab.nexedi.com_kirr_go123:recipe}
-    ${go_gitlab.com_gitlab-org_gitlab-workhorse:recipe}
 
 
 [go_github.com_libgit2_git2go]
@@ -14,7 +13,7 @@ depends_gitfetch  =
 go.importpath = github.com/libgit2/git2go
 repository    = https://github.com/libgit2/git2go.git
 # branch 'next' is required by git-backup
-revision = next-g53594d7581617dbae7bb5960b4ac5f0ff513c184
+revision = next-g5d0a4c752a74258a5f42e40fccd2908ac4e336b8
 
 [go_github.com_pkg_errors]
 <= go-git-package
@@ -26,16 +25,10 @@ revision      = v0.8.0-12-g816c908556
 <= go-git-package
 go.importpath = lab.nexedi.com/kirr/git-backup
 repository    = https://lab.nexedi.com/kirr/git-backup.git
-revision = cc6ac54f451dfa6e343d6340dcfa25aa6eac9565
+revision      = 3f6c4deec8834bdcd2c28c7c5eeacd8211e759b5
 
 [go_lab.nexedi.com_kirr_go123]
 <= go-git-package
 go.importpath = lab.nexedi.com/kirr/go123
 repository    = https://lab.nexedi.com/kirr/go123.git
-revision      = d9250d6332
-
-[go_gitlab.com_gitlab-org_gitlab-workhorse]
-<= go-git-package
-go.importpath = gitlab.com/gitlab-org/gitlab-workhorse
-repository    = https://lab.nexedi.com/nexedi/gitlab-workhorse.git
-revision      = v1.3.0-8-g5f44f59cbb
\ No newline at end of file
+revision      = 56bf8f815a
\ No newline at end of file

software/gitlab/instance-gitlab-export.cfg.in

@@ -44,6 +44,7 @@ command	= ${exporter:wrapper-path}
 recipe = collective.recipe.template
 input = inline: gitlab-shell-work*
   gitlab-work*
+  var/log/**
   var/backup/**
   var/repositories*
   var/repositories/**

software/gitlab/instance.cfg.in

@@ -27,6 +27,7 @@ context =
     import pwd pwd
     import multiprocessing multiprocessing
 
+    key bin_directory           buildout:bin-directory
     key eggs_directory          buildout:eggs-directory
     key develop_eggs_directory  buildout:develop-eggs-directory
     raw gitlab_repository_location          ${gitlab-repository:location}
@@ -36,11 +37,13 @@ context =
     raw bash_bin                    ${bash:location}/bin/bash
     raw bzip2_location              ${bzip2:location}
     raw bundler_4gitlab             ${bundler-4gitlab:bundle}
+    raw bundler_1_17_3_dir          ${bundler-4gitlab:bundle1.17.3}
     raw coreutils_location          ${coreutils:location}
     raw curl_bin                    ${curl:location}/bin/curl
     raw dcron_bin                   ${dcron-output:crond}
     raw git                         ${git:location}/bin/git
     raw git_location                ${git:location}
+    raw gitaly_location             ${gitaly-repository:location}
     raw gitlab_export               ${gitlab-export:rendered}
     raw gitlab_workhorse            ${gowork:bin}/gitlab-workhorse
     raw gopath_bin                  ${gowork:bin}
@@ -51,14 +54,15 @@ context =
     raw logrotate_bin               ${logrotate:location}/usr/sbin/logrotate
     raw nginx_bin                   ${nginx-output:nginx}
     raw nginx_mime_types            ${nginx-output:mime}
-    raw node_bin_location           ${nodejs-8.6.0:location}/bin/
+    raw node_bin_location           ${nodejs-8.12.0:location}/bin/
     raw openssl_bin                 ${openssl-output:openssl}
-    raw postgresql_location         ${postgresql92:location}
+    raw postgresql_location         ${postgresql10:location}
     raw redis_binprefix             ${redis28:location}/bin
     raw ruby_location               ${bundler-4gitlab:ruby-location}
     raw tar_location                ${tar:location}
     raw watcher                     ${watcher:rendered}
     raw xnice_repository_location   ${xnice-repository:location}
+    raw yarn_location               ${yarn:location}
 
 # config files
     raw database_yml_in             ${database.yml.in:target}
@@ -68,6 +72,7 @@ context =
     raw gitlab_shell_config_yml_in  ${gitlab-shell-config.yml.in:target}
     raw gitlab_unicorn_startup_in   ${gitlab-unicorn-startup.in:target}
     raw gitlab_yml_in               ${gitlab.yml.in:target}
+    raw gitaly_config_toml_in       ${gitaly-config.toml.in:target}
     raw macrolib_cfg_in             ${macrolib.cfg.in:target}
     raw nginx_conf_in               ${nginx.conf.in:target}
     raw nginx_gitlab_http_conf_in   ${nginx-gitlab-http.conf.in:target}

software/gitlab/software.cfg

@@ -15,6 +15,7 @@ extends =
     ../../component/openssl/buildout.cfg
     ../../component/nginx/buildout.cfg
     ../../component/zlib/buildout.cfg
+    ../../component/icu/buildout.cfg
     gowork.cfg
 
 #   for instance
@@ -29,10 +30,10 @@ extends =
     ../../component/logrotate/buildout.cfg
 
 parts =
-    ruby2.1
-    golang19
+    ruby2.3
+    golang1.12
     git
-    postgresql92
+    postgresql10
     redis28
     cmake
     icu
@@ -40,6 +41,8 @@ parts =
     nginx-output
 
     gowork
+    gitlab-workhorse
+    gitaly-build
     python-4gitlab
     gitlab-shell/vendor
     gitlab/vendor/bundle
@@ -64,6 +67,13 @@ parts =
 [slapos.cookbook-repository]
 revision = 571d6514f7290e8faa9439c4b86aa2f6c87df261
 
+
+[yarn]
+# need this version of Yarn
+recipe = slapos.recipe.build:download-unpacked
+url = https://github.com/yarnpkg/yarn/releases/download/v1.3.2/yarn-v1.3.2.tar.gz
+md5sum = db82fa09c996e9318f2f1d2ab99228f9
+
 ############################
 #   Software compilation   #
 ############################
@@ -78,20 +88,22 @@ eggs    =
 # rubygemsrecipe with fixed url and this way pinned rubygems version
 [rubygemsrecipe]
 recipe  = rubygemsrecipe
-url     = https://rubygems.org/rubygems/rubygems-2.5.2.zip
-
+url     = https://rubygems.org/rubygems/rubygems-3.1.2.zip
 
 # bundler, that we'll use to
 # - install gems for gitlab
 # - run gitlab services / jobs  (via `bundle exec ...`)
 [bundler-4gitlab]
 <= rubygemsrecipe
-ruby-location = ${ruby2.1:location}
+ruby-location = ${ruby2.3:location}
 ruby-executable = ${:ruby-location}/bin/ruby
-gems    = bundler==1.11.2
+gems    =
+  bundler==1.17.3
 
 # bin installed here
 bundle  = ${buildout:bin-directory}/bundle
+# Gitaly need bundler 1.17.3 which is not the default version at the end
+bundle1.17.3 = ${buildout:parts-directory}/${:_buildout_section_name_}/lib/ruby/gems/1.8/gems/bundler-1.17.3/exe/
 
 # install together with dependencies of gitlab, which we cannot specify using
 #   --with-... gem option
@@ -109,7 +121,8 @@ bundle  = ${buildout:bin-directory}/bundle
 # gitlab (via github-markup) wants to convert rst -> html via running: python2 (with docutils egg)
 # (python-4gitlab puts interpreter into ${buildout:bin-directory})
 environment =
-  PATH    = ${:ruby-location}/bin:${cmake:location}/bin:${pkgconfig:location}/bin:${nodejs-8.6.0:location}/bin:${postgresql92:location}/bin:${redis28:location}/bin:${git:location}/bin:${buildout:bin-directory}:%(PATH)s
+
+  PATH    = ${yarn:location}/bin:${:ruby-location}/bin:${cmake:location}/bin:${pkgconfig:location}/bin:${nodejs-8.12.0:location}/bin:${postgresql10:location}/bin:${redis28:location}/bin:${git:location}/bin:${buildout:bin-directory}:%(PATH)s
 
 
 # gitlab, gitlab-shell & gitlab-workhorse checked out as git repositories
@@ -120,21 +133,31 @@ git-executable = ${git:location}/bin/git
 
 [gitlab-repository]
 <= git-repository
-#repository = https://gitlab.com/gitlab-org/gitlab-ce.git
 repository = https://lab.nexedi.com/nexedi/gitlab-ce.git
-# 8.17.X + NXD patches:
-revision = v8.17.8-12-g611cf13b90
+# 9.5.10 + NXD patches:
+revision = v9.5.10-8-gc290e22a08cb
 location = ${buildout:parts-directory}/gitlab
 
 [gitlab-shell-repository]
 <= git-repository
-#repository = https://gitlab.com/gitlab-org/gitlab-shell.git
-repository = https://lab.nexedi.com/nexedi/gitlab-shell.git
-# gitlab 8.17 wants gitlab-shell 4.1.1
-# 4.1.1 + NXD patches
-revision = v4.1.1-1-g64603b4da2
+#repository = https://lab.nexedi.com/nexedi/gitlab-shell.git
+repository = https://gitlab.com/gitlab-org/gitlab-shell.git
+# gitlab 9.5.10 wants gitlab-shell 5.6.1
+revision = v5.6.1-10-g1e587d3b7f
 location = ${buildout:parts-directory}/gitlab-shell
 
+[gitaly-repository]
+<= git-repository
+repository = https://gitlab.com/gitlab-org/gitaly.git
+# for version v0.35.0 (gitlab 9.5.10)
+revision = v0.35.0-0-gf99a57b19a
+location = ${buildout:parts-directory}/gitaly
+
+[gitlab-workhorse-repository]
+<= git-repository
+repository = https://lab.nexedi.com/nexedi/gitlab-workhorse.git
+revision = v3.0.0-8-g74793ad3cc
+
 # Patch github markup to not call "python2 -S /path/to/rest2html" but only "python2 /path/to/rest2html"
 # NOTE github-markup invokes it as `python2`, that's why we are naming it this way
 # https://github.com/github/markup/blob/5393ae93/lib/github/markups.rb#L36
@@ -158,11 +181,23 @@ bundle  = ${bundler-4gitlab:bundle}
 
 configure-command = cd ${:path} &&
     ${:bundle} config --local build.charlock_holmes --with-icu-dir=${icu:location}  &&
-    ${:bundle} config --local build.pg  --with-pg-config=${postgresql92:location}/bin/pg_config
+    ${:bundle} config --local build.pg --with-pg-config=${postgresql10:location}/bin/pg_config &&
+    ${:bundle} config --local build.re2 --with-re2-dir=${re2:location}
 
 make-binary =
 make-targets= cd ${:path} &&
-    ${:bundle} install --deployment  --without development test  mysql kerberos
+    ${:bundle} install --deployment  --without development test mysql aws kerberos
+environment =
+  PKG_CONFIG_PATH=${openssl-1.0:location}/lib/pkgconfig:${re2:location}/lib/pkgconfig
+  PATH=${pkgconfig:location}/bin:%(PATH)s
+
+################## Google re2
+[re2]
+recipe = slapos.recipe.cmmi
+url = https://github.com/google/re2/archive/2019-12-01.tar.gz
+md5sum = 527eab0c75d6a1a0044c6eefd816b2fb
+configure-command = :
+
 
 [gitlab_npm]
 recipe  = slapos.recipe.cmmi
@@ -173,7 +208,7 @@ make-binary =
 make-targets= cd ${:path} && npm install
 
 environment =
-  PATH=${nodejs-8.6.0:location}/bin/:%(PATH)s
+  PATH=${nodejs-8.12.0:location}/bin/:%(PATH)s
 
 #our go infrastructure not currently supporting submodules, IIRC
 # https://lab.nexedi.com/nexedi/slapos/merge_requests/337
@@ -184,25 +219,39 @@ configure-command = :
 make-binary =
 make-targets= cd ${go_github.com_libgit2_git2go:location}
               && git submodule update --init
+              && sed -i 's/.*--build.*/cmake --build . --target install/' script/build-libgit2-static.sh
               && make install
 environment =
-  PKG_CONFIG_PATH=${openssl:location}/lib/pkgconfig:${zlib:location}/lib/pkgconfig
-  PATH=${cmake:location}/bin:${pkgconfig:location}/bin:${git:location}/bin:${golang19:location}/bin:${buildout:bin-directory}:%(PATH)s
+  PKG_CONFIG_PATH=${openssl-1.0:location}/lib/pkgconfig:${zlib:location}/lib/pkgconfig
+  PATH=${cmake:location}/bin:${pkgconfig:location}/bin:${git:location}/bin:${golang1.12:location}/bin:${buildout:bin-directory}:%(PATH)s
   GOPATH=${gowork:directory}
 
+[gowork.goinstall]
+git2go = ${go_github.com_libgit2_git2go_prepare:path}/vendor/libgit2/install
+command = bash -c ". ${gowork:env.sh} && CGO_CFLAGS=-I${:git2go}/include CGO_LDFLAGS='-L${:git2go}/lib -lgit2' go install ${gowork:buildflags} -v $(echo -n '${gowork:install}' |tr '\n' ' ')"
+
 [gowork]
-golang  = ${golang19:location}
+golang  = ${golang1.12:location}
+gcc-bin-directory = ${golang1.12:gcc-bin-directory}
+# gitlab.com/gitlab-org/gitlab-workhorse
+# gitlab.com/gitlab-org/gitlab-workhorse/cmd/gitlab-zip-cat
+# gitlab.com/gitlab-org/gitlab-workhorse/cmd/gitlab-zip-metadata
 install =
   lab.nexedi.com/kirr/git-backup
-  gitlab.com/gitlab-org/gitlab-workhorse
-  gitlab.com/gitlab-org/gitlab-workhorse/cmd/gitlab-zip-cat
-  gitlab.com/gitlab-org/gitlab-workhorse/cmd/gitlab-zip-metadata
-
 cpkgpath =
-    ${openssl:location}/lib/pkgconfig
+    ${openssl-1.0:location}/lib/pkgconfig
     ${zlib:location}/lib/pkgconfig
-before-install =
-  ${go_github.com_libgit2_git2go_prepare:recipe}
+    ${go_github.com_libgit2_git2go_prepare:path}/vendor/libgit2/install/lib/pkgconfig
+buildflags = --tags "static"
+
+[gitlab-workhorse]
+recipe = slapos.recipe.cmmi
+path = ${gitlab-workhorse-repository:location}
+md5sum = 2988c944d58c4a08880498c4981cc7b7
+configure-command = :
+make-binary =
+make-targets = 
+  . ${gowork:env.sh} && make install PREFIX=${gowork:directory}
 
 [gitlab-backup]
 recipe = plone.recipe.command
@@ -210,6 +259,21 @@ command =
   cp -a ${go_lab.nexedi.com_kirr_git-backup:location}/contrib/gitlab-backup ${gowork:bin}
 update-command = ${:command}
 
+[gitaly-build]
+recipe = slapos.recipe.cmmi
+path = ${gitaly-repository:location}
+bundle  = ${bundler-4gitlab:bundle}
+
+configure-command = cd ${:path}/ruby &&
+    ${:bundle} config --local build.charlock_holmes --with-icu-dir=${icu:location}
+make-binary =
+make-targets =
+  . ${gowork:env.sh} && make
+environment =
+  PKG_CONFIG_PATH=${openssl-1.0:location}/lib/pkgconfig:${icu:location}/lib/pkgconfig
+  PATH=${pkgconfig:location}/bin:${ruby2.3:location}/bin:%(PATH)s
+
+
 [xnice-repository]
 # to get kirr's misc repo containing xnice script for executing processes
 # with lower priority (used for backup script inside the cron)
@@ -231,6 +295,7 @@ make-binary =
 make-targets= cd ${:path} &&
     ${:bundle} install --deployment  --without development test
 
+
 ###############################
 #   Trampoline for instance   #
 ###############################
@@ -293,6 +358,9 @@ destination = ${buildout:directory}/${:_buildout_section_name_}
 [gitlab.yml.in]
 <= download-file
 
+[gitaly-config.toml.in]
+<= download-file
+
 [instance-gitlab.cfg.in]
 <= download-file
 
@@ -336,6 +404,6 @@ strip-top-level-dir = true
 cns.recipe.symlink = 0.2.3
 docutils = 0.12
 plone.recipe.command = 1.1
-rubygemsrecipe  = 0.2.2+slapos001
-slapos.recipe.template = 4.4
+rubygemsrecipe = 0.2.2+slapos002
+slapos.recipe.template = 4.3
 z3c.recipe.scripts = 1.0.1

software/gitlab/template/gitaly-config.toml.in

+# Example Gitaly configuration file
+# Documentation lives at https://docs.gitlab.com/ee/administration/gitaly/ and
+# https://docs.gitlab.com/ee//administration/gitaly/reference
+
+socket_path = "{{ gitaly.socket }}"
+
+# The directory where Gitaly's executables are stored
+bin_dir = "{{ gitaly.location }}"
+
+# # Optional: listen on a TCP socket. This is insecure (no authentication)
+# listen_addr = "localhost:9999"
+# tls_listen_addr = "localhost:8888
+
+# # Optional: export metrics via Prometheus
+# prometheus_listen_addr = "localhost:9236"
+
+
+# # Git settings
+[git]
+bin_path = "{{ git }}"
+
+[[storage]]
+name = "default"
+path = "{{ gitlab.repositories }}"
+
+# # You can optionally configure more storages for this Gitaly instance to serve up
+#
+# [[storage]]
+# name = "other_storage"
+# path = "/mnt/other_storage/repositories"
+#
+
+# # You can optionally configure Gitaly to output JSON-formatted log messages to stdout
+# [logging]
+# format = "json"
+# # Additionally exceptions can be reported to Sentry
+# sentry_dsn = "https://<key>:<secret>@sentry.io/<project>
+
+
+# # You can optionally configure Gitaly to record histogram latencies on GRPC method calls
+# [prometheus]
+# grpc_latency_buckets = [0.001, 0.005, 0.025, 0.1, 0.5, 1.0, 10.0, 30.0, 60.0, 300.0, 1500.0]
+
+[gitaly-ruby]
+# The directory where gitaly-ruby is installed
+dir = "{{ gitaly.location }}/ruby"
+
+
+[gitlab-shell]
+# The directory where gitlab-shell is installed
+dir = "{{ gitlab_shell_work.location }}"
+

software/gitlab/template/gitlab-shell-config.yml.in

@@ -24,7 +24,7 @@ http_settings:
 # Give the canonicalized absolute pathname,
 # REPOS_PATH MUST NOT CONTAIN ANY SYMLINK!!!
 # Check twice that none of the components is a symlink, including "/home".
-repos_path: "{{ gitlab.repositories }}"
+# repos_path: "{{ gitlab.repositories }}"
 
 # File used as authorized_keys for gitlab user
 # NOTE not used in slapos version (all access via https only)
@@ -34,6 +34,9 @@ auth_file: "{{ gitlab.var }}/sshkeys-notused"
 # Default is .gitlab_shell_secret in the root directory.
 secret_file: "{{ gitlab_shell.secret }}"
 
+# Parent directory for global custom hook directories (pre-receive.d, update.d, post-receive.d)
+# Default is hooks in the gitlab-shell directory.
+custom_hooks_dir: "{{ gitlab_shell_work.location }}/hooks/"
 
 # Redis settings used for pushing commit notices to gitlab
 redis:
@@ -41,11 +44,6 @@ redis:
   host: {# <%= @redis_host %> #}
   port: {# <%= @redis_port %> #}
   socket: {{ service_redis.unixsocket }}
-{# we don't use password for redis
-<% if @redis_password %>
-  pass: <%= @redis_password %>
-<% end %>
-#}
   database: {# <%= @redis_database %> #}
   namespace: resque:gitlab
 

software/gitlab/template/gitlab.yml.in

@@ -32,6 +32,29 @@ production: &base
     relative_url_root: <%= @gitlab_relative_url %>
     #}
 
+    # Content Security Policy
+    # See https://guides.rubyonrails.org/security.html#content-security-policy
+    content_security_policy:
+      enabled: true
+      report_only: false
+      directives:
+        base_uri:
+        child_src:
+        connect_src: "'self' http://localhost:* ws://localhost:* wss://localhost:*"
+        default_src: "'self'"
+        font_src:
+        form_action:
+        frame_ancestors: "'self'"
+        frame_src: "'self' https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://content.googleapis.com https://content-compute.googleapis.com https://content-cloudbilling.googleapis.com https://content-cloudresourcemanager.googleapis.com"
+        img_src: "* data: blob:"
+        manifest_src:
+        media_src:
+        object_src: "'none'"
+        script_src: "'self' 'unsafe-eval' http://localhost:* https://www.google.com/recaptcha/ https://www.recaptcha.net/ https://www.gstatic.com/recaptcha/ https://apis.google.com"
+        style_src: "'self' 'unsafe-inline'"
+        worker_src: "'self' blob:"
+        report_uri:
+
     # Trusted Proxies
     # Customize if you have GitLab behind a reverse proxy which is running on a different machine.
     # Add the IP address for your reverse proxy to the list, otherwise users will appear signed in from that address.
@@ -84,7 +107,7 @@ production: &base
       merge_requests:   {{ cfg('default_projects_features.merge_requests') }}
       wiki:             {{ cfg('default_projects_features.wiki') }}
       snippets:         {{ cfg('default_projects_features.snippets') }}
-      builds: false {# builds not supported yet <%= @gitlab_default_projects_features_builds %> #}
+      builds:           {{ cfg('default_projects_features.builds') }}
       {# container_registry: <%= @gitlab_default_projects_features_container_registry %> #}
 
     ## Webhook settings
@@ -148,6 +171,7 @@ production: &base
     storage_path: <%= @lfs_storage_path %>
   #}
 
+
   {# we do not support container registry
   ## Container Registry
   registry:
@@ -191,6 +215,9 @@ production: &base
     ssl_url:   <%= single_quote(@gravatar_ssl_url) %>    # default: https://secure.gravatar.com/avatar/%{hash}?s=%{size}&d=identicon
     #}
 
+  ## Sidekiq
+  sidekiq:
+    log_format: json # (default is the original format)
 
   {# XXX cron jobs are disabled for now - we do not support CI and EE features or we are ok with defaults
   ## Auxiliary jobs
@@ -375,19 +402,18 @@ production: &base
     path: <%= @shared_path %>
   #}
 
+  # Gitaly settings
+  gitaly:
+    # Default Gitaly authentication token. Can be overriden per storage. Can
+    # be left blank when Gitaly is running locally on a Unix socket, which
+    # is the normal way to deploy Gitaly.
+    token:
+
+
   #
   # 4. Advanced settings
   # ==========================
 
-  # GitLab Satellites
-  # Important: keep the satellites.path setting until GitLab 9.0 at
-  # least. This setting is fed to 'rm -rf' in
-  # db/migrate/20151023144219_remove_satellites.rb
-  satellites:
-    # Relative paths are relative to Rails.root (default: tmp/repo_satellites/)
-    path: /dev/null
-    timeout: 0
-
   ## Repositories settings
   repositories:
     # Paths where repositories can be stored. Give the canonicalized absolute pathname.
@@ -395,7 +421,11 @@ production: &base
     # gitlab-shell invokes Dir.pwd inside the repository path and that results
     # real path not the symlink.
     storages: # You must have at least a `default` storage path.
-      default: {{ gitlab.repositories }}
+      default:
+        path: {{ gitlab.repositories }}
+        gitaly_address: unix:{{ gitaly.socket }} # TCP connections are supported too (e.g. tcp://host:port). TLS connections are also supported using the system certificate pool (eg: tls://host:port).
+        # gitaly_token: 'special token' # Optional: override global gitaly.token for this storage.
+
 
   ## Backup settings
   backup:
@@ -420,8 +450,8 @@ production: &base
   ## GitLab Shell settings
   gitlab_shell:
     path: {{ gitlab_shell_work.location }}
+    authorized_keys_file: {{ gitlab.var }}/sshkeys-notused
 
-    # REPOS_PATH MUST NOT BE A SYMLINK!!!
     repos_path: {{ gitlab.repositories }}
     hooks_path: {{ gitlab_shell_work.location }}/hooks/
     secret_file: {{ gitlab_shell.secret }}
@@ -430,6 +460,9 @@ production: &base
     upload_pack: true
     receive_pack: true
 
+    # Git import/fetch timeout, in seconds. Defaults to 3 hours.
+    # git_timeout: 10800
+
     {# Git over SSH is disabled elsewhere (so we don't care about ssh_port)
     # If you use non-standard ssh port you need to specify it
     ssh_port: <%= @gitlab_shell_ssh_port %>
@@ -452,7 +485,6 @@ production: &base
     # Git timeout to read a commit, in seconds
     timeout: {{ cfg('git_timeout') }}
 
-
   #
   # 5. Extra customization
   # ==========================

software/gitlab/template/nginx-gitlab-http.conf.in

@@ -111,26 +111,35 @@ server {
   set_real_ip_from {{ trusted_address }};
   {% endfor %}
 
+  ## HSTS Config
+  ## https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
+  {% if cfg("nginx_hsts_max_age") > 0 -%}
+  {%   if '{{ cfg("nginx_hsts_include_subdomains") }}' == 'true' -%}
+  add_header Strict-Transport-Security "max-age={{ cfg('nginx_hsts_max_age') }}; includeSubDomains"
+  {%   else -%}
+  add_header Strict-Transport-Security "max-age={{ cfg('nginx_hsts_max_age') }}";
+  {%   endif -%}
+  {% endif -%}
+
   ## Individual nginx logs for this GitLab vhost
   access_log  {{ nginx.log }}/gitlab_access.log gitlab_access;
   error_log   {{ nginx.log }}/gitlab_error.log;
   
-  {# we do not support relative URL - path is always "/" #}
-  {% set path = "/" %}
-  location {{ path }} {
   # Set CORS header
   add_header 'Access-Control-Allow-Origin' {{ cfg('nginx_header_allow_origin') }};
   add_header 'Access-Control-Allow-Credentials' true;
-    if ($request_method = OPTIONS ) {
-        add_header Allow "GET, OPTIONS";
-        add_header Content-Type text/plain;
-        add_header 'Access-Control-Allow-Origin' $http_origin;
-        add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Authorization, Content-Type, Accept";
-        return 200;
-    }
-    ## If you use HTTPS make sure you disable gzip compression
-    ## to be safe against BREACH attack.
-    {{ 'gzip off;' if cfg_https else ''}}
+  #{{ 'gzip off;' if cfg_https else ''}}
+  {% if '{{ cfg("nginx_gzip_enabled") }}' == 'true' -%}
+  gzip on;
+  gzip_static on;
+  gzip_comp_level 2;
+  gzip_http_version 1.1;
+  gzip_vary on;
+  gzip_disable "msie6";
+  gzip_min_length 10240;
+  gzip_proxied no-cache no-store private expired auth;
+  gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/json application/xml application/rss+xml;
+  {% endif -%}
 
   ## https://github.com/gitlabhq/gitlabhq/issues/694
   ## Some requests take more than 30 seconds.
@@ -140,11 +149,41 @@ server {
 
   proxy_http_version 1.1;
 
+  {# we do not support relative URL - path is always "/" #}
+  {% set path = "/" %}
+  
+  #if ($http_host = "") {
+  #  set $http_host_with_default "<%= default_host %>";
+  #}
+  #if ($http_host != "") {
+  #  set $http_host_with_default $http_host;
+  #}
+  location ~ (\.git/gitlab-lfs/objects|\.git/info/lfs/objects/batch$) {
+    proxy_set_header    Host                $http_host;
+    proxy_set_header    X-Real-IP           $remote_addr;
+    {% if cfg_https %}
+    proxy_set_header    X-Forwarded-Ssl     on;
+    {% endif %}
+    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
+    proxy_set_header    X-Forwarded-Proto   {{ "https" if cfg_https else "http" }};
+
+    proxy_pass http://gitlab-workhorse;
+  }
+
+  location {{ path }} {
     # NOTE(slapos) proxy headers are defined upstream in omnibus-gitlab in:
     #   - files/gitlab-config-template/gitlab.rb.template       nginx['proxy_set_headers']
     #   - files/gitlab-cookbooks/gitlab/attributes/default.rb   default['gitlab']['nginx']['proxy_set_headers']
     #   - files/gitlab-cookbooks/gitlab/libraries/gitlab.rb     parse_nginx_proxy_headers()
     # (last updated for omnibus-gitlab 8.5.1+ce.0-1-ge732b39)
+    if ($request_method = OPTIONS ) {
+        add_header Allow "GET, OPTIONS";
+        add_header Content-Type text/plain;
+        add_header 'Access-Control-Allow-Origin' $http_origin;
+        add_header Access-Control-Allow-Headers "Origin, X-Requested-With, Authorization, Content-Type, Accept";
+        return 200;
+    }
+    proxy_cache off;
     proxy_set_header    Host                $http_host;
     proxy_set_header    X-Real-IP           $remote_addr;
     {% if cfg_https %}
@@ -156,6 +195,11 @@ server {
     proxy_pass  http://gitlab-workhorse;
   }
 
+  location ~ ^/(assets)/ {
+    proxy_cache off;
+    proxy_pass  http://gitlab-workhorse;
+  }
+
   error_page 404 /404.html;
   error_page 422 /422.html;
   error_page 500 /500.html;
@@ -169,3 +213,4 @@ server {
   <%= @custom_gitlab_server_config %>
   #}
 }
+

software/gitlab/template/nginx.conf.in

@@ -50,6 +50,42 @@ http {
 
   include {{ nginx_gitlab_http_conf }};
 
+  map $http_upgrade $connection_upgrade {
+      default upgrade;
+      ''      close;
+  }
+
+  # Remove private_token from the request URI
+  # In:  /foo?private_token=unfiltered&authenticity_token=unfiltered&rss_token=unfiltered&...
+  # Out: /foo?private_token=[FILTERED]&authenticity_token=unfiltered&rss_token=unfiltered&...
+  map $request_uri $temp_request_uri_1 {
+    default $request_uri;
+    ~(?i)^(?<start>.*)(?<temp>[\?&]private[\-_]token)=[^&]*(?<rest>.*)$ "$start$temp=[FILTERED]$rest";
+  }
+
+  # Remove authenticity_token from the request URI
+  # In:  /foo?private_token=[FILTERED]&authenticity_token=unfiltered&rss_token=unfiltered&...
+  # Out: /foo?private_token=[FILTERED]&authenticity_token=[FILTERED]&rss_token=unfiltered&...
+  map $temp_request_uri_1 $temp_request_uri_2 {
+    default $temp_request_uri_1;
+    ~(?i)^(?<start>.*)(?<temp>[\?&]authenticity[\-_]token)=[^&]*(?<rest>.*)$ "$start$temp=[FILTERED]$rest";
+  }
+
+  # Remove rss_token from the request URI
+  # In:  /foo?private_token=[FILTERED]&authenticity_token=[FILTERED]&rss_token=unfiltered&...
+  # Out: /foo?private_token=[FILTERED]&authenticity_token=[FILTERED]&rss_token=[FILTERED]&...
+  map $temp_request_uri_2 $filtered_request_uri {
+    default $temp_request_uri_2;
+    ~(?i)^(?<start>.*)(?<temp>[\?&]rss[\-_]token)=[^&]*(?<rest>.*)$ "$start$temp=[FILTERED]$rest";
+  }
+
+  # A version of the referer without the query string
+  map $http_referer $filtered_http_referer {
+    default $http_referer;
+    ~^(?<temp>.*)\? $temp;
+  }
+
+
   {# we don't need: ci, pages, mattermost, registry
   include <%= @gitlab_ci_http_config %>
   include <%= @gitlab_pages_http_config %>;

software/gitlab/template/smtp_settings.rb.in

@@ -29,3 +29,4 @@ end
 # SMTP disabled in instance configuration (see `smtp_enable` parameter).
 # Mail sending, if enabled (see `email_enabled`), will be done via sendmail.
 {% endif %}
+

software/gitlab/template/unicorn.rb.in

@@ -17,8 +17,20 @@ working_directory '{{ gitlab_work.location }}'
 # What the timeout for killing busy workers is, in seconds
 timeout {{ cfg('unicorn_worker_timeout') }}
 
-# Whether the app should be pre-loaded
+# combine Ruby 2.0.0dev or REE with "preload_app true" for memory savings
+# http://rubyenterpriseedition.com/faq.html#adapt_apps_for_cow
 preload_app true
+GC.respond_to?(:copy_on_write_friendly=) and
+  GC.copy_on_write_friendly = true
+
+
+# Enable this flag to have unicorn test client connections by writing the
+# beginning of the HTTP headers before calling the application.  This
+# prevents calling the application for connections that have disconnected
+# while queued.  This is only guaranteed to detect clients on the same
+# host unicorn runs on, and unlikely to detect disconnects even on a
+# fast LAN.
+check_client_connection false
 
 # How many worker processes
 worker_processes {{ cfg('unicorn_worker_processes') }}
@@ -35,6 +47,10 @@ before_fork do |server, worker|
   # defined?(ActiveRecord::Base) and
   #   ActiveRecord::Base.connection.disconnect!
 
+  # The following is only recommended for memory/DB-constrained
+  # installations.  It is not needed if your system can house
+  # twice as many worker_processes as you have configured.
+  #
   # This allows a new master process to incrementally
   # phase out the old master process with SIGTTOU to avoid a
   # thundering herd (especially in the "preload_app false" case)
@@ -48,8 +64,15 @@ before_fork do |server, worker|
     rescue Errno::ENOENT, Errno::ESRCH
     end
   end
+  #
+  # Throttle the master from forking too quickly by sleeping.  Due
+  # to the implementation of standard Unix signal handlers, this
+  # helps (but does not completely) prevent identical, repeated signals
+  # from being lost when the receiving process is busy.
+  # sleep 1
 end
 
+
 # What to do after we fork a worker
 after_fork do |server, worker|
   # per-process listener ports for debugging/admin/migrations
@@ -60,6 +83,17 @@ after_fork do |server, worker|
   # # the following is *required* for Rails + "preload_app true",
   # defined?(ActiveRecord::Base) and
   #   ActiveRecord::Base.establish_connection
+  
+  # reset prometheus client, this will cause any opened metrics files to be closed
+  #defined?(::Prometheus::Client.reinitialize_on_pid_change) &&
+  #  Prometheus::Client.reinitialize_on_pid_change
+
+  # if preload_app is true, then you may also want to check and
+  # restart any other shared sockets/descriptors such as Memcached,
+  # and Redis.  TokyoCabinet file handles are safe to reuse
+  # between any number of forked children (assuming your kernel
+  # correctly implements pread()/pwrite() system calls)
+
 end
 
 

software/gitlab/test/README.md

+Tests for Gitlab software release

software/gitlab/test/setup.py

+##############################################################################
+#
+# Copyright (c) 2018 Nexedi SA and Contributors. All Rights Reserved.
+#
+# WARNING: This program as such is intended to be used by professional
+# programmers who take the whole responsibility of assessing all potential
+# consequences resulting from its eventual inadequacies and bugs
+# End users who are looking for a ready-to-use solution with commercial
+# guarantees and support are strongly adviced to contract a Free Software
+# Service Company
+#
+# This program is Free Software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+##############################################################################
+from setuptools import setup, find_packages
+
+version = '0.0.1.dev0'
+name = 'slapos.test.gitlab'
+long_description = open("README.md").read()
+
+setup(
+    name=name,
+    version=version,
+    description="Test for SlapOS' Gitlab",
+    long_description=long_description,
+    long_description_content_type='text/markdown',
+    maintainer="Nexedi",
+    maintainer_email="info@nexedi.com",
+    url="https://lab.nexedi.com/nexedi/slapos",
+    packages=find_packages(),
+    install_requires=[
+        'slapos.core',
+        'slapos.libnetworkcache',
+        'erp5.util',
+        'supervisor',
+        'requests',
+    ],
+    zip_safe=True,
+    test_suite='test',
+)

software/gitlab/test/test.py

+##############################################################################
+#
+# Copyright (c) 2019 Nexedi SA and Contributors. All Rights Reserved.
+#
+# WARNING: This program as such is intended to be used by professional
+# programmers who take the whole responsibility of assessing all potential
+# consequences resulting from its eventual inadequacies and bugs
+# End users who are looking for a ready-to-use solution with commercial
+# guarantees and support are strongly adviced to contract a Free Software
+# Service Company
+#
+# This program is Free Software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+##############################################################################
+
+import os
+import logging
+from six.moves.urllib.parse import urlparse
+
+import requests
+
+from slapos.testing.testcase import makeModuleSetUpAndTestCaseClass
+
+
+setUpModule, SlapOSInstanceTestCase = makeModuleSetUpAndTestCaseClass(
+    os.path.abspath(
+        os.path.join(os.path.dirname(__file__), '..', 'software.cfg')))
+
+
+class TestGitlab(SlapOSInstanceTestCase):
+  __partition_reference__ = 'G'  # solve path too long for postgresql and unicorn
+
+  @classmethod
+  def getInstanceSoftwareType(cls):
+    return 'gitlab-test'
+
+  def setUp(self):
+    self.backend_url = self.computer_partition.getConnectionParameterDict(
+    )['backend_url']
+
+  def test_http_get(self):
+    resp = requests.get(self.backend_url, verify=False)
+    self.assertTrue(
+      resp.status_code in [requests.codes.ok, requests.codes.found])

software/neoppod/buildout.hash.cfg

@@ -30,7 +30,7 @@ md5sum = 9f27195d770b2f57461c60a82c851ab9
 
 [instance-neo]
 filename = instance-neo.cfg.in
-md5sum = 574acb0cae9af8ec2af52825fb2436d8
+md5sum = 512383220488335ac186013c2ffdc7c1
 
 [template-neo-my-cnf]
 filename = my.cnf.in
@@ -46,4 +46,4 @@ md5sum = 5afd326de385563b5aeac81039f23341
 
 [runTestSuite.in]
 _update_hash_filename_ = runTestSuite.in
-md5sum = b656e805c5dbc7f9c73716398b3e032e
+md5sum = 7a0d5d259eb7f90fc0421d1264fbe7b5

software/neoppod/instance-neo-input-schema.json

@@ -114,6 +114,10 @@
               ],
               "type": "string"
             },
+            "private-tmpfs": {
+              "description": "Size of private tmpfs mount to store the database. See filesystems/tmpfs.txt in Linux documentation. Use only for testing.",
+              "type": "string"
+            },
             "mysql": {
               "description": "Dictionary containing parameters for MySQL.",
               "default": {},

software/neoppod/instance-neo.cfg.in

@@ -2,22 +2,36 @@
 {% set part_list = [] -%}
 {% set init_list = [] -%}
 
+{% set private_tmpfs = slapparameter_dict.get('private-tmpfs') -%}
 {% set storage_type = slapparameter_dict.get('storage-type') or (
      'MySQL' if mariadb_location is defined else 'SQLite') -%}
 {% set mysql = storage_type == 'MySQL' -%}
 {% if mysql -%}
 
 [{{ section('mysqld') }}]
+{% if private_tmpfs -%}
+recipe = slapos.cookbook:wrapper
+wrapper-path = ${directory:etc_run}/mariadb
+private-tmpfs = {{ private_tmpfs }} ${my-cnf-parameters:tmp-directory}
+command-line = ${mariadb-ns:rendered}
+[mariadb-ns]
+rendered = ${directory:bin}/mariadb-ns
+{% else -%}
+rendered = ${directory:etc_run}/mariadb
+{% endif -%}
 recipe = slapos.recipe.template:jinja2
 template = {{ template_mysqld_wrapper }}
-rendered = ${directory:etc_run}/mariadb
 context =
   key defaults_file my-cnf:rendered
   key datadir my-cnf-parameters:data-directory
 
 [my-cnf-parameters]
 socket = ${directory:var_run}/mariadb.sock
+{% if private_tmpfs -%}
+data-directory = ${:tmp-directory}/mariadb
+{% else -%}
 data-directory = ${directory:srv}/mariadb
+{% endif -%}
 tmp-directory = ${directory:tmp}
 pid-file = ${directory:var_run}/mariadb.pid
 error-log = ${directory:log}/mariadb_error.log
@@ -105,6 +119,9 @@ logfile = ${directory:log}/{{ 'neostorage-' ~ i }}.log
 {%- if mysql %}
 {%-   do init_list.append('CREATE DATABASE IF NOT EXISTS neo' ~  i ~ ';') %}
 database-parameters = root@neo{{ i }}${my-cnf-parameters:socket}
+{%- elif private_tmpfs %}
+private-tmpfs = {{ private_tmpfs }} ${directory:tmp}
+database-parameters = ${directory:tmp}/db.sqlite
 {%- else %}
 database-parameters = ${directory:db-{{i}}}/db.sqlite
 
@@ -148,6 +165,14 @@ post = ${binary-wrap-mysql:command-line} -e "FLUSH LOGS"
 {% if runTestSuite_in is defined -%}
 # bin/runTestSuite to run NEO tests
 [{{ section('runTestSuite') }}]
+{%- if private_tmpfs %}
+recipe = slapos.cookbook:wrapper
+wrapper-path = ${directory:bin}/${:_buildout_section_name_}
+private-tmpfs = {{ private_tmpfs }} ${directory:tmp}
+command-line = ${runTestSuite-ns:rendered}
+
+[runTestSuite-ns]
+{%- endif %}
 recipe = slapos.recipe.template:jinja2
 rendered = ${directory:bin}/${:_buildout_section_name_}
 template = {{ runTestSuite_in }}
@@ -157,6 +182,13 @@ context =
     section my_cnf_parameters my-cnf-parameters
     raw     bin_directory     {{ bin_directory }}
     raw     prepend_path      {{ mariadb_location }}/bin
+{%- if private_tmpfs %}
+    key     datadir           my-cnf-parameters:data-directory
+    key     results_directory directory:results
+
+[directory]
+results = ${directory:srv}/tests
+{%- endif %}
 {%- endif %}
 
 {%- endif %}

software/neoppod/runTestSuite.in

@@ -2,7 +2,7 @@
 """
   Script to run NEO test suite using Nexedi's test node framework.
 """
-import argparse, os, re, shutil, subprocess, sys, traceback
+import argparse, errno, json, os, re, shutil, subprocess, sys, traceback
 from erp5.util import taskdistribution
 from time import gmtime, sleep, strftime, time
 
@@ -13,12 +13,12 @@ SUMMARY_RE = re.compile(
   r' (.*) (?P<duration>\d+(\.\d*)?|\.\d+)s', re.MULTILINE)
 
 PATH = os.environ['PATH']
-PATH = '{{ prepend_path }}' + (PATH and ':' + PATH)
+PATH = {{ repr(prepend_path) }} + (PATH and ':' + PATH)
 
 # NEO specific environment
-TEMP_DIRECTORY  = '{{directory.tmp}}'
-NEO_DB_SOCKET = '{{my_cnf_parameters.socket}}'
-RUN_NEO_TESTS_COMMAND = '{{ bin_directory }}/neotestrunner'
+TEMP_DIRECTORY  = {{ repr(directory.tmp) }}
+NEO_DB_SOCKET = {{ repr(my_cnf_parameters.socket) }}
+RUN_NEO_TESTS_COMMAND = {{ repr(bin_directory + '/neotestrunner') }}
 
 def parseTestStdOut(data):
   """
@@ -52,6 +52,28 @@ def parseTestStdOut(data):
 
   return test_count, unexpected_count, expected_count, skip_count, duration
 
+class DummyTestResult:
+
+  class DummyTestResultLine:
+
+    def stop(self, **kw):
+      with open(self.name + '.json', 'w') as f:
+        json.dump(kw, f)
+
+  done = 0
+
+  def __init__(self, test_name_list):
+    self.test_name_list = test_name_list
+
+  def start(self):
+    test_result_line = self.DummyTestResultLine()
+    try:
+      test_result_line.name = self.test_name_list[self.done]
+    except IndexError:
+      return
+    self.done += 1
+    return test_result_line
+
 def main():
   parser = argparse.ArgumentParser(description='Run a test suite.')
   parser.add_argument('--test_suite', help='The test suite name')
@@ -67,18 +89,20 @@ def main():
   args = parser.parse_args()
 
   test_suite_title = args.test_suite_title or args.test_suite
-  revision = args.revision
 
   test_name_list = 'SQLite', 'MySQL'
 
+  if args.master_url:
     tool = taskdistribution.TaskDistributor(portal_url = args.master_url)
-  test_result = tool.createTestResult(revision = revision,
-                                      test_name_list = test_name_list,
-                                      node_title = args.test_node_title,
-                                      test_title = test_suite_title,
-                                      project_title = args.project_title)
+    test_result = tool.createTestResult(args.revision,
+                                        test_name_list,
+                                        args.test_node_title,
+                                        test_title=test_suite_title,
+                                        project_title=args.project_title)
     if test_result is None:
       return
+  else:
+    test_result = DummyTestResult(test_name_list)
   # run NEO tests
   while 1:
     test_result_line = test_result.start()
@@ -106,6 +130,14 @@ def main():
           if timeout < time():
             raise RuntimeError("MySQL server not started")
           sleep(1)
+{%- if datadir is defined %}
+        # fake path for neostorage (getTopologyPath)
+        try:
+          os.mkdir({{ repr(datadir) }})
+        except OSError as e:
+          if e.errno != errno.EEXIST:
+            raise
+{%- endif %}
       with open(os.devnull) as stdin:
         p = subprocess.Popen(args, stdin=stdin, stdout=subprocess.PIPE,
                              stderr=subprocess.PIPE, env=env)
@@ -137,6 +169,13 @@ def main():
       date = strftime("%Y/%m/%d %H:%M:%S", gmtime(end)),
       stderr=stderr,
       **status_dict)
+{%- if results_directory is defined %}
+
+    results = {{ repr(results_directory + '/') }} + adapter
+    if os.path.exists(results):
+      shutil.rmtree(results)
+    shutil.move(temp, results)
+{%- endif %}
 
 if __name__ == "__main__":
     main()

software/neoppod/software-common.cfg

@@ -131,14 +131,15 @@ rendered = ${buildout:parts-directory}/${:_buildout_section_name_}/mysqld.in
 mode = 644
 template =
   inline:{% raw %}#!/bin/sh -e
+  basedir='${mariadb:location}'
   datadir='{{datadir}}'
   [ -e "$datadir" ] || {
     rm -vrf "$datadir.new"
-    '${mariadb:location}/scripts/mysql_install_db' \
+    "$basedir/scripts/mysql_install_db" \
       --defaults-file='{{defaults_file}}' \
       --skip-name-resolve \
       --auth-root-authentication-method=normal \
-      --basedir='${mariadb:location}' \
+      --basedir="$basedir" --plugin_dir="$basedir/lib/plugin" \
       --datadir="$datadir.new"
     mv -v "$datadir.new" "$datadir"
   }
@@ -147,9 +148,7 @@ template =
   {{ variable }} \
   {%-   endfor %}
   {%- endif %}
-  exec '${mariadb:location}/bin/mysqld' \
-    --defaults-file='{{defaults_file}}' \
-    "$@"
+  exec "$basedir/bin/mysqld" --defaults-file='{{defaults_file}}' "$@"
   {% endraw %}
 
 [versions]

software/slapos-sr-testing/buildout.hash.cfg

@@ -15,4 +15,4 @@
 
 [template]
 filename = instance.cfg
-md5sum = 1cbab58e896ff63575f6a67db530d183
+md5sum = 3ad1b06673000d9f424a1e7187c6a1fa

software/slapos-sr-testing/instance.cfg

@@ -28,7 +28,7 @@ bin = $${buildout:directory}/bin
 working-dir = $${buildout:directory}/tmp
 
 [test-list]
-path_list = ${slapos.cookbook-setup:setup},${slapos.test.caddy-frontend-setup:setup},${slapos.test.erp5-setup:setup},${slapos.test.slapos-master-setup:setup},${slapos.test.kvm-setup:setup},${slapos.test.monitor-setup:setup},${slapos.test.plantuml-setup:setup},${slapos.test.powerdns-setup:setup},${slapos.test.proftpd-setup:setup},${slapos.test.re6stnet-setup:setup},${slapos.test.seleniumserver-setup:setup},${slapos.test.slaprunner-setup:setup},${slapos.test.helloworld-setup:setup},${slapos.test.jupyter-setup:setup},${slapos.test.nextcloud-setup:setup},${slapos.test.turnserver-setup:setup},${slapos.test.theia-setup:setup},${slapos.test.grafana-setup:setup}
+path_list = ${slapos.cookbook-setup:setup},${slapos.test.caddy-frontend-setup:setup},${slapos.test.erp5-setup:setup},${slapos.test.slapos-master-setup:setup},${slapos.test.kvm-setup:setup},${slapos.test.monitor-setup:setup},${slapos.test.plantuml-setup:setup},${slapos.test.powerdns-setup:setup},${slapos.test.proftpd-setup:setup},${slapos.test.re6stnet-setup:setup},${slapos.test.seleniumserver-setup:setup},${slapos.test.slaprunner-setup:setup},${slapos.test.helloworld-setup:setup},${slapos.test.jupyter-setup:setup},${slapos.test.nextcloud-setup:setup},${slapos.test.turnserver-setup:setup},${slapos.test.theia-setup:setup},${slapos.test.grafana-setup:setup},${slapos.test.gitlab-setup:setup}
 
 [slapos-test-runner]
 recipe = slapos.cookbook:wrapper

software/slapos-sr-testing/software.cfg

@@ -112,6 +112,11 @@ setup = ${slapos-repository:location}/software/theia/test/
 egg = slapos.test.grafana
 setup = ${slapos-repository:location}/software/grafana/test/
 
+[slapos.test.gitlab-setup]
+<= setup-develop-egg
+egg = slapos.test.gitlab
+setup = ${slapos-repository:location}/software/gitlab/test/
+
 [slapos.core-repository]
 <= git-clone-repository
 repository = https://lab.nexedi.com/nexedi/slapos.core.git

stack/erp5/buildout.hash.cfg

@@ -86,7 +86,7 @@ md5sum = 0648e38bd5d3a15bb9f93264932740b9
 
 [template-zope]
 filename = instance-zope.cfg.in
-md5sum = 8b4a15dca7e30ba5a792f1a9622216b0
+md5sum = e9032f39c6e5db684342491fdeb4624c
 
 [template-balancer]
 filename = instance-balancer.cfg.in

stack/erp5/instance-zope.cfg.in

@@ -189,7 +189,10 @@ wrapped-command-line = '{{ bin_directory }}/runwsgi' {% if webdav %}-w{% endif %
 {% else -%}
 wrapped-command-line = '{{ bin_directory }}/runzope' -C '${:configuration-file}'
 {%- endif %}
-private-dev-shm = {{ slapparameter_dict['private-dev-shm'] }}
+{%- set private_dev_shm = slapparameter_dict['private-dev-shm'] %}
+{%- if private_dev_shm %}
+private-tmpfs = {{ private_dev_shm }} /dev/shm
+{%- endif %}
 
 [{{ section('zcml') }}]
 recipe = slapos.cookbook:copyfilelist

stack/slapos.cfg

@@ -137,7 +137,7 @@ pyparsing = 2.2.0
 pytz = 2016.10
 requests = 2.13.0
 six = 1.12.0
-slapos.cookbook = 1.0.138
+slapos.cookbook = 1.0.139
 slapos.core = 1.5.9
 slapos.extension.strip = 0.4
 slapos.extension.shared = 1.0

[Jérome Perrin]

this was not tagged ? https://lab.nexedi.com/nexedi/slapos/commits/1.0.139

I'm lost with release procedure, so this should be checked with @rafael

[Rafael Monnerat]

I seems @alain.takoudjou didn't follow the procedure