Selenium Server

New software release, selenium server, which expose a http server to which clients can connect using [`webdriver.Remote`](https://selenium-python.readthedocs.io/getting-started.html#using-selenium-with-remote-webdriver) and then use standard webdriver API to control the browser.

The server is made of :
 * the "hub" to which webdriver connects
 * a Xvfb server
 * some nodes, for different browsers:
   * Firefox 52
   * Firefox 60
   * Chrome 69
 * an IPv6 https frontend, exposing the admin interface and the hub. Note that python's webdriver binding before 3.14.0 refuse to connect to a hub with a non verified SSL certificate, so for now clients must either use the published `backend-url` (which is IPv4) or use a version of selenium eggs >= 3.14. In the future we'll see how [caucase](https://lab.nexedi.com/nexedi/caucase/) can solve this. Also, this behavior might also change with future version of selenium, because it's wrong that it does not verify SSL certificates.
 * a ssh server, to which you can connect to establish a ssh tunnel with port forward and register more nodes to the hub. This can be used to connect a browser running on your desktop and see the test running, run unsupported browsers, maybe also [appium](http://appium.io/docs/en/advanced-concepts/grid/) to run tests on mobile. This is a kind of "extra feature" that seems working, but I'm not sure we can support it.

To achieve this, we had to :
  * update Xorg components and build gtk with a X11 gdk backend  to be able to run firefox 60. Updating xorg/gdk also fixed a bug in firefox 52, before this, firefox 52 was not able to get the screen size and ` window.screen.width` was 0. Now firefox 52 can get the screen size properly but resizing window does not work.
  * update chrome 69 and fix a few missing `$LD_LIBRARY_PATH` in the wrapper.

Everything seems working, including taking screenshots, uploading files and resizing window - except resizing window does not work on firefox 52.

See the included test suite for some example usage. Basically it's same as what we're doing today, but instead of:

```python
driver = webdriver.Firefox(path_to_firefox)
```

we use:

```python
driver = webdriver.Remote(
   command_executor='http://127.0.0.1:4444/wd/hub', # the url or backend-url parameter published by the instance.
   desired_capabilities={
      'browserName': 'firefox', # or chrome
      'version': '60', # or don't set a version if you want any version (browserName, on the other hand, is required)
   })
```


/reviewed-on nexedi/slapos!420

component/chromedriver/buildout.cfg

+# ChromeDriver - Webdriver for Chrome
+# http://chromedriver.chromium.org/
+# This is a binary download with wrapper scripts.
+
+[buildout]
+extends =
+  ../chromium/buildout.cfg
+  ../nss/buildout.cfg
+  ../nspr/buildout.cfg
+
+parts =
+  chromedriver-wrapper
+
+
+
+[chromedriver-wrapper-2.41]
+<= chromedriver-wrapper
+wrapper-name = chromedriver-2.41
+part = ${chromedriver-2.41:location}
+
+[chromedriver-wrapper]
+# generate a wrapper named ${:wrapper-name} setting $LD_LIBRARY_PATH
+# to run chromedriver installed from ${:part}
+wrapper-name = chromedriver
+part = ${chromedriver:location}
+
+recipe = slapos.recipe.build
+location = ${buildout:bin-directory}/${:wrapper-name}
+script =
+  chromedriver = self.options['part']
+  part = self.buildout[os.path.basename(chromedriver)]
+  with open(%(location)r, 'w') as f:
+    f.write("""#!/bin/sh -e
+  cd {}
+  export LD_LIBRARY_PATH=$PWD:{}
+  export PATH={}:$PATH
+  exec ./chromedriver "$@"
+  """.format(
+    chromedriver,
+    ':'.join(part['library'].split()),
+    ':'.join(part['path'].split()),
+  ))
+    os.fchmod(f.fileno(), 0o755)
+
+
+[chromedriver]
+<= chromedriver-2.41
+
+[chromedriver-2.41]
+<= chromedriver-download
+version = 2.41
+# Supports Chrome v67-69
+md5sum-x86_64 = fbd8b9561575054e0e7e9cc53b680a70
+
+
+[chromedriver-download]
+# Installs chromedriver ${version}.
+# This chromedriver is not usable directly, it needs a wrapper.
+version =
+md5sum-x86_64 =
+
+recipe = slapos.recipe.build
+x86-64 = https://chromedriver.storage.googleapis.com/${:version}/chromedriver_linux64.zip ${:md5sum-x86_64}
+library =
+  ${nss:location}/lib
+  ${nspr:location}/lib
+path =
+
+script =
+  url, md5sum = self.options[guessPlatform()].split()
+  extract_dir = self.extract(self.download(url, md5sum))
+  os.mkdir(%(location)r)
+  shutil.copy(extract_dir + '/chromedriver', %(location)r)
+

component/chromium/buildout.cfg

+# Chromium binary build, from "Downloading old builds of Chrome / Chromium" of
+# https://www.chromium.org/getting-involved/download-chromium
+# Currently, only Linux_x64 is supported.
+
+
 [buildout]
 # chromium binary is linking to libudev, that can be provided by
 # component/systemd.  but current component/systemd can be built only
 # with Linux kernel >= 3.7. so libudev is not included in
 # LD_LIBRARY_PATH below intentionally and libudev should be installed
 # in the system.
-
 extends =
+  ../alsa/buildout.cfg
   ../coreutils/buildout.cfg
   ../cups/buildout.cfg
   ../dbus/buildout.cfg
@@ -14,6 +19,7 @@ extends =
   ../gettext/buildout.cfg
   ../glib/buildout.cfg
   ../gtk-2/buildout.cfg
+  ../gtk-3/buildout.cfg
   ../libexpat/buildout.cfg
   ../libffi/buildout.cfg
   ../libpng/buildout.cfg
@@ -27,77 +33,133 @@ extends =
   ../zlib/buildout.cfg
 
 parts =
-  chromium
+  chromium-wrapper
+
+
+[chromium-wrapper]
+# Install a chromium wrapper named ${:wrapper-name} in buildout's bin-directory,
+# wrapping chromium installed in ${:part}
+# This [chromium-wrapper] installs the default chromium with name `chromium` and
+# can also be used as a macro to install under a different name.
+wrapper-name = chromium
+part = ${chromium:location}
+
+recipe = slapos.recipe.build
+location = ${buildout:bin-directory}/${:wrapper-name}
+script =
+  chromeium = self.options['part']
+  part = self.buildout[os.path.basename(chromeium)]
+  with open(%(location)r, 'w') as f:
+    f.write("""#!/bin/sh -e
+  cd {}
+  # this also needs a $FONTCONFIG_FILE set, otherwise system fonts will be used and if
+  # no system fonts are available, chrome will refuse to start with this error:
+  # FATAL:platform_font_linux.cc(83)] Check failed: InitDefaultFont(). Could not find the default font
+  export LD_LIBRARY_PATH=$PWD:{}
+  export PATH={}:$PATH
+  exec ./chrome --disable-setuid-sandbox --no-sandbox --disable-gpu $@
+  """.format(
+    chromeium,
+    ':'.join(part['library'].split()),
+    ':'.join(part['path'].split()),
+  ))
+    os.fchmod(f.fileno(), 0o755)
+
+[chromium-wrapper-69]
+<= chromium-wrapper
+wrapper-name = chromium-69
+part = ${chromium-69:location}
+
+
+
 
 [chromium]
+<= chromium-69
+
+
+[chromium-69]
+<= chromium-download
+version = 69.0.3497.0
+
+revision_x86-64 = 576753
+md5sum-x86_64 = 08ac27fd40ace4ca8dfbd1db403deccb
+generation-x86_64 = 1532051976706023
+
+
+[chromium-download]
+# macro to download a binary build of chromium and generate a
+# wrapper as chrome-slapos in the part directory
+
+# the full version tag
+version =
+
+# needs the following URL parts:
+# revision is from the "base position"
+revision_x86-64 =
+# generation is in the final download URL
+generation-x86_64 =
+# this is the md5sum of the downloaded archive
+md5sum-x86_64 =
+
+
 recipe = slapos.recipe.build
 slapos_promise =
   file:chrome
-  file:chrome-wrapper
-  file:chrome-slapos
 
-# How to get the revision :
-# stable : https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Linux_x64%2FLAST_CHANGE?alt=media
-# snapshot : https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Linux_x64%2FLAST_CHANGE?alt=media
-revision_x86 = 382014
-revision_x86-64 = 382014
-revision_mac = 381909
 
-linux_x86 = https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Linux%2F${:revision_x86}%2Fchrome-linux.zip?alt=media
-linux_x86-64 = https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Linux_x64%2F${:revision_x86-64}%2Fchrome-linux.zip?alt=media ef2c476b1f059e9aa026bbac1872368d
-mac_x86-64 = https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Mac%2F${:revision_mac}%2Fchrome-linux.zip?alt=media
+x86-64 = https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Linux_x64%2F${:revision_x86-64}%2Fchrome-linux.zip?generation=${:generation-x86_64}&alt=media ${:md5sum-x86_64}
+
+
+library =
+  ${atk:location}/lib
+  ${at-spi2-atk:location}/lib
+  ${alsa:location}/lib
+  ${cairo:location}/lib
+  ${cups:location}/lib
+  ${cups:location}/lib64
+  ${dbus:location}/lib/
+  ${dbus-glib:location}/lib/
+  ${fontconfig:location}/lib/
+  ${gdk-pixbuf:location}/lib
+  ${gettext:location}/lib
+  ${glib:location}/lib
+  ${gtk-3:location}/lib
+  ${harfbuzz:location}/lib
+  ${libX11:location}/lib
+  ${libXau:location}/lib
+  ${libXcomposite:location}/lib
+  ${libXcursor:location}/lib
+  ${libXext:location}/lib
+  ${libXi:location}/lib
+  ${libXrender:location}/lib/
+  ${libXtst:location}/lib
+  ${libXScrnSaver:location}/lib
+  ${libXrandr:location}/lib
+  ${libexpat:location}/lib
+  ${libffi:location}/lib
+  ${libpng:location}/lib
+  ${libpng12:location}/lib
+  ${libxcb:location}/lib
+  ${libxml2:location}/lib
+  ${mesa:location}/lib
+  ${nspr:location}/lib
+  ${nss:location}/lib
+  ${pango:location}/lib
+  ${pcre:location}/lib
+  ${pixman:location}/lib
+  ${sqlite3:location}/lib
+  ${xdamage:location}/lib
+  ${xfixes:location}/lib
+  ${zlib:location}/lib
+path =
+  ${fontconfig:location}/bin
 
 script =
-  #If part directory already exist, will just throw an error.
+  url, md5sum = self.options[guessPlatform()].split()
+  extract_dir = self.extract(self.download(url, md5sum))
+  self.copyTree(guessworkdir(extract_dir), %(location)r)
+  # XXX adjust some permissions
   import os
-  platform = '%%s_%%s' %% (guessOperatingSystem(), guessPlatform())
-  if not self.options.get('url'): self.options['url'], self.options['md5sum'] = self.options[platform].split(' ')
-  extract_dir = self.extract(self.download(self.options['url'], self.options.get('md5sum')))
-  workdir = guessworkdir(extract_dir)
-  self.copyTree(workdir, "%(location)s")
-  wrapper_location = os.path.join("%(location)s", "chrome-slapos")
-  wrapper = open(wrapper_location, 'w')
-  wrapper.write("""#!/bin/sh
-  export LD_LIBRARY_PATH="%(location)s"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${atk:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${cairo:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${cups:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${dbus:location}/lib/"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${dbus-glib:location}/lib/"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${fontconfig:location}/lib/"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${gdk-pixbuf:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${gettext:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${glib:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${gtk-2:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${harfbuzz:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libX11:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXau:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXcomposite:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXcursor:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXext:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXi:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXrender:location}/lib/"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libXtst:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libexpat:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libffi:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libpng:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libpng12:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libxcb:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${libxml2:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${mesa:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${nspr:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${nss:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${pango:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${pcre:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${pixman:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${sqlite3:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${xdamage:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${xfixes:location}/lib"
-  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${zlib:location}/lib"
-  exec %(location)s/chrome --disable-setuid-sandbox --disable-gpu $*""")
-  wrapper.flush()
-  wrapper.close()
   os.system('"${findutils:location}/bin/find" "%(location)s" -type d -exec "${coreutils:location}/bin/chmod" a+rx {} \;')
   os.system('"${findutils:location}/bin/find" "%(location)s" -type f -executable -exec "${coreutils:location}/bin/chmod" a+rx {} \;')
   os.system('"${findutils:location}/bin/find" "%(location)s" -type f -exec "${coreutils:location}/bin/chmod" a+r {} \;')
-  os.chmod(wrapper_location, 0755)

component/firefox/buildout.cfg

@@ -31,7 +31,6 @@ part = ${firefox:location}
 
 recipe = slapos.recipe.build
 location = ${buildout:bin-directory}/${:wrapper-name}
-part = ${firefox:location}
 script =
   firefox = self.options['part']
   part = self.buildout[os.path.basename(firefox)]
@@ -48,6 +47,11 @@ script =
   ))
     os.fchmod(f.fileno(), 0o755)
 
+[firefox-wrapper-60]
+<= firefox-wrapper
+wrapper-name = firefox-60
+part = ${firefox-60:location}
+
 [firefox-wrapper-52]
 <= firefox-wrapper
 wrapper-name = firefox-52
@@ -67,6 +71,12 @@ part = ${firefox-51:location}
 # would not be created.
 <= firefox-52
 
+[firefox-60]
+<= firefox-download
+version = 60.0.2esr
+i686-md5sum = ce7c80716036dfb5c2fb1ca2538556ff
+x86_64-md5sum = 6fe25d9a3fcc82670320242c9047d1da
+
 [firefox-52]
 <= firefox-download
 version = 52.9.0esr
@@ -143,7 +153,25 @@ script =
 
 [geckodriver]
 # Current geckodriver installed as buildout:bin-directory/geckodriver
-<= geckodriver-0.17.0
+<= geckodriver-0.21.0
+
+[geckodriver-0.22.0]
+<= geckodriver-base
+version = 0.22.0
+i686-md5sum = 6de7544753fda56fbaa8382dcac99aaa
+x86_64-md5sum = 81746200ce5841e00cabf3b8ea7db542
+
+[geckodriver-0.21.0]
+<= geckodriver-base
+version = 0.21.0
+i686-md5sum = 9fc1657dd1b94272d0cdb3b29ca80f79
+x86_64-md5sum = 6bc36d4fd4975e296cdb3fa3c5e26a41
+
+[geckodriver-0.19.0]
+<= geckodriver-base
+version = 0.19.0
+i686-md5sum = 07cd383c8aef8ea5ef194a506141afd6
+x86_64-md5sum = ca6935a72fd0527d15a78a17a35e56e8
 
 [geckodriver-0.17.0]
 <= geckodriver-base
@@ -151,6 +179,11 @@ version = 0.17.0
 i686-md5sum = 79b1a158f96d29942a111c0905f1c807
 x86_64-md5sum = be18faeea6e7db9db6990d8667e2298f
 
+[geckodriver-0.16.1]
+<= geckodriver-base
+version = 0.16.1
+i686-md5sum = not not on github
+x86_64-md5sum = 57dfd55d4759d9878eb75b4c0123d00c
 
 [geckodriver-0.14.0]
 <= geckodriver-base

component/fonts/buildout.cfg

 [buildout]
 extends =
   ../xz-utils/buildout.cfg
+  ../p7zip/buildout.cfg
 
 parts =
   liberation-fonts

component/gtk-2/buildout.cfg

@@ -16,6 +16,7 @@ extends =
   ../perl/buildout.cfg
   ../pkgconfig/buildout.cfg
   ../xorg/buildout.cfg
+  ../icu/buildout.cfg
   ../xz-utils/buildout.cfg
   ../zlib/buildout.cfg
 
@@ -49,13 +50,13 @@ recipe = slapos.recipe.cmmi
 shared = true
 url = http://www.freedesktop.org/software/harfbuzz/release/harfbuzz-0.9.35.tar.bz2
 md5sum = 531ee8650626ecddcd90b2a4637e31d4
-pkg_config_depends = ${cairo:location}/lib/pkgconfig:${cairo:pkg_config_depends}
+pkg_config_depends = ${cairo:location}/lib/pkgconfig:${cairo:pkg_config_depends}:${icu4c:location}/lib/pkgconfig
 configure-options =
   --disable-static
   --disable-gtk-doc-html
   --with-cairo
   --with-freetype
-  --without-icu
+  --with-icu
 environment =
   PATH=${glib:location}/bin:${freetype:location}/bin:${pkgconfig:location}/bin:%(PATH)s
   PKG_CONFIG_PATH=${:pkg_config_depends}
@@ -126,5 +127,5 @@ environment =
   PATH=${gdk-pixbuf:location}/bin:${gettext:location}/bin:${glib:location}/bin:${perl:location}/bin:${pkgconfig:location}/bin:${xz-utils:location}/bin:%(PATH)s
   PKG_CONFIG_PATH=${:pkg_config_depends}
 # not taken from pkg-config result...
-  CPPFLAGS=-I${libX11:location}/include/ -I${xproto:location}/include -I${kbproto:location}/include -I${libXrender:location}/include -I${render:location}/include -I${libXext:location}/include
+  CPPFLAGS=-I${libX11:location}/include/ -I${xproto:location}/include -I${kbproto:location}/include -I${libXrender:location}/include -I${renderext:location}/include -I${libXext:location}/include
   LDFLAGS=-L${libX11:location}/lib -L${libXext:location}/lib -L${libXrender:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -L${bzip2:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib

component/gtk-3/buildout.cfg

@@ -35,19 +35,24 @@ environment =
 recipe = slapos.recipe.cmmi
 url = http://ftp.gnome.org/pub/gnome/core/3.22/3.22.2/sources/gtk+-3.22.3.tar.xz
 md5sum = f0e6492896a2ca244501142319adaa95
-pkg_config_depends = ${at-spi2-atk:location}/lib/pkgconfig:${at-spi2-core:location}/lib/pkgconfig:${dbus:location}/lib/pkgconfig:${glib:location}/lib/pkgconfig:${libepoxy:location}/lib/pkgconfig:${pango:location}/lib/pkgconfig:${pango:pkg_config_depends}:${atk:location}/lib/pkgconfig:${gdk-pixbuf:location}/lib/pkgconfig:${libXi:location}/lib/pkgconfig:${pcre:location}/lib/pkgconfig
+pkg_config_depends = ${at-spi2-atk:location}/lib/pkgconfig:${at-spi2-core:location}/lib/pkgconfig:${dbus:location}/lib/pkgconfig:${glib:location}/lib/pkgconfig:${libepoxy:location}/lib/pkgconfig:${pango:location}/lib/pkgconfig:${pango:pkg_config_depends}:${atk:location}/lib/pkgconfig:${gdk-pixbuf:location}/lib/pkgconfig:${libXi:pkg_config_depends}:${libXi:location}/lib/pkgconfig:${pcre:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libX11:pkg_config_depends}:${libX11:location}/lib/pkgconfig:${xorgproto:location}/lib/pkgconfig:${libXext:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${libXrender:location}/lib/pkgconfig:${renderproto:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${xcbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libXinerama:location}/lib/pkgconfig:${libXrandr:pkg_config_depends}:${libXrandr:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${libXcomposite:location}/lib/pkgconfig:${xdamage:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${compositeproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig:${xproto:location}/lib/pkgconfig:${damageproto:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig
 configure-options =
   --disable-static
   --disable-glibtest
   --disable-cups
   --disable-papi
   --enable-explicit-deps
-  --disable-xinerama
   --disable-gtk-doc-html
   --disable-man
+  --enable-x11-backend
+  --enable-xrandr
+  --enable-xfixes
+  --enable-xcomposite
+  --enable-xdamage
 environment =
   PATH=${gdk-pixbuf:location}/bin:${gettext:location}/bin:${glib:location}/bin:${perl:location}/bin:${pkgconfig:location}/bin:${xz-utils:location}/bin:%(PATH)s
   PKG_CONFIG_PATH=${:pkg_config_depends}
 # not taken from pkg-config result...
-  CPPFLAGS=-I${cairo:location}/include -I${inputproto:location}/include -I${libX11:location}/include -I${libXi:location}/include -I${xproto:location}/include -I${kbproto:location}/include -I${libXrender:location}/include -I${render:location}/include -I${libXext:location}/include
-  LDFLAGS=-L${libX11:location}/lib -L${libXi:location}/lib -L${libXext:location}/lib -L${libXrender:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -L${bzip2:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
+  CPPFLAGS=-I${cairo:location}/include -I${inputproto:location}/include -I${libX11:location}/include -I${libXi:location}/include -I${xproto:location}/include -I${kbproto:location}/include -I${libXrender:location}/include -I${renderext:location}/include -I${libXext:location}/include -I${libX11:location}/include  -I${libXinerama:location}/include -I${gdk-pixbuf:location}/include -I${libXrandr:location}/include -I${xfixes:location}/include -I${libXcomposite:location}/include -I${xdamage:location}/include -I${fixesproto:location}/include
+  LDFLAGS=-L${libX11:location}/lib -L${libXi:location}/lib -L${libXext:location}/lib -L${libXrender:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -L${bzip2:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${libX11:location}/lib -Wl,-rpath=${libX11:location}/lib -L${libXinerama:location}/lib -Wl,-rpath=${libXinerama:location}/lib -L${gdk-pixbuf:location}/lib -Wl,-rpath=${gdk-pixbuf:location}/lib -L${libXrandr:location}/lib -Wl,-rpath=${libXrandr:location}/lib -L${xfixes:location}/lib -Wl,-rpath=${xfixes:location}/lib -L${libXcomposite:location}/lib -Wl,-rpath=${libXcomposite:location}/lib -L${xdamage:location}/lib -Wl,-rpath=${xdamage:location}/lib
+

component/gtkmm/buildout.cfg

@@ -26,7 +26,7 @@ configure-options =
 environment =
   PATH=${freetype:location}/bin:${gcc:location}/bin:${perl:location}/bin:${pkgconfig:location}/bin:${xz-utils:location}/bin:%(PATH)s
   PKG_CONFIG_PATH=${:pkg_config_depends}
-  CPPFLAGS=-I${libX11:location}/include -I${libXrender:location}/include -I${render:location}/include -I${xproto:location}/include
+  CPPFLAGS=-I${libX11:location}/include -I${libXrender:location}/include -I${renderext:location}/include -I${xproto:location}/include
   LD_LIBRARY_PATH=${gcc:location}/lib:${gcc:location}/lib64
   LDFLAGS=-Wl,-rpath=${gcc:location}/lib -Wl,-rpath=${gcc:location}/lib64 -L${libX11:location}/lib -Wl,-rpath=${libX11:location}/lib -L${libXext:location}/lib -Wl,-rpath=${libXext:location}/lib -L${libXrender:location}/lib -Wl,-rpath=${libXrender:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
 

component/python-pynacl/buildout.cfg

+[buildout]
+extends =
+  ../libffi/buildout.cfg
+  ../pkgconfig/buildout.cfg
+
+parts =
+  python-pynacl
+
+[python-pynacl]
+recipe = zc.recipe.egg:custom
+egg = pynacl
+environment = python-pynacl-env
+library-dirs =
+  ${libffi:location}/lib/
+rpath =
+  ${libffi:location}/lib/
+
+[python-pynacl-env]
+PATH = ${pkgconfig:location}/bin:%(PATH)s
+PKG_CONFIG_PATH = ${libffi:location}/lib/pkgconfig

component/webp/buildout.cfg

@@ -19,6 +19,7 @@ configure-options =
   --disable-static
   --disable-gl
   --disable-wic
+  --enable-everything
   --with-jpegincludedir=${libjpeg:location}/include
   --with-jpeglibdir=${libjpeg:location}/lib
   --with-tiffincludedir=${libtiff:location}/include

component/xorg/buildout.hash.cfg

@@ -15,7 +15,7 @@
 
 [libxcb_tmp_env.patch]
 filename = libxcb_tmp_env.patch
-md5sum = 03bc7279f2a37bac7bf426af213123b2
+md5sum = 5a2875e62216d2f41a75e92d1bf0ae67
 
 [xorg-server_tmp_env.patch]
 filename = xorg-server_tmp_env.patch

component/xorg/libxcb_tmp_env.patch

-Store Xorg lock files and sockets in different directories
+From 8469ef2a978e19145a56adb54925cf5936ab2d49 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?J=C3=A9rome=20Perrin?= <jerome@nexedi.com>
+Date: Mon, 17 Sep 2018 16:07:02 +0200
+Subject: [PATCH] SlapOS: Store Xorg lock files and sockets in different
+ directories
 
 see xorg-server_tmp_env.patch
+---
+ src/xcb_util.c | 26 ++++++++++++++++++++++++--
+ 1 file changed, 24 insertions(+), 2 deletions(-)
 
---- libxcb-1.7/src/xcb_util.c.back	2012-04-03 13:30:36.000000000 +0200
-+++ libxcb-1.7/src/xcb_util.c	2012-04-03 14:47:00.000000000 +0200
-@@ -147,11 +147,31 @@
+diff --git a/src/xcb_util.c b/src/xcb_util.c
+index a16270c..d05f8a1 100644
+--- a/src/xcb_util.c
++++ b/src/xcb_util.c
+@@ -223,15 +223,37 @@ static int _xcb_open_unix(char *protocol, const char *file);
  static int _xcb_open_abstract(char *protocol, const char *file, size_t filelen);
  #endif
  
@@ -27,14 +36,23 @@ see xorg-server_tmp_env.patch
 +  }
 +  return strncat(path_list[0], suffix, PATH_MAX);
 +}
++
 +
  static int _xcb_open(const char *host, char *protocol, const int display)
  {
      int fd;
+ #ifdef __hpux
+     static const char unix_base[] = "/usr/spool/sockets/X11/";
++    #error "SlapOS: no patch for hpux"
+ #else
 -    static const char unix_base[] = "/tmp/.X11-unix/X";
--    const char *base = unix_base;
 +    char *base;
 +    base = _xcb_getandappendcompatibleenv("XORG_LOCK_DIR", "/tmp", "/.X11-unix/X");
+ #endif
+-    const char *base = unix_base;
      size_t filelen;
      char *file = NULL;
      int actual_filelen;
+-- 
+2.11.0
+

software/erp5testnode/testsuite/seleniumrunner/README.md

+# Selenium Server test
+
+This software release is simply to run the test suite from `../seleniumrunner/test/setup.py`
+
+Nexedi staff can see the results of this test from the test suite
+`SLAPOS-SELENIUMRUNNER-TEST` in test result module.
+

software/erp5testnode/testsuite/seleniumrunner/buildout.hash.cfg

+# THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
+# The only allowed lines here are (regexes):
+# - "^#" comments, copied verbatim
+# - "^[" section beginings, copied verbatim
+# - lines containing an "=" sign which must fit in the following categorie.
+#   - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
+#     Copied verbatim.
+#   - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
+#     by the re-generation script.
+#     Re-generated.
+# - other lines are copied verbatim
+# Substitution (${...:...}), extension ([buildout] extends = ...) and
+# section inheritance (< = ...) are NOT supported (but you should really
+# not need these here).
+
+[template]
+filename = instance.cfg.in
+md5sum = 7e75c9eccb580f278da1784941363432
+

software/erp5testnode/testsuite/seleniumrunner/instance.cfg.in

+[buildout]
+parts =
+  slapos-test-runner
+
+eggs-directory = ${buildout:eggs-directory}
+develop-eggs-directory = ${buildout:develop-eggs-directory}
+offline = true
+
+[slap-configuration]
+recipe = slapos.cookbook:slapconfiguration
+computer = $${slap-connection:computer-id}
+partition = $${slap-connection:partition-id}
+url = $${slap-connection:server-url}
+key = $${slap-connection:key-file}
+cert = $${slap-connection:cert-file}
+
+[download-source]
+recipe = slapos.recipe.build:gitclone
+git-executable = ${git:location}/bin/git
+
+[slapos]
+<= download-source
+repository = ${slapos-repository:location}
+
+[create-directory]
+recipe = slapos.cookbook:mkdirectory
+bin = $${buildout:directory}/bin
+working-dir = $${buildout:directory}/tmp/
+
+[slapos-test-runner]
+recipe = slapos.cookbook:wrapper
+wrapper-path = $${create-directory:bin}/runTestSuite
+command-line =
+  ${buildout:bin-directory}/runTestSuite
+  --python_interpreter=${buildout:bin-directory}/${eggs:interpreter}
+  --source_code_path_list=$${slapos:location}/software/seleniumrunner/test
+
+
+# XXX we need "standard" path entries to compile softwares inside test
+# XXX can't we just inherit $PATH ?
+environment =
+  PATH=${buildout:bin-directory}:/bin/:/usr/bin/
+  LOCAL_IPV4=$${slap-configuration:ipv4-random}
+  GLOBAL_IPV6=$${slap-configuration:ipv6-random}
+  SLAPOS_TEST_WORKING_DIR=$${create-directory:working-dir}

software/erp5testnode/testsuite/seleniumrunner/software.cfg

+[buildout]
+
+extends =
+  ../../../../component/git/buildout.cfg
+  ../../../../component/pillow/buildout.cfg
+  ../../../../component/python-pynacl/buildout.cfg
+  ../../../../component/bcrypt/buildout.cfg
+  ../../../../stack/slapos.cfg
+  ./buildout.hash.cfg
+
+parts =
+  slapos-cookbook
+  eggs
+  template
+
+[setup-develop-egg]
+recipe = zc.recipe.egg:develop
+
+[slapos.test.seleniumrunner-setup]
+<= setup-develop-egg
+egg = slapos.test.seleniumrunner
+setup = ${slapos-repository:location}/software/seleniumrunner/test/
+
+[eggs]
+recipe = zc.recipe.egg
+eggs =
+  ${pillow-python:egg}
+  ${python-pynacl:egg}
+  ${bcrypt:egg}
+  ${slapos.test.seleniumrunner-setup:egg}
+  slapos.core
+entry-points =
+  runTestSuite=erp5.util.testsuite:runTestSuite
+scripts =
+  runTestSuite
+  slapos
+interpreter=
+  python_for_test
+
+[git-clone-repository]
+recipe = slapos.recipe.build:gitclone
+git-executable = ${git:location}/bin/git
+forbid-download-cache = true
+branch = master
+
+[slapos-repository]
+<= git-clone-repository
+repository = https://lab.nexedi.com/nexedi/slapos.git
+
+[template]
+recipe = slapos.recipe.template
+url = ${:_profile_base_location_}/${:filename}
+output = ${buildout:directory}/template.cfg
+mode = 640
+
+[versions]
+erp5.util = 0.4.56
+slapos.recipe.template = 4.3
+
+image = 1.5.25
+Pillow = 5.3.0
+selenium = 3.14.1
+urllib3 = 1.23
+# Django 1.11 is python 2 compatible
+Django = 1.11
+paramiko = 2.4.2
+pyasn1 = 0.4.2
+PyNaCl = 1.3.0
+bcrypt = 3.1.4
\ No newline at end of file

software/seleniumrunner/buildout.hash.cfg

@@ -19,4 +19,4 @@ md5sum = c4ac5de141ae6a64848309af03e51d88
 
 [template-selenium]
 filename = instance-selenium.cfg.in
-md5sum = 8f06bef6ed0737afa7916bc5ac7db537
+md5sum = 4167621b473f81892d38389ac427c6ba

software/seleniumrunner/software.cfg

+# Selenium server
+# https://seleniumhq.github.io/docs/grid.html
+
 [buildout]
 extends =
    ../../component/xorg/buildout.cfg
    ../../component/lxml-python/buildout.cfg
    ../../component/firefox/buildout.cfg
-   ../../component/dash/buildout.cfg
+   ../../component/chromium/buildout.cfg
+   ../../component/chromedriver/buildout.cfg
+   ../../component/coreutils/buildout.cfg
+   ../../component/java/buildout.cfg
+   ../../component/caddy/buildout.cfg
+   ../../component/openssh/buildout.cfg
    ../../stack/slapos.cfg
    ./buildout.hash.cfg
 
-# develop += /opt/slapdev
-
 parts =
    slapos-cookbook
    template
-   eggs
-   instance-recipe-egg
-   xserver
-   firefox-wrapper
-   geckodriver
-   xwd
+
 
 [instance-recipe]
 egg = slapos.cookbook
 module = seleniumrunner
 
-[instance-recipe-egg]
-recipe = zc.recipe.egg
-eggs = ${instance-recipe:egg}
-
-[eggs]
-recipe = zc.recipe.egg
-eggs =
-  ${lxml-python:egg}
-
+[selenium-server]
+recipe = slapos.recipe.build:download
+version = 3.14.0
+md5sum = 376450bd517510442b60018646deadfe
+filename = selenium-server-standalone-${:version}.jar
+url = https://selenium-release.storage.googleapis.com/3.14/${:filename}
 
 [macro-template]
 recipe = slapos.recipe.template

software/seleniumrunner/test/README.md

+Tests for SeleniumRunner software release

software/seleniumrunner/test/setup.py

+##############################################################################
+#
+# Copyright (c) 2018 Nexedi SA and Contributors. All Rights Reserved.
+#
+# WARNING: This program as such is intended to be used by professional
+# programmers who take the whole responsibility of assessing all potential
+# consequences resulting from its eventual inadequacies and bugs
+# End users who are looking for a ready-to-use solution with commercial
+# guarantees and support are strongly adviced to contract a Free Software
+# Service Company
+#
+# This program is Free Software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+#
+##############################################################################
+from setuptools import setup, find_packages
+
+version = '0.0.1.dev0'
+name = 'slapos.test.seleniumrunner'
+long_description = open("README.md").read()
+
+setup(name=name,
+      version=version,
+      description="Test for SlapOS' Seleniumrunner",
+      long_description=long_description,
+      long_description_content_type='text/markdown',
+      maintainer="Nexedi",
+      maintainer_email="info@nexedi.com",
+      url="https://lab.nexedi.com/nexedi/slapos",
+      packages=find_packages(),
+      install_requires=[
+        'slapos.core',
+        'supervisor',
+        'slapos.libnetworkcache',
+        'erp5.util',
+        'selenium',
+        'psutil',
+        'image',
+        'requests',
+        'paramiko',
+        ],
+      zip_safe=True,
+      test_suite='test',
+    )