Skip to content

slapos
slapos
Commit 160d9762 authored by Thomas Gambier's avatar Thomas Gambier 🚴🏼
Browse files
Options

Update Release Candidate

parents 25ce84fc abce372e
Show whitespace changes
Inline Side-by-side
Showing with 1019 additions and 669 deletions
component/lxml-python/buildout.cfg
View file @ 160d9762
...@@ -8,7 +8,8 @@ parts = ...@@ -8,7 +8,8 @@ parts =
lxml-python lxml-python
[lxml-python-env] [lxml-python-env]
XSLT_CONFIG = ${libxslt:location}/bin/xslt-config WITH_XSLT_CONFIG = ${libxslt:location}/bin/xslt-config
WITH_XML2_CONFIG = ${libxml2:location}/bin/xml2-config
[lxml-python] [lxml-python]
recipe = zc.recipe.egg:custom recipe = zc.recipe.egg:custom
......
component/qemu-kvm/buildout.cfg
View file @ 160d9762
...@@ -72,11 +72,11 @@ md5sum = 6097fdb9cbab47c96471274b9044e983 ...@@ -72,11 +72,11 @@ md5sum = 6097fdb9cbab47c96471274b9044e983
# XXX: This is not the latest version because # XXX: This is not the latest version because
# Debian does not provide a stable URL for it. # Debian does not provide a stable URL for it.
<= debian-amd64-netinst-base <= debian-amd64-netinst-base
version = 10.5.0 version = 10.7.0
md5sum = a3ebc76aec372808ad80000108a2593a md5sum = 7227c779619e6c8a0a1b0f55d10c6270
[debian-amd64-testing-netinst.iso] [debian-amd64-testing-netinst.iso]
<= debian-amd64-netinst-base <= debian-amd64-netinst-base
release = bullseye_di_alpha2 release = bullseye_di_alpha3
version = bullseye-DI-alpha2 version = bullseye-DI-alpha3
md5sum = 3d7f45ac47e36212f5f7924b41c47f61 md5sum = bff147077791586fa7c102267da9f2d2
component/trafficserver/buildout.cfg
View file @ 160d9762
...@@ -28,7 +28,7 @@ md5sum = 4f4d1e7de19c77157be0c2a825b31026 ...@@ -28,7 +28,7 @@ md5sum = 4f4d1e7de19c77157be0c2a825b31026
shared = true shared = true
patch-options = -p1 patch-options = -p1
patches = patches =
https://github.com/apache/trafficserver/commit/3d0492f7276bb93aff499f637ff843c45908b193.patch#f23dd950acc0395fe7bc72264987c013 https://github.com/apache/trafficserver/commit/254e9e22181ca369673407bd3fcd93e7287275ac.patch#1c324f76464d33d334a15a69490ccd55
configure-options = configure-options =
--with-openssl=${openssl:location} --with-openssl=${openssl:location}
--with-pcre=${pcre:location} --with-pcre=${pcre:location}
......
release-sr
View file @ 160d9762
#!/bin/bash #!/bin/bash
git fetch set -e
git fetch --all
# Go to master
git checkout master
# Clean up 1.0
git branch -D 1.0 || echo
# Checkout 1.0
git checkout origin/1.0 -b 1.0
git checkout 1.0
git reset origin/1.0 --hard git reset origin/1.0 --hard
# for now this script is hardcoded to release on 1.0.x versions intentionally # for now this script is hardcoded to release on 1.0.x versions intentionally
......
software/caddy-frontend/CHANGES.caddy_frontend.rst
View file @ 160d9762
...@@ -3,6 +3,25 @@ Changes ...@@ -3,6 +3,25 @@ Changes
Here are listed the most important changes, which might affect upgrades. Here are listed the most important changes, which might affect upgrades.
1.0.XXX (XXXX-XX-XX)
--------------------
* fix: exposed log file names are stabilised
* feature: in case of not found instance more information are provided
* feature: telemetry is fully disabled
* feature: Apache Traffic Server 8.0 is used
* feature: backend-haproxy statistic for haproxy's frontend is available
* fix: slave publication has been fixed in case of mixed case slave reference
* feature: running test/test.py resolves with starting backend used in tests
* fix: automatic caucase-updater usage has been fixed
* fix/workaround: reconnect to backend-haproxy from Caddy and Apache Traffic Server
* fix/feature: use explicitly Apache Traffic Server simulation of stale-if-error, as in reality Apache Traffic Server does not support it
* feature: dropped not used parameters
* feature: Strict-Transport-Security aka HSTS
* fix: use kedifa with with for file with multiple CAs
* feature: support query string (the characters after ? in the url) in url and https-url
* fix: by having unique acl names fix rare bug of directing traffic to https-url instead of url or otherwise
1.0.164 (2020-09-24) 1.0.164 (2020-09-24)
-------------------- --------------------
......
software/caddy-frontend/README.caddy_frontend.rst
View file @ 160d9762
...@@ -239,14 +239,14 @@ Necessary to activate cache. ...@@ -239,14 +239,14 @@ Necessary to activate cache.
``enable_cache`` is an optional parameter. ``enable_cache`` is an optional parameter.
backend-active-check-* health-check-*
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
This set of parameters is used to control the way how the backend checks will be done. Such active checks can be really useful for `stale-if-error` caching technique and especially in case if backend is very slow to reply or to connect to. This set of parameters is used to control the way how the backend checks will be done. Such active checks can be really useful for `stale-if-error` caching technique and especially in case if backend is very slow to reply or to connect to.
`backend-active-check-http-method` can be used to configure the HTTP method used to check the backend. Special method `CONNECT` can be used to check only for connection attempt. `health-check-http-method` can be used to configure the HTTP method used to check the backend. Special method `CONNECT` can be used to check only for connection attempt.
Please be aware that the `backend-active-check-timeout` is really short by default, so in case if `/` of the backend is slow to reply configure proper path with `backend-active-check-http-path` to not mark such backend down too fast, before increasing the check timeout. Please be aware that the `health-check-timeout` is really short by default, so in case if `/` of the backend is slow to reply configure proper path with `health-check-http-path` to not mark such backend down too fast, before increasing the check timeout.
Examples Examples
======== ========
......
software/caddy-frontend/buildout.hash.cfg
View file @ 160d9762
...@@ -26,11 +26,11 @@ md5sum = a6a626fd1579fd1d4b80ea67433ca16a ...@@ -26,11 +26,11 @@ md5sum = a6a626fd1579fd1d4b80ea67433ca16a
[profile-caddy-replicate] [profile-caddy-replicate]
filename = instance-apache-replicate.cfg.in filename = instance-apache-replicate.cfg.in
md5sum = 7cb8157d2b368ab3b281ea42f743eb9c md5sum = 9cc78e7ce1960691e37f103855ff0dc9
[profile-slave-list] [profile-slave-list]
_update_hash_filename_ = templates/apache-custom-slave-list.cfg.in _update_hash_filename_ = templates/apache-custom-slave-list.cfg.in
md5sum = 772c04c165fdae91299fd909e061f926 md5sum = eb98ffd96b2768cc6a5cf664b23aabd3
[profile-replicate-publish-slave-information] [profile-replicate-publish-slave-information]
_update_hash_filename_ = templates/replicate-publish-slave-information.cfg.in _update_hash_filename_ = templates/replicate-publish-slave-information.cfg.in
...@@ -50,7 +50,7 @@ md5sum = a0ae858a3db8825c22d33d323392f588 ...@@ -50,7 +50,7 @@ md5sum = a0ae858a3db8825c22d33d323392f588
[template-backend-haproxy-configuration] [template-backend-haproxy-configuration]
_update_hash_filename_ = templates/backend-haproxy.cfg.in _update_hash_filename_ = templates/backend-haproxy.cfg.in
md5sum = 0923a9227c131d2f1e11d7ddd5b15673 md5sum = 8c4e2548a12c8fd7dba74f940201745a
[template-empty] [template-empty]
_update_hash_filename_ = templates/empty.in _update_hash_filename_ = templates/empty.in
......
software/caddy-frontend/instance-apache-replicate.cfg.in
View file @ 160d9762
...@@ -123,33 +123,33 @@ context = ...@@ -123,33 +123,33 @@ context =
{% elif slave_type not in [None, '', 'default', 'zope', 'redirect', 'notebook', 'websocket'] %} {% elif slave_type not in [None, '', 'default', 'zope', 'redirect', 'notebook', 'websocket'] %}
{% do slave_error_list.append('type:%s is not supported' % (slave_type,)) %} {% do slave_error_list.append('type:%s is not supported' % (slave_type,)) %}
{% endif %} {% endif %}
{# Check backend-active-check-* #} {# Check health-check-* #}
{% set backend_active_check = (str(slave.get('backend-active-check', False)) or 'false').lower() %} {% set health_check = (str(slave.get('health-check', False)) or 'false').lower() %}
{% if backend_active_check in TRUE_VALUES %} {% if health_check in TRUE_VALUES %}
{% set backend_active_check_http_method = slave.get('backend-active-check-http-method') or 'GET' %} {% set health_check_http_method = slave.get('health-check-http-method') or 'GET' %}
{% if backend_active_check_http_method not in ['GET', 'OPTIONS', 'CONNECT', 'POST'] %} {% if health_check_http_method not in ['GET', 'OPTIONS', 'CONNECT', 'POST'] %}
{% do slave_error_list.append('Wrong backend-active-check-http-method %s' % (backend_active_check_http_method,)) %} {% do slave_error_list.append('Wrong health-check-http-method %s' % (health_check_http_method,)) %}
{% endif %} {% endif %}
{% set backend_active_check_http_path = slave.get('backend-active-check-http-path') or '/' %} {% set health_check_http_path = slave.get('health-check-http-path') or '/' %}
{% set backend_active_check_http_version = slave.get('backend-active-check-http-version') or 'HTTP/1.1' %} {% set health_check_http_version = slave.get('health-check-http-version') or 'HTTP/1.1' %}
{% if backend_active_check_http_version not in ['HTTP/1.1', 'HTTP/1.0'] %} {% if health_check_http_version not in ['HTTP/1.1', 'HTTP/1.0'] %}
{% do slave_error_list.append('Wrong backend-active-check-http-version %s' % (backend_active_check_http_version,)) %} {% do slave_error_list.append('Wrong health-check-http-version %s' % (health_check_http_version,)) %}
{% endif %} {% endif %}
{% set backend_active_check_timeout = (slave.get('backend-active-check-timeout') or '2') | int(false) %} {% set health_check_timeout = (slave.get('health-check-timeout') or '2') | int(false) %}
{% if backend_active_check_timeout is false or backend_active_check_timeout <= 0 %} {% if health_check_timeout is false or health_check_timeout <= 0 %}
{% do slave_error_list.append('Wrong backend-active-check-timeout %s' % (slave.get('backend-active-check-timeout'),)) %} {% do slave_error_list.append('Wrong health-check-timeout %s' % (slave.get('health-check-timeout'),)) %}
{% endif %} {% endif %}
{% set backend_active_check_interval = (slave.get('backend-active-check-interval') or '5') | int(false) %} {% set health_check_interval = (slave.get('health-check-interval') or '5') | int(false) %}
{% if backend_active_check_interval is false or backend_active_check_interval <= 0 %} {% if health_check_interval is false or health_check_interval <= 0 %}
{% do slave_error_list.append('Wrong backend-active-check-interval %s' % (slave.get('backend-active-check-interval'),)) %} {% do slave_error_list.append('Wrong health-check-interval %s' % (slave.get('health-check-interval'),)) %}
{% endif %} {% endif %}
{% set backend_active_check_rise = (slave.get('backend-active-check-rise') or '1') | int(false) %} {% set health_check_rise = (slave.get('health-check-rise') or '1') | int(false) %}
{% if backend_active_check_rise is false or backend_active_check_rise <= 0 %} {% if health_check_rise is false or health_check_rise <= 0 %}
{% do slave_error_list.append('Wrong backend-active-check-rise %s' % (slave.get('backend-active-check-rise'),)) %} {% do slave_error_list.append('Wrong health-check-rise %s' % (slave.get('health-check-rise'),)) %}
{% endif %} {% endif %}
{% set backend_active_check_fall = (slave.get('backend-active-check-fall') or '1') | int(false) %} {% set health_check_fall = (slave.get('health-check-fall') or '1') | int(false) %}
{% if backend_active_check_fall is false or backend_active_check_fall <= 0 %} {% if health_check_fall is false or health_check_fall <= 0 %}
{% do slave_error_list.append('Wrong backend-active-check-fall %s' % (slave.get('backend-active-check-fall'),)) %} {% do slave_error_list.append('Wrong health-check-fall %s' % (slave.get('health-check-fall'),)) %}
{% endif %} {% endif %}
{% endif %} {% endif %}
{# Check virtualhostroot-http-port and virtualhostroot-https-port #} {# Check virtualhostroot-http-port and virtualhostroot-https-port #}
......
software/caddy-frontend/instance-slave-caddy-input-schema.json
View file @ 160d9762
...@@ -8,14 +8,14 @@ ...@@ -8,14 +8,14 @@
"type": "string" "type": "string"
}, },
"url": { "url": {
"description": "Url of the backend", "description": "URL of the backend",
"pattern": "^(http|https|ftp)://", "pattern": "^(http|https|ftp)://",
"title": "Backend URL", "title": "Backend URL",
"type": "string" "type": "string"
}, },
"type": { "type": {
"default": "", "default": "",
"description": "Type of slave. If redirect, the slave will redirect to the given url. If zope, the rewrite rules will be compatible with Virtual Host Monster. Implemented are default, zope, redirect, notebook and websocket, not implemneted is eventsource.", "description": "Type of slave. If redirect, the slave will redirect to the given URL. If zope, the rewrite rules will be compatible with Virtual Host Monster. Implemented are default, zope, redirect, notebook and websocket, not implemneted is eventsource.",
"enum": [ "enum": [
"", "",
"zope", "zope",
...@@ -96,7 +96,7 @@ ...@@ -96,7 +96,7 @@
"type": "string" "type": "string"
}, },
"https-url": { "https-url": {
"description": "HTTPS Url of the backend if it is diferent from url parameter", "description": "HTTPS URL of the backend if it is different from url parameter",
"pattern": "^(http|https|ftp)://", "pattern": "^(http|https|ftp)://",
"title": "HTTPS Backend URL", "title": "HTTPS Backend URL",
"type": "string" "type": "string"
...@@ -224,8 +224,8 @@ ...@@ -224,8 +224,8 @@
"title": "Authenticate to backend", "title": "Authenticate to backend",
"type": "string" "type": "string"
}, },
"backend-active-check": { "health-check": {
"title": "Backend Active Check", "title": "Health Check",
"description": "Enables active checks of the backend. For HTTP level checks the HTTP code shall be 2xx or 3xx, otherwise backend will be considered down.", "description": "Enables active checks of the backend. For HTTP level checks the HTTP code shall be 2xx or 3xx, otherwise backend will be considered down.",
"enum": [ "enum": [
"false", "false",
...@@ -234,8 +234,8 @@ ...@@ -234,8 +234,8 @@
"default": "false", "default": "false",
"type": "string" "type": "string"
}, },
"backend-active-check-http-method": { "health-check-http-method": {
"title": "Backend Active Check HTTP Metod", "title": "Health Check HTTP Metod",
"description": "Selects method to do the active check. CONNECT means that connection will be enough for the check, otherwise it's HTTP method.", "description": "Selects method to do the active check. CONNECT means that connection will be enough for the check, otherwise it's HTTP method.",
"enum": [ "enum": [
"GET", "GET",
...@@ -246,14 +246,14 @@ ...@@ -246,14 +246,14 @@
"default": "GET", "default": "GET",
"type": "string" "type": "string"
}, },
"backend-active-check-http-path": { "health-check-http-path": {
"title": "Backend Active Check HTTP Path", "title": "Health Check HTTP Path",
"description": "A path on which do the active check, unused in case of CONNECT.", "description": "A path on which do the active check, unused in case of CONNECT.",
"default": "/", "default": "/",
"type": "string" "type": "string"
}, },
"backend-active-check-http-version": { "health-check-http-version": {
"title": "Backend Active Check HTTP Version", "title": "Health Check HTTP Version",
"description": "A HTTP version to use to check the backend, unused in case of CONNECT.", "description": "A HTTP version to use to check the backend, unused in case of CONNECT.",
"enum": [ "enum": [
"HTTP/1.1", "HTTP/1.1",
...@@ -262,26 +262,26 @@ ...@@ -262,26 +262,26 @@
"default": "HTTP/1.1", "default": "HTTP/1.1",
"type": "string" "type": "string"
}, },
"backend-active-check-timeout": { "health-check-timeout": {
"title": "Backend Active Check Timeout (seconds)", "title": "Health Check Timeout (seconds)",
"description": "A timeout to for the request to be fulfilled, after connection happen.", "description": "A timeout to for the request to be fulfilled, after connection happen.",
"default": "2", "default": "2",
"type": "integer" "type": "integer"
}, },
"backend-active-check-interval": { "health-check-interval": {
"title": "Backend Active Check Interval (seconds)", "title": "Health Check Interval (seconds)",
"description": "An interval of backend active check.", "description": "An interval of health check.",
"default": "5", "default": "5",
"type": "integer" "type": "integer"
}, },
"backend-active-check-rise": { "health-check-rise": {
"title": "Backend Active Check Rise", "title": "Health Check Rise",
"description": "Amount of correct responses from the backend to consider it up.", "description": "Amount of correct responses from the backend to consider it up.",
"default": "1", "default": "1",
"type": "integer" "type": "integer"
}, },
"backend-active-check-fall": { "health-check-fall": {
"title": "Backend Active Check Fall", "title": "Health Check Fall",
"description": "Amount of bad responses from the backend to consider it down.", "description": "Amount of bad responses from the backend to consider it down.",
"default": "1", "default": "1",
"type": "integer" "type": "integer"
......
software/caddy-frontend/software.cfg
View file @ 160d9762
...@@ -31,7 +31,7 @@ parts += ...@@ -31,7 +31,7 @@ parts +=
recipe = slapos.recipe.build:gitclone recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/kedifa.git repository = https://lab.nexedi.com/nexedi/kedifa.git
git-executable = ${git:location}/bin/git git-executable = ${git:location}/bin/git
revision = d6bbd7db215e12871c1536f22a8fbf994227270c revision = 1e38bcb3bf56f544262780be7e8858f3e6ceedc5
[kedifa-develop] [kedifa-develop]
recipe = zc.recipe.egg:develop recipe = zc.recipe.egg:develop
...@@ -228,15 +228,6 @@ mode = 0644 ...@@ -228,15 +228,6 @@ mode = 0644
[versions] [versions]
# Modern KeDiFa requires zc.lockfile # Modern KeDiFa requires zc.lockfile
zc.lockfile = 1.4 zc.lockfile = 1.4
# Versions pinned for kedifa need urllib3 >= 1.18
urllib3 = 1.24
requests = 2.20.0
certifi = 2018.10.15
idna = 2.7
chardet = 3.0.4
# ipaddress is patching IPAddress so IPv6 match works
ipaddress = 1.0.22
# Versions pinned for kedifa need urllib3 >= 1.18
validators = 0.12.2 validators = 0.12.2
PyRSS2Gen = 1.1 PyRSS2Gen = 1.1
......
software/caddy-frontend/templates/apache-custom-slave-list.cfg.in
View file @ 160d9762
...@@ -54,7 +54,7 @@ context = ...@@ -54,7 +54,7 @@ context =
{#- * stabilise values for backend #} {#- * stabilise values for backend #}
{%- for key, prefix in [('url', 'http_backend'), ('https-url', 'https_backend')] %} {%- for key, prefix in [('url', 'http_backend'), ('https-url', 'https_backend')] %}
{%- set parsed = urlparse_module.urlparse(slave_instance.get(key, '').strip()) %} {%- set parsed = urlparse_module.urlparse(slave_instance.get(key, '').strip()) %}
{%- set info_dict = {'scheme': parsed.scheme, 'hostname': parsed.hostname, 'port': parsed.port or DEFAULT_PORT[parsed.scheme], 'path': parsed.path, 'fragment': parsed.fragment} %} {%- set info_dict = {'scheme': parsed.scheme, 'hostname': parsed.hostname, 'port': parsed.port or DEFAULT_PORT[parsed.scheme], 'path': parsed.path, 'fragment': parsed.fragment, 'query': parsed.query} %}
{%- do slave_instance.__setitem__(prefix, info_dict) %} {%- do slave_instance.__setitem__(prefix, info_dict) %}
{%- endfor %} {%- endfor %}
{%- do slave_instance.__setitem__('ssl_proxy_verify', ('' ~ slave_instance.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES) %} {%- do slave_instance.__setitem__('ssl_proxy_verify', ('' ~ slave_instance.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES) %}
...@@ -136,32 +136,32 @@ context = ...@@ -136,32 +136,32 @@ context =
{%- do slave_instance.__setitem__('strict-transport-security', int(slave_instance['strict-transport-security'])) %} {%- do slave_instance.__setitem__('strict-transport-security', int(slave_instance['strict-transport-security'])) %}
{%- do slave_instance.__setitem__('authenticate-to-backend', ('' ~ slave_instance.get('authenticate-to-backend', '')).lower() in TRUE_VALUES) %} {%- do slave_instance.__setitem__('authenticate-to-backend', ('' ~ slave_instance.get('authenticate-to-backend', '')).lower() in TRUE_VALUES) %}
{#- Setup active check #} {#- Setup active check #}
{%- do slave_instance.__setitem__('backend-active-check', ('' ~ slave_instance.get('backend-active-check', '')).lower() in TRUE_VALUES) %} {%- do slave_instance.__setitem__('health-check', ('' ~ slave_instance.get('health-check', '')).lower() in TRUE_VALUES) %}
{%- if slave_instance['backend-active-check'] %} {%- if slave_instance['health-check'] %}
{%- if 'backend-active-check-http-method' not in slave_instance %} {%- if 'health-check-http-method' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-http-method', 'GET') %} {%- do slave_instance.__setitem__('health-check-http-method', 'GET') %}
{%- endif %} {%- endif %}
{%- if 'backend-active-check-http-version' not in slave_instance %} {%- if 'health-check-http-version' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-http-version', 'HTTP/1.1') %} {%- do slave_instance.__setitem__('health-check-http-version', 'HTTP/1.1') %}
{%- endif %} {%- endif %}
{%- if 'backend-active-check-interval' not in slave_instance %} {%- if 'health-check-interval' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-interval', '5') %} {%- do slave_instance.__setitem__('health-check-interval', '5') %}
{%- endif %} {%- endif %}
{%- if 'backend-active-check-rise' not in slave_instance %} {%- if 'health-check-rise' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-rise', '1') %} {%- do slave_instance.__setitem__('health-check-rise', '1') %}
{%- endif %} {%- endif %}
{%- if 'backend-active-check-fall' not in slave_instance %} {%- if 'health-check-fall' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-fall', '2') %} {%- do slave_instance.__setitem__('health-check-fall', '2') %}
{%- endif %} {%- endif %}
{%- if 'backend-active-check-timeout' not in slave_instance %} {%- if 'health-check-timeout' not in slave_instance %}
{%- do slave_instance.__setitem__('backend-active-check-timeout', '2') %} {%- do slave_instance.__setitem__('health-check-timeout', '2') %}
{%- endif %} {%- endif %}
{%- do slave_instance.__setitem__('backend-active-check-http-path', slave_instance.get('backend-active-check-http-path') or '/') %} {%- do slave_instance.__setitem__('health-check-http-path', slave_instance.get('health-check-http-path') or '/') %}
{%- else %} {%- else %}
{%- do slave_instance.__setitem__('backend-active-check-http-method', '') %} {%- do slave_instance.__setitem__('health-check-http-method', '') %}
{%- do slave_instance.__setitem__('backend-active-check-http-version', '') %} {%- do slave_instance.__setitem__('health-check-http-version', '') %}
{%- do slave_instance.__setitem__('backend-active-check-http-path', '') %} {%- do slave_instance.__setitem__('health-check-http-path', '') %}
{%- endif %} {# if backend_active_check #} {%- endif %} {# if slave_instance['health-check'] #}
{#- Set Up log files #} {#- Set Up log files #}
{%- do slave_parameter_dict.__setitem__('access_log', '/'.join([caddy_log_directory, '%s_access_log' % slave_reference])) %} {%- do slave_parameter_dict.__setitem__('access_log', '/'.join([caddy_log_directory, '%s_access_log' % slave_reference])) %}
{%- do slave_parameter_dict.__setitem__('error_log', '/'.join([caddy_log_directory, '%s_error_log' % slave_reference])) %} {%- do slave_parameter_dict.__setitem__('error_log', '/'.join([caddy_log_directory, '%s_error_log' % slave_reference])) %}
......
software/caddy-frontend/templates/backend-haproxy.cfg.in
View file @ 160d9762
...@@ -25,14 +25,14 @@ defaults ...@@ -25,14 +25,14 @@ defaults
{%- if wildcard and host.startswith('*.') %} {%- if wildcard and host.startswith('*.') %}
{%- do matched.__setitem__('count', matched['count'] + 1) %} {%- do matched.__setitem__('count', matched['count'] + 1) %}
# match wildcard {{ host }} # match wildcard {{ host }}
acl is_{{ slave_instance['slave_reference'] }} hdr_reg(host) -i {{ host[2:] }}($|:.*) acl is_{{ slave_instance['slave_reference'] }}_{{ scheme }} hdr_reg(host) -i {{ host[2:] }}($|:.*)
{%- elif not wildcard and not host.startswith('*.') %} {%- elif not wildcard and not host.startswith('*.') %}
{%- do matched.__setitem__('count', matched['count'] + 1) %} {%- do matched.__setitem__('count', matched['count'] + 1) %}
acl is_{{ slave_instance['slave_reference'] }} hdr_reg(host) -i ^{{ host }}($|:.*) acl is_{{ slave_instance['slave_reference'] }}_{{ scheme }} hdr_reg(host) -i ^{{ host }}($|:.*)
{%- endif %} {%- endif %}
{%- endfor %} {%- endfor %}
{%- if matched['count'] > 0 %} {%- if matched['count'] > 0 %}
use_backend {{ slave_instance['slave_reference'] }}-{{ scheme }} if is_{{ slave_instance['slave_reference'] }} use_backend {{ slave_instance['slave_reference'] }}-{{ scheme }} if is_{{ slave_instance['slave_reference'] }}_{{ scheme }}
{%- endif %} {%- endif %}
{%- endif %} {%- endif %}
{%- endmacro %} {%- endmacro %}
...@@ -92,22 +92,27 @@ frontend https-backend ...@@ -92,22 +92,27 @@ frontend https-backend
backend {{ slave_instance['slave_reference'] }}-{{ scheme }} backend {{ slave_instance['slave_reference'] }}-{{ scheme }}
{%- set hostname = info_dict['hostname'] %} {%- set hostname = info_dict['hostname'] %}
{%- set port = info_dict['port'] %} {%- set port = info_dict['port'] %}
{%- set path = info_dict['path'].rstrip('/') %} {%- set path_list = [info_dict['path'].rstrip('/')] %}
{%- set query = info_dict['query'] %}
{%- if query %}
{%- do path_list.append(query) %}
{%- endif %}
{%- set path = '?'.join(path_list) %}
{%- if hostname and port %} {%- if hostname and port %}
timeout server {{ slave_instance['request-timeout'] }}s timeout server {{ slave_instance['request-timeout'] }}s
timeout connect {{ slave_instance['backend-connect-timeout'] }}s timeout connect {{ slave_instance['backend-connect-timeout'] }}s
retries {{ slave_instance['backend-connect-retries'] }} retries {{ slave_instance['backend-connect-retries'] }}
{%- set active_check_list = [] %} {%- set active_check_list = [] %}
{%- set active_check_option_list = [] %} {%- set active_check_option_list = [] %}
{%- if slave_instance['backend-active-check'] %} {%- if slave_instance['health-check'] %}
{%- do active_check_list.append('check') %} {%- do active_check_list.append('check') %}
{%- do active_check_list.append('inter %ss' % (slave_instance['backend-active-check-interval'])) %} {%- do active_check_list.append('inter %ss' % (slave_instance['health-check-interval'])) %}
{%- do active_check_list.append('rise %s' % (slave_instance['backend-active-check-rise'])) %} {%- do active_check_list.append('rise %s' % (slave_instance['health-check-rise'])) %}
{%- do active_check_list.append('fall %s' % (slave_instance['backend-active-check-fall'])) %} {%- do active_check_list.append('fall %s' % (slave_instance['health-check-fall'])) %}
{%- if slave_instance['backend-active-check-http-method'] != 'CONNECT' %} {%- if slave_instance['health-check-http-method'] != 'CONNECT' %}
{%- do active_check_option_list.append('option httpchk %s %s %s' % (slave_instance['backend-active-check-http-method'], slave_instance['backend-active-check-http-path'] | urlencode, slave_instance['backend-active-check-http-version'])) %} {%- do active_check_option_list.append('option httpchk %s %s %s' % (slave_instance['health-check-http-method'], slave_instance['health-check-http-path'] | urlencode, slave_instance['health-check-http-version'])) %}
{%- endif %} {%- endif %}
{%- do active_check_option_list.append('timeout check %ss' % (slave_instance['backend-active-check-timeout'])) %} {%- do active_check_option_list.append('timeout check %ss' % (slave_instance['health-check-timeout'])) %}
{%- endif %} {%- endif %}
server {{ slave_instance['slave_reference'] }}-backend {{ hostname }}:{{ port }} {{ ' '.join(ssl_list) }} {{ ' ' + ' '.join(active_check_list)}} server {{ slave_instance['slave_reference'] }}-backend {{ hostname }}:{{ port }} {{ ' '.join(ssl_list) }} {{ ' ' + ' '.join(active_check_list)}}
{%- for active_check_option in active_check_option_list %} {%- for active_check_option in active_check_option_list %}
......
software/caddy-frontend/test/test.py
View file @ 160d9762
...@@ -37,6 +37,7 @@ from unittest import skip ...@@ -37,6 +37,7 @@ from unittest import skip
import ssl import ssl
from BaseHTTPServer import HTTPServer from BaseHTTPServer import HTTPServer
from BaseHTTPServer import BaseHTTPRequestHandler from BaseHTTPServer import BaseHTTPRequestHandler
from SocketServer import ThreadingMixIn
import time import time
import tempfile import tempfile
import ipaddress import ipaddress
...@@ -198,6 +199,10 @@ def createCSR(common_name, ip=None): ...@@ -198,6 +199,10 @@ def createCSR(common_name, ip=None):
return key, key_pem, csr, csr_pem return key, key_pem, csr, csr_pem
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
pass
class CertificateAuthority(object): class CertificateAuthority(object):
def __init__(self, common_name): def __init__(self, common_name):
self.key, self.key_pem = createKey() self.key, self.key_pem = createKey()
...@@ -419,13 +424,21 @@ def fakeHTTPSResult(domain, path, port=HTTPS_PORT, ...@@ -419,13 +424,21 @@ def fakeHTTPSResult(domain, path, port=HTTPS_PORT,
try: try:
add_custom_dns(domain, port, TEST_IP) add_custom_dns(domain, port, TEST_IP)
socket.getaddrinfo = new_getaddrinfo socket.getaddrinfo = new_getaddrinfo
return session.get( # Use a prepared request, to disable path normalization.
'https://%s:%s/%s' % (domain, port, path), # We need this because some test checks requests with paths like
verify=False, # /test-path/deep/.././deeper but we don't want the client to send
allow_redirects=False, # /test-path/deeper
# See also https://github.com/psf/requests/issues/5289
url = 'https://%s:%s/%s' % (domain, port, path)
req = requests.Request(
method='GET',
url=url,
headers=headers, headers=headers,
cookies=cookies cookies=cookies,
) )
prepped = req.prepare()
prepped.url = url
return session.send(prepped, verify=False, allow_redirects=False)
finally: finally:
socket.getaddrinfo = socket_getaddrinfo socket.getaddrinfo = socket_getaddrinfo
...@@ -447,17 +460,25 @@ def fakeHTTPResult(domain, path, port=HTTP_PORT, ...@@ -447,17 +460,25 @@ def fakeHTTPResult(domain, path, port=HTTP_PORT,
new_source = source.SourceAddressAdapter(source_ip) new_source = source.SourceAddressAdapter(source_ip)
session.mount('http://', new_source) session.mount('http://', new_source)
session.mount('https://', new_source) session.mount('https://', new_source)
return session.get(
'http://%s:%s/%s' % (TEST_IP, port, path), # Use a prepared request, to disable path normalization.
headers=headers, url = 'http://%s:%s/%s' % (TEST_IP, port, path)
allow_redirects=False, req = requests.Request(method='GET', url=url, headers=headers)
) prepped = req.prepare()
prepped.url = url
return session.send(prepped, allow_redirects=False)
class TestHandler(BaseHTTPRequestHandler): class TestHandler(BaseHTTPRequestHandler):
identification = None identification = None
configuration = {} configuration = {}
def log_message(self, *args):
if os.environ.get('SLAPOS_TEST_DEBUG'):
return BaseHTTPRequestHandler.log_message(self, *args)
else:
return
def do_DELETE(self): def do_DELETE(self):
config = self.configuration.pop(self.path, None) config = self.configuration.pop(self.path, None)
if config is None: if config is None:
...@@ -471,7 +492,7 @@ class TestHandler(BaseHTTPRequestHandler): ...@@ -471,7 +492,7 @@ class TestHandler(BaseHTTPRequestHandler):
def do_PUT(self): def do_PUT(self):
config = { config = {
'status_code': self.headers.dict.get('status-code', '200') 'status_code': self.headers.dict.get('x-reply-status-code', '200')
} }
prefix = 'x-reply-header-' prefix = 'x-reply-header-'
length = len(prefix) length = len(prefix)
...@@ -583,11 +604,11 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase): ...@@ -583,11 +604,11 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
@classmethod @classmethod
def startServerProcess(cls): def startServerProcess(cls):
server = HTTPServer( server = ThreadedHTTPServer(
(cls._ipv4_address, cls._server_http_port), (cls._ipv4_address, cls._server_http_port),
TestHandler) TestHandler)
server_https = HTTPServer( server_https = ThreadedHTTPServer(
(cls._ipv4_address, cls._server_https_port), (cls._ipv4_address, cls._server_https_port),
TestHandler) TestHandler)
...@@ -626,6 +647,48 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase): ...@@ -626,6 +647,48 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
cls.logger.warning( cls.logger.warning(
'Process %s still alive' % (process, )) 'Process %s still alive' % (process, ))
def startAuthenticatedServerProcess(self):
master_parameter_dict = self.parseConnectionParameterDict()
caucase_url = master_parameter_dict['backend-client-caucase-url']
ca_certificate = requests.get(caucase_url + '/cas/crt/ca.crt.pem')
assert ca_certificate.status_code == httplib.OK
ca_certificate_file = os.path.join(
self.working_directory, 'ca-backend-client.crt.pem')
with open(ca_certificate_file, 'w') as fh:
fh.write(ca_certificate.text)
class OwnTestHandler(TestHandler):
identification = 'Auth Backend'
server_https_auth = ThreadedHTTPServer(
(self._ipv4_address, self._server_https_auth_port),
OwnTestHandler)
server_https_auth.socket = ssl.wrap_socket(
server_https_auth.socket,
certfile=self.test_server_certificate_file.name,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=ca_certificate_file,
server_side=True)
self.backend_https_auth_url = 'https://%s:%s/' \
% server_https_auth.server_address
self.server_https_auth_process = multiprocessing.Process(
target=server_https_auth.serve_forever, name='HTTPSServerAuth')
self.server_https_auth_process.start()
self.logger.debug('Started process %s' % (self.server_https_auth_process,))
def stopAuthenticatedServerProcess(self):
self.logger.debug('Stopping process %s' % (
self.server_https_auth_process,))
self.server_https_auth_process.join(10)
self.server_https_auth_process.terminate()
time.sleep(0.1)
if self.server_https_auth_process.is_alive():
self.logger.warning(
'Process %s still alive' % (self.server_https_auth_process, ))
@classmethod @classmethod
def setUpMaster(cls): def setUpMaster(cls):
# run partition until AIKC finishes # run partition until AIKC finishes
...@@ -887,12 +950,12 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase): ...@@ -887,12 +950,12 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
return parsed_parameter_dict return parsed_parameter_dict
def getMasterPartitionPath(self): def getMasterPartitionPath(self):
return '/' + os.path.join( # partition w/o etc/trafficserver, but with buildout.cfg
*glob.glob( return [
os.path.join( q for q in glob.glob(os.path.join(self.instance_path, '*',))
self.instance_path, '*', 'etc', 'Caddyfile-rejected-slave' if not os.path.exists(
) os.path.join(q, 'etc', 'trafficserver')) and os.path.exists(
)[0].split('/')[:-2]) os.path.join(q, 'buildout.cfg'))][0]
def parseConnectionParameterDict(self): def parseConnectionParameterDict(self):
return self.parseParameterDict( return self.parseParameterDict(
...@@ -1220,7 +1283,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1220,7 +1283,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'Url': { 'Url': {
# make URL "incorrect", with whitespace, nevertheless it shall be # make URL "incorrect", with whitespace, nevertheless it shall be
# correctly handled # correctly handled
'url': ' ' + cls.backend_url + ' ', 'url': ' ' + cls.backend_url + '/?a=b&c=' + ' ',
# authenticating to http backend shall be no-op # authenticating to http backend shall be no-op
'authenticate-to-backend': True, 'authenticate-to-backend': True,
}, },
...@@ -1340,23 +1403,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1340,23 +1403,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'type': 'zope', 'type': 'zope',
'https-only': 'false', 'https-only': 'false',
}, },
'type-zope-ssl-proxy-verify_ssl_proxy_ca_crt': {
'url': cls.backend_https_url,
'type': 'zope',
'ssl-proxy-verify': True,
'ssl_proxy_ca_crt': cls.test_server_ca.certificate_pem,
},
'type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified': {
'url': cls.backend_https_url,
'type': 'zope',
'ssl-proxy-verify': True,
'ssl_proxy_ca_crt': cls.another_server_ca.certificate_pem,
},
'type-zope-ssl-proxy-verify-unverified': {
'url': cls.backend_https_url,
'type': 'zope',
'ssl-proxy-verify': True,
},
'type-zope-virtualhostroot-http-port': { 'type-zope-virtualhostroot-http-port': {
'url': cls.backend_url, 'url': cls.backend_url,
'type': 'zope', 'type': 'zope',
...@@ -1443,23 +1489,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1443,23 +1489,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'url': cls.backend_url, 'url': cls.backend_url,
'enable-http2': False, 'enable-http2': False,
}, },
'enable_cache-ssl-proxy-verify-unverified': {
'url': cls.backend_https_url,
'enable_cache': True,
'ssl-proxy-verify': True,
},
'enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt': {
'url': cls.backend_https_url,
'enable_cache': True,
'ssl_proxy_ca_crt': cls.test_server_ca.certificate_pem,
'ssl-proxy-verify': True,
},
'enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified': {
'url': cls.backend_https_url,
'enable_cache': True,
'ssl_proxy_ca_crt': cls.another_server_ca.certificate_pem,
'ssl-proxy-verify': True,
},
'enable-http2-default': { 'enable-http2-default': {
'url': cls.backend_url, 'url': cls.backend_url,
}, },
...@@ -1583,14 +1612,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1583,14 +1612,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
got_status_code_list got_status_code_list
) )
def getMasterPartitionPath(self):
# partition w/o etc/trafficserver, but with buildout.cfg
return [
q for q in glob.glob(os.path.join(self.instance_path, '*',))
if not os.path.exists(
os.path.join(q, 'etc', 'trafficserver')) and os.path.exists(
os.path.join(q, 'buildout.cfg'))][0]
def getSlavePartitionPath(self): def getSlavePartitionPath(self):
# partition w/ etc/trafficserver # partition w/ etc/trafficserver
return [ return [
...@@ -1660,15 +1681,15 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1660,15 +1681,15 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'monitor-base-url': 'https://[%s]:8401' % self._ipv6_address, 'monitor-base-url': 'https://[%s]:8401' % self._ipv6_address,
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address, 'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
'domain': 'example.com', 'domain': 'example.com',
'accepted-slave-amount': '56', 'accepted-slave-amount': '50',
'rejected-slave-amount': '0', 'rejected-slave-amount': '0',
'slave-amount': '56', 'slave-amount': '50',
'rejected-slave-dict': { 'rejected-slave-dict': {
}, },
'warning-slave-dict': { 'warning-slave-dict': {
'_Url': [ '_Url': [
"slave url ' %(backend)s ' has been converted to '%(backend)s'" % { "slave url ' %(backend)s/?a=b&c= ' has been converted to "
'backend': self.backend_url}]} "'%(backend)s/?a=b&c='" % {'backend': self.backend_url}]}
} }
self.assertEqual( self.assertEqual(
...@@ -1817,7 +1838,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1817,7 +1838,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'secure_access': 'https://%s.example.com' % (hostname, ), 'secure_access': 'https://%s.example.com' % (hostname, ),
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address, 'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
'warning-list': [ 'warning-list': [
"slave url ' %s ' has been converted to '%s'" % ( "slave url ' %s/?a=b&c= ' has been converted to '%s/?a=b&c='" % (
self.backend_url, self.backend_url)], self.backend_url, self.backend_url)],
}, },
parameter_dict parameter_dict
...@@ -1829,6 +1850,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1829,6 +1850,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
headers={ headers={
'Timeout': '10', # more than default backend-connect-timeout == 5 'Timeout': '10', # more than default backend-connect-timeout == 5
'Accept-Encoding': 'gzip', 'Accept-Encoding': 'gzip',
'User-Agent': 'TEST USER AGENT',
} }
) )
...@@ -1837,7 +1859,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1837,7 +1859,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert)) der2pem(result.peercert))
self.assertNotIn('Strict-Transport-Security', result.headers) self.assertNotIn('Strict-Transport-Security', result.headers)
self.assertEqualResultJson(result, 'Path', '/test-path/deeper') self.assertEqualResultJson(result, 'Path', '?a=b&c=/test-path/deeper')
try: try:
j = result.json() j = result.json()
...@@ -1862,7 +1884,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1862,7 +1884,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
log_regexp = r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} - - ' \ log_regexp = r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} - - ' \
r'\[\d{2}\/.{3}\/\d{4}\:\d{2}\:\d{2}\:\d{2} \+\d{4}\] ' \ r'\[\d{2}\/.{3}\/\d{4}\:\d{2}\:\d{2}\:\d{2} \+\d{4}\] ' \
r'"GET \/test-path\/deep\/..\/.\/deeper HTTP\/1.1" \d{3} ' \ r'"GET \/test-path\/deep\/..\/.\/deeper HTTP\/1.1" \d{3} ' \
r'\d+ "-" "python-requests.*" \d+' r'\d+ "-" "TEST USER AGENT" \d+'
self.assertRegexpMatches( self.assertRegexpMatches(
open(log_file, 'r').readlines()[-1], open(log_file, 'r').readlines()[-1],
...@@ -1879,7 +1901,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1879,7 +1901,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
r'http-backend _Url-http\/_Url-backend ' \ r'http-backend _Url-http\/_Url-backend ' \
r'\d+/\d+\/\d+\/\d+\/\d+ ' \ r'\d+/\d+\/\d+\/\d+\/\d+ ' \
r'200 \d+ - - ---- ' \ r'200 \d+ - - ---- ' \
r'\d\/\d\/\d\/\d\/\d \d\/\d ' \ r'\d+\/\d+\/\d+\/\d+\/\d+ \d+\/\d+ ' \
r'"GET /test-path/deeper HTTP/1.1"' r'"GET /test-path/deeper HTTP/1.1"'
self.assertRegexpMatches( self.assertRegexpMatches(
...@@ -1920,49 +1942,19 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1920,49 +1942,19 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
def test_auth_to_backend(self): def test_auth_to_backend(self):
parameter_dict = self.assertSlaveBase('auth-to-backend') parameter_dict = self.assertSlaveBase('auth-to-backend')
# 1. fetch certificate from backend-client-caucase-url
master_parameter_dict = self.parseConnectionParameterDict()
caucase_url = master_parameter_dict['backend-client-caucase-url']
ca_certificate = requests.get(caucase_url + '/cas/crt/ca.crt.pem')
assert ca_certificate.status_code == httplib.OK
ca_certificate_file = os.path.join(
self.working_directory, 'ca-backend-client.crt.pem')
with open(ca_certificate_file, 'w') as fh:
fh.write(ca_certificate.text)
# 2. start backend with this certificate
class OwnTestHandler(TestHandler):
identification = 'Auth Backend'
server_https_auth = HTTPServer( self.startAuthenticatedServerProcess()
(self._ipv4_address, self._server_https_auth_port),
OwnTestHandler)
server_https_auth.socket = ssl.wrap_socket(
server_https_auth.socket,
certfile=self.test_server_certificate_file.name,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=ca_certificate_file,
server_side=True)
backend_https_auth_url = 'https://%s:%s/' \
% server_https_auth.server_address
server_https_auth_process = multiprocessing.Process(
target=server_https_auth.serve_forever, name='HTTPSServerAuth')
server_https_auth_process.start()
self.logger.debug('Started process %s' % (server_https_auth_process,))
try: try:
# 3. assert that you can't fetch nothing without key # assert that you can't fetch nothing without key
try: try:
requests.get(backend_https_auth_url, verify=False) requests.get(self.backend_https_auth_url, verify=False)
except Exception: except Exception:
pass pass
else: else:
self.fail( self.fail(
'Access to %r shall be not possible without certificate' % ( 'Access to %r shall be not possible without certificate' % (
backend_https_auth_url,)) self.backend_https_auth_url,))
# 4. check that you can access this backend via frontend # check that you can access this backend via frontend
# (so it means that auth to backend worked) # (so it means that auth to backend worked)
result = fakeHTTPSResult( result = fakeHTTPSResult(
parameter_dict['domain'], parameter_dict['domain'],
...@@ -1999,59 +1991,22 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -1999,59 +1991,22 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
result.headers['X-Backend-Identification'] result.headers['X-Backend-Identification']
) )
finally: finally:
self.logger.debug('Stopping process %s' % (server_https_auth_process,)) self.stopAuthenticatedServerProcess()
server_https_auth_process.join(10)
server_https_auth_process.terminate()
time.sleep(0.1)
if server_https_auth_process.is_alive():
self.logger.warning(
'Process %s still alive' % (server_https_auth_process, ))
def test_auth_to_backend_not_configured(self): def test_auth_to_backend_not_configured(self):
parameter_dict = self.assertSlaveBase('auth-to-backend-not-configured') parameter_dict = self.assertSlaveBase('auth-to-backend-not-configured')
# 1. fetch certificate from backend-client-caucase-url self.startAuthenticatedServerProcess()
master_parameter_dict = self.parseConnectionParameterDict()
caucase_url = master_parameter_dict['backend-client-caucase-url']
ca_certificate = requests.get(caucase_url + '/cas/crt/ca.crt.pem')
assert ca_certificate.status_code == httplib.OK
ca_certificate_file = os.path.join(
self.working_directory, 'ca-backend-client.crt.pem')
with open(ca_certificate_file, 'w') as fh:
fh.write(ca_certificate.text)
# 2. start backend with this certificate
class OwnTestHandler(TestHandler):
identification = 'Auth Backend'
server_https_auth = HTTPServer(
(self._ipv4_address, self._server_https_auth_port),
OwnTestHandler)
server_https_auth.socket = ssl.wrap_socket(
server_https_auth.socket,
certfile=self.test_server_certificate_file.name,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=ca_certificate_file,
server_side=True)
backend_https_auth_url = 'https://%s:%s/' \
% server_https_auth.server_address
server_https_auth_process = multiprocessing.Process(
target=server_https_auth.serve_forever, name='HTTPSServerAuth')
server_https_auth_process.start()
self.logger.debug('Started process %s' % (server_https_auth_process,))
try: try:
# 3. assert that you can't fetch nothing without key # assert that you can't fetch nothing without key
try: try:
requests.get(backend_https_auth_url, verify=False) requests.get(self.backend_https_auth_url, verify=False)
except Exception: except Exception:
pass pass
else: else:
self.fail( self.fail(
'Access to %r shall be not possible without certificate' % ( 'Access to %r shall be not possible without certificate' % (
backend_https_auth_url,)) self.backend_https_auth_url,))
# 4. check that you can access this backend via frontend # check that you can access this backend via frontend
# (so it means that auth to backend worked) # (so it means that auth to backend worked)
result = fakeHTTPSResult( result = fakeHTTPSResult(
parameter_dict['domain'], parameter_dict['domain'],
...@@ -2071,13 +2026,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -2071,13 +2026,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
httplib.BAD_GATEWAY httplib.BAD_GATEWAY
) )
finally: finally:
self.logger.debug('Stopping process %s' % (server_https_auth_process,)) self.stopAuthenticatedServerProcess()
server_https_auth_process.join(10)
server_https_auth_process.terminate()
time.sleep(0.1)
if server_https_auth_process.is_alive():
self.logger.warning(
'Process %s still alive' % (server_https_auth_process, ))
def test_auth_to_backend_backend_ignore(self): def test_auth_to_backend_backend_ignore(self):
parameter_dict = self.assertSlaveBase('auth-to-backend-backend-ignore') parameter_dict = self.assertSlaveBase('auth-to-backend-backend-ignore')
...@@ -2141,7 +2090,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -2141,7 +2090,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'secure_access': 'https://%s.example.com' % (hostname, ), 'secure_access': 'https://%s.example.com' % (hostname, ),
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address, 'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
'warning-list': [ 'warning-list': [
"slave url ' %s ' has been converted to '%s'" % ( "slave url ' %s/?a=b&c= ' has been converted to '%s/?a=b&c='" % (
self.backend_url, self.backend_url)], self.backend_url, self.backend_url)],
}, },
parameter_dict parameter_dict
...@@ -2193,7 +2142,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -2193,7 +2142,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
'secure_access': 'https://%s.example.com' % (hostname, ), 'secure_access': 'https://%s.example.com' % (hostname, ),
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address, 'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
'warning-list': [ 'warning-list': [
"slave url ' %s ' has been converted to '%s'" % ( "slave url ' %s/?a=b&c= ' has been converted to '%s/?a=b&c='" % (
self.backend_url, self.backend_url)], self.backend_url, self.backend_url)],
}, },
parameter_dict parameter_dict
...@@ -3496,208 +3445,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -3496,208 +3445,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
result.status_code result.status_code
) )
def test_enable_cache_ssl_proxy_verify_ssl_proxy_ca_crt_unverified(self):
parameter_dict = self.parseSlaveParameterDict(
'enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified')
self.assertLogAccessUrlWithPop(parameter_dict)
self.assertKedifaKeysWithPop(parameter_dict)
self.assertEqual(
{
'domain':
'enablecachesslproxyverifysslproxycacrtunverified.example.com',
'replication_number': '1',
'url':
'http://enablecachesslproxyverifysslproxycacrtunverified.example.com',
'site_url':
'http://enablecachesslproxyverifysslproxycacrtunverified.example.com',
'secure_access':
'https://enablecachesslproxyverifysslproxycacrtunverified.example.com',
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
},
parameter_dict
)
result = fakeHTTPSResult(
parameter_dict['domain'],
'test-path/deep/.././deeper')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
result_http = fakeHTTPResult(
parameter_dict['domain'],
'test-path/deeper')
self.assertEqual(
httplib.FOUND,
result_http.status_code
)
self.assertEqual(
'https://enablecachesslproxyverifysslproxycacrtunverified.example.com'
':%s/test-path/deeper' % (HTTP_PORT,),
result_http.headers['Location']
)
def test_enable_cache_ssl_proxy_verify_ssl_proxy_ca_crt(self):
parameter_dict = self.assertSlaveBase(
'enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt')
result = fakeHTTPSResult(
parameter_dict['domain'],
'test-path/deep/.././deeper')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
self.assertEqualResultJson(result, 'Path', '/test-path/deeper')
headers = result.headers.copy()
self.assertKeyWithPop('Server', headers)
self.assertKeyWithPop('Date', headers)
self.assertKeyWithPop('Age', headers)
# drop keys appearing randomly in headers
headers.pop('Transfer-Encoding', None)
headers.pop('Content-Length', None)
headers.pop('Connection', None)
headers.pop('Keep-Alive', None)
self.assertEqual(
{
'Content-type': 'application/json',
'Set-Cookie': 'secured=value;secure, nonsecured=value',
},
headers
)
def test_enable_cache_ssl_proxy_verify_unverified(self):
parameter_dict = self.assertSlaveBase(
'enable_cache-ssl-proxy-verify-unverified')
result = fakeHTTPSResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
def test_type_zope_ssl_proxy_verify_ssl_proxy_ca_crt_unverified(self):
parameter_dict = self.parseSlaveParameterDict(
'type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified')
self.assertLogAccessUrlWithPop(parameter_dict)
self.assertKedifaKeysWithPop(parameter_dict)
self.assertEqual(
{
'domain': 'typezopesslproxyverifysslproxycacrtunverified.example.com',
'replication_number': '1',
'url':
'http://typezopesslproxyverifysslproxycacrtunverified.example.com',
'site_url':
'http://typezopesslproxyverifysslproxycacrtunverified.example.com',
'secure_access':
'https://typezopesslproxyverifysslproxycacrtunverified.example.com',
'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
},
parameter_dict
)
result = fakeHTTPSResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
result_http = fakeHTTPResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
httplib.FOUND,
result_http.status_code
)
self.assertEqual(
'https://typezopesslproxyverifysslproxycacrtunverified.example.com:%s/'
'test-path' % (HTTP_PORT,),
result_http.headers['Location']
)
def test_type_zope_ssl_proxy_verify_ssl_proxy_ca_crt(self):
parameter_dict = self.assertSlaveBase(
'type-zope-ssl-proxy-verify_ssl_proxy_ca_crt')
result = fakeHTTPSResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
try:
j = result.json()
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (result.text,))
self.assertBackendHeaders(j['Incoming Headers'], parameter_dict['domain'])
self.assertEqualResultJson(
result,
'Path',
'/VirtualHostBase/https//'
'typezopesslproxyverifysslproxycacrt.example.com:443/'
'/VirtualHostRoot/test-path'
)
result = fakeHTTPResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
httplib.FOUND,
result.status_code
)
self.assertEqual(
'https://typezopesslproxyverifysslproxycacrt.example.com:'
'%s/test-path' % (HTTP_PORT,),
result.headers['Location']
)
def test_type_zope_ssl_proxy_verify_unverified(self):
parameter_dict = self.assertSlaveBase(
'type-zope-ssl-proxy-verify-unverified')
result = fakeHTTPSResult(
parameter_dict['domain'], 'test-path')
self.assertEqual(
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
def test_monitor_ipv6_test(self): def test_monitor_ipv6_test(self):
parameter_dict = self.assertSlaveBase('monitor-ipv6-test') parameter_dict = self.assertSlaveBase('monitor-ipv6-test')
...@@ -4069,7 +3816,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -4069,7 +3816,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
backend_url = self.getSlaveParameterDictDict()['enable_cache']['url'] backend_url = self.getSlaveParameterDictDict()['enable_cache']['url']
result = requests.put(backend_url + path, headers={ result = requests.put(backend_url + path, headers={
'X-Reply-Header-Cache-Control': 'max-age=%s, public' % (max_age,), 'X-Reply-Header-Cache-Control': 'max-age=%s, public' % (max_age,),
'Status-Code': status_code, 'X-Reply-Status-Code': status_code,
'X-Reply-Body': base64.b64encode(body), 'X-Reply-Body': base64.b64encode(body),
}) })
self.assertEqual(result.status_code, httplib.CREATED) self.assertEqual(result.status_code, httplib.CREATED)
...@@ -4678,21 +4425,19 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -4678,21 +4425,19 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
retries 5""" in content) retries 5""" in content)
@skip('Impossible to instantiate cluster with stopped partition')
class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin): class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin):
@classmethod instance_parameter_dict = {
def getInstanceParameterDict(cls):
return {
'domain': 'example.com', 'domain': 'example.com',
'-frontend-quantity': 2,
'-sla-2-computer_guid': cls.slap._computer_id,
'-frontend-2-state': 'stopped',
'port': HTTPS_PORT, 'port': HTTPS_PORT,
'plain_http_port': HTTP_PORT, 'plain_http_port': HTTP_PORT,
'kedifa_port': KEDIFA_PORT, 'kedifa_port': KEDIFA_PORT,
'caucase_port': CAUCASE_PORT, 'caucase_port': CAUCASE_PORT,
} }
@classmethod
def getInstanceParameterDict(cls):
return cls.instance_parameter_dict
@classmethod @classmethod
def getSlaveParameterDictDict(cls): def getSlaveParameterDictDict(cls):
return { return {
...@@ -4703,6 +4448,29 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -4703,6 +4448,29 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin):
} }
def test(self): def test(self):
# now instantiate 2nd partition in started state
# and due to port collision, stop the first one...
self.instance_parameter_dict.update({
'-frontend-quantity': 2,
'-sla-2-computer_guid': self.slap._computer_id,
'-frontend-1-state': 'stopped',
'-frontend-2-state': 'started',
})
self.requestDefaultInstance()
self.requestSlaves()
self.slap.waitForInstance(self.instance_max_retry)
# ...and be nice, put back the first one online
self.instance_parameter_dict.update({
'-frontend-1-state': 'started',
'-frontend-2-state': 'stopped',
})
self.requestDefaultInstance()
self.slap.waitForInstance(self.instance_max_retry)
self.slap.waitForInstance(self.instance_max_retry)
self.slap.waitForInstance(self.instance_max_retry)
self.updateSlaveConnectionParameterDictDict()
# the real assertions follow...
parameter_dict = self.parseSlaveParameterDict('replicate') parameter_dict = self.parseSlaveParameterDict('replicate')
self.assertLogAccessUrlWithPop(parameter_dict) self.assertLogAccessUrlWithPop(parameter_dict)
self.assertKedifaKeysWithPop(parameter_dict) self.assertKedifaKeysWithPop(parameter_dict)
...@@ -4729,7 +4497,7 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -4729,7 +4497,7 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin):
result_http = fakeHTTPResult( result_http = fakeHTTPResult(
parameter_dict['domain'], 'test-path') parameter_dict['domain'], 'test-path')
self.assertEqualResultJson(result_http, 'Path', '/test-path') self.assertEqual(httplib.FOUND, result_http.status_code)
# prove 2nd frontend by inspection of the instance # prove 2nd frontend by inspection of the instance
slave_configuration_name = '_replicate.conf' slave_configuration_name = '_replicate.conf'
...@@ -4744,21 +4512,19 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -4744,21 +4512,19 @@ class TestReplicateSlave(SlaveHttpFrontendTestCase, TestDataMixin):
2, len(slave_configuration_file_list), slave_configuration_file_list) 2, len(slave_configuration_file_list), slave_configuration_file_list)
@skip('Impossible to instantiate cluster with destroyed partition')
class TestReplicateSlaveOtherDestroyed(SlaveHttpFrontendTestCase): class TestReplicateSlaveOtherDestroyed(SlaveHttpFrontendTestCase):
@classmethod instance_parameter_dict = {
def getInstanceParameterDict(cls):
return {
'domain': 'example.com', 'domain': 'example.com',
'-frontend-quantity': 2,
'-sla-2-computer_guid': cls.slap._computer_id,
'-frontend-2-state': 'destroyed',
'port': HTTPS_PORT, 'port': HTTPS_PORT,
'plain_http_port': HTTP_PORT, 'plain_http_port': HTTP_PORT,
'kedifa_port': KEDIFA_PORT, 'kedifa_port': KEDIFA_PORT,
'caucase_port': CAUCASE_PORT, 'caucase_port': CAUCASE_PORT,
} }
@classmethod
def getInstanceParameterDict(cls):
return cls.instance_parameter_dict
@classmethod @classmethod
def getSlaveParameterDictDict(cls): def getSlaveParameterDictDict(cls):
return { return {
...@@ -4769,6 +4535,28 @@ class TestReplicateSlaveOtherDestroyed(SlaveHttpFrontendTestCase): ...@@ -4769,6 +4535,28 @@ class TestReplicateSlaveOtherDestroyed(SlaveHttpFrontendTestCase):
} }
def test_extra_slave_instance_list_not_present_destroyed_request(self): def test_extra_slave_instance_list_not_present_destroyed_request(self):
# now instantiate 2nd partition in started state
# and due to port collision, stop the first one
self.instance_parameter_dict.update({
'-frontend-quantity': 2,
'-sla-2-computer_guid': self.slap._computer_id,
'-frontend-1-state': 'stopped',
'-frontend-2-state': 'started',
})
self.requestDefaultInstance()
self.slap.waitForInstance(self.instance_max_retry)
# now start back first instance, and destroy 2nd one
self.instance_parameter_dict.update({
'-frontend-1-state': 'started',
'-frontend-2-state': 'destroyed',
})
self.requestDefaultInstance()
self.slap.waitForInstance(self.instance_max_retry)
self.slap.waitForInstance(self.instance_max_retry)
self.slap.waitForInstance(self.instance_max_retry)
buildout_file = os.path.join( buildout_file = os.path.join(
self.getMasterPartitionPath(), 'buildout-switch-softwaretype.cfg') self.getMasterPartitionPath(), 'buildout-switch-softwaretype.cfg')
with open(buildout_file) as fh: with open(buildout_file) as fh:
...@@ -5021,19 +4809,20 @@ class TestRe6stVerificationUrlDefaultSlave(SlaveHttpFrontendTestCase, ...@@ -5021,19 +4809,20 @@ class TestRe6stVerificationUrlDefaultSlave(SlaveHttpFrontendTestCase,
) )
@skip('New test system cannot be used with failing promises')
class TestRe6stVerificationUrlSlave(SlaveHttpFrontendTestCase, class TestRe6stVerificationUrlSlave(SlaveHttpFrontendTestCase,
TestDataMixin): TestDataMixin):
@classmethod instance_parameter_dict = {
def getInstanceParameterDict(cls):
return {
'port': HTTPS_PORT, 'port': HTTPS_PORT,
'domain': 'example.com',
'plain_http_port': HTTP_PORT, 'plain_http_port': HTTP_PORT,
're6st-verification-url': 'some-re6st-verification-url',
'kedifa_port': KEDIFA_PORT, 'kedifa_port': KEDIFA_PORT,
'caucase_port': CAUCASE_PORT, 'caucase_port': CAUCASE_PORT,
} }
@classmethod
def getInstanceParameterDict(cls):
return cls.instance_parameter_dict
@classmethod @classmethod
def getSlaveParameterDictDict(cls): def getSlaveParameterDictDict(cls):
return { return {
...@@ -5044,16 +4833,28 @@ class TestRe6stVerificationUrlSlave(SlaveHttpFrontendTestCase, ...@@ -5044,16 +4833,28 @@ class TestRe6stVerificationUrlSlave(SlaveHttpFrontendTestCase,
} }
def test_default(self): def test_default(self):
self.instance_parameter_dict[
're6st-verification-url'] = 'some-re6st-verification-url'
# re-request instance with updated parameters
self.requestDefaultInstance()
# run once instance, it's only needed for later checks
try:
self.slap.waitForInstance()
except Exception:
pass
parameter_dict = self.parseSlaveParameterDict('default') parameter_dict = self.parseSlaveParameterDict('default')
self.assertLogAccessUrlWithPop(parameter_dict) self.assertLogAccessUrlWithPop(parameter_dict)
self.assertKedifaKeysWithPop(parameter_dict) self.assertKedifaKeysWithPop(parameter_dict)
self.assertEqual( self.assertEqual(
{ {
'domain': 'default.None', 'backend-client-caucase-url': 'http://[%s]:8990' % self._ipv6_address,
'domain': 'default.example.com',
'replication_number': '1', 'replication_number': '1',
'url': 'http://default.None', 'url': 'http://default.example.com',
'site_url': 'http://default.None', 'site_url': 'http://default.example.com',
'secure_access': 'https://default.None', 'secure_access': 'https://default.example.com',
}, },
parameter_dict parameter_dict
) )
...@@ -6408,45 +6209,45 @@ class TestSlaveRejectReportUnsafeDamaged(SlaveHttpFrontendTestCase): ...@@ -6408,45 +6209,45 @@ class TestSlaveRejectReportUnsafeDamaged(SlaveHttpFrontendTestCase):
'ssl_key': '${section:option}ssl_keyunsafe\nunsafe', 'ssl_key': '${section:option}ssl_keyunsafe\nunsafe',
'ssl_crt': '${section:option}ssl_crtunsafe\nunsafe', 'ssl_crt': '${section:option}ssl_crtunsafe\nunsafe',
}, },
'backend-active-check-http-method': { 'health-check-http-method': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-http-method': 'WRONG', 'health-check-http-method': 'WRONG',
}, },
'backend-active-check-http-version': { 'health-check-http-version': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-http-version': 'WRONG/1.1', 'health-check-http-version': 'WRONG/1.1',
}, },
'backend-active-check-timeout': { 'health-check-timeout': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-timeout': 'WRONG', 'health-check-timeout': 'WRONG',
}, },
'backend-active-check-timeout-negative': { 'health-check-timeout-negative': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-timeout': '-2', 'health-check-timeout': '-2',
}, },
'backend-active-check-interval': { 'health-check-interval': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-interval': 'WRONG', 'health-check-interval': 'WRONG',
}, },
'backend-active-check-interval-negative': { 'health-check-interval-negative': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-interval': '-2', 'health-check-interval': '-2',
}, },
'backend-active-check-rise': { 'health-check-rise': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-rise': 'WRONG', 'health-check-rise': 'WRONG',
}, },
'backend-active-check-rise-negative': { 'health-check-rise-negative': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-rise': '-2', 'health-check-rise': '-2',
}, },
'backend-active-check-fall': { 'health-check-fall': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-fall': 'WRONG', 'health-check-fall': 'WRONG',
}, },
'backend-active-check-fall-negative': { 'health-check-fall-negative': {
'backend-active-check': True, 'health-check': True,
'backend-active-check-fall': '-2', 'health-check-fall': '-2',
} }
} }
...@@ -6502,26 +6303,26 @@ class TestSlaveRejectReportUnsafeDamaged(SlaveHttpFrontendTestCase): ...@@ -6502,26 +6303,26 @@ class TestSlaveRejectReportUnsafeDamaged(SlaveHttpFrontendTestCase):
'_EMPTY-BACKEND': [ '_EMPTY-BACKEND': [
"slave https-url '' invalid", "slave https-url '' invalid",
"slave url '' invalid"], "slave url '' invalid"],
'_backend-active-check-fall': [ '_health-check-fall': [
'Wrong backend-active-check-fall WRONG'], 'Wrong health-check-fall WRONG'],
'_backend-active-check-fall-negative': [ '_health-check-fall-negative': [
'Wrong backend-active-check-fall -2'], 'Wrong health-check-fall -2'],
'_backend-active-check-http-method': [ '_health-check-http-method': [
'Wrong backend-active-check-http-method WRONG'], 'Wrong health-check-http-method WRONG'],
'_backend-active-check-http-version': [ '_health-check-http-version': [
'Wrong backend-active-check-http-version WRONG/1.1'], 'Wrong health-check-http-version WRONG/1.1'],
'_backend-active-check-interval': [ '_health-check-interval': [
'Wrong backend-active-check-interval WRONG'], 'Wrong health-check-interval WRONG'],
'_backend-active-check-interval-negative': [ '_health-check-interval-negative': [
'Wrong backend-active-check-interval -2'], 'Wrong health-check-interval -2'],
'_backend-active-check-rise': [ '_health-check-rise': [
'Wrong backend-active-check-rise WRONG'], 'Wrong health-check-rise WRONG'],
'_backend-active-check-rise-negative': [ '_health-check-rise-negative': [
'Wrong backend-active-check-rise -2'], 'Wrong health-check-rise -2'],
'_backend-active-check-timeout': [ '_health-check-timeout': [
'Wrong backend-active-check-timeout WRONG'], 'Wrong health-check-timeout WRONG'],
'_backend-active-check-timeout-negative': [ '_health-check-timeout-negative': [
'Wrong backend-active-check-timeout -2'], 'Wrong health-check-timeout -2'],
}, },
'warning-slave-dict': { 'warning-slave-dict': {
'_SSL_CA_CRT_ONLY': [ '_SSL_CA_CRT_ONLY': [
...@@ -7223,7 +7024,7 @@ class TestPassedRequestParameter(HttpFrontendTestCase): ...@@ -7223,7 +7024,7 @@ class TestPassedRequestParameter(HttpFrontendTestCase):
) )
class TestSlaveBackendActiveCheck(SlaveHttpFrontendTestCase, TestDataMixin): class TestSlaveHealthCheck(SlaveHttpFrontendTestCase, TestDataMixin):
@classmethod @classmethod
def getInstanceParameterDict(cls): def getInstanceParameterDict(cls):
return { return {
...@@ -7240,28 +7041,28 @@ class TestSlaveBackendActiveCheck(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -7240,28 +7041,28 @@ class TestSlaveBackendActiveCheck(SlaveHttpFrontendTestCase, TestDataMixin):
def getSlaveParameterDictDict(cls): def getSlaveParameterDictDict(cls):
cls.setUpAssertionDict() cls.setUpAssertionDict()
return { return {
'backend-active-check-disabled': { 'health-check-disabled': {
'url': cls.backend_url, 'url': cls.backend_url,
}, },
'backend-active-check-default': { 'health-check-default': {
'url': cls.backend_url, 'url': cls.backend_url,
'backend-active-check': True, 'health-check': True,
}, },
'backend-active-check-connect': { 'health-check-connect': {
'url': cls.backend_url, 'url': cls.backend_url,
'backend-active-check': True, 'health-check': True,
'backend-active-check-http-method': 'CONNECT', 'health-check-http-method': 'CONNECT',
}, },
'backend-active-check-custom': { 'health-check-custom': {
'url': cls.backend_url, 'url': cls.backend_url,
'backend-active-check': True, 'health-check': True,
'backend-active-check-http-method': 'POST', 'health-check-http-method': 'POST',
'backend-active-check-http-path': '/POST-path to be encoded', 'health-check-http-path': '/POST-path to be encoded',
'backend-active-check-http-version': 'HTTP/1.0', 'health-check-http-version': 'HTTP/1.0',
'backend-active-check-timeout': '7', 'health-check-timeout': '7',
'backend-active-check-interval': '15', 'health-check-interval': '15',
'backend-active-check-rise': '3', 'health-check-rise': '3',
'backend-active-check-fall': '7', 'health-check-fall': '7',
}, },
} }
...@@ -7269,35 +7070,35 @@ class TestSlaveBackendActiveCheck(SlaveHttpFrontendTestCase, TestDataMixin): ...@@ -7269,35 +7070,35 @@ class TestSlaveBackendActiveCheck(SlaveHttpFrontendTestCase, TestDataMixin):
def setUpAssertionDict(cls): def setUpAssertionDict(cls):
backend = urlparse.urlparse(cls.backend_url).netloc backend = urlparse.urlparse(cls.backend_url).netloc
cls.assertion_dict = { cls.assertion_dict = {
'backend-active-check-disabled': """\ 'health-check-disabled': """\
backend _backend-active-check-disabled-http backend _health-check-disabled-http
timeout server 12s timeout server 12s
timeout connect 5s timeout connect 5s
retries 3 retries 3
server _backend-active-check-disabled-backend %s""" % (backend,), server _health-check-disabled-backend %s""" % (backend,),
'backend-active-check-connect': """\ 'health-check-connect': """\
backend _backend-active-check-connect-http backend _health-check-connect-http
timeout server 12s timeout server 12s
timeout connect 5s timeout connect 5s
retries 3 retries 3
server _backend-active-check-connect-backend %s check inter 5s""" server _health-check-connect-backend %s check inter 5s"""
""" rise 1 fall 2 """ rise 1 fall 2
timeout check 2s""" % (backend,), timeout check 2s""" % (backend,),
'backend-active-check-custom': """\ 'health-check-custom': """\
backend _backend-active-check-custom-http backend _health-check-custom-http
timeout server 12s timeout server 12s
timeout connect 5s timeout connect 5s
retries 3 retries 3
server _backend-active-check-custom-backend %s check inter 15s""" server _health-check-custom-backend %s check inter 15s"""
""" rise 3 fall 7 """ rise 3 fall 7
option httpchk POST /POST-path%%20to%%20be%%20encoded HTTP/1.0 option httpchk POST /POST-path%%20to%%20be%%20encoded HTTP/1.0
timeout check 7s""" % (backend,), timeout check 7s""" % (backend,),
'backend-active-check-default': """\ 'health-check-default': """\
backend _backend-active-check-default-http backend _health-check-default-http
timeout server 12s timeout server 12s
timeout connect 5s timeout connect 5s
retries 3 retries 3
server _backend-active-check-default-backend %s check inter 5s""" server _health-check-default-backend %s check inter 5s"""
""" rise 1 fall 2 """ rise 1 fall 2
option httpchk GET / HTTP/1.1 option httpchk GET / HTTP/1.1
timeout check 2s""" % (backend, ) timeout check 2s""" % (backend, )
...@@ -7329,28 +7130,28 @@ backend _backend-active-check-default-http ...@@ -7329,28 +7130,28 @@ backend _backend-active-check-default-http
self.assertEqualResultJson(result, 'Path', '/test-path/deeper') self.assertEqualResultJson(result, 'Path', '/test-path/deeper')
def test_backend_active_check_disabled(self): def test_health_check_disabled(self):
self._test('backend-active-check-disabled') self._test('health-check-disabled')
def test_backend_active_check_default(self): def test_health_check_default(self):
self._test('backend-active-check-default') self._test('health-check-default')
def test_backend_active_check_connect(self): def test_health_check_connect(self):
self._test('backend-active-check-connect') self._test('health-check-connect')
def test_backend_active_check_custom(self): def test_health_check_custom(self):
self._test('backend-active-check-custom') self._test('health-check-custom')
if __name__ == '__main__': if __name__ == '__main__':
class HTTP6Server(HTTPServer): class HTTP6Server(ThreadedHTTPServer):
address_family = socket.AF_INET6 address_family = socket.AF_INET6
ip, port = sys.argv[1], int(sys.argv[2]) ip, port = sys.argv[1], int(sys.argv[2])
if ':' in ip: if ':' in ip:
klass = HTTP6Server klass = HTTP6Server
url_template = 'http://[%s]:%s/' url_template = 'http://[%s]:%s/'
else: else:
klass = HTTPServer klass = ThreadedHTTPServer
url_template = 'http://%s:%s/' url_template = 'http://%s:%s/'
server = klass((ip, port), TestHandler) server = klass((ip, port), TestHandler)
......
software/caddy-frontend/test/test_data/test.TestSlaveBackendActiveCheck.test_file_list_log-CADDY.txt → software/caddy-frontend/test/test_data/test.TestRe6stVerificationUrlSlave.test_file_list_log-CADDY.txt
View file @ 160d9762
...@@ -9,18 +9,9 @@ T-2/var/log/backend-haproxy.log ...@@ -9,18 +9,9 @@ T-2/var/log/backend-haproxy.log
T-2/var/log/expose-csr_id.log T-2/var/log/expose-csr_id.log
T-2/var/log/frontend-access.log T-2/var/log/frontend-access.log
T-2/var/log/frontend-error.log T-2/var/log/frontend-error.log
T-2/var/log/httpd/_backend-active-check-connect_access_log T-2/var/log/httpd/_default_access_log
T-2/var/log/httpd/_backend-active-check-connect_backend_log T-2/var/log/httpd/_default_backend_log
T-2/var/log/httpd/_backend-active-check-connect_error_log T-2/var/log/httpd/_default_error_log
T-2/var/log/httpd/_backend-active-check-custom_access_log
T-2/var/log/httpd/_backend-active-check-custom_backend_log
T-2/var/log/httpd/_backend-active-check-custom_error_log
T-2/var/log/httpd/_backend-active-check-default_access_log
T-2/var/log/httpd/_backend-active-check-default_backend_log
T-2/var/log/httpd/_backend-active-check-default_error_log
T-2/var/log/httpd/_backend-active-check-disabled_access_log
T-2/var/log/httpd/_backend-active-check-disabled_backend_log
T-2/var/log/httpd/_backend-active-check-disabled_error_log
T-2/var/log/monitor-httpd-access.log T-2/var/log/monitor-httpd-access.log
T-2/var/log/monitor-httpd-error.log T-2/var/log/monitor-httpd-error.log
T-2/var/log/slave-introspection-access.log T-2/var/log/slave-introspection-access.log
......
software/caddy-frontend/test/test_data/test.TestReplicateSlave.test_file_list_etc_cron_d-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/etc/cron.d/logrotate
T-0/etc/cron.d/monitor-configurator
T-0/etc/cron.d/monitor-globalstate
T-0/etc/cron.d/monitor_collect
T-1/etc/cron.d/logrotate
T-1/etc/cron.d/monitor-configurator
T-1/etc/cron.d/monitor-globalstate
T-1/etc/cron.d/monitor_collect
T-2/etc/cron.d/logrotate
T-2/etc/cron.d/monitor-configurator
T-2/etc/cron.d/monitor-globalstate
T-2/etc/cron.d/monitor_collect
T-2/etc/cron.d/trafficserver-logrotate
T-3/etc/cron.d/logrotate
T-3/etc/cron.d/monitor-configurator
T-3/etc/cron.d/monitor-globalstate
T-3/etc/cron.d/monitor_collect
T-3/etc/cron.d/trafficserver-logrotate
software/caddy-frontend/test/test_data/test.TestReplicateSlave.test_file_list_log-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/var/log/monitor-httpd-access.log
T-0/var/log/monitor-httpd-error.log
T-0/var/log/slapgrid-T-0-error.log
T-1/var/log/expose-csr_id.log
T-1/var/log/kedifa.log
T-1/var/log/monitor-httpd-access.log
T-1/var/log/monitor-httpd-error.log
T-2/var/log/backend-haproxy.log
T-2/var/log/expose-csr_id.log
T-2/var/log/frontend-access.log
T-2/var/log/frontend-error.log
T-2/var/log/httpd/_replicate_access_log
T-2/var/log/httpd/_replicate_backend_log
T-2/var/log/httpd/_replicate_error_log
T-2/var/log/monitor-httpd-access.log
T-2/var/log/monitor-httpd-error.log
T-2/var/log/slave-introspection-access.log
T-2/var/log/slave-introspection-error.log
T-2/var/log/trafficserver/manager.log
T-3/var/log/backend-haproxy.log
T-3/var/log/expose-csr_id.log
T-3/var/log/frontend-access.log
T-3/var/log/frontend-error.log
T-3/var/log/httpd/_replicate_access_log
T-3/var/log/httpd/_replicate_error_log
T-3/var/log/monitor-httpd-access.log
T-3/var/log/monitor-httpd-error.log
T-3/var/log/slave-introspection-access.log
T-3/var/log/slave-introspection-error.log
T-3/var/log/trafficserver/manager.log
software/caddy-frontend/test/test_data/test.TestReplicateSlave.test_file_list_plugin-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/etc/plugin/__init__.py
T-0/etc/plugin/aibcc-user-caucase-updater.py
T-0/etc/plugin/aikc-user-caucase-updater.py
T-0/etc/plugin/buildout-T-0-status.py
T-0/etc/plugin/caucased-backend-client.py
T-0/etc/plugin/check-backend-haproxy-statistic-url-caddy-frontend-1.py
T-0/etc/plugin/check-backend-haproxy-statistic-url-caddy-frontend-2.py
T-0/etc/plugin/check-free-disk-space.py
T-0/etc/plugin/monitor-bootstrap-status.py
T-0/etc/plugin/monitor-http-frontend.py
T-0/etc/plugin/monitor-httpd-listening-on-tcp.py
T-0/etc/plugin/rejected-slave-publish-ip-port-listening.py
T-0/etc/plugin/rejected-slave.py
T-1/etc/plugin/__init__.py
T-1/etc/plugin/buildout-T-1-status.py
T-1/etc/plugin/caucased.py
T-1/etc/plugin/check-free-disk-space.py
T-1/etc/plugin/expose-csr_id-ip-port-listening.py
T-1/etc/plugin/kedifa-http-reply.py
T-1/etc/plugin/monitor-bootstrap-status.py
T-1/etc/plugin/monitor-http-frontend.py
T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
T-1/etc/plugin/promise-logrotate-setup.py
T-2/etc/plugin/__init__.py
T-2/etc/plugin/backend-client-caucase-updater.py
T-2/etc/plugin/backend-haproxy-configuration.py
T-2/etc/plugin/backend_haproxy_http.py
T-2/etc/plugin/backend_haproxy_https.py
T-2/etc/plugin/buildout-T-2-status.py
T-2/etc/plugin/caddy_frontend_ipv4_http.py
T-2/etc/plugin/caddy_frontend_ipv4_https.py
T-2/etc/plugin/caddy_frontend_ipv6_http.py
T-2/etc/plugin/caddy_frontend_ipv6_https.py
T-2/etc/plugin/caucase-updater.py
T-2/etc/plugin/check-free-disk-space.py
T-2/etc/plugin/expose-csr_id-ip-port-listening.py
T-2/etc/plugin/frontend-caddy-configuration-promise.py
T-2/etc/plugin/monitor-bootstrap-status.py
T-2/etc/plugin/monitor-http-frontend.py
T-2/etc/plugin/monitor-httpd-listening-on-tcp.py
T-2/etc/plugin/promise-logrotate-setup.py
T-2/etc/plugin/re6st-connectivity.py
T-2/etc/plugin/slave-introspection-configuration.py
T-2/etc/plugin/slave_introspection_https.py
T-2/etc/plugin/trafficserver-cache-availability.py
T-2/etc/plugin/trafficserver-port-listening.py
T-3/etc/plugin/__init__.py
T-3/etc/plugin/backend-client-caucase-updater.py
T-3/etc/plugin/backend-haproxy-configuration.py
T-3/etc/plugin/backend_haproxy_http.py
T-3/etc/plugin/backend_haproxy_https.py
T-3/etc/plugin/buildout-T-3-status.py
T-3/etc/plugin/caddy_frontend_ipv4_http.py
T-3/etc/plugin/caddy_frontend_ipv4_https.py
T-3/etc/plugin/caddy_frontend_ipv6_http.py
T-3/etc/plugin/caddy_frontend_ipv6_https.py
T-3/etc/plugin/caucase-updater.py
T-3/etc/plugin/check-free-disk-space.py
T-3/etc/plugin/expose-csr_id-ip-port-listening.py
T-3/etc/plugin/frontend-caddy-configuration-promise.py
T-3/etc/plugin/monitor-bootstrap-status.py
T-3/etc/plugin/monitor-http-frontend.py
T-3/etc/plugin/monitor-httpd-listening-on-tcp.py
T-3/etc/plugin/promise-logrotate-setup.py
T-3/etc/plugin/re6st-connectivity.py
T-3/etc/plugin/slave-introspection-configuration.py
T-3/etc/plugin/slave_introspection_https.py
T-3/etc/plugin/trafficserver-cache-availability.py
T-3/etc/plugin/trafficserver-port-listening.py
software/caddy-frontend/test/test_data/test.TestReplicateSlave.test_file_list_run-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/var/run/monitor-httpd.pid
T-1/var/run/kedifa.pid
T-1/var/run/monitor-httpd.pid
T-2/var/run/backend-haproxy-rsyslogd.pid
T-2/var/run/backend-haproxy.pid
T-2/var/run/backend_haproxy_configuration_last_state
T-2/var/run/backend_haproxy_graceful_configuration_state_signature
T-2/var/run/bhlog.sck
T-2/var/run/graceful_configuration_state_signature
T-2/var/run/httpd.pid
T-2/var/run/monitor-httpd.pid
T-2/var/run/slave-introspection.pid
T-2/var/run/slave_introspection_configuration_last_state
T-2/var/run/slave_introspection_graceful_configuration_state_signature
T-3/var/run/backend-haproxy.pid
T-3/var/run/backend_haproxy_configuration_last_state
T-3/var/run/backend_haproxy_graceful_configuration_state_signature
T-3/var/run/graceful_configuration_state_signature
T-3/var/run/slave_introspection_configuration_last_state
T-3/var/run/slave_introspection_graceful_configuration_state_signature
software/caddy-frontend/test/test_data/test.TestReplicateSlave.test_supervisor_state-CADDY.txt 0 → 100644
View file @ 160d9762
T-0:aibcc-user-caucase-updater-on-watch RUNNING
T-0:aikc-user-caucase-updater-on-watch RUNNING
T-0:bootstrap-monitor EXITED
T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
T-0:certificate_authority-{hash-generic}-on-watch RUNNING
T-0:crond-{hash-generic}-on-watch RUNNING
T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
T-0:monitor-httpd-graceful EXITED
T-0:rejected-slave-publish-{hash-rejected-slave-publish}-on-watch RUNNING
T-1:bootstrap-monitor EXITED
T-1:caucase-updater-on-watch RUNNING
T-1:caucased-{hash-generic}-on-watch RUNNING
T-1:certificate_authority-{hash-generic}-on-watch RUNNING
T-1:crond-{hash-generic}-on-watch RUNNING
T-1:expose-csr_id-{hash-generic}-on-watch RUNNING
T-1:kedifa-{hash-generic}-on-watch RUNNING
T-1:kedifa-reloader EXITED
T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
T-1:monitor-httpd-graceful EXITED
T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
T-2:backend-haproxy-safe-graceful EXITED
T-2:bootstrap-monitor EXITED
T-2:certificate_authority-{hash-generic}-on-watch RUNNING
T-2:crond-{hash-generic}-on-watch RUNNING
T-2:expose-csr_id-{hash-generic}-on-watch RUNNING
T-2:frontend-caddy-safe-graceful EXITED
T-2:frontend_caddy-{hash-caddy-T-2}-on-watch RUNNING
T-2:kedifa-login-certificate-caucase-updater-on-watch RUNNING
T-2:kedifa-updater-{hash-generic}-on-watch RUNNING
T-2:monitor-httpd-{hash-generic}-on-watch RUNNING
T-2:monitor-httpd-graceful EXITED
T-2:slave-instrospection-nginx-{hash-generic}-on-watch RUNNING
T-2:slave-introspection-safe-graceful EXITED
T-2:trafficserver-{hash-generic}-on-watch RUNNING
T-2:trafficserver-reload EXITED
T-3:6tunnel-11080-{hash-generic}-on-watch STOPPED
T-3:6tunnel-11443-{hash-generic}-on-watch STOPPED
T-3:backend-client-login-certificate-caucase-updater-on-watch STOPPED
T-3:backend-haproxy-{hash-generic}-on-watch STOPPED
T-3:backend-haproxy-rsyslogd-{hash-generic}-on-watch STOPPED
T-3:backend-haproxy-safe-graceful EXITED
T-3:bootstrap-monitor EXITED
T-3:certificate_authority-{hash-generic}-on-watch STOPPED
T-3:crond-{hash-generic}-on-watch STOPPED
T-3:expose-csr_id-{hash-generic}-on-watch STOPPED
T-3:frontend-caddy-safe-graceful EXITED
T-3:frontend_caddy-{hash-caddy-T-3}-on-watch STOPPED
T-3:kedifa-login-certificate-caucase-updater-on-watch STOPPED
T-3:kedifa-updater-{hash-generic}-on-watch STOPPED
T-3:monitor-httpd-{hash-generic}-on-watch STOPPED
T-3:monitor-httpd-graceful EXITED
T-3:slave-instrospection-nginx-{hash-generic}-on-watch STOPPED
T-3:slave-introspection-safe-graceful EXITED
T-3:trafficserver-{hash-generic}-on-watch STOPPED
T-3:trafficserver-reload EXITED
software/caddy-frontend/test/test_data/test.TestSlave.test_file_list_log-CADDY.txt
View file @ 160d9762
...@@ -55,15 +55,6 @@ T-2/var/log/httpd/_enable_cache-disable-no-cache-request_error_log ...@@ -55,15 +55,6 @@ T-2/var/log/httpd/_enable_cache-disable-no-cache-request_error_log
T-2/var/log/httpd/_enable_cache-disable-via-header_access_log T-2/var/log/httpd/_enable_cache-disable-via-header_access_log
T-2/var/log/httpd/_enable_cache-disable-via-header_backend_log T-2/var/log/httpd/_enable_cache-disable-via-header_backend_log
T-2/var/log/httpd/_enable_cache-disable-via-header_error_log T-2/var/log/httpd/_enable_cache-disable-via-header_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
T-2/var/log/httpd/_enable_cache_access_log T-2/var/log/httpd/_enable_cache_access_log
T-2/var/log/httpd/_enable_cache_backend_log T-2/var/log/httpd/_enable_cache_backend_log
T-2/var/log/httpd/_enable_cache_custom_domain_access_log T-2/var/log/httpd/_enable_cache_custom_domain_access_log
...@@ -150,15 +141,6 @@ T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend-https-only_error_lo ...@@ -150,15 +141,6 @@ T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend-https-only_error_lo
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_access_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_access_log
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_backend_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_backend_log
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_error_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_access_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_access_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_backend_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_backend_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_error_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_error_log
......
software/caddy-frontend/test/test_data/test.TestSlaveGlobalDisableHttp2.test_file_list_log-CADDY.txt
View file @ 160d9762
...@@ -55,15 +55,6 @@ T-2/var/log/httpd/_enable_cache-disable-no-cache-request_error_log ...@@ -55,15 +55,6 @@ T-2/var/log/httpd/_enable_cache-disable-no-cache-request_error_log
T-2/var/log/httpd/_enable_cache-disable-via-header_access_log T-2/var/log/httpd/_enable_cache-disable-via-header_access_log
T-2/var/log/httpd/_enable_cache-disable-via-header_backend_log T-2/var/log/httpd/_enable_cache-disable-via-header_backend_log
T-2/var/log/httpd/_enable_cache-disable-via-header_error_log T-2/var/log/httpd/_enable_cache-disable-via-header_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify-unverified_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_backend_log
T-2/var/log/httpd/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
T-2/var/log/httpd/_enable_cache_access_log T-2/var/log/httpd/_enable_cache_access_log
T-2/var/log/httpd/_enable_cache_backend_log T-2/var/log/httpd/_enable_cache_backend_log
T-2/var/log/httpd/_enable_cache_custom_domain_access_log T-2/var/log/httpd/_enable_cache_custom_domain_access_log
...@@ -150,15 +141,6 @@ T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend-https-only_error_lo ...@@ -150,15 +141,6 @@ T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend-https-only_error_lo
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_access_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_access_log
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_backend_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_backend_log
T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_error_log T-2/var/log/httpd/_type-zope-prefer-gzip-encoding-to-backend_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify-unverified_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_backend_log
T-2/var/log/httpd/_type-zope-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_access_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_access_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_backend_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_backend_log
T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_error_log T-2/var/log/httpd/_type-zope-virtualhostroot-http-port_error_log
......
software/caddy-frontend/test/test_data/test.TestSlaveHealthCheck.test_file_list_etc_cron_d-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/etc/cron.d/logrotate
T-0/etc/cron.d/monitor-configurator
T-0/etc/cron.d/monitor-globalstate
T-0/etc/cron.d/monitor_collect
T-1/etc/cron.d/logrotate
T-1/etc/cron.d/monitor-configurator
T-1/etc/cron.d/monitor-globalstate
T-1/etc/cron.d/monitor_collect
T-2/etc/cron.d/logrotate
T-2/etc/cron.d/monitor-configurator
T-2/etc/cron.d/monitor-globalstate
T-2/etc/cron.d/monitor_collect
T-2/etc/cron.d/trafficserver-logrotate
software/caddy-frontend/test/test_data/test.TestSlaveHealthCheck.test_file_list_log-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/var/log/monitor-httpd-access.log
T-0/var/log/monitor-httpd-error.log
T-0/var/log/slapgrid-T-0-error.log
T-1/var/log/expose-csr_id.log
T-1/var/log/kedifa.log
T-1/var/log/monitor-httpd-access.log
T-1/var/log/monitor-httpd-error.log
T-2/var/log/backend-haproxy.log
T-2/var/log/expose-csr_id.log
T-2/var/log/frontend-access.log
T-2/var/log/frontend-error.log
T-2/var/log/httpd/_health-check-connect_access_log
T-2/var/log/httpd/_health-check-connect_backend_log
T-2/var/log/httpd/_health-check-connect_error_log
T-2/var/log/httpd/_health-check-custom_access_log
T-2/var/log/httpd/_health-check-custom_backend_log
T-2/var/log/httpd/_health-check-custom_error_log
T-2/var/log/httpd/_health-check-default_access_log
T-2/var/log/httpd/_health-check-default_backend_log
T-2/var/log/httpd/_health-check-default_error_log
T-2/var/log/httpd/_health-check-disabled_access_log
T-2/var/log/httpd/_health-check-disabled_backend_log
T-2/var/log/httpd/_health-check-disabled_error_log
T-2/var/log/monitor-httpd-access.log
T-2/var/log/monitor-httpd-error.log
T-2/var/log/slave-introspection-access.log
T-2/var/log/slave-introspection-error.log
T-2/var/log/trafficserver/manager.log
software/caddy-frontend/test/test_data/test.TestSlaveHealthCheck.test_file_list_plugin-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/etc/plugin/__init__.py
T-0/etc/plugin/aibcc-user-caucase-updater.py
T-0/etc/plugin/aikc-user-caucase-updater.py
T-0/etc/plugin/buildout-T-0-status.py
T-0/etc/plugin/caucased-backend-client.py
T-0/etc/plugin/check-backend-haproxy-statistic-url-caddy-frontend-1.py
T-0/etc/plugin/check-free-disk-space.py
T-0/etc/plugin/monitor-bootstrap-status.py
T-0/etc/plugin/monitor-http-frontend.py
T-0/etc/plugin/monitor-httpd-listening-on-tcp.py
T-0/etc/plugin/rejected-slave-publish-ip-port-listening.py
T-0/etc/plugin/rejected-slave.py
T-1/etc/plugin/__init__.py
T-1/etc/plugin/buildout-T-1-status.py
T-1/etc/plugin/caucased.py
T-1/etc/plugin/check-free-disk-space.py
T-1/etc/plugin/expose-csr_id-ip-port-listening.py
T-1/etc/plugin/kedifa-http-reply.py
T-1/etc/plugin/monitor-bootstrap-status.py
T-1/etc/plugin/monitor-http-frontend.py
T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
T-1/etc/plugin/promise-logrotate-setup.py
T-2/etc/plugin/__init__.py
T-2/etc/plugin/backend-client-caucase-updater.py
T-2/etc/plugin/backend-haproxy-configuration.py
T-2/etc/plugin/backend_haproxy_http.py
T-2/etc/plugin/backend_haproxy_https.py
T-2/etc/plugin/buildout-T-2-status.py
T-2/etc/plugin/caddy_frontend_ipv4_http.py
T-2/etc/plugin/caddy_frontend_ipv4_https.py
T-2/etc/plugin/caddy_frontend_ipv6_http.py
T-2/etc/plugin/caddy_frontend_ipv6_https.py
T-2/etc/plugin/caucase-updater.py
T-2/etc/plugin/check-free-disk-space.py
T-2/etc/plugin/expose-csr_id-ip-port-listening.py
T-2/etc/plugin/frontend-caddy-configuration-promise.py
T-2/etc/plugin/monitor-bootstrap-status.py
T-2/etc/plugin/monitor-http-frontend.py
T-2/etc/plugin/monitor-httpd-listening-on-tcp.py
T-2/etc/plugin/promise-logrotate-setup.py
T-2/etc/plugin/re6st-connectivity.py
T-2/etc/plugin/slave-introspection-configuration.py
T-2/etc/plugin/slave_introspection_https.py
T-2/etc/plugin/trafficserver-cache-availability.py
T-2/etc/plugin/trafficserver-port-listening.py
software/caddy-frontend/test/test_data/test.TestSlaveHealthCheck.test_file_list_run-CADDY.txt 0 → 100644
View file @ 160d9762
T-0/var/run/monitor-httpd.pid
T-1/var/run/kedifa.pid
T-1/var/run/monitor-httpd.pid
T-2/var/run/backend-haproxy-rsyslogd.pid
T-2/var/run/backend-haproxy.pid
T-2/var/run/backend_haproxy_configuration_last_state
T-2/var/run/backend_haproxy_graceful_configuration_state_signature
T-2/var/run/bhlog.sck
T-2/var/run/graceful_configuration_state_signature
T-2/var/run/httpd.pid
T-2/var/run/monitor-httpd.pid
T-2/var/run/slave-introspection.pid
T-2/var/run/slave_introspection_configuration_last_state
T-2/var/run/slave_introspection_graceful_configuration_state_signature
software/caddy-frontend/test/test_data/test.TestSlaveHealthCheck.test_supervisor_state-CADDY.txt 0 → 100644
View file @ 160d9762
T-0:aibcc-user-caucase-updater-on-watch RUNNING
T-0:aikc-user-caucase-updater-on-watch RUNNING
T-0:bootstrap-monitor EXITED
T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
T-0:certificate_authority-{hash-generic}-on-watch RUNNING
T-0:crond-{hash-generic}-on-watch RUNNING
T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
T-0:monitor-httpd-graceful EXITED
T-0:rejected-slave-publish-{hash-rejected-slave-publish}-on-watch RUNNING
T-1:bootstrap-monitor EXITED
T-1:caucase-updater-on-watch RUNNING
T-1:caucased-{hash-generic}-on-watch RUNNING
T-1:certificate_authority-{hash-generic}-on-watch RUNNING
T-1:crond-{hash-generic}-on-watch RUNNING
T-1:expose-csr_id-{hash-generic}-on-watch RUNNING
T-1:kedifa-{hash-generic}-on-watch RUNNING
T-1:kedifa-reloader EXITED
T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
T-1:monitor-httpd-graceful EXITED
T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
T-2:backend-haproxy-safe-graceful EXITED
T-2:bootstrap-monitor EXITED
T-2:certificate_authority-{hash-generic}-on-watch RUNNING
T-2:crond-{hash-generic}-on-watch RUNNING
T-2:expose-csr_id-{hash-generic}-on-watch RUNNING
T-2:frontend-caddy-safe-graceful EXITED
T-2:frontend_caddy-{hash-caddy-T-2}-on-watch RUNNING
T-2:kedifa-login-certificate-caucase-updater-on-watch RUNNING
T-2:kedifa-updater-{hash-generic}-on-watch RUNNING
T-2:monitor-httpd-{hash-generic}-on-watch RUNNING
T-2:monitor-httpd-graceful EXITED
T-2:slave-instrospection-nginx-{hash-generic}-on-watch RUNNING
T-2:slave-introspection-safe-graceful EXITED
T-2:trafficserver-{hash-generic}-on-watch RUNNING
T-2:trafficserver-reload EXITED
software/caucase/buildout.hash.cfg
View file @ 160d9762
...@@ -15,8 +15,8 @@ ...@@ -15,8 +15,8 @@
# not need these here). # not need these here).
[instance-caucased] [instance-caucased]
filename = instance-caucased.cfg.jinja2 filename = instance-caucased.cfg.jinja2
md5sum = 2277c891a71534e00487468f5048d196 md5sum = 8081efb41f12592033283db9841178ea
[instance] [instance]
filename = instance.cfg.jinja2 filename = instance.cfg.jinja2
md5sum = d40bed5ccc457ff7dc99b618bf29b189 md5sum = 687ed460bebc18768ff7b7315e9b6de5
software/caucase/instance-caucased.cfg.jinja2
View file @ 160d9762
...@@ -27,3 +27,4 @@ parts = ...@@ -27,3 +27,4 @@ parts =
publish publish
caucased caucased
caucased-promise caucased-promise
extends = {{ template_monitor }}
software/caucase/instance.cfg.jinja2
View file @ 160d9762
...@@ -20,10 +20,12 @@ instance-caucased = {{ dumps(instance_caucased) }} ...@@ -20,10 +20,12 @@ instance-caucased = {{ dumps(instance_caucased) }}
recipe = slapos.recipe.template:jinja2 recipe = slapos.recipe.template:jinja2
template = ${context:instance-caucased} template = ${context:instance-caucased}
rendered = ${buildout:parts-directory}/instance-caucased.cfg rendered = ${buildout:parts-directory}/instance-caucased.cfg
monitor = {{ template_monitor }}
context = context =
key ipv6_set slap-configuration:ipv6 key ipv6_set slap-configuration:ipv6
key slapparameter_dict slap-configuration:configuration key slapparameter_dict slap-configuration:configuration
key bin_directory context:bin-directory key bin_directory context:bin-directory
key template_monitor :monitor
import-list = import-list =
file caucase context:caucase-jinja2-library file caucase context:caucase-jinja2-library
......
software/caucase/software.cfg
View file @ 160d9762
...@@ -2,6 +2,8 @@ ...@@ -2,6 +2,8 @@
extends = extends =
buildout.hash.cfg buildout.hash.cfg
../../stack/caucase/buildout.cfg ../../stack/caucase/buildout.cfg
# Monitoring stack (keep on bottom)
../../stack/monitor/buildout.cfg
parts += parts +=
instance instance
...@@ -24,3 +26,4 @@ context = ...@@ -24,3 +26,4 @@ context =
key eggs_directory buildout:eggs-directory key eggs_directory buildout:eggs-directory
key caucase_jinja2_library caucase-jinja2-library:target key caucase_jinja2_library caucase-jinja2-library:target
key instance_caucased instance-caucased:target key instance_caucased instance-caucased:target
key template_monitor monitor2-template:rendered
software/caucase/test/setup.py 0 → 100644
View file @ 160d9762
##############################################################################
#
# Copyright (c) 2018 Nexedi SA and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from setuptools import setup, find_packages
version = '0.0.1.dev0'
name = 'slapos.test.caucase'
setup(
name=name,
version=version,
description="Test for SlapOS' Caucase",
maintainer="Nexedi",
maintainer_email="info@nexedi.com",
url="https://lab.nexedi.com/nexedi/slapos",
packages=find_packages(),
install_requires=[
'slapos.core',
'slapos.libnetworkcache',
'erp5.util',
'supervisor',
'pexpect',
'requests',
],
zip_safe=True,
test_suite='test',
)
software/caucase/test/test.py 0 → 100644
View file @ 160d9762
##############################################################################
#
# Copyright (c) 2019 Nexedi SA and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from __future__ import unicode_literals
import json
import os
import requests
import httplib
from slapos.testing.testcase import makeModuleSetUpAndTestCaseClass
setUpModule, SlapOSInstanceTestCase = makeModuleSetUpAndTestCaseClass(
os.path.abspath(
os.path.join(os.path.dirname(__file__), '..', 'software.cfg')))
class TestCaucase(SlapOSInstanceTestCase):
def deserializeConnectionParameter(self):
return json.loads(
self.computer_partition.getConnectionParameter('_'))
def test(self):
connection_parameter_dict = self.deserializeConnectionParameter()
self.assertEqual(
connection_parameter_dict,
{'url': 'http://[%s]:8009' % (self._ipv6_address,)}
)
result = requests.get(connection_parameter_dict['url'])
self.assertEqual(result.status_code, httplib.OK)
self.assertEqual(
result.json(),
{
'_links': {
'self': {
'href': 'http://[%s]:8009' % (self._ipv6_address,)
},
'getCAUHAL': {
'href': 'http://[%s]:8009//cau' % (self._ipv6_address,),
'title': 'cau'
},
'getCASHAL': {
'href': 'http://[%s]:8009//cas' % (self._ipv6_address,),
'title': 'cas'
}
}
}
)
software/erp5/test/test/test_balancer.py
View file @ 160d9762
...@@ -750,7 +750,7 @@ class TestFrontendXForwardedFor(BalancerTestCase): ...@@ -750,7 +750,7 @@ class TestFrontendXForwardedFor(BalancerTestCase):
).json() ).json()
self.assertNotEqual(result['Incoming Headers'].get('x-forwarded-for', '').split(', ')[0], '1.2.3.4') self.assertNotEqual(result['Incoming Headers'].get('x-forwarded-for', '').split(', ')[0], '1.2.3.4')
balancer_url = json.loads(self.computer_partition.getConnectionParameterDict()['_'])['default-auth'] balancer_url = json.loads(self.computer_partition.getConnectionParameterDict()['_'])['default-auth']
with self.assertRaises(OpenSSL.SSL.Error): with self.assertRaisesRegexp(Exception, "certificate required"):
requests.get( requests.get(
balancer_url, balancer_url,
headers={'X-Forwarded-For': '1.2.3.4'}, headers={'X-Forwarded-For': '1.2.3.4'},
......
software/erp5/test/test/test_mariadb.py
View file @ 160d9762
...@@ -63,6 +63,7 @@ class MariaDBTestCase(ERP5InstanceTestCase): ...@@ -63,6 +63,7 @@ class MariaDBTestCase(ERP5InstanceTestCase):
return { return {
'tcpv4-port': 3306, 'tcpv4-port': 3306,
'max-connection-count': 5, 'max-connection-count': 5,
'long-query-time': 3,
'max-slowqueries-threshold': 1, 'max-slowqueries-threshold': 1,
'slowest-query-threshold': 0.1, 'slowest-query-threshold': 0.1,
# XXX what is this ? should probably not be needed here # XXX what is this ? should probably not be needed here
...@@ -115,12 +116,13 @@ class TestCrontabs(MariaDBTestCase, CrontabMixin): ...@@ -115,12 +116,13 @@ class TestCrontabs(MariaDBTestCase, CrontabMixin):
# run logrotate a first time so that it create state files # run logrotate a first time so that it create state files
self._executeCrontabAtDate('logrotate', '2000-01-01') self._executeCrontabAtDate('logrotate', '2000-01-01')
# make two slow queries # make two slow queries. We are using long-query-time=3, so the queries
# must take more than 3 seconds to be logged.
cnx = self.getDatabaseConnection() cnx = self.getDatabaseConnection()
with contextlib.closing(cnx): with contextlib.closing(cnx):
cnx.query("SELECT SLEEP(1.1)") cnx.query("SELECT SLEEP(3.1)")
cnx.store_result() cnx.store_result()
cnx.query("SELECT SLEEP(1.2)") cnx.query("SELECT SLEEP(3.2)")
# slow query crontab depends on crontab for log rotation # slow query crontab depends on crontab for log rotation
# to be executed first. # to be executed first.
...@@ -171,7 +173,7 @@ class TestCrontabs(MariaDBTestCase, CrontabMixin): ...@@ -171,7 +173,7 @@ class TestCrontabs(MariaDBTestCase, CrontabMixin):
"""\ """\
Threshold is lower than expected: Threshold is lower than expected:
Expected total queries : 1.0 and current is: 2 Expected total queries : 1.0 and current is: 2
Expected slowest query : 0.1 and current is: 1 Expected slowest query : 0.1 and current is: 3
""") """)
......
software/jstestnode/buildout.hash.cfg
View file @ 160d9762
...@@ -27,4 +27,4 @@ md5sum = 98faa5ad8cfb23a11d97a459078a1d05 ...@@ -27,4 +27,4 @@ md5sum = 98faa5ad8cfb23a11d97a459078a1d05
[template-runTestSuite] [template-runTestSuite]
filename = runTestSuite.in filename = runTestSuite.in
md5sum = bb3f053b6cdb0a8888e9d32e63085ed5 md5sum = 73ef758825563b7a6a1d660d4d5229b8
software/jstestnode/runTestSuite.in
View file @ 160d9762
...@@ -97,8 +97,8 @@ def main(): ...@@ -97,8 +97,8 @@ def main():
executable_path='${geckodriver:location}') executable_path='${geckodriver:location}')
else: else:
assert target == 'selenium-server', "Unsupported target {}".format(test_runner['target']) assert target == 'selenium-server', "Unsupported target {}".format(test_runner['target'])
# use a remote connection which verifies TLS certificate # use a remote connection which optionally verifies TLS certificate
# workaround for https://github.com/SeleniumHQ/selenium/issues/6534 # and optionally against a user provided CA.
executor = RemoteConnection(test_runner['server-url'], keep_alive=True) executor = RemoteConnection(test_runner['server-url'], keep_alive=True)
cert_reqs = 'CERT_REQUIRED' cert_reqs = 'CERT_REQUIRED'
ca_certs = certifi.where() ca_certs = certifi.where()
......
software/jstestnode/software.cfg
View file @ 160d9762
...@@ -117,7 +117,4 @@ output = ${buildout:directory}/template-nginx.cfg.in ...@@ -117,7 +117,4 @@ output = ${buildout:directory}/template-nginx.cfg.in
output = ${buildout:directory}/runTestSuite.in output = ${buildout:directory}/runTestSuite.in
[versions] [versions]
selenium = 3.14.1 selenium = 3.14.1
urllib3 = 1.24
certifi = 2018.10.15
software/kvm/buildout.hash.cfg
View file @ 160d9762
...@@ -23,7 +23,7 @@ md5sum = 23493c541efef97ac5fe435114910b8e ...@@ -23,7 +23,7 @@ md5sum = 23493c541efef97ac5fe435114910b8e
[template-kvm-cluster] [template-kvm-cluster]
filename = instance-kvm-cluster.cfg.jinja2.in filename = instance-kvm-cluster.cfg.jinja2.in
md5sum = bdf8549a76ec61e125d51a05e611e004 md5sum = 28a00c28a972f42627849b25c2792abb
[template-kvm-resilient] [template-kvm-resilient]
filename = instance-kvm-resilient.cfg.jinja2 filename = instance-kvm-resilient.cfg.jinja2
......
software/kvm/instance-kvm-cluster.cfg.jinja2.in
View file @ 160d9762
...@@ -94,7 +94,15 @@ config-auto-ballooning = {{ dumps(kvm_parameter_dict.get('auto-ballooning', True ...@@ -94,7 +94,15 @@ config-auto-ballooning = {{ dumps(kvm_parameter_dict.get('auto-ballooning', True
{{ setconfig('disk-cache', kvm_parameter_dict.get('disk-cache', '')) }} {{ setconfig('disk-cache', kvm_parameter_dict.get('disk-cache', '')) }}
{{ setconfig('disk-device-path', kvm_parameter_dict.get('disk-device-path', '')) }} {{ setconfig('disk-device-path', kvm_parameter_dict.get('disk-device-path', '')) }}
{% set nat_rules_list = kvm_parameter_dict.get('nat-rules', []) -%} {# Note: dirty_nat_rules_list is cleaned up later, as the UI generated by JSON schema #}
{# gives freedom to the user to enter values separated by spaces and newlines #}
{# but on UI level they are only supported when separated by newlines, which #}
{# leads to cryptic failures of the cluster #}
{% set dirty_nat_rules_list = kvm_parameter_dict.get('nat-rules', []) -%}
{% set nat_rules_list = [] %}
{% for nat_rule in dirty_nat_rules_list %}
{% do nat_rules_list.extend(nat_rule.split()) %}
{% endfor %}
{{ setconfig('nat-rules', nat_rules_list | join(' ')) }} {{ setconfig('nat-rules', nat_rules_list | join(' ')) }}
config-publish-nat-url = True config-publish-nat-url = True
config-use-nat = {{ dumps(use_nat) }} config-use-nat = {{ dumps(use_nat) }}
......
software/kvm/test/test.py
View file @ 160d9762
...@@ -924,3 +924,78 @@ class TestCpuMemMaxDynamic(InstanceTestCase): ...@@ -924,3 +924,78 @@ class TestCpuMemMaxDynamic(InstanceTestCase):
self.assertIn('smp_max_count = 3', kvm_raw) self.assertIn('smp_max_count = 3', kvm_raw)
self.assertIn('ram_size = 2048', kvm_raw) self.assertIn('ram_size = 2048', kvm_raw)
self.assertIn("ram_max_size = '2560'", kvm_raw) self.assertIn("ram_max_size = '2560'", kvm_raw)
@skipUnlessKvm
class TestNatRules(InstanceTestCase):
__partition_reference__ = 'nr'
@classmethod
def getInstanceParameterDict(cls):
return {
'nat-rules': '100 200',
}
def test(self):
connection_parameter_dict = self.computer_partition\
.getConnectionParameterDict()
self.assertIn('nat-rule-port-tcp-100', connection_parameter_dict)
self.assertIn('nat-rule-port-tcp-200', connection_parameter_dict)
self.assertEqual(
'%s : 10100' % (self._ipv6_address,),
connection_parameter_dict['nat-rule-port-tcp-100']
)
self.assertEqual(
'%s : 10200' % (self._ipv6_address,),
connection_parameter_dict['nat-rule-port-tcp-200']
)
@skipUnlessKvm
class TestNatRulesKvmCluster(InstanceTestCase):
__partition_reference__ = 'nrkc'
nat_rules = ["100", "200", "300"]
@classmethod
def getInstanceSoftwareType(cls):
return 'kvm-cluster'
@classmethod
def getInstanceParameterDict(cls):
return {'_': json.dumps({
"kvm-partition-dict": {
"KVM0": {
"nat-rules": cls.nat_rules,
"disable-ansible-promise": True,
}
}
})}
def getRunningHostFwd(self):
with self.slap.instance_supervisor_rpc as instance_supervisor:
kvm_pid = [q for q in instance_supervisor.getAllProcessInfo()
if 'kvm-' in q['name']][0]['pid']
kvm_process = psutil.Process(kvm_pid)
for entry in kvm_process.cmdline():
if 'hostfwd' in entry:
return entry
def test(self):
host_fwd_entry = self.getRunningHostFwd()
self.assertIn(
'hostfwd=tcp:%s:10100-:100' % (self._ipv4_address,),
host_fwd_entry)
self.assertIn(
'hostfwd=tcp:%s:10200-:200' % (self._ipv4_address,),
host_fwd_entry)
self.assertIn(
'hostfwd=tcp:%s:10300-:300' % (self._ipv4_address,),
host_fwd_entry)
@skipUnlessKvm
class TestNatRulesKvmClusterComplex(TestNatRulesKvmCluster):
__partition_reference__ = 'nrkcc'
nat_rules = ["100", "200 300"]
software/monitor/software.cfg
View file @ 160d9762
...@@ -81,12 +81,7 @@ surykatka = 0.5.0 ...@@ -81,12 +81,7 @@ surykatka = 0.5.0
# For surykatka 0.5.0 # For surykatka 0.5.0
click = 7.0 click = 7.0
certifi = 2019.11.28
chardet = 3.0.4
dnspython = 1.16.0 dnspython = 1.16.0
forcediphttpsadapter = 1.0.1 forcediphttpsadapter = 1.0.1
idna = 2.8
miniupnpc = 2.0.2 miniupnpc = 2.0.2
peewee = 3.13.1 peewee = 3.13.1
requests = 2.22.0
urllib3 = 1.25.8
software/seleniumserver/test/test.py
View file @ 160d9762
...@@ -46,8 +46,10 @@ from PIL import Image ...@@ -46,8 +46,10 @@ from PIL import Image
from selenium import webdriver from selenium import webdriver
from selenium.webdriver.common.by import By from selenium.webdriver.common.by import By
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.remote.remote_connection import RemoteConnection
from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support.ui import WebDriverWait
import urllib3
from slapos.testing.testcase import makeModuleSetUpAndTestCaseClass from slapos.testing.testcase import makeModuleSetUpAndTestCaseClass
from slapos.testing.utils import findFreeTCPPort, ImageComparisonTestCase, ManagedHTTPServer from slapos.testing.utils import findFreeTCPPort, ImageComparisonTestCase, ManagedHTTPServer
...@@ -318,8 +320,13 @@ class TestFrontend(WebServerMixin, SeleniumServerTestCase): ...@@ -318,8 +320,13 @@ class TestFrontend(WebServerMixin, SeleniumServerTestCase):
self.assertEqual('selenium', parsed.username) self.assertEqual('selenium', parsed.username)
self.assertTrue(parsed.password) self.assertTrue(parsed.password)
# XXX we are using a self signed certificate, but selenium 3.141.0 does
# not expose API to ignore certificate verification
executor = RemoteConnection(webdriver_url, keep_alive=True)
executor._conn = urllib3.PoolManager(cert_reqs='CERT_NONE', ca_certs=None)
driver = webdriver.Remote( driver = webdriver.Remote(
command_executor=webdriver_url, command_executor=executor,
desired_capabilities=DesiredCapabilities.CHROME) desired_capabilities=DesiredCapabilities.CHROME)
driver.get(self.server_url) driver.get(self.server_url)
......
software/slapos-sr-testing/software.cfg
View file @ 160d9762
...@@ -167,6 +167,11 @@ setup = ${slapos-repository:location}/software/repman/test/ ...@@ -167,6 +167,11 @@ setup = ${slapos-repository:location}/software/repman/test/
repository = https://lab.nexedi.com/nexedi/slapos.core.git repository = https://lab.nexedi.com/nexedi/slapos.core.git
branch = master branch = master
[slapos.test.caucase-setup]
<= setup-develop-egg
egg = slapos.test.caucase
setup = ${slapos-repository:location}/software/caucase/test/
[slapos.core-setup] [slapos.core-setup]
<= setup-develop-egg <= setup-develop-egg
egg = slapos.core egg = slapos.core
...@@ -209,6 +214,7 @@ extra-eggs = ...@@ -209,6 +214,7 @@ extra-eggs =
${slapos.test.dream-setup:egg} ${slapos.test.dream-setup:egg}
${slapos.test.metabase-setup:egg} ${slapos.test.metabase-setup:egg}
${slapos.test.repman-setup:egg} ${slapos.test.repman-setup:egg}
${slapos.test.caucase-setup:egg}
# We don't name this interpreter `python`, so that when we run slapos node # We don't name this interpreter `python`, so that when we run slapos node
# software, installation scripts running `python` use a python without any # software, installation scripts running `python` use a python without any
...@@ -278,6 +284,7 @@ extra = ...@@ -278,6 +284,7 @@ extra =
gitlab ${slapos.test.gitlab-setup:setup} gitlab ${slapos.test.gitlab-setup:setup}
cloudooo ${slapos.test.cloudooo-setup:setup} cloudooo ${slapos.test.cloudooo-setup:setup}
dream ${slapos.test.dream-setup:setup} dream ${slapos.test.dream-setup:setup}
caucase ${slapos.test.caucase-setup:setup}
[versions] [versions]
# slapos.core is used from the clone always # slapos.core is used from the clone always
...@@ -298,16 +305,7 @@ selenium = 3.141.0 ...@@ -298,16 +305,7 @@ selenium = 3.141.0
# Patched eggs # Patched eggs
PyPDF2 = 1.26.0+SlapOSPatched001 PyPDF2 = 1.26.0+SlapOSPatched001
# modern versions for SSL fixed support # caucase and its dependencies
requests = 2.20.0
certifi = 2018.10.15
idna = 2.7
chardet = 3.0.4
# ipaddress is patching IPAddress so IPv6 match works
ipaddress = 1.0.22
# cacuase and its dependencies
caucase = 0.9.4 caucase = 0.9.4
pem = 18.2.0 pem = 18.2.0
PyJWT = 1.6.4 PyJWT = 1.6.4
...@@ -315,10 +313,6 @@ PyJWT = 1.6.4 ...@@ -315,10 +313,6 @@ PyJWT = 1.6.4
# Django 1.11 is python 2 compatible # Django 1.11 is python 2 compatible
Django = 1.11 Django = 1.11
# Required by:
# selenium==3.141.0
urllib3 = 1.24.1
mock = 2.0.0 mock = 2.0.0
testfixtures = 6.11 testfixtures = 6.11
funcsigs = 1.0.2 funcsigs = 1.0.2
......
software/slapos-testing/software.cfg
View file @ 160d9762
...@@ -203,22 +203,29 @@ output = ${buildout:directory}/template.cfg ...@@ -203,22 +203,29 @@ output = ${buildout:directory}/template.cfg
mode = 640 mode = 640
[versions] [versions]
# clear version pins of tested eggs, to make sure buildout always use the # When possible, clear version pins of tested eggs, to make sure buildout
# git checkout version. # always use the git checkout version.
# This is not possible for buildout extensions, or for buildout recipes
# that are also used in this profile, so we keep version pins for these.
# We need to keep the versions in the profiles (in stack/slapos.cfg, or
# maybe here during development) in sync with the latest version from their
# setup.py , because if the version in setup.py is newer from the version
# pin, buildout will install the egg from pypi and run the test against
# released egg
caucase = caucase =
erp5.util = erp5.util =
kedifa = kedifa =
slapos.cookbook = slapos.cookbook =
slapos.core = slapos.core =
slapos.libnetworkcache = # slapos.libnetworkcache =
slapos.rebootstrap = # slapos.rebootstrap =
slapos.recipe.build = # slapos.recipe.build =
slapos.recipe.cmmi = # slapos.recipe.cmmi =
slapos.recipe.template = # slapos.recipe.template =
slapos.toolbox = slapos.toolbox =
rubygemsrecipe = rubygemsrecipe =
# All depencies should be pinned. # All other depencies should be pinned.
Pygments = 2.1.3 Pygments = 2.1.3
zc.lockfile = 1.4 zc.lockfile = 1.4
bcrypt = 3.1.4 bcrypt = 3.1.4
......
stack/erp5/zope-versions.cfg
View file @ 160d9762
...@@ -5,7 +5,6 @@ Acquisition = 2.13.12 ...@@ -5,7 +5,6 @@ Acquisition = 2.13.12
DateTime = 2.12.8 DateTime = 2.12.8
DocumentTemplate = 2.13.6 DocumentTemplate = 2.13.6
ExtensionClass = 2.13.2 ExtensionClass = 2.13.2
Jinja2 = 2.8.1
MarkupSafe = 1.1.1 MarkupSafe = 1.1.1
Missing = 2.13.1 Missing = 2.13.1
MultiMapping = 2.13.0 MultiMapping = 2.13.0
......
stack/erp5/ztk-versions.cfg
View file @ 160d9762
...@@ -74,7 +74,6 @@ zope.documenttemplate = 3.4.3 ...@@ -74,7 +74,6 @@ zope.documenttemplate = 3.4.3
ClientForm = 0.2.10 ClientForm = 0.2.10
distribute = 0.6.49 distribute = 0.6.49
docutils = 0.7 docutils = 0.7
Jinja2 = 2.5.5
# Newer versions of mechanize are not fully py24 compatible. # Newer versions of mechanize are not fully py24 compatible.
mechanize = 0.1.11 mechanize = 0.1.11
Paste = 1.7.5.1 Paste = 1.7.5.1
......
stack/slapos.cfg
View file @ 160d9762
...@@ -145,8 +145,8 @@ hexagonit.recipe.download = 1.7.post4 ...@@ -145,8 +145,8 @@ hexagonit.recipe.download = 1.7.post4
Jinja2 = 2.11.2 Jinja2 = 2.11.2
Importing = 1.10 Importing = 1.10
MarkupSafe = 1.0 MarkupSafe = 1.0
PyYAML = 3.13 PyYAML = 5.4.1
Werkzeug = 0.12 Werkzeug = 1.0.1
asn1crypto = 1.3.0 asn1crypto = 1.3.0
atomicwrites = 1.4.0 atomicwrites = 1.4.0
backports.functools-lru-cache = 1.6.1 backports.functools-lru-cache = 1.6.1
...@@ -159,7 +159,7 @@ collective.recipe.shelloutput = 0.1 ...@@ -159,7 +159,7 @@ collective.recipe.shelloutput = 0.1
collective.recipe.template = 2.0 collective.recipe.template = 2.0
configparser = 4.0.2 configparser = 4.0.2
contextlib2 = 0.6.0.post1 contextlib2 = 0.6.0.post1
cryptography = 2.9.2 cryptography = 3.3.1
dateparser = 0.7.6 dateparser = 0.7.6
decorator = 4.3.0 decorator = 4.3.0
funcsigs = 1.0.2 funcsigs = 1.0.2
...@@ -171,7 +171,7 @@ importlib-metadata = 1.7.0 ...@@ -171,7 +171,7 @@ importlib-metadata = 1.7.0
inotify-simple = 1.1.1 inotify-simple = 1.1.1
itsdangerous = 0.24 itsdangerous = 0.24
lock-file = 2.0 lock-file = 2.0
lxml = 4.4.3 lxml = 4.6.2
meld3 = 1.0.2 meld3 = 1.0.2
more-itertools = 5.0.0 more-itertools = 5.0.0
netaddr = 0.7.19 netaddr = 0.7.19
...@@ -179,7 +179,7 @@ pathlib2 = 2.3.5 ...@@ -179,7 +179,7 @@ pathlib2 = 2.3.5
pbr = 2.0.0 pbr = 2.0.0
plone.recipe.command = 1.1 plone.recipe.command = 1.1
prettytable = 0.7.2 prettytable = 0.7.2
psutil = 5.6.3 psutil = 5.8.0
pluggy = 0.13.1 pluggy = 0.13.1
py = 1.9.0 py = 1.9.0
pyOpenSSL = 19.1.0 pyOpenSSL = 19.1.0
...@@ -200,7 +200,7 @@ slapos.rebootstrap = 4.5 ...@@ -200,7 +200,7 @@ slapos.rebootstrap = 4.5
slapos.recipe.build = 0.46 slapos.recipe.build = 0.46
slapos.recipe.cmmi = 0.16 slapos.recipe.cmmi = 0.16
slapos.recipe.template = 4.5 slapos.recipe.template = 4.5
slapos.toolbox = 0.115 slapos.toolbox = 0.116
stevedore = 1.21.0 stevedore = 1.21.0
subprocess32 = 3.5.3 subprocess32 = 3.5.3
unicodecsv = 0.14.1 unicodecsv = 0.14.1
...@@ -217,7 +217,7 @@ msgpack = 0.6.2 ...@@ -217,7 +217,7 @@ msgpack = 0.6.2
# Required by: # Required by:
# slapos.core==1.5.0 # slapos.core==1.5.0
Flask = 0.12 Flask = 1.1.2
# Required by: # Required by:
# slapos.toolbox==0.94 # slapos.toolbox==0.94
...@@ -251,8 +251,6 @@ croniter = 0.3.25 ...@@ -251,8 +251,6 @@ croniter = 0.3.25
# slapos.toolbox==0.94 # slapos.toolbox==0.94
dnspython = 1.16.0 dnspython = 1.16.0
# Required by:
# cryptography==1.8.1
enum34 = 1.1.10 enum34 = 1.1.10
# Required by: # Required by:
...@@ -279,8 +277,6 @@ pyrsistent = 0.14.5 ...@@ -279,8 +277,6 @@ pyrsistent = 0.14.5
# jsonschema==3.0.2 # jsonschema==3.0.2
setuptools-scm = 3.5.0 setuptools-scm = 3.5.0
# Required by:
# cryptography==1.8.1
ipaddress = 1.0.23 ipaddress = 1.0.23
# Required by: # Required by:
...@@ -296,8 +292,6 @@ lockfile = 0.12.2 ...@@ -296,8 +292,6 @@ lockfile = 0.12.2
# XXX 'slapos node format' raises an exception with netifaces 0.10.5. # XXX 'slapos node format' raises an exception with netifaces 0.10.5.
netifaces = 0.10.7 netifaces = 0.10.7
# Required by:
# cryptography==1.8.1
packaging = 16.8 packaging = 16.8
# Required by: # Required by:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7