Skip to content

slapos
slapos
Commit 6cc11cdf authored by Łukasz Nowak's avatar Łukasz Nowak
Browse files
Options

caddy-frontend: Put haproxy just before the backend 

This is needed in order to provide future support for client certificates
to the backend.

Also it means that haproxy is used in all cases, with or without cache, and as
a result the "cached" version of caddy is dropped.

fixup! caddy-frontend: Put haproxy just before the backend
parent 00288e68
Show whitespace changes
Inline Side-by-side
Showing with 416 additions and 366 deletions
software/caddy-frontend/README.caddy_frontend.rst
View file @ 6cc11cdf
......@@ -472,11 +472,22 @@ Instantiating caddy-frontend results with a cluster in various partitions:
* kedifa (contains kedifa server)
* caddy-frontend-N which contains the running processes to serve sites - this partition can be replicated by ``-frontend-quantity`` parameter
So it means sites are served in `caddy-frontend-N` partition, and this partition is structured as:
It means sites are served in `caddy-frontend-N` partition, and this partition is structured as:
* Caddy serving the browser
* (optional) Apache Traffic Server for caching
* Caddy connected to the backend
* Caddy serving the browser [frontend-caddy]
* (optional) Apache Traffic Server for caching [ats]
* Haproxy as a way to communicate to the backend [backend-haproxy]
* some other additional tools (6tunnel, monitor, etc)
In case of slaves without cache (`enable_cache = False`) the request will travel as follows::
frontend-caddy --> backend-haproxy --> backend
In case of slaves using cache (`enable_cache = True`) the request will travel as follows::
frontend-caddy --> ats --> backend-haproxy --> backend
Usage of Haproxy as a relay to the backend allows much better control of the backend, removes the hassle of checking the backend from Caddy and allows future developments like client SSL certificates to the backend or even health checks.
Kedifa implementation
---------------------
......
software/caddy-frontend/buildout.hash.cfg
View file @ 6cc11cdf
......@@ -14,7 +14,7 @@
# not need these here).
[template]
filename = instance.cfg.in
md5sum = 20f7a925e686949092823595c79a0523
md5sum = 3214dec50a312d5acc27f7947fcc6f0c
[template-common]
filename = instance-common.cfg.in
......@@ -22,7 +22,7 @@ md5sum = c801b7f9f11f0965677c22e6bbe9281b
[template-apache-frontend]
filename = instance-apache-frontend.cfg.in
md5sum = 0851faa528eb4f21330a6f23f77dea7f
md5sum = c0325f8cabba57b399c3f5ed2c6a01c4
[template-caddy-replicate]
filename = instance-apache-replicate.cfg.in
......@@ -30,7 +30,7 @@ md5sum = 6d7113ebf0c46b0e4c72c128ebb647db
[template-slave-list]
_update_hash_filename_ = templates/apache-custom-slave-list.cfg.in
md5sum = 9da1616d203e4909af37e658aa923d95
md5sum = c72c8d6d0189aa19a743c96e8d7215cf
[template-replicate-publish-slave-information]
_update_hash_filename_ = templates/replicate-publish-slave-information.cfg.in
......@@ -38,7 +38,7 @@ md5sum = 7e3ee70c447f8203273d78f66ab519c3
[template-caddy-frontend-configuration]
_update_hash_filename_ = templates/Caddyfile.in
md5sum = f0faf6d2e6c187df7e25bf717676f9df
md5sum = 2503056e35463e045db3329bb8b6fae8
[caddy-backend-url-validator]
filename = templates/caddy-backend-url-validator.in
......@@ -50,11 +50,11 @@ md5sum = f20d6c3d2d94fb685f8d26dfca1e822b
[template-default-slave-virtualhost]
_update_hash_filename_ = templates/default-virtualhost.conf.in
md5sum = a72e9056eeda3c7c794f6f6560056380
md5sum = 82c3908c9f6346356179646211058bf5
[template-cached-slave-virtualhost]
_update_hash_filename_ = templates/cached-virtualhost.conf.in
md5sum = e839ca3cb308f7fcdfa06c2f1b95e93f
[template-backend-haproxy-configuration]
_update_hash_filename_ = templates/backend-haproxy.cfg.in
md5sum = eb517d1f051f0631744a512af84b9a3b
[template-log-access]
_update_hash_filename_ = templates/template-log-access.conf.in
......@@ -94,7 +94,7 @@ md5sum = 061cc244558fd3af2b6bacf17cae5555
[template-validate-script]
_update_hash_filename_ = templates/validate-script.sh.in
md5sum = f26e11574f266c7437c9c89e3c93825a
md5sum = 53e5d7ba2827bff003051f74f24ffe4f
[template-configuration-state-script]
_update_hash_filename_ = templates/configuration-state-script.sh.in
......
software/caddy-frontend/common.cfg
View file @ 6cc11cdf
......@@ -10,6 +10,7 @@ extends =
../../component/trafficserver/buildout.cfg
../../component/6tunnel/buildout.cfg
../../component/xz-utils/buildout.cfg
../../component/haproxy/buildout.cfg
../../stack/caucase/buildout.cfg
# Monitoring stack (keep on bottom)
......@@ -94,6 +95,7 @@ bin_directory = ${buildout:bin-directory}
sixtunnel = ${6tunnel:location}
caddy = ${caddy:output}
caddy_location = ${caddy:location}
haproxy_executable = ${haproxy:location}/sbin/haproxy
curl = ${curl:location}
dash = ${dash:location}
gzip = ${gzip:location}
......@@ -108,7 +110,7 @@ kedifa-csr = ${:bin_directory}/kedifa-csr
xz_location = ${xz-utils:location}
monitor_template = ${monitor-template:output}
template_cached_slave_virtualhost = ${template-cached-slave-virtualhost:target}
template_backend_haproxy_configuration = ${template-backend-haproxy-configuration:target}
template_caddy_frontend_configuration = ${template-caddy-frontend-configuration:target}
template_graceful_script = ${template-graceful-script:target}
template_validate_script = ${template-validate-script:target}
......@@ -184,7 +186,7 @@ mode = 640
[template-default-slave-virtualhost]
<=download-template
[template-cached-slave-virtualhost]
[template-backend-haproxy-configuration]
<=download-template
[template-log-access]
......
software/caddy-frontend/instance-apache-frontend.cfg.in
View file @ 6cc11cdf
......@@ -21,8 +21,6 @@ parts =
promise-caddy-frontend-v4-http
promise-caddy-frontend-v6-https
promise-caddy-frontend-v6-http
promise-caddy-frontend-cached
promise-caddy-frontend-ssl-cached
trafficserver-launcher
trafficserver-reload
......@@ -43,6 +41,12 @@ parts =
monitor-caddy-server-status-wrapper
monitor-verify-re6st-connectivity
backend-haproxy
backend-haproxy-graceful
promise-backend-haproxy-http
promise-backend-haproxy-https
promise-backend-haproxy-configuration
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
......@@ -147,7 +151,7 @@ context =
[software-release-path]
template-empty = {{ parameter_dict['template_empty'] }}
template-default-slave-virtualhost = {{ parameter_dict['template_default_slave_virtualhost'] }}
template-cached-slave-virtualhost = {{ parameter_dict['template_cached_slave_virtualhost'] }}
template-backend-haproxy-configuration = {{ parameter_dict['template_backend_haproxy_configuration'] }}
caddy-location = {{ parameter_dict['caddy_location'] }}
[kedifa-login-config]
......@@ -210,13 +214,11 @@ kedifa-updater = {{ parameter_dict['kedifa-updater'] }}
kedifa-updater-mapping-file = ${directory:etc}/kedifa_updater_mapping.txt
kedifa-updater-state-file = ${directory:srv}/kedifa_updater_state.json
kedifa-csr = {{ parameter_dict['kedifa-csr'] }}
service_directory = ${directory:service}
extra-context =
import urlparse_module urlparse
key kedifa_caucase_ca_certificate kedifa-login-config:ca-certificate
key kedifa_login_certificate kedifa-login-config:certificate
key caddy_configuration_directory caddy-directory:slave-configuration
key caddy_cached_configuration_directory caddy-directory:slave-with-cache-configuration
key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
key kedifa_updater :kedifa-updater
key kedifa_updater_mapping_file :kedifa-updater-mapping-file
key kedifa_updater_state_file :kedifa-updater-state-file
......@@ -248,7 +250,6 @@ extra-context =
key global_ipv6 slap-network-information:global-ipv6
key empty_template software-release-path:template-empty
key template_default_slave_configuration software-release-path:template-default-slave-virtualhost
key template_cached_slave_configuration software-release-path:template-cached-slave-virtualhost
key software_type :software_type
key frontend_lazy_graceful_reload frontend-caddy-lazy-graceful:rendered
key frontend_graceful_reload caddy-configuration:frontend-graceful-command
......@@ -260,20 +261,21 @@ extra-context =
key enable_http2_by_default configuration:enable-http2-by-default
key global_disable_http2 configuration:global-disable-http2
key ciphers configuration:ciphers
key request_timeout configuration:request-timeout
key proxy_try_duration configuration:proxy-try-duration
key proxy_try_interval configuration:proxy-try-interval
key access_log caddy-configuration:access-log
key error_log caddy-configuration:error-log
key sixtunnel_executable :sixtunnel_executable
key service_directory directory:service
key run_directory directory:etc-run
key not_found_file caddy-configuration:not-found-file
key custom_ssl_directory caddy-directory:custom-ssl-directory
# BBB: SlapOS Master non-zero knowledge BEGIN
key bbb_ssl_directory directory:bbb-ssl-dir
key apache_certificate apache-certificate:rendered
# BBB: SlapOS Master non-zero knowledge END
## backend haproxy
key backend_haproxy_configuration_file backend-haproxy-configuration:file
key template_backend_haproxy_configuration software-release-path:template-backend-haproxy-configuration
section backend_haproxy_configuration backend-haproxy-configuration
## full configuration
section configuration configuration
# Deploy Caddy Frontend with Jinja power
[dynamic-caddy-frontend-template]
......@@ -288,9 +290,6 @@ extra-context =
key master_certificate caddy-configuration:master-certificate
key access_log caddy-configuration:access-log
key slave_configuration_directory caddy-directory:slave-configuration
key cached_port caddy-configuration:cache-through-port
key ssl_cached_port caddy-configuration:ssl-cache-through-port
key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
section frontend_configuration frontend-configuration
key http_port configuration:plain_http_port
key https_port configuration:port
......@@ -343,7 +342,6 @@ command = ln -sf {{ parameter_dict['template_not_found_html'] }} ${caddy-direct
recipe = slapos.cookbook:mkdirectory
document-root = ${directory:srv}/htdocs
slave-configuration = ${directory:etc}/caddy-slave-conf.d/
slave-with-cache-configuration = ${directory:etc}/caddy-slave-with-cache-conf.d/
cache = ${directory:var}/cache
mod-ssl = ${:cache}/httpd_mod_ssl
slave-log = ${directory:log}/httpd
......@@ -362,8 +360,6 @@ not-found-file = ${caddy-directory:document-root}/${not-found-html:filename}
master-certificate = ${caddy-directory:master-autocert-dir}/master.pem
# Communication with ATS
cache-port = ${trafficserver-variable:input-port}
cache-through-port = 26011
ssl-cache-through-port = 26012
# BBB: SlapOS Master non-zero knowledge BEGIN
[get-self-signed-fallback-access]
......@@ -415,9 +411,6 @@ reload-path = ${directory:etc-run}/trafficserver-reload
local-ip = {{ instance_parameter['ipv4-random'] }}
input-port = 23432
hostname = ${configuration:frontend-name}
remap = map /HTTPS/ http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:ssl-cache-through-port}
map / http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:cache-through-port}
plugin-config =
ip-allow-config = src_ip=0.0.0.0-255.255.255.255 action=ip_allow
cache-path = ${trafficserver-directory:cache-path}
......@@ -475,11 +468,18 @@ template = {{ parameter_dict['template_trafficserver_logging_config'] }}
filename = logging.config
[trafficserver-remap-config]
< = trafficserver-jinja2-template-base
template = {{ parameter_dict['template_empty'] }}
<= trafficserver-jinja2-template-base
{%- raw %}
template = inline:
map /HTTPS/ http://{{ ipv4 }}:{{ https_port }}
map / http://{{ ipv4 }}:{{ http_port }}
{%- endraw %}
extra-context =
raw ipv4 {{ instance_parameter['ipv4-random'] }}
key https_port backend-haproxy-configuration:https-port
key http_port backend-haproxy-configuration:http-port
filename = remap.config
context =
key content trafficserver-variable:remap
[trafficserver-plugin-config]
< = trafficserver-jinja2-template-base
......@@ -547,7 +547,7 @@ template = {{ parameter_dict['template_configuration_state_script'] }}
rendered = ${directory:bin}/${:_buildout_section_name_}
mode = 0700
path_list = ${caddy-configuration:frontend-configuration} ${frontend-configuration:log-access-configuration} ${caddy-directory:slave-configuration}/*.conf ${caddy-directory:slave-with-cache-configuration}/*.conf ${caddy-directory:master-autocert-dir}/*.key ${caddy-directory:master-autocert-dir}/*.crt ${caddy-directory:master-autocert-dir}/*.pem ${caddy-directory:autocert}/*.pem ${caddy-directory:custom-ssl-directory}/*.proxy_ca_crt ${directory:bbb-ssl-dir}/*.crt
path_list = ${caddy-configuration:frontend-configuration} ${frontend-configuration:log-access-configuration} ${caddy-directory:slave-configuration}/*.conf ${caddy-directory:master-autocert-dir}/*.key ${caddy-directory:master-autocert-dir}/*.crt ${caddy-directory:master-autocert-dir}/*.pem ${caddy-directory:autocert}/*.pem ${caddy-directory:custom-ssl-directory}/*.proxy_ca_crt ${directory:bbb-ssl-dir}/*.crt
sha256sum = {{ parameter_dict['sha256sum'] }}
extra-context =
......@@ -579,9 +579,10 @@ template = {{ parameter_dict['template_validate_script'] }}
rendered = ${directory:bin}/frontend-caddy-validate
mode = 0700
last_state_file = ${directory:run}/caddy_configuration_last_state
validate_command = ${caddy-wrapper:rendered} -validate
extra-context =
key wrapper caddy-wrapper:rendered
key caddy_configuration_state frontend-caddy-configuration-state-validate:rendered
key validate_command :validate_command
key configuration_state_command frontend-caddy-configuration-state-validate:rendered
key last_state_file :last_state_file
[frontend-caddy-lazy-graceful]
......@@ -642,19 +643,117 @@ name = caddy_frontend_ipv6_http.py
config-hostname = {{ instance_parameter['ipv6-random'] }}
config-port = ${configuration:plain_http_port}
[promise-caddy-frontend-cached]
[promise-backend-haproxy-http]
<= monitor-promise-base
module = check_port_listening
name = caddy_cached.py
name = backend_haproxy_http.py
config-hostname = {{ instance_parameter['ipv4-random'] }}
config-port = ${caddy-configuration:cache-through-port}
config-port = ${backend-haproxy-configuration:http-port}
[promise-caddy-frontend-ssl-cached]
[promise-backend-haproxy-https]
<= monitor-promise-base
module = check_port_listening
name = caddy_ssl_cached.py
name = backend_haproxy_https.py
config-hostname = {{ instance_parameter['ipv4-random'] }}
config-port = ${caddy-configuration:ssl-cache-through-port}
config-port = ${backend-haproxy-configuration:https-port}
[backend-haproxy-configuration]
file = ${directory:etc}/backend-haproxy.cfg
pid-file = ${directory:run}/backend-haproxy.pid
graceful-command = ${backend-haproxy-validate:rendered} && kill -USR2 $(cat ${:pid-file})
http-port = ${configuration:backend-haproxy-http-port}
https-port = ${configuration:backend-haproxy-https-port}
[backend-haproxy-wrapper]
recipe = slapos.recipe.template:jinja2
template = inline:
#!/bin/sh
ulimit -n $(ulimit -Hn)
exec {{ parameter_dict['haproxy_executable'] }} \
-f ${backend-haproxy-configuration:file} \
"$@"
rendered = ${directory:bin}/backend-haproxy-wrapper
mode = 0755
[backend-haproxy]
recipe = slapos.cookbook:wrapper
command-line = ${backend-haproxy-wrapper:rendered}
wrapper-path = ${directory:service}/backend-haproxy
hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
hash-files = ${backend-haproxy-wrapper:rendered}
[backend-haproxy-configuration-state]
<= jinja2-template-base
template = {{ parameter_dict['template_configuration_state_script'] }}
rendered = ${directory:bin}/${:_buildout_section_name_}
mode = 0700
path_list = ${backend-haproxy-configuration:file}
sha256sum = {{ parameter_dict['sha256sum'] }}
extra-context =
key path_list :path_list
key sha256sum :sha256sum
key signature_file :signature_file
[backend-haproxy-configuration-state-graceful]
<= backend-haproxy-configuration-state
signature_file = ${directory:run}/backend_haproxy_graceful_configuration_state_signature
[backend-haproxy-configuration-state-validate]
<= backend-haproxy-configuration-state
signature_file = ${directory:run}/backend_haproxy_validate_configuration_state_signature
[backend-haproxy-graceful]
< = jinja2-template-base
template = {{ parameter_dict['template_graceful_script'] }}
rendered = ${directory:etc-run}/backend-haproxy-safe-graceful
mode = 0700
extra-context =
key graceful_reload_command backend-haproxy-configuration:graceful-command
key caddy_configuration_state backend-haproxy-configuration-state-graceful:rendered
[backend-haproxy-validate]
<= jinja2-template-base
template = {{ parameter_dict['template_validate_script'] }}
rendered = ${directory:bin}/backend-haproxy-validate
mode = 0700
last_state_file = ${directory:run}/backend_haproxy_configuration_last_state
validate_command = ${backend-haproxy-wrapper:rendered} -c
extra-context =
key validate_command :validate_command
key configuration_state_command backend-haproxy-configuration-state-validate:rendered
key last_state_file :last_state_file
[backend-haproxy-lazy-graceful]
< = jinja2-template-base
template = {{ parameter_dict['template_caddy_lazy_script_call'] }}
rendered = ${directory:bin}/backend-haproxy-lazy-graceful
mode = 0700
pid-file = ${directory:run}/backend-haproxy-lazy-graceful.pid
wait_time = 60
extra-context =
key pid_file :pid-file
key wait_time :wait_time
key lazy_command backend-haproxy-configuration:graceful-command
[promise-backend-haproxy-configuration]
<= monitor-promise-base
module = validate_frontend_configuration
name = backend-haproxy-configuration.py
config-verification-script = ${promise-backend-haproxy-configuration-helper:rendered}
[promise-backend-haproxy-configuration-helper]
< = jinja2-template-base
template = {{ parameter_dict['template_empty'] }}
rendered = ${directory:bin}/backend-haproxy-read-last-configuration-state
mode = 0700
content =
#!/bin/sh
exit `cat ${backend-haproxy-validate:last_state_file}`
context =
key content :content
#######
# Monitoring sections
......
software/caddy-frontend/instance-caddy-input-schema.json
View file @ 6cc11cdf
......@@ -66,16 +66,16 @@
"title": "Test Verification URL",
"type": "string"
},
"proxy-try-duration": {
"timeout-backend-connect": {
"default": 5,
"description": "A time during which Caddy will try to establish connection with a backend. Setting it to 0 will result with immediate return of 502 EOF error to the browser, even if it would be possible to (re)connect to the backend during few moments. More info in https://caddyserver.com/docs/proxy try_durtion.",
"description": "A time for wait for the backend to be connected..",
"title": "Duration in seconds of trying a backend",
"type": "integer"
},
"proxy-try-interval": {
"default": 250,
"description": "How often Caddy will try to establish connection with a backend during proxy-try-duration. More info in https://caddyserver.com/docs/proxy try_interval",
"title": "Interval in milliseconds of tries during proxy-try-duration",
"timeout-backend-connect-retries": {
"default": 3,
"description": "Amount of retries to connect to the backend. The amount of timeout-backend-connect*timeout-backend-connect-retries seconds will be spent to connect to the backend.",
"title": "Amount of retries to connect to the backend.",
"type": "integer"
},
"automatic-internal-kedifa-caucase-csr": {
......
software/caddy-frontend/instance-slave-caddy-input-schema.json
View file @ 6cc11cdf
......@@ -195,6 +195,16 @@
"title": "type:zope virtualhostroot-https-port",
"type": "integer"
},
"timeout-backend-connect": {
"description": "A time for wait for the backend to be connected..",
"title": "Duration in seconds of trying a backend",
"type": "integer"
},
"timeout-backend-connect-retries": {
"description": "Amount of retries to connect to the backend. The amount of timeout-backend-connect*timeout-backend-connect-retries seconds will be spent to connect to the backend.",
"title": "Amount of retries to connect to the backend.",
"type": "integer"
},
"ciphers": {
"description": "List of ciphers. Empty defaults to cluster list of ciphers, which by default are Caddy list of ciphers. See https://caddyserver.com/docs/tls for more information.",
"title": "Ordered space separated list of ciphers",
......
software/caddy-frontend/instance.cfg.in
View file @ 6cc11cdf
......@@ -117,5 +117,7 @@ configuration.ciphers =
configuration.request-timeout = 600
configuration.mpm-graceful-shutdown-timeout = 5
configuration.frontend-name =
configuration.proxy-try-duration = 5
configuration.proxy-try-interval = 250
configuration.timeout-backend-connect = 5
configuration.timeout-backend-connect-retries = 3
configuration.backend-haproxy-http-port = 21080
configuration.backend-haproxy-https-port = 21443
software/caddy-frontend/templates/Caddyfile.in
View file @ 6cc11cdf
......@@ -2,10 +2,8 @@
import {{frontend_configuration.get('log-access-configuration')}}
import {{ slave_configuration_directory }}/*.conf
import {{ slave_with_cache_configuration_directory }}/*.conf
{% for port in [https_port] %}
:{{ port }} {
:{{ https_port }} {
tls {{ master_certificate }} {{ master_certificate }}
bind {{ local_ipv4 }}
status 404 /
......@@ -17,10 +15,8 @@ import {{ slave_with_cache_configuration_directory }}/*.conf
* {{ not_found_file }}
}
}
{%- endfor %}
{% for port in [http_port, cached_port, ssl_cached_port] %}
:{{ port }} {
:{{ http_port }} {
bind {{ local_ipv4 }}
status 404 /
log / {{ access_log }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
......@@ -31,7 +27,6 @@ import {{ slave_with_cache_configuration_directory }}/*.conf
* {{ not_found_file }}
}
}
{%- endfor %}
# Access to server-status Caddy-style
https://[{{ global_ipv6 }}]:{{ https_port }}/server-status, https://{{ local_ipv4 }}:{{ https_port }}/server-status {
......
software/caddy-frontend/templates/apache-custom-slave-list.cfg.in
View file @ 6cc11cdf
{%- if software_type == slap_software_type %}
{%- set kedifa_updater_mapping = [] %}
{%- set cached_server_dict = {} %}
{%- set backend_slave_list = [] %}
{%- set part_list = [] %}
{%- set cache_port = caddy_configuration.get('cache-port') %}
{%- set cached_port = caddy_configuration.get('cache-through-port') %}
{%- set ssl_cached_port = caddy_configuration.get('ssl-cache-through-port') %}
{%- set cache_access = "http://%s:%s" % (local_ipv4, cache_port) %}
{%- set ssl_cache_access = "http://%s:%s/HTTPS" % (local_ipv4, cache_port) %}
{%- set backend_haproxy_http_url = 'http://%s:%s' % (local_ipv4, backend_haproxy_configuration['http-port']) %}
{%- set backend_haproxy_https_url = 'http://%s:%s' % (local_ipv4, backend_haproxy_configuration['https-port']) %}
{%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] %}
{%- set generic_instance_parameter_dict = { 'cache_access': cache_access, 'local_ipv4': local_ipv4, 'http_port': http_port, 'https_port': https_port} %}
{%- set slave_log_dict = {} %}
......@@ -47,7 +48,15 @@ create = true
[slave-log-cache-direct-directory-dict]
{#- Loop thought slave list to set up slaves #}
{%- set DEFAULT_PORT = {'http': 80, 'https': 443, '': None} %}
{%- for slave_instance in slave_instance_list %}
{#- prepare backend parameters #}
{%- for key, prefix in [('url', 'http_backend'), ('https-url', 'https_backend')] %}
{%- set parsed = urlparse_module.urlparse(slave_instance.get(key, '')) %}
{%- set info_dict = {'scheme': parsed.scheme, 'hostname': parsed.hostname, 'port': parsed.port or DEFAULT_PORT[parsed.scheme], 'path': parsed.path, 'fragment': parsed.fragment} %}
{%- do slave_instance.__setitem__(prefix, info_dict) %}
{%- endfor %}
{%- do slave_instance.__setitem__('ssl_proxy_verify', ('' ~ slave_instance.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES) %}
{#- Manage ciphers #}
{%- set slave_ciphers = slave_instance.get('ciphers', '').strip().split() %}
{%- if slave_ciphers %}
......@@ -56,10 +65,34 @@ create = true
{%- set slave_cipher_list = ciphers.strip() %}
{%- endif %}
{%- do slave_instance.__setitem__('cipher_list', slave_cipher_list) %}
{#- Manage common instance parameters #}
{%- set slave_type = slave_instance.get('type', '') %}
{%- set enable_cache = (('' ~ slave_instance.get('enable_cache', '')).lower() in TRUE_VALUES and slave_type != 'redirect') %}
{%- set slave_reference = slave_instance.get('slave_reference') %}
{%- set slave_kedifa = slave_kedifa_information.get(slave_reference) %}
{#- Setup backend URLs for front facing Caddy #}
{%- if slave_type == 'redirect' %}
{%- do slave_instance.__setitem__('backend-http-url', slave_instance.get('url', '').rstrip('/')) %}
{%- if slave_instance.get('https-url') %}
{%- do slave_instance.__setitem__('backend-https-url', slave_instance.get['https-url'].rstrip('/')) %}
{%- endif %}
{%- elif enable_cache %}
{%- if 'domain' in slave_instance %}
{%- if not slave_instance.get('custom_domain') %}
{%- do slave_instance.__setitem__('custom_domain', slave_instance.get('domain')) %}
{%- endif %}
{%- endif %}
{%- do slave_instance.__setitem__('backend-http-url', cache_access) %}
{%- if slave_instance.get('https-url') %}
{%- do slave_instance.__setitem__('backend-https-url', ssl_cache_access) %}
{%- endif %}
{%- do cached_server_dict.__setitem__(slave_reference, slave_configuration_section_name) %}
{%- else %}
{%- do slave_instance.__setitem__('backend-http-url', backend_haproxy_http_url) %}
{%- if slave_instance.get('https-url') %}
{%- do slave_instance.__setitem__('backend-https-url', backend_haproxy_https_url) %}
{%- endif %}
{%- endif %}
{%- if slave_kedifa %}
{%- set key_download_url = slave_kedifa.get('key-download-url') %}
{%- else %}
......@@ -84,9 +117,14 @@ create = true
{#- Pass HTTP2 switch #}
{%- do slave_instance.__setitem__('enable_http2_by_default', enable_http2_by_default) %}
{%- do slave_instance.__setitem__('global_disable_http2', global_disable_http2) %}
{#- Pass proxy_try_duration and proxy_try_interval #}
{%- do slave_instance.__setitem__('proxy_try_duration', proxy_try_duration) %}
{%- do slave_instance.__setitem__('proxy_try_interval', proxy_try_interval) %}
{#- Pass backend timeout values #}
{#- XXX: maybe support request timeout on master/slave level #}
{%- do slave_instance.__setitem__('request-timeout', ('' ~ configuration['request-timeout'])) %}
{%- for key in ['timeout-backend-connect', 'timeout-backend-connect-retries'] %}
{%- if slave_instance.get(key, '') == '' %}
{%- do slave_instance.__setitem__(key, configuration[key]) %}
{%- endif %}
{%- endfor %}
{#- Set Up log files #}
{%- do slave_parameter_dict.__setitem__('access_log', '/'.join([caddy_log_directory, '%s_access_log' % slave_reference])) %}
{%- do slave_parameter_dict.__setitem__('error_log', '/'.join([caddy_log_directory, '%s_error_log' % slave_reference])) %}
......@@ -109,18 +147,6 @@ create = true
{%- set domain_prefix = slave_instance.get('slave_reference').replace("-", "").replace("_", "").lower() %}
{%- do slave_instance.__setitem__('custom_domain', "%s.%s" % (domain_prefix, slapparameter_dict.get('domain'))) %}
{%- endif %}
{%- if enable_cache and 'url' in slave_instance %}
{%- if 'domain' in slave_instance %}
{%- if not slave_instance.get('custom_domain') %}
{%- do slave_instance.__setitem__('custom_domain', slave_instance.get('domain')) %}
{%- endif %}
{%- endif %}
{%- do slave_instance.__setitem__('backend_url', slave_instance.get('url')) %}
{%- do slave_instance.__setitem__('https_backend_url', slave_instance.get('https-url', slave_instance.get('url'))) %}
{%- do slave_instance.__setitem__('url', cache_access) %}
{%- do slave_instance.__setitem__('https-url', ssl_cache_access) %}
{%- do cached_server_dict.__setitem__(slave_reference, slave_configuration_section_name) %}
{%- endif %}
{%- do slave_publish_dict.__setitem__('domain', slave_instance.get('custom_domain')) %}
{%- do slave_publish_dict.__setitem__('url', "http://%s" % slave_instance.get('custom_domain')) %}
{%- do slave_publish_dict.__setitem__('site_url', "http://%s" % slave_instance.get('custom_domain')) %}
......@@ -221,9 +247,6 @@ certificate = {{ certificate }}
https_port = {{ dumps('' ~ https_port) }}
http_port = {{ dumps('' ~ http_port) }}
local_ipv4 = {{ dumps('' ~ local_ipv4) }}
cached_port = {{ dumps('' ~ cached_port) }}
ssl_cached_port = {{ ('' ~ ssl_cached_port) }}
request_timeout = {{ ('' ~ request_timeout) }}
{%- for key, value in slave_instance.iteritems() %}
{%- if value is not none %}
{{ key }} = {{ dumps('' ~ value) }}
......@@ -282,6 +305,7 @@ recipe = slapos.cookbook:publish
{%- else %}
{%- do slave_instance_information_list.append(slave_publish_dict) %}
{%- endif %}
{%- do backend_slave_list.append(slave_instance) %}
{%- endfor %} {# Slave iteration ends for slave_instance in slave_instance_list #}
[slave-log-directories]
......@@ -290,34 +314,15 @@ recipe = slapos.cookbook:mkdirectory
{%- do part_list.append('slave-log-directories') %}
[slave-log-cache-direct-directories]
<= slave-log-cache-direct-directory-dict
recipe = slapos.cookbook:mkdirectory
{%- do part_list.append('slave-log-cache-direct-directories') %}
{%- do part_list.append('caddy-log-access') %}
{#- ############################################## #}
{#- ## Prepare virtualhost for slaves using cache #}
{%- for slave_reference, slave_configuration_section_name in cached_server_dict.iteritems() %}
{%- set cached_slave_configuration_section_title = '%s-cached-virtualhost' % slave_reference %}
{%- do part_list.append(cached_slave_configuration_section_title) %}
[{{ cached_slave_configuration_section_title }}]
< = jinja2-template-base
template = {{ template_cached_slave_configuration }}
filename = {{ '%s.conf' % slave_reference }}
rendered = {{ caddy_cached_configuration_directory }}/${:filename}
extensions = jinja2.ext.do
extra-context =
section slave_parameter {{ slave_configuration_section_name }}
{{ '\n' }}
{%- endfor %}
{#- Define IPv6 to IPV4 tunneling #}
[tunnel-6to4-base]
recipe = slapos.cookbook:wrapper
ipv4 = ${slap-network-information:local-ipv4}
ipv6 = ${slap-network-information:global-ipv6}
wrapper-path = {{ service_directory}}/6tunnel-${:ipv6-port}
wrapper-path = {{ directory['service'] }}/6tunnel-${:ipv6-port}
command-line = {{ sixtunnel_executable }} -6 -4 -d -l ${:ipv6} ${:ipv6-port} ${:ipv4} ${:ipv4-port}
hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
......@@ -331,16 +336,6 @@ ipv6-port = {{ http_port }}
ipv4-port = {{ https_port }}
ipv6-port = {{ https_port }}
[tunnel-6to4-base-cached_port]
<= tunnel-6to4-base
ipv4-port = {{ cached_port }}
ipv6-port = {{ cached_port }}
[tunnel-6to4-base-ssl_cached_port]
<= tunnel-6to4-base
ipv4-port = {{ ssl_cached_port }}
ipv6-port = {{ ssl_cached_port }}
{#- Define log access #}
[caddy-log-access-parameters]
......@@ -388,7 +383,7 @@ command-line = {{ kedifa_updater }}
${kedifa-updater-mapping:file}
{{ kedifa_updater_state_file }}
wrapper-path = {{ service_directory }}/kedifa-updater
wrapper-path = {{ directory['service'] }}/kedifa-updater
hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
[kedifa-updater-run]
......@@ -414,6 +409,24 @@ rendered = ${:file}
template = inline: # This file contain directives to serve directories with log files for shared instances, but no shared instances are defined yet.
rendered = {{frontend_configuration.get('log-access-configuration')}}
##<Backend haproxy>
[backend-haproxy-configuration]
< = jinja2-template-base
template = {{ template_backend_haproxy_configuration }}
rendered = ${backend-haproxy-config:file}
backend_slave_list = {{ dumps(sorted(backend_slave_list)) }}
extra-context =
key backend_slave_list :backend_slave_list
section configuration backend-haproxy-config
[backend-haproxy-config]
file = {{ backend_haproxy_configuration['file'] }}
pid-file = {{ backend_haproxy_configuration['pid-file'] }}
local-ipv4 = {{ dumps('' ~ local_ipv4) }}
http-port = {{ ('' ~ backend_haproxy_configuration['http-port']) }}
https-port = {{ ('' ~ backend_haproxy_configuration['https-port']) }}
##<Backend haproxy>
[buildout]
extends =
{{ common_profile }}
......@@ -423,6 +436,7 @@ extends =
parts +=
kedifa-updater
kedifa-updater-run
backend-haproxy-configuration
{%- for part in part_list %}
{{ ' %s' % part }}
{%- endfor %}
......@@ -432,8 +446,6 @@ parts +=
publish-caddy-information
tunnel-6to4-base-http_port
tunnel-6to4-base-https_port
tunnel-6to4-base-cached_port
tunnel-6to4-base-ssl_cached_port
expose-csr_id
promise-expose-csr_id-ip-port
......@@ -506,7 +518,7 @@ command-line = {{ caddy_executable }}
-disable-tls-alpn-challenge
-root {{ directory_csr_id }}
wrapper-path = {{ service_directory }}/expose-csr_id
wrapper-path = {{ directory['service'] }}/expose-csr_id
hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
[get-csr_id-certificate]
......
software/caddy-frontend/templates/backend-haproxy.cfg.in 0 → 100644
View file @ 6cc11cdf
# NON PROD CONFIG
global
maxconn 4096
pidfile {{ configuration['pid-file'] }}
# master-worker is compatible with foreground with process management
master-worker
log stderr local0
defaults
log global
mode http
option httplog
option dontlognull
retries 1
option redispatch
cookie SERVERID rewrite
balance roundrobin
stats uri /haproxy
stats realm Global\ statistics
timeout queue 60s
option httpclose
{%- macro frontend_entry(slave_instance, scheme) %}
{%- set host_list = slave_instance.get('server-alias', '').split() %}
{%- if slave_instance.get('custom_domain') not in host_list %}
{%- do host_list.append(slave_instance.get('custom_domain')) %}
{%- endif %}
{%- for host in host_list %}
{%- if host.startswith('*.') %}
{#- hdr_sub has to be used, as anyway something.example.com shall match *.example.com[:port], with optional port #}
acl is_{{ slave_instance['slave_reference'] }} hdr_sub(host) -i {{ host[2:] }}
{%- else %}
acl is_{{ slave_instance['slave_reference'] }} hdr_dom(host) -i {{ host }}
{%- endif %}
{%- endfor %}
use_backend {{ slave_instance['slave_reference'] }}-{{ scheme }} if is_{{ slave_instance['slave_reference'] }}
{%- endmacro %}
frontend http-backend
bind {{ configuration['local-ipv4'] }}:{{ configuration['http-port'] }}
{%- for slave_instance in backend_slave_list %}
{{ frontend_entry(slave_instance, 'http') }}
{%- endfor %}
frontend https-backend
bind {{ configuration['local-ipv4'] }}:{{ configuration['https-port'] }}
{%- for slave_instance in backend_slave_list %}
{{ frontend_entry(slave_instance, 'https') }}
{%- endfor %}
{%- for slave_instance in backend_slave_list %}
{%- for (scheme, prefix) in [('http', 'http_backend'), ('https', 'https_backend')] %}
{%- set info_dict = slave_instance[prefix] %}
{%- if info_dict['scheme'] == 'https' %}
{%- set ssl = ['ssl verify'] %}
{%- set path_to_ssl_proxy_ca_crt = slave_instance.get('path_to_ssl_proxy_ca_crt') %}
{%- if slave_instance['ssl_proxy_verify'] %}
{%- if path_to_ssl_proxy_ca_crt %}
{%- do ssl.append('required ca-file %s' % (path_to_ssl_proxy_ca_crt,)) %}
{%- else %}
{#- Backend SSL shall be verified, but not CA provided, disallow connection #}
{#- Simply dropping hostname from the dict will result with ignoring it... #}
{%- do info_dict.__setitem__('hostname', '') %}
{%- endif %}
{%- else %}
{%- do ssl.append('none') %}
{%- endif %}
{%- set ssl = ' '.join(ssl) %}
{%- else %}
{%- set ssl = '' %}
{%- endif %}
backend {{ slave_instance['slave_reference'] }}-{{ scheme }}
timeout server {{ slave_instance['request-timeout'] }}s
timeout client {{ slave_instance['request-timeout'] }}s
timeout connect {{ slave_instance['timeout-backend-connect'] }}s
retries {{ slave_instance['timeout-backend-connect-retries'] }}s
{%- set hostname = info_dict['hostname'] %}
{%- set port = info_dict['port'] %}
{%- set path = info_dict['path'].rstrip('/') %}
{%- if hostname and port %}
server backend {{ hostname }}:{{ port }} {{ ssl }}
{%- if path %}
http-request set-path {{ path }}%[path]
{%- endif %}
{%- endif %}
{%- endfor %}
{%- endfor %}
software/caddy-frontend/templates/cached-virtualhost.conf.in deleted 100644 → 0
View file @ 00288e68
{%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] %}
{%- set server_alias_list = slave_parameter.get('server-alias', '').split() %}
{%- set ssl_proxy_verify = ('' ~ slave_parameter.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES %}
{%- set host_list = [] %}
{%- for host in [slave_parameter.get('custom_domain')] + server_alias_list %}
{%- if host not in host_list %}
{%- do host_list.append(host) %}
{%- endif %}
{%- endfor %}
{%- set http_backend_host_list = [] %}
{%- set https_backend_host_list = [] %}
{%- for host in host_list %}
{%- do http_backend_host_list.append('http://%s:%s' % (host, slave_parameter['cached_port'])) %}
{%- do https_backend_host_list.append('http://%s:%s' % (host, slave_parameter['ssl_cached_port'])) %}
{%- endfor %}
# SSL-disabled backends
{{ http_backend_host_list|join(', ') }} {
bind {{ slave_parameter['local_ipv4'] }}
# Rewrite part
proxy / {{ slave_parameter.get('backend_url', '') }} {
try_duration {{ slave_parameter['proxy_try_duration'] }}s
try_interval {{ slave_parameter['proxy_try_interval'] }}ms
header_upstream Host {host}
{# header_upstream -X-Forwarded-For - caddy behaviour while removing and setting header is unstable, so for now original header has to be kept, even if in that case it comes from after ATS caddy itself #}
header_upstream X-Forwarded-For {>X-Forwarded-For-Real}
header_upstream -X-Forwarded-For-Real
timeout {{ slave_parameter['request_timeout'] }}s
{%- if ssl_proxy_verify %}
{%- if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
{%- endif %}
{%- else %}
insecure_skip_verify
{%- endif %}
}
log / {{ slave_parameter.get('access_log_cache_direct') }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
rotate_size 0
}
errors {{ slave_parameter.get('error_log_cache_direct') }} {
rotate_size 0
}
}
# SSL-enabled backends
{{ https_backend_host_list|join(', ') }} {
bind {{ slave_parameter['local_ipv4'] }}
proxy / {{ slave_parameter.get('https_backend_url', '') }} {
try_duration {{ slave_parameter['proxy_try_duration'] }}s
try_interval {{ slave_parameter['proxy_try_interval'] }}ms
header_upstream Host {host}
{# header_upstream -X-Forwarded-For - caddy behaviour while removing and setting header is unstable, so for now original header has to be kept, even if in that case it comes from after ATS caddy itself #}
header_upstream X-Forwarded-For {>X-Forwarded-For-Real}
header_upstream -X-Forwarded-For-Real
timeout {{ slave_parameter['request_timeout'] }}s
{%- if ssl_proxy_verify %}
{%- if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
{%- endif %}
{%- else %}
insecure_skip_verify
{%- endif %}
}
log / {{ slave_parameter.get('access_log_cache_direct') }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
rotate_size 0
}
errors {{ slave_parameter.get('error_log_cache_direct') }} {
rotate_size 0
}
}
software/caddy-frontend/templates/default-virtualhost.conf.in
View file @ 6cc11cdf
......@@ -9,7 +9,6 @@
{%- endif %} {#- if prefer_gzip #}
{%- set server_alias_list = slave_parameter.get('server-alias', '').split() %}
{%- set enable_h2 = slave_parameter['global_disable_http2'].lower() not in TRUE_VALUES and slave_parameter.get('enable-http2', slave_parameter['enable_http2_by_default']).lower() in TRUE_VALUES %}
{%- set ssl_proxy_verify = slave_parameter.get('ssl-proxy-verify', '').lower() in TRUE_VALUES %}
{%- set disabled_cookie_list = slave_parameter.get('disabled-cookie-list', '').split() %}
{%- set https_only = slave_parameter.get('https-only', 'true').lower() in TRUE_VALUES %}
{%- set slave_type = slave_parameter.get('type', '') %}
......@@ -41,31 +40,18 @@
{%- endif %}
{%- macro proxy_header() %}
try_duration {{ slave_parameter['proxy_try_duration'] }}s
try_interval {{ slave_parameter['proxy_try_interval'] }}ms
timeout {{ slave_parameter['request_timeout'] }}s
{%- if ssl_proxy_verify %}
{%- if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
{%- endif %} {#- if 'path_to_ssl_proxy_ca_crt' in slave_parameter #}
{%- else %} {#- if ssl_proxy_verify #}
insecure_skip_verify
{%- endif %} {#- if ssl_proxy_verify #}
timeout {{ slave_parameter['request-timeout'] }}s
# force reset of X-Forwarded-For
header_upstream X-Forwarded-For {remote}
{%- if enable_cache %}
# provide a header for other components
header_upstream X-Forwarded-For-Real {remote}
{%- endif %}
{%- endmacro %} {# proxy_header #}
{%- for tls in [True, False] %}
{%- if tls %}
{%- set backend_url = slave_parameter.get('https-url', slave_parameter.get('url', '')).rstrip('/') %}
{%- set backend_url = slave_parameter.get('backend-https-url', slave_parameter.get('backend-http-url')) %}
# SSL enabled hosts
{{ https_host_list|join(', ') }} {
{%- else %}
{%- set backend_url = slave_parameter.get('url', '').rstrip('/') %}
{%- set backend_url = slave_parameter['backend-http-url'] %}
# SSL-disabled hosts
{{ http_host_list|join(', ') }} {
{%- endif %}
......@@ -178,11 +164,13 @@
{%- endif %}
} {# rewrite #}
{%- endif %} {#- if prefer_gzip #}
{%- elif slave_type == 'redirect' and backend_url %} {#- if slave_type == 'zope' and backend_url #}
{%- elif slave_type == 'redirect' %}
{%- if backend_url %}
# Redirect configuration
redir 302 {
/ {{ backend_url }}{rewrite_uri}
} {# redir #}
}
{%- endif %}
{%- elif slave_type == 'notebook' %}
proxy / {{ backend_url }} {
{{ proxy_header() }}
......@@ -204,6 +192,8 @@
{{ proxy_header() }}
{%- if websocket_transparent %}
transparent
{%- else %}
header_upstream Host {host}
{%- endif %}
}
{%- for websocket_path in websocket_path_list %}
......@@ -212,6 +202,8 @@
websocket
{%- if websocket_transparent %}
transparent
{%- else %}
header_upstream Host {host}
{%- endif %}
}
{%- endfor %}
......@@ -221,6 +213,8 @@
websocket
{%- if websocket_transparent %}
transparent
{%- else %}
header_upstream Host {host}
{%- endif %}
}
{%- endif %}
......
software/caddy-frontend/templates/validate-script.sh.in
View file @ 6cc11cdf
......@@ -10,10 +10,10 @@ if [ -f $LAST_STATE_FILE ] ; then
old_found=$(find $LAST_STATE_FILE -mmin +120 | wc -l)
fi
if [ "$old_found" -eq 1 ] || {{ caddy_configuration_state }} ; then
if [ "$old_found" -eq 1 ] || {{ configuration_state_command }} ; then
# do not catch errors during validation
set +e
{{ wrapper }} -validate
{{ validate_command }}
echo $? > $LAST_STATE_FILE
set -e
fi
......
software/caddy-frontend/test/test.py
View file @ 6cc11cdf
......@@ -537,15 +537,7 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
return "RootSoftwareInstance"
@classmethod
def startServerProcess(cls):
server = HTTPServer(
(cls._ipv4_address, findFreeTCPPort(cls._ipv4_address)),
TestHandler)
server_https = HTTPServer(
(cls._ipv4_address, findFreeTCPPort(cls._ipv4_address)),
TestHandler)
def prepareCertificate(cls):
cls.another_server_ca = CertificateAuthority("Another Server Root CA")
cls.test_server_ca = CertificateAuthority("Test Server Root CA")
key, key_pem, csr, csr_pem = createCSR(
......@@ -560,6 +552,17 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
cls.test_server_certificate_pem + key_pem
)
cls.test_server_certificate_file.close()
@classmethod
def startServerProcess(cls):
server = HTTPServer(
(cls._ipv4_address, cls._server_http_port),
TestHandler)
server_https = HTTPServer(
(cls._ipv4_address, cls._server_https_port),
TestHandler)
server_https.socket = ssl.wrap_socket(
server_https.socket,
certfile=cls.test_server_certificate_file.name,
......@@ -578,9 +581,12 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
cls.logger.debug('Started process %s' % (cls.server_https_process,))
@classmethod
def stopServerProcess(cls):
def cleanUpCertificate(cls):
if getattr(cls, 'test_server_certificate_file', None) is not None:
os.unlink(cls.test_server_certificate_file.name)
@classmethod
def stopServerProcess(cls):
for server in ['server_process', 'server_https_process']:
process = getattr(cls, server, None)
if process is not None:
......@@ -831,6 +837,7 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
@classmethod
def _cleanup(cls, snapshot_name):
cls.cleanUpCertificate()
cls.stopServerProcess()
super(HttpFrontendTestCase, cls)._cleanup(snapshot_name)
......@@ -838,6 +845,10 @@ class HttpFrontendTestCase(SlapOSInstanceTestCase):
def setUpClass(cls):
try:
cls.createWildcardExampleComCertificate()
cls.prepareCertificate()
# find ports once to be able startServerProcess many times
cls._server_http_port = findFreeTCPPort(cls._ipv4_address)
cls._server_https_port = findFreeTCPPort(cls._ipv4_address)
cls.startServerProcess()
except BaseException:
cls.logger.exception("Error during setUpClass")
......@@ -1081,14 +1092,14 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
}
@classmethod
def startServerProcess(cls):
def prepareCertificate(cls):
cls.ca = CertificateAuthority('TestSlave')
_, cls.customdomain_ca_key_pem, csr, _ = createCSR(
'customdomainsslcrtsslkeysslcacrt.example.com')
_, cls.customdomain_ca_certificate_pem = cls.ca.signCSR(csr)
_, cls.customdomain_key_pem, _, cls.customdomain_certificate_pem = \
createSelfSignedCertificate(['customdomainsslcrtsslkey.example.com'])
super(TestSlave, cls).startServerProcess()
super(TestSlave, cls).prepareCertificate()
@classmethod
def getSlaveParameterDictDict(cls):
......@@ -1551,7 +1562,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(httplib.NOT_FOUND, result.status_code)
self.assertEqual(httplib.SERVICE_UNAVAILABLE, result.status_code)
# check that log file contains verbose log
log_file = glob.glob(
......@@ -1561,7 +1572,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
log_regexp = r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} - - ' \
r'\[\d{2}\/.{3}\/\d{4}\:\d{2}\:\d{2}\:\d{2} \+\d{4}\] ' \
r'"GET \/test-path HTTP\/1.1" 404 \d+ "-" '\
r'"GET \/test-path HTTP\/1.1" \d{3} \d+ "-" '\
r'"python-requests.*" \d+'
self.assertRegexpMatches(
......@@ -1616,16 +1627,12 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
if ignore_header_list is None:
ignore_header_list = []
self.assertFalse('remote_user' in backend_header_dict.keys())
self.assertFalse('x-forwarded-for-real' in backend_header_dict.keys())
if 'Host' not in ignore_header_list:
self.assertEqual(
backend_header_dict['host'],
'%s:%s' % (domain, port))
# XXX It's really hard to play with Caddy headers, thus we have to keep
# some of them. As other solutions will come in future, more control
# over sent X-Forwarded-For will be possible
self.assertEqual(
backend_header_dict['x-forwarded-for'].split(',')[0],
backend_header_dict['x-forwarded-for'],
source_ip
)
self.assertEqual(
......@@ -1683,13 +1690,16 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
result_http.headers['Location']
)
# check that try_duration == 5 in the test_url slave
slave_configuration_file = glob.glob(os.path.join(
self.instance_path, '*', 'etc', '*slave-conf.d', '_Url.conf'))[0]
with open(slave_configuration_file) as fh:
# check that timeouts are correctly set in the haproxy configuration
backend_configuration_file = glob.glob(os.path.join(
self.instance_path, '*', 'etc', 'backend-haproxy.cfg'))[0]
with open(backend_configuration_file) as fh:
content = fh.read()
self.assertTrue('try_duration 5s' in content)
self.assertTrue('try_interval 250ms' in content)
self.assertTrue("""backend _Url-http
timeout server 12s
timeout client 12s
timeout connect 5s
retries 3s""" in content)
def test_compressed_result(self):
parameter_dict = self.assertSlaveBase('Url')
......@@ -2649,7 +2659,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (result.text,))
self.assertBackendHeaders(j['Incoming Headers'], parameter_dict['domain'])
self.assertFalse('connection' in j['Incoming Headers'].keys())
self.assertTrue('x-real-ip' in j['Incoming Headers'])
result = fakeHTTPSResult(
......@@ -2724,7 +2733,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
self.assertBackendHeaders(
j['Incoming Headers'], parsed.hostname, port='17', proto='irc',
ignore_header_list=['Host'])
self.assertFalse('connection' in j['Incoming Headers'].keys())
self.assertFalse('x-real-ip' in j['Incoming Headers'])
result = fakeHTTPSResult(
......@@ -2872,7 +2880,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -2949,7 +2957,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -2984,7 +2992,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -3037,47 +3045,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
headers
)
result_direct = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path',
port=26011)
self.assertEqualResultJson(result_direct, 'Path', '/test-path')
try:
j = result_direct.json()
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (result_direct.text,))
self.assertFalse('remote_user' in j['Incoming Headers'].keys())
self.assertFalse('Content-Encoding' in result_direct.headers)
self.assertEqual(
'secured=value;secure, nonsecured=value',
result_direct.headers['Set-Cookie']
)
result_direct_https_backend = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path',
port=26012)
self.assertEqualResultJson(
result_direct_https_backend, 'Path', '/test-path')
try:
j = result_direct_https_backend.json()
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (
result_direct_https_backend.text,))
self.assertFalse('remote_user' in j['Incoming Headers'].keys())
self.assertFalse(
'Content-Encoding' in result_direct_https_backend.headers)
self.assertEqual(
'secured=value;secure, nonsecured=value',
result_direct_https_backend.headers['Set-Cookie']
)
def test_enable_cache_ssl_proxy_verify_unverified(self):
parameter_dict = self.assertSlaveBase(
'enable_cache-ssl-proxy-verify-unverified')
......@@ -3090,7 +3057,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -3123,7 +3090,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -3192,7 +3159,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
der2pem(result.peercert))
self.assertEqual(
httplib.BAD_GATEWAY,
httplib.SERVICE_UNAVAILABLE,
result.status_code
)
......@@ -3206,7 +3173,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(httplib.NOT_FOUND, result.status_code)
self.assertEqual(httplib.SERVICE_UNAVAILABLE, result.status_code)
result_http = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path')
......@@ -3243,7 +3210,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(httplib.NOT_FOUND, result.status_code)
self.assertEqual(httplib.SERVICE_UNAVAILABLE, result.status_code)
result_http = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path')
......@@ -3281,7 +3248,7 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
self.certificate_pem,
der2pem(result.peercert))
self.assertEqual(httplib.NOT_FOUND, result.status_code)
self.assertEqual(httplib.SERVICE_UNAVAILABLE, result.status_code)
result_http = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path')
......@@ -3472,37 +3439,24 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
# real check: cache access provides old data, access cache directly, as
# caddy has to be stopped
try:
# stop caddy, to have error on while connecting to the backend
caddy_process_name = [
':'.join([q['group'], q['name']]) for q in
self.callSupervisorMethod('getAllProcessInfo')
if 'caddy' in q['name'] and 'on-watch' in q['name']][0]
self.callSupervisorMethod('stopProcess', caddy_process_name)
# sanity check: see that it is impossible to connect to caddy
with self.assertRaises(requests.ConnectionError):
fakeHTTPResult(
# stop the backend, to have error on while connecting to it
self.stopServerProcess()
result = fakeHTTPSResult(
parameter_dict['domain'], parameter_dict['public-ipv4'],
'test-path/deep/.././deeper', headers={
'X-Reply-Header-Cache-Control': 'max-age=1, stale-while-'
'revalidate=3600, stale-if-error=3600'})
result = fakeHTTPResult(
# append with :HTTP_PORT to mimic access in ATS
parameter_dict['domain'] + ':' + HTTPS_PORT,
parameter_dict['public-ipv4'],
# prepend with HTTPS to mimic access via https in ATS
# use simple path, as it is changed in Caddy
'HTTPS/test-path/deeper',
port=23432, headers={
'X-Reply-Header-Cache-Control': 'max-age=1, stale-while-'
'revalidate=3600, stale-if-error=3600'})
'revalidate=3600, stale-if-error=3600',
},
source_ip=source_ip
)
self.assertEqual(result.status_code, httplib.OK)
self.assertEqualResultJson(result, 'Path', '/test-path/deeper')
headers = result.headers.copy()
self.assertKeyWithPop('Server', headers)
self.assertKeyWithPop('Date', headers)
self.assertKeyWithPop('Age', headers)
self.assertKeyWithPop('Expires', headers)
# drop keys appearing randomly in headers
headers.pop('Transfer-Encoding', None)
headers.pop('Content-Length', None)
......@@ -3518,7 +3472,6 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
# 'Set-Cookie': 'secured=value;secure, nonsecured=value',
'Cache-Control': 'max-age=1, stale-while-revalidate=3600, '
'stale-if-error=3600',
'Warning': '111 ApacheTrafficServer/7.1.6'
},
headers
)
......@@ -3532,52 +3485,9 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
r'^http\/1.1 caddy-frontend-1\[.*\] \(ApacheTrafficServer\/7.1.6\)$'
)
finally:
self.callSupervisorMethod('startProcess', caddy_process_name)
# give few moments for caddy to start
# XXX: convert to a loop which awaits caddy to be ready
time.sleep(2)
self.startServerProcess()
# END: check stale-if-error support
result_direct = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path',
port=26011)
self.assertEqualResultJson(result_direct, 'Path', '/test-path')
try:
j = result_direct.json()
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (result_direct.text,))
self.assertFalse('remote_user' in j['Incoming Headers'].keys())
self.assertFalse('Content-Encoding' in result_direct.headers)
self.assertEqual(
'secured=value;secure, nonsecured=value',
result_direct.headers['Set-Cookie']
)
result_direct_https_backend = fakeHTTPResult(
parameter_dict['domain'], parameter_dict['public-ipv4'], 'test-path',
port=26012)
self.assertEqualResultJson(
result_direct_https_backend, 'Path', '/test-path')
try:
j = result_direct_https_backend.json()
except Exception:
raise ValueError('JSON decode problem in:\n%s' % (
result_direct_https_backend.text,))
self.assertFalse('remote_user' in j['Incoming Headers'].keys())
self.assertFalse('Content-Encoding' in result_direct_https_backend.headers)
self.assertEqual(
'secured=value;secure, nonsecured=value',
result_direct_https_backend.headers['Set-Cookie']
)
def test_enable_cache_ats_timeout(self):
parameter_dict = self.assertSlaveBase('enable_cache')
# check that timeout seen by ATS does not result in many queries done
......@@ -3595,21 +3505,21 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
result.status_code
)
caddy_log_file = glob.glob(
backend_haproxy_log_file = glob.glob(
os.path.join(
self.instance_path, '*', 'var', 'log', 'httpd-cache-direct',
'_enable_cache_access_log'
self.instance_path, '*', 'var', 'log', 'backend-haproxy.log'
))[0]
matching_line_amount = 0
pattern = re.compile(
r'.*GET .test_enable_cache_ats_timeout.*" 499.*')
with open(caddy_log_file) as fh:
r'.* _enable_cache-http.backend .* 504 .*'
'"GET .test_enable_cache_ats_timeout HTTP.1.1"$')
with open(backend_haproxy_log_file) as fh:
for line in fh.readlines():
if pattern.match(line):
matching_line_amount += 1
# Caddy used between ATS and the backend received maximum one connection
# Haproxy backend received maximum one connection
self.assertIn(matching_line_amount, [0, 1])
timeout = 5
......@@ -5281,7 +5191,7 @@ class TestSlaveSlapOSMasterCertificateCompatibility(
# Do not upload certificates for the master partition
@classmethod
def startServerProcess(cls):
def prepareCertificate(cls):
_, cls.ssl_from_slave_key_pem, _, cls.ssl_from_slave_certificate_pem = \
createSelfSignedCertificate(
[
......@@ -5325,7 +5235,7 @@ class TestSlaveSlapOSMasterCertificateCompatibility(
createSelfSignedCertificate(['customdomainsslcrtsslkey.example.com'])
super(
TestSlaveSlapOSMasterCertificateCompatibility, cls).startServerProcess()
TestSlaveSlapOSMasterCertificateCompatibility, cls).prepareCertificate()
@classmethod
def getInstanceParameterDict(cls):
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7