Skip to content

slapos-caddy
slapos-caddy
Commit 22396c5f authored by Łukasz Nowak's avatar Łukasz Nowak
Browse files
Options

caddy-frontend: Minimal implementation 

Note: This is adaptation of apache-frontend

ACHIEVEMENT UNLOCKED: Requesting working instance to any simple redirect
                      like backend (like Webrunner).

Features:

 * added a lot of bugs
 * simple, Webrunner like, backends are supported
 * buildout templates are named by their usage in instance buildout (master,
   slave)
 * file templates names are minimised and try to express functionality
 * follows Caddy per-instance self-contained configuration
 * redirection from http to https
 * annotated slave buildout with readable part names
 * comptability with software/erp5testnode/testsuite/ infrastructure
 * same request parameters as for apache-frontend are supported

Caddy executed with:

 * disabled automatic SSL
 * pidfile location
 * log location
 * specified http and https ports
 * site root

Slave improvements:

 * SSL files are in each slave configuration
 * same for IPv4 and IPV6 information
 * pass proper variables to slave list
 * remove not needed varfiables from slave list
 * per-slave SSL certificates supported
   This seems to be hacky, there is need to clean up the whole infrastructure,
   to follow Caddy per-instance self-containment approach.
parent f4e24b78
Hide whitespace changes
Inline Side-by-side
Showing with 1011 additions and 0 deletions
software/caddy-frontend/buildout.hash.cfg 0 → 100644
View file @ 22396c5f
# THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
# The only allowed lines here are (regexes):
# - "^#" comments, copied verbatim
# - "^[" section beginings, copied verbatim
# - lines containing an "=" sign which must fit in the following categorie.
# - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
# But avoid directories, they are not portable.
# Copied verbatim.
# - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
# by the re-generation script.
# Re-generated.
# - other lines are copied verbatim
# Substitution (${...:...}), extension ([buildout] extends = ...) and
# section inheritance (< = ...) are NOT supported (but you should really
# not need these here).
[template]
filename = instance.cfg
md5sum = ef9b6d129eaf47f98d3bc518d78f72fa
[template-caddy-slave]
filename = instance-caddy-slave.cfg
md5sum = 76ea12dd93a143a74220a93ff9231962
[template-caddy-master]
filename = instance-caddy-master.cfg.in
md5sum = 41f6c773181fab2c4fce25901704f5b1
[template-master-publish-slave-information]
filename = templates/master-publish-slave-information.cfg.in
md5sum = 4f25aa89209dd73cb6d6c20802b6a3b4
[template-slave-list]
filename = templates/custom-slave-list.cfg.in
md5sum = f0b49784efc41edfc8207e4b418099cf
[template-empty]
filename = templates/empty.in
md5sum = c2314c3a9c3412a38d14b312d3df83c1
[template-slave-configuration]
filename = templates/slave-configuration.conf.in
md5sum = a63777da492a9f67cfd6040c6d854f57
[template-default-slave-virtualhost]
filename = templates/default-slave-virtualhost.conf.in
md5sum = 009940c6dff79ffadb507371d1ba3489
[template-lazy-script-call]
filename = templates/lazy-script-call.sh.in
md5sum = ebe5d3d19923eb812a40019cb11276d8
[template-caddyfile]
filename = templates/Caddyfile.in
md5sum = e3f1d5d5ab3eb3a1a26782403c7373a3
software/caddy-frontend/instance-caddy-master.cfg.in 0 → 100644
View file @ 22396c5f
{% if slap_software_type in software_type -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:directory}/${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameter:slap_software_type
key slave_instance_list slap-parameter:slave_instance_list
${:extra-context}
{% set part_list = [] -%}
{% set single_type_key = 'single-' %}
{% if slap_software_type == "replicate" %}
{% set frontend_type = slapparameter_dict.pop('-frontend-type', 'single-default') -%}
{% elif slap_software_type in ['default', 'RootSoftwareInstance'] -%}
{% set frontend_type = "%s%s" % (single_type_key, 'custom-personal') -%}
{% else -%}
{% set frontend_type = "%s%s" % (single_type_key, slap_software_type) -%}
{% endif -%}
{% set frontend_quantity = slapparameter_dict.pop('-frontend-quantity', '1') | int -%}
{% set slave_list_name = 'extra_slave_instance_list' -%}
{% set frontend_list = [] %}
{% set frontend_section_list = [] %}
{% set request_dict = {} %}
{% set namebase = 'caddy-frontend' -%}
# XXX Dirty hack, not possible to define default value before
{% set sla_computer_caddy_1_key = '-sla-1-computer_guid' -%}
{% if not sla_computer_caddy_1_key in slapparameter_dict -%}
{% do slapparameter_dict.__setitem__(sla_computer_caddy_1_key, '${slap-connection:computer-id}') -%}
{% endif -%}
# Here we request individualy each frontend.
# The presence of sla parameters is checked and added if found
{% for i in range(1, frontend_quantity + 1) -%}
{% set frontend_name = "%s-%s" % (namebase, i) -%}
{% set request_section_title = 'request-%s' % frontend_name -%}
{% set sla_key = "-sla-%s-" % i -%}
{% set sla_key_length = sla_key | length %}
{% set sla_dict = {} %}
{% set config_key = "-frontend-config-%s-" % i %}
{% set config_key_length = config_key | length %}
{% set config_dict = {} %}
{% for key in slapparameter_dict.keys() %}
{% if key.startswith(sla_key) %}
{% do sla_dict.__setitem__(key[sla_key_length:], slapparameter_dict.pop(key)) %}
# We check for specific configuration regarding the frontend
{% elif key.startswith(config_key) -%}
{% do config_dict.__setitem__(key[config_key_length:], slapparameter_dict.pop(key)) %}
{% endif -%}
{% endfor -%}
{% do frontend_list.append(frontend_name) -%}
{% do frontend_section_list.append(request_section_title) -%}
{% do part_list.append(request_section_title) -%}
# Filling request dict for slave
{% set state_key = "-frontend-%s-state" % i %}
{% do request_dict.__setitem__(request_section_title,
{
'config': config_dict,
'name': frontend_name,
'sla': sla_dict,
'state': slapparameter_dict.pop(state_key, None)
}) %}
{% endfor -%}
{% set authorized_slave_string = slapparameter_dict.pop('-frontend-authorized-slave-string', '') -%}
{% set authorized_slave_list = [] %}
{% set rejected_slave_list = [] %}
{% for slave in slave_instance_list %}
{% if not (slave.has_key('caddy_custom_http') and not slave.get('slave_reference') in authorized_slave_string) %}
{% do authorized_slave_list.append(slave) %}
{% else %}
{% do rejected_slave_list.append(slave.get('slave_reference')) %}
{% endif %}
{% endfor -%}
[frontend-base]
<= slap-connection
recipe = slapos.cookbook:requestoptional
{% set frontend_software_url_key = "-frontend-software-release-url" %}
{% if slapparameter_dict.has_key(frontend_software_url_key) %}
software-url = {{ slapparameter_dict.pop(frontend_software_url_key) }}
{% else %}
software-url = ${slap-connection:software-release-url}
{% endif %}
software-type = {{frontend_type}}
return = private-ipv4 public-ipv4 slave-instance-information-list
{% for section, frontend_request in request_dict.iteritems() %}
[{{section}}]
<= frontend-base
name = {{ frontend_request.get('name') }}
{% if frontend_request.get('state') %}
state = {{ frontend_request.get('state') }}
{% endif%}
{% set slave_configuration_dict = frontend_request.get('config') %}
{% do slave_configuration_dict.update(**slapparameter_dict) %}
{% do slave_configuration_dict.__setitem__(slave_list_name, json_module.dumps(authorized_slave_list)) %}
{% do slave_configuration_dict.__setitem__("frontend-name", frontend_request.get('name')) %}
config-_ = {{ json_module.dumps(slave_configuration_dict) }}
{% if frontend_request.get('sla') %}
{% for parameter, value in frontend_request.get('sla').iteritems() -%}
sla-{{ parameter }} = {{ value }}
{% endfor -%}
{% endif -%}
{% endfor -%}
[publish-information]
recipe = slapos.cookbook:publish
domain = {{ slapparameter_dict.get('domain') }}
slave-amount = {{ slave_instance_list | length }}
accepted-slave-amount = {{ authorized_slave_list | length }}
rejected-slave-amount = {{ rejected_slave_list | length }}
rejected-slave-list = {{ json_module.dumps(rejected_slave_list) }}
# Monitor publication
# would use <= monitor-publish and <= monitor-publish-parameters but none
# provide all what needed, thus copy&paste
monitor-setup-url = ${monitor-instance-parameter:interface-url}/#page=settings_configurator&url=${:monitor-url}&username=${:monitor-user}&password=${:monitor-password}
monitor-base-url = ${monitor-base:base-url}
monitor-url = ${:monitor-base-url}/public/feeds
monitor-user = ${monitor-instance-parameter:username}
monitor-password = ${monitor-instance-parameter:password}
#----------------------------
#--
#-- Publish slave information
[publish-slave-information]
recipe = slapos.cookbook:softwaretype
default = ${dynamic-publish-slave-information:rendered}
RootSoftwareInstance = ${dynamic-publish-slave-information:rendered}
replicate = ${dynamic-publish-slave-information:rendered}
custom-personal = ${dynamic-publish-slave-information:rendered}
custom-group = ${dynamic-publish-slave-information:rendered}
[slave-information]
{% for frontend_section in frontend_section_list -%}
{{ frontend_section }} = {{ "${%s:connection-slave-instance-information-list}" % frontend_section }}
{% endfor -%}
[dynamic-publish-slave-information]
< = jinja2-template-base
template = {{ template_publish_slave_information }}
filename = dynamic-publish-slave-information.cfg
extensions = jinja2.ext.do
extra-context =
section slave_information slave-information
[buildout]
parts =
publish-slave-information
publish-information
monitor-base
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
extends =
{{ template_monitor }}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap_connection]
# Kept for backward compatiblity
computer_id = ${slap-connection:computer-id}
partition_id = ${slap-connection:partition-id}
server_url = ${slap-connection:server-url}
software_release_url = ${slap-connection:software-release-url}
key_file = ${slap-connection:key-file}
cert_file = ${slap-connection:cert-file}
[slap-parameter]
slave_instance_list =
-frontend-quantity = 1
-frontend-type = single-default
{%- endif %}
software/caddy-frontend/instance-caddy-slave.cfg 0 → 100644
View file @ 22396c5f
[buildout]
parts =
directory
logrotate
cron
cron-entry-logrotate
ca-frontend
certificate-authority
caddy-frontend
switch-caddy-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
# improtant parts for profile generation
[switch-caddy-softwaretype]
recipe = slapos.cookbook:softwaretype
single-default = $${dynamic-custom-personal-template-slave-list:rendered}
single-custom-personal = $${dynamic-custom-personal-template-slave-list:rendered}
[dynamic-custom-personal-template-slave-list]
# Configures each requested slave
< = jinja2-template-base
template = ${template-slave-list:target}
filename = custom-personal-instance-slave-list.cfg
extensions = jinja2.ext.do
# XXX? content to be removed or tackled:
# * key domain
extra-context =
key caddy_slave_configuration_directory caddy-directory:slave-configuration
key caddy_log_directory caddy-directory:slave-log
key custom_ssl_directory caddy-directory:vh-ssl
key domain instance-parameter:configuration.domain
key empty_template software-release-path:template-empty
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key frontend_lazy_graceful_reload frontend-caddy-lazy-graceful:rendered
key global_ipv6 slap-network-information:global-ipv6
key local_ipv6 instance-parameter:ipv6-random
key local_ipv4 instance-parameter:ipv4-random
key promise_directory directory:promise
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key ssl_ca_crt ca-custom-frontend:rendered
key ssl_certificate ca-frontend:cert-file
key ssl_key ca-frontend:key-file
key template_custom_slave_configuration software-release-path:template-slave-configuration
key template_default_slave_configuration software-release-path:template-default-slave-virtualhost
raw bin_directory ${buildout:bin-directory}
raw software_type single-custom-personal
section logrotate_dict logrotate
[caddyfile]
# Main Caddyfile
< = jinja2-template-base
template = ${template-caddyfile:target}
rendered = $${caddy-configuration:frontend-configuration}
extra-context =
key slave_configuration_directory caddy-directory:slave-configuration
[caddy-frontend]
recipe = slapos.cookbook:wrapper
command-line = ${caddy:output}
-http-port $${instance-parameter:configuration.plain_http_port}
-https-port $${instance-parameter:configuration.port}
-conf $${caddyfile:rendered}
-disable-http-challenge
-disable-tls-sni-challenge
-pidfile $${caddy-configuration:pid-file}
-log $${caddy-configuration:frontend-log}
-root $${caddy-directory:document-root}
wrapper-path = $${directory:service}/frontend_caddy
wait-for-files =
$${ca-frontend:cert-file}
$${ca-frontend:key-file}
[frontend-caddy-lazy-graceful]
< = jinja2-template-base
template = ${template-lazy-script-call:target}
rendered = $${directory:bin}/frontend-caddy-lazy-graceful
mode = 0700
pid-file = $${directory:run}/lazy-graceful.pid
extra-context =
key pid_file :pid-file
raw wait_time 60
key lazy_command caddy-configuration:frontend-graceful-command
[ca-custom-frontend]
< = jinja2-template-base
template = ${template-empty:target}
rendered = $${cadirectory:certs}/caddy_frontend.ca.crt
extra-context =
key content instance-parameter:configuration.apache-ca-certificate
# configuration
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
configuration.domain = example.org
configuration.public-ipv4 =
configuration.port = 4443
configuration.plain_http_port = 8080
## Caddy own variables -- TODO!!
#configuration.caddy_custom_https = ""
#configuration.caddy_custom_http = ""
#configuration.caddy-key =
#configuration.caddy-certificate =
#configuration.caddy-ca-certificate =
# Apache compatbility
configuration.apache_custom_https = ""
configuration.apache_custom_http = ""
configuration.apache-key =
configuration.apache-certificate =
configuration.apache-ca-certificate =
configuration.open-port = 80 443
configuration.extra_slave_instance_list =
configuration.re6st-verification-url = http://[2001:67c:1254:4::1]/index.html
# common and shared parts for slave
[directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin/
etc = $${buildout:directory}/etc/
srv = $${buildout:directory}/srv/
var = $${buildout:directory}/var/
template = $${buildout:directory}/template/
backup = $${:srv}/backup
log = $${:var}/log
run = $${:var}/run
service = $${:etc}/service
etc-run = $${:etc}/run
promise = $${:etc}/promise
logrotate-backup = $${:backup}/logrotate
logrotate-entries = $${:etc}/logrotate.d
cron-entries = $${:etc}/cron.d
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
ca-dir = $${:srv}/ssl
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type instance-parameter:slap-software-type
key slapparameter_dict instance-parameter:configuration
section directory directory
$${:extra-context}
[caddy-directory]
recipe = slapos.cookbook:mkdirectory
document-root = $${directory:srv}/htdocs
slave-configuration = $${directory:etc}/caddy-slave-conf.d/
vh-ssl = $${:slave-configuration}/ssl
slave-log = $${directory:log}/httpd
[caddy-configuration]
frontend-configuration = $${directory:etc}/Caddyfile
frontend-log = $${directory:log}/frontend.log
pid-file = $${directory:run}/httpd.pid
protected-path = /
access-control-string = none
frontend-configuration-verification = ${caddy:location}/bin/httpd -Sf $${:frontend-configuration} > /dev/null
frontend-graceful-command = $${:frontend-configuration-verification}; if [ $? -eq 0 ]; then kill -USR1 $(cat $${:pid-file}); fi
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:service}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[software-release-path]
template-empty = ${template-empty:target}
template-slave-configuration = ${template-slave-configuration:target}
template-default-slave-virtualhost = ${template-default-slave-virtualhost:target}
[ca-frontend]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/caddy_frontend.key
cert-file = $${cadirectory:certs}/caddy_frontend.crt
executable = $${directory:service}/frontend_caddy
wrapper = $${directory:service}/frontend_caddy
key-content = $${instance-parameter:configuration.apache-key}
cert-content = $${instance-parameter:configuration.apache-certificate}
# Put domain name
name = $${instance-parameter:configuration.domain}
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = ${logrotate:location}/sbin/logrotate
gzip-binary = ${gzip:location}/bin/gzip
gunzip-binary = ${gzip:location}/bin/gunzip
# Directories
wrapper = $${directory:bin}/logrotate
conf = $${directory:etc}/logrotate.conf
logrotate-entries = $${directory:logrotate-entries}
backup = $${directory:logrotate-backup}
state-file = $${directory:srv}/logrotate.status
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = ${dcron:location}/sbin/crond
cron-entries = $${directory:cron-entries}
crontabs = $${directory:crontabs}
cronstamps = $${directory:cronstamps}
catcher = $${cron-simplelogger:wrapper}
binary = $${directory:service}/crond
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = $${directory:bin}/cron_simplelogger
log = $${directory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = $${logrotate:wrapper}
[slap_connection]
# Kept for backward compatiblity
computer_id = $${slap-connection:computer-id}
partition_id = $${slap-connection:partition-id}
server_url = $${slap-connection:server-url}
software_release_url = $${slap-connection:software-release-url}
key_file = $${slap-connection:key-file}
cert_file = $${slap-connection:cert-file}
software/caddy-frontend/instance.cfg 0 → 100644
View file @ 22396c5f
[buildout]
parts =
dynamic-template-caddy-master
switch_softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-configuration:slap-software-type
key slapparameter_dict slap-configuration:configuration
key slave_instance_list slap-configuration:slave-instance-list
$${:extra-context}
[dynamic-template-caddy-master]
< = jinja2-template-base
template = ${template-caddy-master:target}
filename = instance-caddy-master.cfg
extensions = jinja2.ext.do
extra-context =
raw template_publish_slave_information ${template-master-publish-slave-information:target}
# Must match the key id in [switch_softwaretype] which uses this section.
raw software_type RootSoftwareInstance-default-custom-personal-master
raw template_monitor ${monitor2-template:rendered}
[switch_softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${dynamic-template-caddy-master:rendered}
RootSoftwareInstance = $${dynamic-template-caddy-master:rendered}
custom-personal = $${dynamic-template-caddy-master:rendered}
single-default = ${template-caddy-slave:output}
single-custom-personal = ${template-caddy-slave:output}
replicate = $${dynamic-template-caddy-master:rendered}
software/caddy-frontend/software.cfg 0 → 100644
View file @ 22396c5f
[buildout]
extends =
buildout.hash.cfg
../../stack/slapos.cfg
../../component/caddy/buildout.cfg
# Monitoring stack (keep on bottom)
../../stack/monitor/buildout.cfg
parts +=
template
template-caddy-slave
template-caddy-master
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/${:filename}
output = ${buildout:directory}/template.cfg
mode = 0644
[template-caddy-slave]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/${:filename}
output = ${buildout:directory}/instance-caddy-slave.cfg
mode = 0644
[template-caddy-master]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/${:filename}
mode = 0640
[download-template]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/${:filename}
mode = 640
[template-master-publish-slave-information]
<=download-template
filename = master-publish-slave-information.cfg.in
[template-slave-list]
<=download-template
filename = custom-slave-list.cfg.in
[template-empty]
<=download-template
filename = empty.in
[template-slave-configuration]
<=download-template
filename = slave-configuration.conf.in
[template-default-slave-virtualhost]
<=download-template
filename = default-slave-virtualhost.conf.in
[template-lazy-script-call]
<=download-template
filename = lazy-script-call.sh.in
[template-caddyfile]
<=download-template
filename = Caddyfile.in
[versions]
slapos.recipe.template = 4.3
apache-libcloud = 2.3.0
gitdb2 = 2.0.3
smmap2 = 2.0.3
# Required by:
# slapos.toolbox==0.75
GitPython = 2.1.9
# Required by:
# slapos.toolbox==0.75
atomize = 0.2.0
# Required by:
# slapos.toolbox==0.75
dnspython = 1.15.0
# Required by:
# slapos.toolbox==0.75
erp5.util = 0.4.50
# Required by:
# slapos.toolbox==0.75
feedparser = 5.2.1
# Required by:
# slapos.toolbox==0.75
lockfile = 0.12.2
# Required by:
# slapos.toolbox==0.75
passlib = 1.7.1
software/caddy-frontend/templates/Caddyfile.in 0 → 100644
View file @ 22396c5f
# Automatically generated Caddyfile
import {{ slave_configuration_directory }}/*.conf
software/caddy-frontend/templates/custom-slave-list.cfg.in 0 → 100644
View file @ 22396c5f
{% if software_type == slap_software_type -%}
{% set part_list = [] -%}
{% set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{% set slave_log_dict = {} -%}
{% if extra_slave_instance_list -%}
{% set slave_instance_information_list = [] -%}
{% set slave_instance_list = slave_instance_list + json_module.loads(extra_slave_instance_list) -%}
{% endif -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
extensions = jinja2.ext.do
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
${:extra-context}
{% do logrotate_dict.pop('recipe') -%}
[logrotate]
{% for key, value in logrotate_dict.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
post = {{ frontend_lazy_graceful_reload }} &
frequency = daily
rotatep-num = 30
sharedscripts = true
notifempty = true
create = true
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = {{ custom_ssl_directory }}/requests/
private = {{ custom_ssl_directory }}/private/
certs = {{ custom_ssl_directory }}/certs/
newcerts = {{ custom_ssl_directory }}/newcerts/
crl = {{ custom_ssl_directory }}/crl/
{# Loop throught slave list to set up slaves -#}
{% for slave_instance in slave_instance_list -%}
{# # Do all set and do upper, so it makes easy to read the file later -#}
{% set slave_reference = slave_instance.get('slave_reference') -%}
{% set slave_type = slave_instance.get('type', '') -%}
{% set slave_section_title = 'dynamic-template-slave-instance-%s' % slave_reference -%}
{% set slave_parameter_dict = {} -%}
{% set slave_publish_dict = {} -%}
{% set slave_configuration_section_name = 'slave-instance-%s-configuration' % slave_reference -%}
{% set slave_logrotate_section = slave_reference + "-logs" -%}
{% set slave_ln_section = slave_reference + "-ln" -%}
{# extend parts -#}
{% do part_list.extend([slave_logrotate_section, slave_section_title]) -%}
{% set slave_log_folder = logrotate_dict.get('backup') + '/' + slave_reference + "-logs" -%}
{# Set Up log files -#}
{% do slave_parameter_dict.__setitem__('access_log', '/'.join([caddy_log_directory, '%s_access_log' % slave_reference])) -%}
{% do slave_parameter_dict.__setitem__('error_log', '/'.join([caddy_log_directory, '%s_error_log' % slave_reference])) -%}
{% do slave_instance.__setitem__('access_log', slave_parameter_dict.get('access_log')) -%}
{% do slave_instance.__setitem__('error_log', slave_parameter_dict.get('error_log')) -%}
{# Set Up ip addresses -#}
{% do slave_parameter_dict.__setitem__('local_ipv4', local_ipv4) -%}
{% do slave_parameter_dict.__setitem__('local_ipv6', global_ipv6) -%}
{% do slave_instance.__setitem__('local_ipv4', slave_parameter_dict.get('local_ipv4')) -%}
{% do slave_instance.__setitem__('local_ipv6', slave_parameter_dict.get('local_ipv6')) -%}
{# Pass SSL information -#}
{% if 'ssl_key' in slave_instance and 'ssl_crt' in slave_instance -%}
{# Case: local certificate for an slave -#}
{% set cert_title = '%s-crt' % (slave_reference) -%}
{% set key_title = '%s-key' % (slave_reference) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% set key_file = '/'.join([custom_ssl_directory, key_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_parameter_dict.__setitem__("ssl_certificate", cert_file) -%}
{% do slave_parameter_dict.__setitem__("ssl_key", key_file) -%}
{% do slave_instance.__setitem__('path_to_ssl_certificate', cert_file) -%}
{% do slave_instance.__setitem__('path_to_ssl_key', key_file) -%}
[{{cert_title}}]
recipe = slapos.cookbook:certificate_authority.request
#openssl-binary = ${openssl:location}/bin/openssl
requests-directory = ${cadirectory:requests}
ca-private = ${cadirectory:private}
ca-certs = ${cadirectory:certs}
ca-newcerts = ${cadirectory:newcerts}
ca-crl = ${cadirectory:crl}
key-file = {{ key_file }}
cert-file = {{ cert_file }}
key-content = {{ dumps(slave_instance.get('ssl_key')) }}
cert-content = {{ dumps(slave_instance.get('ssl_crt')) }}
{% else -%}
{# Case: shared certificate for an slave -#}
{% do slave_parameter_dict.__setitem__('path_to_ssl_certificate', ssl_certificate) -%}
{% do slave_parameter_dict.__setitem__('path_to_ssl_key', ssl_key) -%}
{% do slave_parameter_dict.__setitem__('path_to_ssl_ca_crt', ssl_ca_crt) -%}
{% do slave_instance.__setitem__('path_to_ssl_certificate', slave_parameter_dict.get('path_to_ssl_certificate')) -%}
{% do slave_instance.__setitem__('path_to_ssl_key', slave_parameter_dict.get('path_to_ssl_key')) -%}
{% do slave_instance.__setitem__('path_to_ssl_ca_crt', slave_parameter_dict.get('path_to_ssl_ca_crt')) -%}
{% endif -%}
{% do slave_publish_dict.__setitem__('slave-reference', slave_reference) -%}
{% do slave_publish_dict.__setitem__('public-ipv4', public_ipv4) -%}
{# Set slave domain if none was defined -#}
{% if slave_instance.get('custom_domain', None) == None -%}
{% set domain_prefix = slave_instance.get('slave_reference').replace("-", "").lower() -%}
{% do slave_instance.__setitem__('custom_domain', "%s.%s" % (domain_prefix, slapparameter_dict.get('domain'))) -%}
{% endif -%}
{% if not slave_instance.has_key('caddy_custom_http') -%}
{% do slave_publish_dict.__setitem__('domain', slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('url', "http://%s" % slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('site_url', "http://%s" % slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('secure_access', 'https://%s' % slave_instance.get('custom_domain')) -%}
{% endif -%}
[slave-log-directories]
{{slave_reference}} = {{ slave_log_folder }}
{# Set slave logrotate entry -#}
[{{slave_logrotate_section}}]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = ${:_buildout_section_name_}
log = {{slave_parameter_dict.get('access_log')}} {{slave_parameter_dict.get('error_log')}}
backup = {{ slave_log_folder }}
{# integrate current logs inside -#}
[{{slave_ln_section}}]
recipe = plone.recipe.command
stop-on-error = false
command = ln -s {{slave_parameter_dict.get('error_log')}} {{ slave_log_folder }}/caddy-error.log && ln -s {{slave_parameter_dict.get('access_log')}} {{ slave_log_folder }}/caddy-access.log
{# ################################################## -#}
{# Set Slave Certificates if needed -#}
{# Set ssl certificates for each slave -#}
{% for cert_name in ('ssl_ca_crt', 'ssl_csr', 'ssl_proxy_ca_crt')-%}
{% if cert_name in slave_instance -%}
{% set cert_title = '%s-%s' % (slave_reference, cert_name.replace('ssl_', '')) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_parameter_dict.__setitem__(cert_name, cert_file) -%}
{% do slave_instance.__setitem__('path_to_' + cert_name, cert_file) -%}
{# Store certificates on fs -#}
[{{ cert_title }}]
< = jinja2-template-base
template = {{ empty_template }}
rendered = {{ cert_file }}
extra-context =
key content {{ cert_title + '-config:value' }}
# Store certificate in config
[{{ cert_title + '-config' }}]
value = {{ dumps(slave_instance.get(cert_name)) }}
{% endif -%}
{% endfor -%}
{# ########################################## -#}
{# Set Slave Configuration -#}
[{{ slave_configuration_section_name }}]
{% set caddy_custom_http = ((slave_instance.pop('caddy_custom_http', '')) % slave_parameter_dict) -%}
{% set caddy_custom_https = ((slave_instance.pop('caddy_custom_https', '')) % slave_parameter_dict) -%}
caddy_custom_http = {{ dumps(caddy_custom_http) }}
caddy_custom_https = {{ dumps(caddy_custom_https) }}
{{ '\n' }}
{% for key, value in slave_instance.iteritems() -%}
{{ key }} = {{ dumps(value) }}
{% endfor -%}
[{{ slave_section_title }}]
< = jinja2-template-base
rendered = {{ caddy_slave_configuration_directory }}/${:filename}
{% if caddy_custom_http -%}
template = {{ template_custom_slave_configuration }}
{% else -%}
template = {{ template_default_slave_configuration }}
{% endif -%}
filename = {{ '%s.conf' % slave_reference }}
extra-context =
raw global_ipv6 {{ global_ipv6 }}
raw local_ipv4 {{ local_ipv4 }}
section slave_parameter {{ slave_configuration_section_name }}
{{ '\n' }}
{% set re6st_optimal_test = slave_instance.get('re6st-optimal-test', '') -%}
{% if re6st_optimal_test -%}
{% set re6st_ipv6, re6st_ipv4 = re6st_optimal_test.split(",") -%}
{% set re6st_optimal_test_section_title = 'check-%s-re6st-optimal-test' % slave_instance.get('slave_reference') -%}
{% do part_list.append(re6st_optimal_test_section_title) -%}
[{{ re6st_optimal_test_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/check-re6st-optimal-status -4 {{re6st_ipv4}} -6 {{re6st_ipv6}}
filename = {{ re6st_optimal_test_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% endif -%}
{# ############################### -#}
{# Publish Slave Information -#}
{% if not extra_slave_instance_list -%}
{% set publish_section_title = 'publish-%s-connection-information' % slave_instance.get('slave_reference') -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
{% for key, value in slave_publish_dict.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% else -%}
{% do slave_instance_information_list.append(slave_publish_dict) -%}
{% endif -%}
{# End of the main for loop#}
{% endfor -%}
[slave-log-directories]
recipe = slapos.cookbook:mkdirectory
{# Publish information for the instance -#}
[publish-caddy-information]
recipe = slapos.cookbook:publish
public-ipv4 = {{ public_ipv4 }}
private-ipv4 = {{ local_ipv4 }}
{% if extra_slave_instance_list -%}
slave-instance-information-list = {{ json_module.dumps(slave_instance_information_list) }}
{% endif -%}
[buildout]
parts +=
slave-log-directories
publish-caddy-information
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
{% endif -%}
software/caddy-frontend/templates/default-slave-virtualhost.conf.in 0 → 100644
View file @ 22396c5f
{%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{%- set disable_no_cache_header = ('' ~ slave_parameter.get('disable-no-cache-request', '')).lower() in TRUE_VALUES -%}
{%- set disable_via_header = ('' ~ slave_parameter.get('disable-via-header', '')).lower() in TRUE_VALUES -%}
{%- set prefer_gzip = ('' ~ slave_parameter.get('prefer-gzip-encoding-to-backend', '')).lower() in TRUE_VALUES -%}
{%- set server_alias_list = slave_parameter.get('server-alias', '').split() -%}
{%- set enable_h2 = ('' ~ slave_parameter.get('enable-http2', 'true')).lower() in TRUE_VALUES -%}
{%- set ssl_proxy_verify = ('' ~ slave_parameter.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES -%}
{%- set disabled_cookie_list = slave_parameter.get('disabled-cookie-list', '').split() -%}
{%- set https_only = ('' ~ slave_parameter.get('https-only', '')).lower() in TRUE_VALUES -%}
{%- set slave_type = slave_parameter.get('type', '') -%}
https://{{ slave_parameter.get('custom_domain') }} {
proxy / {{ slave_parameter.get('https-url', slave_parameter.get('url', '')) }} {
transparent
{% if not ssl_proxy_verify -%}
insecure_skip_verify
{% endif %}
}
log / {{ slave_parameter.get('access_log') }} {combined}
errors {{ slave_parameter.get('error_log') }}
bind {{ slave_parameter.get('local_ipv4') }}
{# TODO: only one bind is supported https://github.com/mholt/caddy/issues/864 -#}
{# bind {{ slave_parameter.get('local_ipv6') }} -#}
tls {{ slave_parameter.get('path_to_ssl_certificate') }} {{ slave_parameter.get('path_to_ssl_key') }}
}
http://{{ slave_parameter.get('custom_domain') }} {
bind {{ slave_parameter.get('local_ipv4') }}
{# TODO: only one bind is supported https://github.com/mholt/caddy/issues/864 -#}
{# bind {{ slave_parameter.get('local_ipv6') }} -#}
# XXX --> hardcoded redirection
redir / https://{host}{uri}
log / {{ slave_parameter.get('access_log') }} {combined}
errors {{ slave_parameter.get('error_log') }}
}
\ No newline at end of file
software/caddy-frontend/templates/empty.in 0 → 100644
View file @ 22396c5f
{{ content }}
\ No newline at end of file
software/caddy-frontend/templates/lazy-script-call.sh.in 0 → 100644
View file @ 22396c5f
#!/bin/bash
PID=$$
PIDFILE={{ pid_file }}
sleep $((RANDOM % 10))
if [ ! -f $PIDFILE ]; then
echo $PID > $PIDFILE
sleep {{ wait_time }}
{{ lazy_command }}
else
ps --pid `cat $PIDFILE` &>/dev/null
if [ $? -eq 0 ]; then
echo "Skipped"
else
echo $PID > $PIDFILE
sleep {{ wait_time }}
{{ lazy_command }}
fi
fi
software/caddy-frontend/templates/log-access.conf.in 0 → 100644
View file @ 22396c5f
{% for slave, directory in slave_log_directory.iteritems() %}
Alias /{{slave}}/ {{directory}}/
<Directory {{directory}}>
Order Deny,Allow
Deny from env=AUTHREQUIRED
<Files ".??*">
Order Allow,Deny
Deny from all
</Files>
AuthType Basic
AuthName "Log Access {{slave}}"
Options Indexes FollowSymLinks
Satisfy all
</Directory>
{% endfor %}
software/caddy-frontend/templates/master-publish-slave-information.cfg.in 0 → 100644
View file @ 22396c5f
{% set part_list = [] -%}
{% set slave_information_dict = {} -%}
# regroup slave information from all frontends
{%- for frontend, slave_list_raw in slave_information.iteritems() -%}
{% if slave_list_raw %}
{% set slave_list = json_module.loads(slave_list_raw) -%}
{% else %}
{% set slave_list = [] %}
{% endif %}
{% for slave_dict in slave_list -%}
{% set slave_reference = slave_dict.pop('slave-reference') %}
{% set current_slave_dict = slave_information_dict.get(slave_reference, {}) %}
{% do current_slave_dict.update(slave_dict) -%}
{% do current_slave_dict.__setitem__(
'replication_number',
current_slave_dict.get('replication_number', 0) + 1
) -%}
{% do slave_information_dict.__setitem__(slave_reference, current_slave_dict) -%}
{% endfor -%}
{% endfor %}
# Publish information for each slave
{% for slave_reference, slave_information in slave_information_dict.iteritems() %}
{% set publish_section_title = 'publish-%s' % slave_reference -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_reference }}
{% for key, value in slave_information.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% endfor %}
[buildout]
parts =
{% for part in part_list %}
{{ ' %s' % part }}
{% endfor %}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
software/caddy-frontend/templates/slave-configuration.conf.in 0 → 100644
View file @ 22396c5f
{{ slave_parameter.get('caddy_custom_https', '') }}
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7