pax_global_header 0000666 0000000 0000000 00000000064 13344470245 0014520 g ustar 00root root 0000000 0000000 52 comment=38ed675aef44c876cc23a5b810087f5472550031
slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/ 0000775 0000000 0000000 00000000000 13344470245 0022315 5 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/ 0000775 0000000 0000000 00000000000 13344470245 0023422 5 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/ 0000775 0000000 0000000 00000000000 13344470245 0025111 5 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/README.md 0000664 0000000 0000000 00000026745 13344470245 0026406 0 ustar 00root root 0000000 0000000 Monitor stack
=============
* This stack has for purpose to know if all promises, services, custom monitoring scripts went/are ok.
* It also provides a web interface, to see which promises, instance and hosting subscription status. It also provide a rss feed to easily know the actual state of your instance, and to know when it started to went bad. You can also add your own monitoring scripts.
Implementation :
----------------
1/ In your software.cfg, extends the stack:
[buildout]
extends =
...
../../stack/monitor/buildout.cfg
2/ In your instance.cfg file or instance template (instance-xxx.cfg.jinja2, ...),
Extend monitor template and a monitor-base to parts:
[buildout]
extends =
${monitor-template:rendered}
parts =
...
monitor-base
Override monitor configuration by adding monitor-instance-parameter section to define your custom parameters.
[monitor-instance-parameter]
monitor-title = ${slap-configuration:instance-title}
monitor-httpd-ipv6 = ${slap-configuration:ipv6-random}
monitor-httpd-port = ...
monitor-base-url = ${monitor-frontend-promise:url}
root-instance-title = ${slap-configuration:root-instance-title}
monitor-url-list =
cors-domains = monitor.app.officejs.com
collector-db = ...
password = ${monitor-htpasswd:passwd}
username = admin
instance-configuration = ...
configuration-file-path = ...
interface-url = ...
You don't need to define all parameters, you can only set what is required to be changed. ie:
[monitor-instance-parameter]
monitor-httpd-port = 8333
- monitor-title: is the title of the current software instance.
- root-instance-title: is the title of the hosting subscription.
- monitor-httpd-ipv6: is the ipv6 of the computer partition.
- monitor-httpd-port: the port to bind monitor httpd server on.
- monitor-base-url: this url will be used/showed in monitor interface. This url is present in some monitor generated output files. There can be two value, the default: ${monitor-frontend-promise:url} which access monitor httpd server through the frontend and ${monitor-httpd-conf-parameter:url} which is the url with ipv6 (https://[IPv6]:port/).
- monitor-url-list: set list of Monitor Base URL of monitor sub-instances, if this is the root instance with at least one child.
- cors-domains: the domain used by the monitor web interface. The default is: monitor.app.officejs.com.
- username: monitor username, this should be the same in all sub-instances. Default is: admin.
- password: monitor password, this should be the same in all sub-instances. Default is generated (${monitor-instance-parameter:username}).
- instance-configuration: instance custom information or configuration to show in monitor web interface. There is many possibility:
raw CONFIG_KEY VALUE => non editable configuration, ie: raw monitor-password resqdsdsd34
file CONFIG_KEY PATH_TO_RESULT_FILE => editable configuration.
httpdcors CONFIG_KEY PATH_TO_HTTP_CORS_CFG_FILE PATH_HTTPD_GRACEFUL_WRAPPER => show/edit cors domain in monitor
- configuration-file-path: path of knowledge0 cfg file where instance configuration will be written.
- interface-url: The URL of monitor web interface. This URL will be present in generated JSON files.
**Multiple Monitors**
If you have sub-instances, you should collect the base monitor url from all instances with monitor and send it to monitor-url-list or you can override "monitor-base-url-dict" section and add all the urls as key/value pairs in the root instance.
[monitor-base-url-list]
monitor1-url = https://[xxxx:xxx:xxxx:e:11::1fb1]:4200
monitor2-url = https://[xxxx:xxx:xxxx:e:22::2fb2]:4200
..
..
Also, All monitors of the sub instances need to have same password as the password of the root instance monitor.
NB: You should use double $ (ex: $${monitor-template:rendered}) instead of one $ in your instance template file if it's not a jinja template. See:
- Jinja template file exemple, use one $: https://lab.nexedi.com/nexedi/slapos/blob/master/software/slaprunner/instance-resilient-test.cfg.jinja2
- Non Jinja template file, use $$: https://lab.nexedi.com/nexedi/slapos/blob/master/software/slaprunner/instance.cfg
Add a promise
-------------
To learn how to write a promise in SlapOS, please read this document:
https://www.erp5.com/slapos-TechnicalNote.General.SlapOS.Monitoring.Specifications
Writing a promise consists of defining a class called RunPromise which inherits from GenericPromise class and defining methods: anomaly(), sense() and test(). Python promises should be placed into the folder etc/plugin of the computer partition.
New promises should be placed into the folder etc/plugin, legacy promise are into the folder etc/promise. Legacy promises are bash or other executable promises script which does not use GenericPromise class.
You will use slapos.cookbook:promise.plugin to generate your promise script into `etc/plugin` directory. Add promise will look like this:
[promise-check-site]
recipe = slapos.cookbook:promise.plugin
eggs =
slapos.toolbox
output = ${directory:plugins}/promise-check-mysite-status.py
content =
from slapos.promise.plugin.check_site_state import RunPromise
config-site-url = ${publish:site-url}
config-connection-timeout = 20
config-foo = bar
mode = 600
Then you will have to add `promise-check-site` section to buildout parts, so it will be installed.
In your promise code, you will be able to call `self.getConfig('site-url')`, `self.getConfig('connection-timeout')` and `self.getConfig('foo')`. The
returned value is `None` if the config parameter is not set.
Slapgrid will run each promise every time a partition is processed (every minutes in theory), if the partition is up to date, slapgrid will only run promises anomaly check and save the result in a json file. Here is an exemple of promise result:
{"result": {"date": "2018-03-22T15:35:07", "failed": false, "message": "buildout is OK", "type": "Test Result"}, "path": "PARTITION_DIRECTORY/etc/plugin/buildout-slappart0-status.py", "name": "buildout-slappart0-status.py", "execution-time": 0.1, "title": "buildout-slappart0-status"}
The promise execution time should be short, by default promise-timeout in slapgrid is to 20 seconds. If a promise runs in more than the defined promise-timeout, the process is killed and a "promise timed out" message is returned.
JSON in the folder `PARTITION_DIRECTORY/.slapgrid/promise/result`, and promise logs are in `PARTITION_DIRECTORY/.slapgrid/promise/log`.
Monitor will expose promise JSON result in web public folder, access URL is: `MONITOR_BASE_URL/public/promise/PROMISE_TITLE.status.json`. Log files are exposed in monitor private web folder,
access URL is: `MONITOR_BASE_URL/private/log/monitor/promise`
Add custom file or directory to monitor
---------------------------------------
Log or others files can be added in monitor public or private directory:
[monitor-conf-parameters]
public-path-list =
/path/to/my/public/directory/
...
private-path-list =
${directory:log}
...
files in public directory are accessible at `MONITOR_BASE_URL/public`, and for private directory: `MONITOR_BASE_URL/private`.
Monitor promise history, RSS and OPML Feed
------------------------------------------
Monitor read every minutes JSON promises result files to build Rss and full instance state. The Rss feed URL is
`MONITOR_BASE_URL/public/feed`
OPML Feed is used to aggregate many feed URL, this is used on monitor to link many single monitor instances. For example, a resilient
webrunner has 3 instances at least, each instance has a monitor which leads to 3 monitor instances too. One main instance (usally the root instance)
will collect rss feed of all others monitor's in a single OPML file. This file is published and used to configure a monitor backend to the web interface.
The URL of OPML feed is: `MONITOR_BASE_URL/public/feeds`
Everytime monitor will also produce history of for each promise in a single JSON file.
To access promise history file as JSON, use URL `MONITOR_BASE_URL/public/PROMISE_TITLE.history.json`
Monitor Base web directory tree
-------------------------------
MONITOR_BASE_URL
|
--------------------------
| | |
share public private
(webdav) X Y
|
---------------------------------
| |
public private
X Y
MONITOR_BASE_URL/public or private is for normal HTTPS.
MONITOR_BASE_URL/share is the webdav URL. public/ and private/ are linked to public and private directories.
Access to Monitor
-----------------
In monitor instance.cfg file, the section [monitor-publish-parameters] contain information about monitor access.
Usefull information are monitor-base-url, monitor-url, monitor-user and monitor-password.
- ${monitor-publish-parameters:monitor-base-url} is the url of monitor httpd server.
- ${monitor-publish-parameters:monitor-base-url}/public/feed is the RSS url of this monitor instance.
- ${monitor-publish-parameters:monitor-base-url}/public/feeds is the OPML URL of this monitor instance. To setup monitor instance in your monitoring interface, use OPML URL of the root instance. It should contain URL to others monitor instances.
- ${monitor-publish-parameters:monitor-base-url}/private is the monitor private directory. Username and password are reqired to connect.
The section [monitor-publish] contain parameters to publish with your instance connection information. It will publish "monitor-base-url" and
"monitor-setup-url" which is used to configure your instance to monitor interface in one click.
To publish configuration URL in your instance.cfg, you can do like this:
[publish-connection-information]
<= monitor-publish
...
custom-parameter-one = xxxxx
custom-parameter-two = yyyyy
Send parameters to monitor interface
------------------------------------
Monitor has a paramters called "instance-configuration" from the section [monitor-instance-parameter]
that can be updated to specify which parameters will be deplayed on monitor web interface.
Parameters can be editable (except raw parameter) directly from monitor interface. The change will be updated into the related file. Here are some examples:
[monitor-instance-parameter]
instance-configuration =
raw init-user ${publish-connection-information:init-user}
htpasswd monitor-password ${httpd-monitor-htpasswd:password-file} ${monitor-instance-parameter:username} ${httpd-monitor-htpasswd:htpasswd-path}
file promise-timeout ${monitor-promise-timeout-file:file}
The user will see parameters:
- init-user (non editable)
- monitor-password (editable)
- promise-timeout (editable)
htpasswd: is used to change apache htpasswd directly from monitor interface. The syntax is like:
htpasswd PARAMETER_ID PASSWORD_TEXT_FILE HTPASSWD_USER_NAME HTPASSWD_FILE
PASSWORD_TEXT_FILE contain the password which is showed to the user.
file: is used to edit a parameter directly into file. Parameter is read and write into the provided file
file PARAMETER_ID PATH_TO_THE_FILE
raw: is a non editable paramter.
raw PARAMETER_ID TEXT_VALUE
httpdcors: used to edit an apache http_cors.conf file, this file should be include in the main apache configuration file
httpdcors PARAMETER_ID PATH_TO_HTTP_CORS_CFG_FILE PATH_HTTPD_GRACEFUL_WRAPPER
PATH_HTTPD_GRACEFUL_WRAPPER will be executed to reload apache configuration after modification is done.
slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/buildout.cfg 0000664 0000000 0000000 00000007424 13344470245 0027430 0 ustar 00root root 0000000 0000000 [buildout]
extends =
buildout.hash.cfg
../../component/apache/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/openssl/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/pycurl/buildout.cfg
../../component/python-cryptography/buildout.cfg
../../stack/logrotate/buildout.cfg
parts =
slapos-cookbook
monitor-eggs
extra-eggs
monitor2-template
[monitor-download-base]
recipe = hexagonit.recipe.download
ignore-existing = true
download-only = true
url = ${:_profile_base_location_}/${:filename}
mode = 0644
[monitor-template-base]
<= monitor-download-base
url = ${:_profile_base_location_}/templates/${:filename}
[monitor-template-script]
<= monitor-download-base
url = ${:_profile_base_location_}/scripts/${:filename}
destination = ${buildout:parts-directory}/monitor-scripts
on-update = true
[monitor-eggs]
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
${pycurl:egg}
${python-cryptography:egg}
plone.recipe.command
collective.recipe.template
cns.recipe.symlink
slapos.toolbox
slapos.core
# Do no generate any scripts here as all of them are generated by extraeggs
scripts =
[extra-eggs]
recipe = zc.recipe.egg
interpreter = pythonwitheggs
eggs =
${monitor-eggs:eggs}
psutil
PyRSS2Gen
Jinja2
# Monitor templates files
[monitor-httpd-conf]
<= monitor-template-base
md5sum = b5f42503799e7e770afce4097d3b75ae
filename = monitor-httpd.conf.in
[monitor-template-wrapper]
<= monitor-template-base
filename = wrapper.in
md5sum = 1695c9a06a2b11ccfe893d7a224e489d
[monitor-conf]
<= monitor-template-base
filename = monitor.conf.in
md5sum = 91c4c9bba1f7df788b9b7a059ed89ac2
[monitor-httpd-cors]
<= monitor-template-base
filename = httpd-cors.cfg.in
md5sum = 683ea85fc054094248baf5752dd089bf
# End templates files
# XXX keep compatibility (with software/ipython_notebook/software.cfg )
[monitor-template]
rendered = ${monitor2-template:rendered}
output = ${monitor2-template:rendered}
[monitor2-template]
recipe = slapos.recipe.template:jinja2
filename = template-monitor.cfg
template = ${:_profile_base_location_}/instance-monitor.cfg.jinja2.in
rendered = ${buildout:directory}/template-monitor.cfg
context =
key apache_location apache:location
key gzip_location gzip:location
key template_logrotate_base template-logrotate-base:rendered
raw monitor_bin ${buildout:directory}/bin/monitor.bootstrap
raw monitor_collect ${buildout:directory}/bin/monitor.collect
raw monitor_statistic ${buildout:directory}/bin/monitor.statistic
raw monitor_runpromise ${buildout:directory}/bin/monitor.runpromise
raw monitor_genstatus ${buildout:directory}/bin/monitor.genstatus
raw monitor_configwrite ${buildout:directory}/bin/monitor.configwrite
raw monitor_conf_template ${monitor-conf:location}/${monitor-conf:filename}
raw monitor_https_cors ${monitor-httpd-cors:location}/${monitor-httpd-cors:filename}
raw curl_executable_location ${curl:location}/bin/curl
raw dash_executable_location ${dash:location}/bin/dash
raw dcron_executable_location ${dcron:location}/sbin/crond
raw logrotate_executable_location ${logrotate:location}/usr/sbin/logrotate
raw monitor_httpd_template ${monitor-httpd-conf:location}/${monitor-httpd-conf:filename}
raw openssl_executable_location ${openssl:location}/bin/openssl
raw python_executable ${buildout:executable}
raw python_with_eggs ${buildout:directory}/bin/${extra-eggs:interpreter}
raw template_wrapper ${monitor-template-wrapper:location}/${monitor-template-wrapper:filename}
raw check_disk_space ${buildout:directory}/bin/check-free-disk
raw bin_directory ${buildout:directory}/bin
depends =
${monitor-eggs:eggs}
[versions]
PyRSS2Gen = 1.1
cns.recipe.symlink = 0.2.3
pycurl = 7.43.0
slapos.toolbox = 0.81
pyasn1 = 0.3.7
slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/buildout.hash.cfg 0000664 0000000 0000000 00000001453 13344470245 0030346 0 ustar 00root root 0000000 0000000 # THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
# The only allowed lines here are (regexes):
# - "^#" comments, copied verbatim
# - "^[" section beginings, copied verbatim
# - lines containing an "=" sign which must fit in the following categorie.
# - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
# Copied verbatim.
# - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
# by the re-generation script.
# Re-generated.
# - other lines are copied verbatim
# Substitution (${...:...}), extension ([buildout] extends = ...) and
# section inheritance (< = ...) are NOT supported (but you should really
# not need these here).
[monitor2-template]
filename = instance-monitor.cfg.jinja2.in
md5sum = a2da18b7e1d4ae1332e19e5d1b70fcdf
instance-monitor.cfg.jinja2.in 0000664 0000000 0000000 00000035617 13344470245 0032601 0 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor [slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
# XXX Default values if doesn't exists
root-instance-title = UNKNOWN H-S
instance-title = UNKNOWN Instance
[directory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
bin = ${buildout:directory}/bin
srv = ${buildout:directory}/srv
var = ${buildout:directory}/var
run = ${:var}/run
log = ${:var}/log
scripts = ${:etc}/run
services = ${:etc}/service
promises = ${:etc}/promise
plugins = ${:etc}/plugin
monitor = ${:srv}/monitor
[monitor-directory]
recipe = slapos.cookbook:mkdirectory
bin = ${directory:bin}
etc = ${directory:etc}
promises = ${directory:etc}/monitor-promise
reports = ${directory:etc}/monitor-report
pids = ${directory:run}/monitor
webdav = ${directory:monitor}/webdav
public = ${directory:monitor}/public
private = ${directory:monitor}/private
documents = ${:private}/documents
log = ${directory:log}/monitor
promise-result = ${buildout:directory}/.slapgrid/promise/result
promise-log = ${buildout:directory}/.slapgrid/promise/log
[ca-directory]
recipe = slapos.cookbook:mkdirectory
root = ${directory:srv}/ssl
requests = ${:root}/requests
private = ${:root}/private
certs = ${:root}/certs
newcerts = ${:root}/newcerts
crl = ${:root}/crl
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = {{ openssl_executable_location }}
ca-dir = ${ca-directory:root}
requests-directory = ${ca-directory:requests}
wrapper = ${directory:services}/certificate_authority
ca-private = ${ca-directory:private}
ca-certs = ${ca-directory:certs}
ca-newcerts = ${ca-directory:newcerts}
ca-crl = ${ca-directory:crl}
[ca-monitor-httpd]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = ${monitor-httpd-conf-parameter:key-file}
cert-file = ${monitor-httpd-conf-parameter:cert-file}
executable = ${monitor-httpd-wrapper:wrapper-path}
wrapper = ${directory:services}/monitor-httpd
[monitor-conf-parameters]
title = ${monitor-instance-parameter:monitor-title}
root-title = ${monitor-instance-parameter:root-instance-title}
public-folder = ${monitor-directory:public}
private-folder = ${monitor-directory:private}
webdav-folder = ${monitor-directory:webdav}
base-url = ${monitor-instance-parameter:monitor-base-url}
service-pid-folder = ${monitor-directory:pids}
crond-folder = ${logrotate-directory:cron-entries}
log-folder = ${monitor-directory:log}
document-folder = ${monitor-directory:documents}
pid-file = ${monitor-directory:pids}/monitor-bootstrap.pid
public-path-list =
private-path-list = ${directory:log}
monitor-url-list = ${monitor-instance-parameter:monitor-url-list}
parameter-file-path = ${monitor-instance-parameter:configuration-file-path}
parameter-list =
raw monitor-user ${monitor-instance-parameter:username}
htpasswd monitor-password ${httpd-monitor-htpasswd:password-file} ${monitor-instance-parameter:username} ${httpd-monitor-htpasswd:htpasswd-path}
file min-free-disk-MB ${promise-check-free-disk-space:config-threshold-file}
${monitor-instance-parameter:instance-configuration}
# htpasswd entry: htpasswd key password-file username htpasswd-file
promise-output-file = ${directory:monitor}/monitor-bootstrap-status
[monitor-promise-conf]
output-folder = ${monitor-directory:public}/promise
history-folder = ${monitor-directory:public}
promise-folder = ${directory:plugins}
legacy-promise-folder = ${directory:promises}
pid-path = ${monitor-directory:pids}/runpromise.pid
partition-folder = ${buildout:directory}
master-url = ${slap-connection:server-url}
partition-cert = ${slap-connection:cert-file}
partition-key = ${slap-connection:key-file}
partition-id = ${slap-connection:partition-id}
computer-id = ${slap-connection:computer-id}
ipv4 = ${slap-configuration:ipv4-random}
ipv6 = ${slap-configuration:ipv6-random}
software-release = ${slap-connection:software-release-url}
software-type = ${slap-configuration:slap-software-type}
[monitor-base-url-dict]
# place holder to be used to collect erp5 monitor urls
[monitor-conf]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_conf_template }}
rendered = ${directory:etc}/${:filename}
filename = monitor.conf
context = section parameter_dict monitor-conf-parameters
section promise_parameter_dict monitor-promise-conf
section monitor_base_urls monitor-base-url-dict
[start-monitor]
recipe = slapos.cookbook:wrapper
command-line = {{ monitor_bin }} -c ${monitor-conf:rendered}
name = bootstrap-monitor
wrapper-path = ${directory:scripts}/${:name}
[monitor-htpasswd]
recipe = slapos.cookbook:generate.password
storage-path = ${directory:etc}/.monitor_pwd
bytes = 8
[httpd-monitor-htpasswd]
recipe = plone.recipe.command
stop-on-error = true
password-file = ${directory:etc}/.monitor_pwd
htpasswd-path = ${monitor-directory:etc}/monitor-htpasswd
# Keep multiple lines as password can end with newline char.
command =
if [ ! -s "${:htpasswd-path}" ]; then
{{ apache_location }}/bin/htpasswd -cb ${:htpasswd-path} ${:user} ${:password}
fi
if [ ! -s "${:password-file}" ]; then echo "${monitor-instance-parameter:password}" > ${:password-file}; fi
update-command = ${:command}
user = ${monitor-instance-parameter:username}
password = ${monitor-instance-parameter:password}
[monitor-symlink]
recipe = cns.recipe.symlink
symlink =
${monitor-directory:promise-result} = ${monitor-directory:public}/promise
${monitor-directory:promise-log} = ${monitor-directory:log}/promise
[monitor-httpd-conf-parameter]
listening-ip = ${monitor-instance-parameter:monitor-httpd-ipv6}
port = ${monitor-instance-parameter:monitor-httpd-port}
pid-file = ${directory:run}/monitor-httpd.pid
access-log = ${directory:log}/monitor-httpd-access.log
error-log = ${directory:log}/monitor-httpd-error.log
cert-file = ${ca-directory:certs}/httpd.crt
key-file = ${ca-directory:certs}/httpd.key
htpasswd-file = ${httpd-monitor-htpasswd:htpasswd-path}
url = https://[${monitor-instance-parameter:monitor-httpd-ipv6}]:${:port}
httpd-cors-config-file = ${monitor-httpd-cors:rendered}
httpd-include-file =
[monitor-httpd-conf]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_httpd_template }}
rendered = ${monitor-directory:etc}/monitor-httpd.conf
mode = 0744
context =
section directory monitor-directory
section parameter_dict monitor-httpd-conf-parameter
[monitor-httpd-cors]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_https_cors }}
rendered = ${directory:etc}/httpd-cors.cfg
mode = 0600
context =
key domain monitor-instance-parameter:cors-domains
[monitor-httpd-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ apache_location }}/bin/httpd -f ${monitor-httpd-conf:rendered} -DFOREGROUND
wrapper-path = ${directory:bin}/monitor-httpd
wait-for-files =
${ca-directory:certs}/httpd.key
${ca-directory:certs}/httpd.crt
${monitor-httpd-graceful-wrapper:rendered}
[monitor-httpd-graceful-wrapper]
recipe = slapos.recipe.template:jinja2
template = {{ template_wrapper }}
rendered = ${directory:scripts}/monitor-httpd-graceful
mode = 0700
context =
key content :command
raw dash_binary {{ dash_executable_location }}
command = kill -USR1 $(cat ${monitor-httpd-conf-parameter:pid-file})
[logrotate-entry-monitor-httpd]
<= logrotate-entry-base
name = monitor-apache
log = ${directory:log}/monitor-httpd-*.log
post = test ! -s ${monitor-httpd-conf-parameter:pid-file} || {{ bin_directory }}/slapos-kill --pidfile ${monitor-httpd-conf-parameter:pid-file} -s USR1
[xnice-bin]
recipe = collective.recipe.template
input = inline:#!/bin/sh
# run something at lowest possible priority
exec nice -19 chrt --idle 0 ionice -c3 "$@"
output = ${directory:bin}/xnice
mode = 700
[promise-monitor-httpd-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/is-process-older-than-dependency-set ${monitor-httpd-conf-parameter:pid-file}
wrapper-path = ${directory:promises}/promise-monitor-httpd-is-process-older-than-dependency-set
[monitor-globalstate-wrapper]
recipe = slapos.cookbook:wrapper
command-line = ${xnice-bin:output} {{ monitor_genstatus }} '${monitor-conf:rendered}'
wrapper-path = ${directory:bin}/monitor-globalstate
[monitor-configurator-wrapper]
recipe = slapos.cookbook:wrapper
# XXX - hard coded path
command-line = ${xnice-bin:output} {{ monitor_configwrite }}
--config_folder '${monitor-conf-parameters:private-folder}/config/.jio_documents'
--output_cfg_file '${monitor-instance-parameter:configuration-file-path}'
--htpasswd_bin '{{ apache_location }}/bin/htpasswd'
--monitor_https_cors {{ monitor_https_cors }}
wrapper-path = ${directory:bin}/monitor-configurator
[monitor-collect-wrapper]
recipe = slapos.cookbook:wrapper
command-line = ${xnice-bin:output} {{ monitor_collect }}
--output_folder ${monitor-directory:documents}
--collector_db ${monitor-instance-parameter:collector-db}
--pid_file ${monitor-directory:pids}/monitor-collect.pid
wrapper-path = ${directory:bin}/monitor-collect
[monitor-globalstate-cron-entry]
recipe = slapos.cookbook:cron.d
cron-entries = ${cron:cron-entries}
name = monitor-globalstate
frequency = */2 * * * *
command = {{ bin_directory }}/randomsleep 20 && ${monitor-globalstate-wrapper:wrapper-path}
[monitor-configurator-cron-entry]
recipe = slapos.cookbook:cron.d
cron-entries = ${cron:cron-entries}
name = monitor-configurator
frequency = * * * * *
command = {{ bin_directory }}/randomsleep 10 && ${monitor-configurator-wrapper:wrapper-path}
[monitor-collect-cron-entry]
recipe = slapos.cookbook:cron.d
cron-entries = ${cron:cron-entries}
name = monitor_collect
frequency = * * * * *
command = {{ bin_directory }}/randomsleep 40 && ${monitor-collect-wrapper:wrapper-path}
[logrotate-entry-monitor-data]
recipe = collective.recipe.template
name = monitor.data
log = ${monitor-directory:private}/*.data.json ${monitor-directory:documents}/*.data.json
input = inline:${:log} {
weekly
nocreate
olddir ${monitor-directory:documents}
rotate 104
nocompress
missingok
extension .json
dateext
dateformat -%Y-%m-%d
notifempty
}
output = ${logrotate-directory:logrotate-entries}/${:name}
mode = 600
[logrotate-entry-monitor-promise-history]
<= logrotate-entry-base
name = monitor.service.status
log = ${monitor-directory:public}/*.history.json
rotate-num = 0
frequency = weekly
pre = {{ monitor_statistic }} --history_folder ${monitor-directory:public}
[monitor-httpd-promise]
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/${:filename}
filename = monitor-httpd-listening-on-tcp
url = ${monitor-httpd-conf-parameter:url}
check-secure = 1
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
[monitor-publish-parameters]
# XXX depends on monitor-base section
monitor-base-url = ${monitor-base:base-url}
monitor-url = ${:monitor-base-url}/public/feeds
monitor-user = ${monitor-instance-parameter:username}
monitor-password = ${monitor-instance-parameter:password}
[monitor-instance-parameter]
monitor-title = ${slap-configuration:instance-title}
monitor-httpd-ipv6 = ${slap-configuration:ipv6-random}
monitor-httpd-port = 8196
# XXX - Set monitor-base-url = ${monitor-httpd-conf-parameter:url} => https://[ipv6]:port
monitor-base-url = ${monitor-frontend-promise:url}
#monitor-base-url = ${monitor-httpd-conf-parameter:url}
root-instance-title = ${slap-configuration:root-instance-title}
monitor-url-list =
cors-domains = monitor.app.officejs.com
# XXX Hard coded parameter
collector-db = /srv/slapgrid/var/data-log/collector.db
# Credentials
password = ${monitor-htpasswd:passwd}
username = admin
# XXX: type key value
# ex raw monitor-password resqdsdsd34
instance-configuration =
configuration-file-path = ${monitor-directory:etc}/monitor_knowledge0.cfg
interface-url = https://monitor.app.officejs.com
[monitor-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Monitor Frontend ${monitor-instance-parameter:monitor-title}
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = ${monitor-httpd-conf-parameter:url}
config-https-only = true
#software-type = custom-personal
return = domain secure_access
[monitor-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/monitor-http-frontend
url = ${monitor-frontend:connection-secure_access}
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
check-secure = 1
[monitor-bootstrap-promise]
recipe = slapos.cookbook:promise.plugin
eggs =
slapos.toolbox
file = ${monitor-conf-parameters:promise-output-file}
content =
from slapos.promise.plugin.monitor_bootstrap_status import RunPromise
output = ${directory:plugins}/monitor-bootstrap-status.py
mode = 600
config-process-pid-file = ${monitor-conf-parameters:pid-file}
config-process-name = ${start-monitor:name}
config-status-file = ${:file}
[promise-check-slapgrid]
recipe = slapos.cookbook:promise.plugin
eggs =
slapos.toolbox
output = ${directory:plugins}/buildout-${slap-connection:partition-id}-status.py
content =
from slapos.promise.plugin.check_partition_deployment_state import RunPromise
config-monitor-url = ${monitor-instance-parameter:monitor-base-url}
mode = 600
[promise-check-free-disk-space]
recipe = slapos.cookbook:promise.plugin
eggs =
slapos.toolbox
output = ${directory:plugins}/check-free-disk-space.py
content =
from slapos.promise.plugin.check_free_disk_space import RunPromise
mode = 600
config-collectordb = ${monitor-instance-parameter:collector-db}
config-threshold-file = ${directory:etc}/min-free-disk-size
[monitor-base]
# create dependencies between required monitor parts
recipe = plone.recipe.command
command = true
update-command =
base-url = ${monitor-conf-parameters:base-url}
depends =
${monitor-globalstate-cron-entry:name}
${monitor-configurator-cron-entry:name}
${monitor-collect-cron-entry:name}
${cron-entry-logrotate:name}
${logrotate-entry-cron:name}
${certificate-authority:wrapper}
${monitor-conf:rendered}
${start-monitor:wrapper-path}
${ca-monitor-httpd:wrapper}
${monitor-httpd-promise:filename}
${monitor-bootstrap-promise:file}
${monitor-symlink:recipe}
${promise-check-slapgrid:recipe}
${promise-monitor-httpd-is-process-older-than-dependency-set:wrapper-path}
${logrotate-entry-monitor-httpd:name}
${logrotate-entry-monitor-data:name}
${logrotate-entry-monitor-promise-history:name}
[monitor-publish]
monitor-base-url = ${monitor-publish-parameters:monitor-base-url}
monitor-setup-url = ${monitor-instance-parameter:interface-url}/#page=settings_configurator&url=${monitor-publish-parameters:monitor-url}&username=${monitor-publish-parameters:monitor-user}&password=${monitor-publish-parameters:monitor-password}
[buildout]
extends =
{{ template_logrotate_base }}
slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/templates/ 0000775 0000000 0000000 00000000000 13344470245 0027107 5 ustar 00root root 0000000 0000000 httpd-cors.cfg.in 0000664 0000000 0000000 00000001435 13344470245 0032210 0 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/templates {% if domain -%}
{% set allow_domain = '|'.join(domain.replace('.', '\.').split()) -%}
SetEnvIf Origin "^http(s)?://(.+\.)?({{ allow_domain }})$" ORIGIN_DOMAIN=$0
Header always set Access-Control-Allow-Origin "%{ORIGIN_DOMAIN}e" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Credentials "true" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Methods "PROPFIND, PROPPATCH, COPY, MOVE, DELETE, MKCOL, LOCK, UNLOCK, PUT, GETLIB, VERSION-CONTROL, CHECKIN, CHECKOUT, UNCHECKOUT, REPORT, UPDATE, CANCELUPLOAD, HEAD, OPTIONS, GET, POST" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Headers "Overwrite, Destination, Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control, Authorization" env=ORIGIN_DOMAIN
{% endif -%}
monitor-httpd.conf.in 0000664 0000000 0000000 00000006557 13344470245 0033131 0 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/templates PidFile "{{ parameter_dict.get('pid-file') }}"
StartServers 1
ServerLimit 1
MaxRequestWorkers 4
ThreadLimit 4
ThreadsPerChild 4
ServerName example.com
ServerAdmin someone@email
Listen [{{ parameter_dict.get('listening-ip') }}]:{{ parameter_dict.get('port') }}
Define MonitorPort
DocumentRoot "{{ directory.get('private') }}"
ErrorLog "{{ parameter_dict.get('error-log') }}"
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule mime_module modules/mod_mime.so
LoadModule dir_module modules/mod_dir.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule alias_module modules/mod_alias.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule headers_module modules/mod_headers.so
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule env_module modules/mod_env.so
LoadModule setenvif_module modules/mod_setenvif.so
# SSL Configuration
Define SSLConfigured
SSLCertificateFile {{ parameter_dict.get('cert-file') }}
SSLCertificateKeyFile {{ parameter_dict.get('key-file') }}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
AddType application/hal+json .haljson
SSLEngine On
Include {{ parameter_dict.get('httpd-cors-config-file') }}
Header set Vary Origin
Header set Cache-Control "private, max-age=40"
Header set Access-Control-Max-Age "40"
DavLockDB {{ directory.get('monitor-var') }}/DavLock
Alias /share {{ directory.get('webdav') }}
DirectoryIndex disabled
DAV On
Options Indexes FollowSymLinks
AuthType Basic
AuthName "webdav"
AuthUserFile "{{ parameter_dict.get('htpasswd-file') }}"
Require valid-user
Allow from all
Satisfy any
Alias /private {{ directory.get('private') }}/
Order Deny,Allow
Deny from env=AUTHREQUIRED
Order Allow,Deny
Deny from all
AuthType Basic
AuthName "Private access"
AuthUserFile "{{ parameter_dict.get('htpasswd-file') }}"
Require valid-user
Options Indexes FollowSymLinks
Satisfy all
Alias /public {{ directory.get('public') }}/
Options Indexes FollowSymLinks
Order Allow,Deny
Allow from all
{% if parameter_dict.get('httpd-include-file', '') -%}
Include {{ parameter_dict.get('httpd-include-file') }}
{% endif -%}
monitor.conf.in 0000664 0000000 0000000 00000001045 13344470245 0031773 0 ustar 00root root 0000000 0000000 slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/templates [monitor]
{% set monitor_url_list = parameter_dict.pop("monitor-url-list", "").strip() -%}
{% for key, value in parameter_dict.items() -%}
{{ key }} = {{ value.strip().replace("\n", "\n ") }}
{% endfor -%}
monitor-url-list =
{% if monitor_url_list -%}
{{ ' ' ~ monitor_url_list.replace("\n", "\n ") }}
{% else -%}
{% for key, value in monitor_base_urls.items() -%}
{{ ' ' ~ value }}
{% endfor -%}
{% endif %}
[promises]
{% for key, value in promise_parameter_dict.items() -%}
{{ key }} = {{ value.strip().replace("\n", "\n ") }}
{% endfor -%} slapos-38ed675aef44c876cc23a5b810087f5472550031-stack-monitor/stack/monitor/templates/wrapper.in 0000664 0000000 0000000 00000000041 13344470245 0031112 0 ustar 00root root 0000000 0000000 #!{{ dash_binary }}
{{ content }}