pax_global_header 0000666 0000000 0000000 00000000064 13173525115 0014515 g ustar 00root root 0000000 0000000 52 comment=56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4
slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/ 0000775 0000000 0000000 00000000000 13173525115 0023017 5 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/ 0000775 0000000 0000000 00000000000 13173525115 0024124 5 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/ 0000775 0000000 0000000 00000000000 13173525115 0025613 5 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/README.md 0000664 0000000 0000000 00000030705 13173525115 0027077 0 ustar 00root root 0000000 0000000 Monitor stack
=============
* This stack has for purpose to know if all promises, services, custom monitoring scripts went/are ok.
* It also provides a web interface, to see which promises, instance and hosting subscription status. It also provide a rss feed to easily know the actual state of your instance, and to know when it started to went bad. You can also add your own monitoring scripts.
Implementation :
----------------
1/ In your software.cfg, extends the stack:
[buildout]
extends =
...
../../stack/monitor/buildout.cfg
2/ In your instance.cfg file or instance template (instance-xxx.cfg.jinja2, ...),
Extend monitor template and a monitor-base to parts:
[buildout]
extends =
${monitor-template:rendered}
parts =
...
monitor-base
Override monitor configuration by adding monitor-instance-parameter section to define your custom parameters.
[monitor-instance-parameter]
monitor-title = ${slap-configuration:instance-title}
monitor-httpd-ipv6 = ${slap-configuration:ipv6-random}
monitor-httpd-port = ...
monitor-base-url = ${monitor-frontend-promise:url}
root-instance-title = ${slap-configuration:root-instance-title}
monitor-url-list =
cors-domains = monitor.app.officejs.com
collector-db = ...
password = ${monitor-htpasswd:passwd}
username = admin
instance-configuration = ...
configuration-file-path = ...
interface-url = ...
You don't need to define all parameters, you can only set what is required to be changed. ie:
[monitor-instance-parameter]
monitor-httpd-port = 8333
- monitor-title: is the title of the current software instance.
- root-instance-title: is the title of the hosting subscription.
- monitor-httpd-ipv6: is the ipv6 of the computer partition.
- monitor-httpd-port: the port to bind monitor httpd server on.
- monitor-base-url: this url will be used/showed in monitor interface. This url is present in some monitor generated output files. There can be two value, the default: ${monitor-frontend-promise:url} which access monitor httpd server through the frontend and ${monitor-httpd-conf-parameter:url} which is the url with ipv6 (https://[IPv6]:port/).
- monitor-url-list: set list of Monitor Base URL of monitor sub-instances, if this is the root instance with at least one child.
- cors-domains: the domain used by the monitor web interface. The default is: monitor.app.officejs.com.
- username: monitor username, this should be the same in all sub-instances. Default is: admin.
- password: monitor password, this should be the same in all sub-instances. Default is generated (${monitor-instance-parameter:username}).
- instance-configuration: instance custom information or configuration to show in monitor web interface. There is many possibility:
raw CONFIG_KEY VALUE => non editable configuration, ie: raw monitor-password resqdsdsd34
file CONFIG_KEY PATH_TO_RESULT_FILE => editable configuration.
httpdcors CONFIG_KEY PATH_TO_HTTP_CORS_CFG_FILE PATH_HTTPD_GRACEFUL_WRAPPER => show/edit cors domain in monitor
- configuration-file-path: path of knowledge0 cfg file where instance configuration will be written.
- interface-url: The URL of monitor web interface. This URL will be present in generated JSON files.
**Multiple Monitors**
If you have sub-instances, you should collect the base monitor url from all instances with monitor and send it to monitor-url-list or you can override "monitor-base-url-dict" section and add all the urls as key/value pairs in the root instance.
[monitor-base-url-list]
monitor1-url = https://[xxxx:xxx:xxxx:e:11::1fb1]:4200
monitor2-url = https://[xxxx:xxx:xxxx:e:22::2fb2]:4200
..
..
Also, All monitors of the sub instances need to have same password as the password of the root instance monitor.
NB: You should use double $ (ex: $${monitor-template:rendered}) instead of one $ in your instance template file if it's not a jinja template. See:
- Jinja template file exemple, use one $: https://lab.nexedi.com/nexedi/slapos/blob/master/software/slaprunner/instance-resilient-test.cfg.jinja2
- Non Jinja template file, use $$: https://lab.nexedi.com/nexedi/slapos/blob/master/software/slaprunner/instance.cfg
Add a promise
-------------
Monitor stack will include slapos promise directory etc/promise to promise folder. All files in that directory will be considered as a promise.
This mean that all slapos promises will be checked frequently by monitor.
[monitor-conf-parameters]
...
promise-folder = ${directory:promises}
...
Monitor will run each promise every minutes and save the result in a json file. Here is an exemple of promise result:
{"status": "ERROR", "change-time": 1466415901.53, "hosting_subscription": "XXXX", "title": "vnc_promise", "start-date": "2016-06-21 10:47:01", "instance": "XXXX-title", "_links": {"monitor": {"href": "MONITOR_PRIVATE_URL"}}, "message": "PROMISE_OUPT_MESSAGE", "type": "status"}
A promise will be ran during a short time and report the status: ERROR or OK, plus an ouput message which says what was good or bad.
The promise should not run for more that 20 seconds, else it will be interrupted because of time out. However this value can be modified from monitoring web interface, see parameter "promise-timeout" of your hosting subscription.
On slapos, the default timeout value is also 20 seconds, if the value is modified on monitor (ex: to 50 seconds), it will still fails when slapgrid will process instance if the promise execution exceed 20 seconds.
Promises result are published in web public folder, access URL is: MONITOR_BASE_URL/private/PROMISE_NAME.status.json
Everytime monitor will run a promise an history of result will be also updated. The promise history will be updated during one day, after that a new history will be created.
To access promise history file as JSON, use URL MONITOR_BASE_URL/private/PROMISE_NAME.history.json
Add a promise: monitor promise
------------------------------
Monitor promise is also a promise like normal promise script but it will be placed into the folder ${monitor-directory:promises}:
[monitor-promise-xxxxx]
recipe = slapos.recipe.template:jinja2
rendered = ${monitor-directory:promises}/my-custom-monitor-promise
Theses promises will be executed only by monitor (not slapos) every minutes and will report same infor as default promises. This is another way to
add more custom promises to check if server is overloaded, or if network start to be slow, etc...
Add custom scripts to monitor
-----------------------------
Custom script will be automatically run by the monitor and result will be reported in monitor private folder. To add your custom script in ${monitor-directory:reports} folder. Here is an example:
[monitor-check-webrunner-internal-instance]
recipe = slapos.recipe.template:jinja2
template = ${monitor-check-webrunner-internal-instance:location}/${monitor-check-webrunner-internal-instance:filename}
rendered = $${monitor-directory:reports}/$${:filename}
filename = monitor-check-webrunner-internal-instance
mode = 0744
The script will be executed every minutes by default. To change, put periodicity in script name:
- monitor-check-webrunner-internal-instance_every_1_minute
- monitor-check-webrunner-internal-instance_every_25_minute
- monitor-check-webrunner-internal-instance_every_1_hour
- monitor-check-webrunner-internal-instance_every_3_hour
- ...
the script name should end with _every_XX_hour or _every_XX_minute. With this, we can set filename as:
filename = monitor-check-webrunner-internal-instance_every_2_minute
You can get custom script results files at MONITOR_BASE_URL/private/FILE_NAME.
Add custom file or directory to monitor
---------------------------------------
Log or others files can be added in monitor public or private directory:
[monitor-conf-parameters]
public-path-list =
/path/to/my/public/directory/
...
private-path-list =
${directory:log}
...
files in public directory are accessible at MONITOR_BASE_URL/public, and for private directory: MONITOR_BASE_URL/private.
Monitor RSS and OPML Feed
-------------------------
Monitor generate rss containing latest result for all promises, this feed will be upaded every minutes. The Rss fee URL is
MONITOR_BASE_URL/public/feed
OPML Feed is used to aggregate many feed URL, this is used on monitor to link many single monitor instances. For example, a resilient
webrunner has 5 instances at least, each instance has a monitor which leads to 5 monitor instances too. One main instance (usally the root instance)
will collect rss feeds of all others monitor's in a single OPML file. This file is published and used to configure a monitor backend to the web interface.
The URL of OPML feed is: MONITOR_BASE_URL/public/feeds
Monitor Base web directory tree
-------------------------------
MONITOR_BASE_URL
|
--------------------------
| | |
share public private
(webdav) X Y
|
---------------------------------
| | | |
public private jio_public jio_private
X Y | |
.jio_documents .jio_documents
X Y
MONITOR_BASE_URL/public or private is for normal HTTPS.
MONITOR_BASE_URL/share is the webdav URL. public/ and private/ are linked to public and private directories.
webdav also has jio_public/.jio_documents and jio_private/.jio_documents which are linked to public and private directory and they works with jio webdav pluging.
Access to Monitor
-----------------
In monitor instance.cfg file, the section [monitor-publish-parameters] contain information about monitor access.
Usefull information are monitor-base-url, monitor-url, monitor-user and monitor-password.
- ${monitor-publish-parameters:monitor-base-url} is the url of monitor httpd server.
- ${monitor-publish-parameters:monitor-base-url}/public/feed is the RSS url of this monitor instance.
- ${monitor-publish-parameters:monitor-base-url}/public/feeds is the OPML URL of this monitor instance. To setup monitor instance in your monitoring interface, use OPML URL of the root instance. It should contain URL to others monitor instances.
- ${monitor-publish-parameters:monitor-base-url}/private is the monitor private directory. Username and password are reqired to connect.
The section [monitor-publish] contain parameters to publish with your instance connection information. It will publish "monitor-base-url" and
"monitor-setup-url" which is used to configure your instance to monitor interface in one click.
To publish configuration URL in your instance.cfg, you can do like this:
[publish-connection-information]
<= monitor-publish
...
custom-parameter-one = xxxxx
custom-parameter-two = yyyyy
Send parameters to monitor interface
------------------------------------
Monitor has a paramters called "instance-configuration" from the section [monitor-instance-parameter]
that can be updated to specify which parameters will be deplayed on monitor web interface.
Parameters can be editable (except raw parameter) directly from monitor interface. The change will be updated into the related file. Here are some examples:
[monitor-instance-parameter]
instance-configuration =
raw init-user ${publish-connection-information:init-user}
htpasswd monitor-password ${httpd-monitor-htpasswd:password-file} ${monitor-instance-parameter:username} ${httpd-monitor-htpasswd:htpasswd-path}
file promise-timeout ${monitor-promise-timeout-file:file}
The user will see parameters:
- init-user (non editable)
- monitor-password (editable)
- promise-timeout (editable)
htpasswd: is used to change apache htpasswd directly from monitor interface. The syntax is like:
htpasswd PARAMETER_ID PASSWORD_TEXT_FILE HTPASSWD_USER_NAME HTPASSWD_FILE
PASSWORD_TEXT_FILE contain the password which is showed to the user.
file: is used to edit a parameter directly into file. Parameter is read and write into the provided file
file PARAMETER_ID PATH_TO_THE_FILE
raw: is a non editable paramter.
raw PARAMETER_ID TEXT_VALUE
httpdcors: used to edit an apache http_cors.conf file, this file should be include in the main apache configuration file
httpdcors PARAMETER_ID PATH_TO_HTTP_CORS_CFG_FILE PATH_HTTPD_GRACEFUL_WRAPPER
PATH_HTTPD_GRACEFUL_WRAPPER will be executed to reload apache configuration after modification is done.
slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/buildout.cfg 0000664 0000000 0000000 00000010213 13173525115 0030120 0 ustar 00root root 0000000 0000000 [buildout]
extends =
buildout.hash.cfg
../../component/apache/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/openssl/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/pycurl/buildout.cfg
../../component/python-cryptography/buildout.cfg
../../stack/logrotate/buildout.cfg
parts =
slapos-cookbook
monitor-eggs
extra-eggs
monitor2-template
[monitor-download-base]
recipe = hexagonit.recipe.download
ignore-existing = true
download-only = true
url = ${:_profile_base_location_}/${:filename}
mode = 0644
[monitor-template-base]
<= monitor-download-base
url = ${:_profile_base_location_}/templates/${:filename}
[monitor-template-script]
<= monitor-download-base
url = ${:_profile_base_location_}/scripts/${:filename}
destination = ${buildout:parts-directory}/monitor-scripts
on-update = true
[monitor-eggs]
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
${pycurl:egg}
${python-cryptography:egg}
plone.recipe.command
collective.recipe.template
cns.recipe.symlink
slapos.toolbox
# Do no generate any scripts here as all of them are generated by extraeggs
scripts =
[extra-eggs]
recipe = zc.recipe.egg
interpreter = pythonwitheggs
eggs =
${monitor-eggs:eggs}
psutil
PyRSS2Gen
Jinja2
# Monitor templates files
[monitor-httpd-conf]
<= monitor-template-base
md5sum = f2d6951670733de3b37c0ebe9eee343b
filename = monitor-httpd.conf.in
[monitor-template-wrapper]
<= monitor-template-base
filename = wrapper.in
md5sum = 1695c9a06a2b11ccfe893d7a224e489d
[monitor-conf]
<= monitor-template-base
filename = monitor.conf.in
md5sum = fc65084dd1c2fe2487b58a003b576f61
[monitor-instance-info]
<= monitor-template-base
filename = instance-info.conf.in
md5sum = 1bdb4e05c6be04f4e5766c64467fbcec
[monitor-httpd-cors]
<= monitor-template-base
filename = httpd-cors.cfg.in
md5sum = 683ea85fc054094248baf5752dd089bf
[monitor-check-free-disk-space]
<= monitor-template-base
filename = check_free_disk.in
md5sum = bab457ac4d139ed31d0b343a7d14d996
# End templates files
# XXX keep compatibility (with software/ipython_notebook/software.cfg )
[monitor-template]
rendered = ${monitor2-template:rendered}
output = ${monitor2-template:rendered}
[monitor2-template]
recipe = slapos.recipe.template:jinja2
filename = template-monitor.cfg
template = ${:_profile_base_location_}/instance-monitor.cfg.jinja2.in
rendered = ${buildout:directory}/template-monitor.cfg
context =
key apache_location apache:location
key gzip_location gzip:location
key template_logrotate_base template-logrotate-base:rendered
raw monitor_bin ${buildout:directory}/bin/monitor.bootstrap
raw monitor_collect ${buildout:directory}/bin/monitor.collect
raw monitor_statistic ${buildout:directory}/bin/monitor.statistic
raw monitor_runpromise ${buildout:directory}/bin/monitor.runpromise
raw monitor_genstatus ${buildout:directory}/bin/monitor.genstatus
raw monitor_configwrite ${buildout:directory}/bin/monitor.configwrite
raw monitor_conf_template ${monitor-conf:location}/${monitor-conf:filename}
raw monitor_https_cors ${monitor-httpd-cors:location}/${monitor-httpd-cors:filename}
raw monitor_instance_info ${monitor-instance-info:location}/${monitor-instance-info:filename}
raw curl_executable_location ${curl:location}/bin/curl
raw dash_executable_location ${dash:location}/bin/dash
raw dcron_executable_location ${dcron:location}/sbin/crond
raw logrotate_executable_location ${logrotate:location}/usr/sbin/logrotate
raw monitor_httpd_template ${monitor-httpd-conf:location}/${monitor-httpd-conf:filename}
raw openssl_executable_location ${openssl:location}/bin/openssl
raw python_executable ${buildout:executable}
raw python_with_eggs ${buildout:directory}/bin/${extra-eggs:interpreter}
raw template_wrapper ${monitor-template-wrapper:location}/${monitor-template-wrapper:filename}
raw template_check_disk_space ${monitor-check-free-disk-space:location}/${monitor-check-free-disk-space:filename}
raw bin_directory ${buildout:directory}/bin
depends =
${monitor-eggs:eggs}
[versions]
PyRSS2Gen = 1.1
cns.recipe.symlink = 0.2.3
pycurl = 7.43.0
slapos.toolbox = 0.73
slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/buildout.hash.cfg 0000664 0000000 0000000 00000001537 13173525115 0031053 0 ustar 00root root 0000000 0000000 # THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
# The only allowed lines here are (regexes):
# - "^#" comments, copied verbatim
# - "^[" section beginings, copied verbatim
# - lines containing an "=" sign which must fit in the following categorie.
# - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
# But avoid directories, they are not portable.
# Copied verbatim.
# - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
# by the re-generation script.
# Re-generated.
# - other lines are copied verbatim
# Substitution (${...:...}), extension ([buildout] extends = ...) and
# section inheritance (< = ...) are NOT supported (but you should really
# not need these here).
[monitor2-template]
filename = instance-monitor.cfg.jinja2.in
md5sum = 5027f0b1abcc28ce3817cd70fb667a3b
instance-monitor.cfg.jinja2.in 0000664 0000000 0000000 00000034550 13173525115 0033276 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor [slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
# XXX Default values if doesn't exists
root-instance-title = UNKNOWN H-S
instance-title = UNKNOWN Instance
[directory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
bin = ${buildout:directory}/bin
srv = ${buildout:directory}/srv
var = ${buildout:directory}/var
run = ${:var}/run
log = ${:var}/log
scripts = ${:etc}/run
services = ${:etc}/service
promises = ${:etc}/promise
monitor = ${:srv}/monitor
monitor-promise = ${:etc}/monitor-promise
monitor-report = ${:etc}/monitor-report
[monitor-directory]
recipe = slapos.cookbook:mkdirectory
bin = ${directory:bin}
etc = ${directory:etc}
run = ${directory:monitor}/run
#run = ${directory:scripts}
promises = ${directory:monitor-promise}
reports = ${directory:monitor-report}
pids = ${directory:run}/monitor
cgi-bin = ${directory:monitor}/cgi-bin
webdav = ${directory:monitor}/webdav
public = ${directory:monitor}/public
private = ${directory:monitor}/private
services = ${directory:services}
services-conf = ${directory:etc}/monitor.conf.d
log = ${directory:log}/monitor
monitor-var = ${directory:var}/monitor
[ca-directory]
recipe = slapos.cookbook:mkdirectory
root = ${directory:srv}/ssl
requests = ${:root}/requests
private = ${:root}/private
certs = ${:root}/certs
newcerts = ${:root}/newcerts
crl = ${:root}/crl
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = {{ openssl_executable_location }}
ca-dir = ${ca-directory:root}
requests-directory = ${ca-directory:requests}
wrapper = ${monitor-directory:services}/certificate_authority
ca-private = ${ca-directory:private}
ca-certs = ${ca-directory:certs}
ca-newcerts = ${ca-directory:newcerts}
ca-crl = ${ca-directory:crl}
[ca-monitor-httpd]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = ${monitor-httpd-conf-parameter:key-file}
cert-file = ${monitor-httpd-conf-parameter:cert-file}
executable = ${monitor-httpd-wrapper:wrapper-path}
wrapper = ${directory:services}/monitor-httpd
[monitor-conf-parameters]
title = ${monitor-instance-parameter:monitor-title}
root-title = ${monitor-instance-parameter:root-instance-title}
public-folder = ${monitor-directory:public}
private-folder = ${monitor-directory:private}
webdav-folder = ${monitor-directory:webdav}
report-folder = ${monitor-directory:reports}
base-url = ${monitor-instance-parameter:monitor-base-url}
monitor-hal-json = ${monitor-directory:public}/monitor.hal.json
service-pid-folder = ${monitor-directory:pids}
crond-folder = ${logrotate-directory:cron-entries}
logrotate-folder = ${logrotate-directory:logrotate-entries}
promise-runner = {{ monitor_runpromise }}
promise-folder = ${directory:promises}
monitor-promise-folder = ${monitor-directory:promises}
promises-timeout-file = ${monitor-promise-timeout-file:file}
pid-file = ${monitor-directory:pids}/monitor-bootstrap.pid
public-path-list =
private-path-list =
${directory:log}
#
monitor-url-list =
${monitor-instance-parameter:monitor-url-list}
parameter-file-path = ${monitor-instance-parameter:configuration-file-path}
parameter-list =
raw monitor-user ${monitor-instance-parameter:username}
htpasswd monitor-password ${httpd-monitor-htpasswd:password-file} ${monitor-instance-parameter:username} ${httpd-monitor-htpasswd:htpasswd-path}
file promise-timeout ${monitor-promise-timeout-file:file}
file min-free-disk-MB ${promise-check-free-disk-space:config-file}
${monitor-instance-parameter:instance-configuration}
# htpasswd entry: htpasswd key password-file username htpasswd-file
collector-db = ${monitor-instance-parameter:collector-db}
collect-script = {{ monitor_collect }}
statistic-script = {{ monitor_statistic }}
python = {{ python_with_eggs }}
nice-cmd = ${xnice-bin:output}
promise-output-file = ${directory:monitor}/monitor-bootstrap-status
[monitor-promise-timeout-file]
recipe = plone.recipe.command
file = ${directory:etc}/promise_timeout
command =
if [ ! -s "${:file}" ]; then
echo "20" > ${:file}
fi
[monitor-base-url-dict]
# place holder to be used to collect erp5 monitor urls
[monitor-conf]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_conf_template }}
rendered = ${directory:etc}/${:filename}
filename = monitor.conf
context = section parameter_dict monitor-conf-parameters
section monitor_base_urls monitor-base-url-dict
[instance-info-parameters]
name = ${monitor-instance-parameter:monitor-title}
root-name = ${monitor-instance-parameter:root-instance-title}
computer-id = ${slap-connection:computer-id}
ipv4 = ${slap-configuration:ipv4-random}
ipv6 = ${slap-configuration:ipv6-random}
software-release = ${slap-connection:software-release-url}
software-type = ${slap-configuration:slap-software-type}
partition-id = ${slap-connection:partition-id}
[monitor-instance-info]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_instance_info }}
rendered = ${directory:etc}/${:filename}
filename = instance-info.conf
context =
section instance_dict instance-info-parameters
[python-symlink]
recipe = plone.recipe.command
target = ${directory:bin}
command = ln -sf {{ python_with_eggs }} ${:target}/python
update-command = ${:command}
[start-monitor]
recipe = slapos.cookbook:wrapper
command-line = {{ python_executable }} {{ monitor_bin }} --config_file ${monitor-conf:rendered}
name = bootstrap-monitor
wrapper-path = ${directory:scripts}/${:name}
environment =
PATH=${python-symlink:target}:/usr/local/bin:/usr/bin:/bin
[monitor-htpasswd]
recipe = slapos.cookbook:generate.password
storage-path = ${directory:etc}/.monitor_pwd
bytes = 8
[httpd-monitor-htpasswd]
recipe = plone.recipe.command
stop-on-error = true
password-file = ${directory:etc}/.monitor_pwd
htpasswd-path = ${monitor-directory:etc}/monitor-htpasswd
# Keep multiple lines as password can end with newline char.
command =
if [ ! -s "${:htpasswd-path}" ]; then
{{ apache_location }}/bin/htpasswd -cb ${:htpasswd-path} ${:user} ${:password}
fi
if [ ! -s "${:password-file}" ]; then echo "${monitor-instance-parameter:password}" > ${:password-file}; fi
update-command = ${:command}
user = ${monitor-instance-parameter:username}
password = ${monitor-instance-parameter:password}
[monitor-httpd-conf-parameter]
listening-ip = ${monitor-instance-parameter:monitor-httpd-ipv6}
port = ${monitor-instance-parameter:monitor-httpd-port}
pid-file = ${directory:run}/monitor-httpd.pid
access-log = ${monitor-directory:log}/monitor-httpd-access.log
error-log = ${monitor-directory:log}/monitor-httpd-error.log
cert-file = ${ca-directory:certs}/httpd.crt
key-file = ${ca-directory:certs}/httpd.key
htpasswd-file = ${httpd-monitor-htpasswd:htpasswd-path}
url = https://[${monitor-instance-parameter:monitor-httpd-ipv6}]:${:port}
httpd-cors-config-file = ${monitor-httpd-cors:rendered}
httpd-include-file =
[monitor-httpd-conf]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_httpd_template }}
rendered = ${monitor-directory:etc}/monitor-httpd.conf
mode = 0744
context =
section directory monitor-directory
section parameter_dict monitor-httpd-conf-parameter
[monitor-httpd-cors]
recipe = slapos.recipe.template:jinja2
template = {{ monitor_https_cors }}
rendered = ${directory:etc}/httpd-cors.cfg
mode = 0600
context =
key domain monitor-instance-parameter:cors-domains
[monitor-httpd-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ apache_location }}/bin/httpd -f ${monitor-httpd-conf:rendered} -DFOREGROUND
wrapper-path = ${directory:bin}/monitor-httpd
wait-for-files =
${ca-directory:certs}/httpd.key
${ca-directory:certs}/httpd.crt
${monitor-httpd-graceful-wrapper:rendered}
[monitor-httpd-graceful-wrapper]
recipe = slapos.recipe.template:jinja2
template = {{ template_wrapper }}
rendered = ${directory:scripts}/monitor-httpd-graceful
mode = 0700
context =
key content :command
raw dash_binary {{ dash_executable_location }}
command = kill -USR1 $(cat ${monitor-httpd-conf-parameter:pid-file})
[xnice-bin]
recipe = collective.recipe.template
input = inline:#!/bin/sh
# run something at lowest possible priority
exec nice -19 chrt --idle 0 ionice -c3 "$@"
output = ${directory:bin}/xnice
mode = 700
[promise-monitor-httpd-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/is-process-older-than-dependency-set ${monitor-httpd-conf-parameter:pid-file}
wrapper-path = ${directory:promises}/promise-monitor-httpd-is-process-older-than-dependency-set
parameters-extra = true
[monitor-globalstate-wrapper]
recipe = slapos.cookbook:wrapper
command-line = ${xnice-bin:output} {{ monitor_genstatus }} '${monitor-conf:rendered}' '${monitor-instance-info:rendered}'
wrapper-path = ${directory:bin}/monitor-globalstate
[monitor-configurator-wrapper]
recipe = slapos.cookbook:wrapper
# XXX - hard coded path
command-line = {{ monitor_configwrite }} --config_folder '${monitor-conf-parameters:private-folder}/config/.jio_documents' --output_cfg_file '${monitor-instance-parameter:configuration-file-path}' --htpasswd_bin '{{ apache_location }}/bin/htpasswd' --monitor_https_cors {{ monitor_https_cors }}
wrapper-path = ${directory:bin}/monitor-configurator
[monitor-globalstate-cron-entry]
recipe = slapos.cookbook:cron.d
cron-entries = ${cron:cron-entries}
name = monitor-globalstate
frequency = * * * * *
command = ${monitor-globalstate-wrapper:wrapper-path}
[monitor-configurator-cron-entry]
recipe = slapos.cookbook:cron.d
cron-entries = ${cron:cron-entries}
name = monitor-configurator
frequency = * * * * *
command = ${monitor-configurator-wrapper:wrapper-path}
[monitor-httpd-promise]
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/${:filename}
filename = monitor-httpd-listening-on-tcp
url = ${monitor-httpd-conf-parameter:url}
check-secure = 1
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
[monitor-publish-parameters]
# XXX depends on monitor-base section
monitor-base-url = ${monitor-base:base-url}
monitor-url = ${:monitor-base-url}/public/feeds
monitor-user = ${monitor-instance-parameter:username}
monitor-password = ${monitor-instance-parameter:password}
[monitor-instance-parameter]
monitor-title = ${slap-configuration:instance-title}
monitor-httpd-ipv6 = ${slap-configuration:ipv6-random}
monitor-httpd-port = 8196
# XXX - Set monitor-base-url = ${monitor-httpd-conf-parameter:url} => https://[ipv6]:port
monitor-base-url = ${monitor-frontend-promise:url}
#monitor-base-url = ${monitor-httpd-conf-parameter:url}
root-instance-title = ${slap-configuration:root-instance-title}
monitor-url-list =
cors-domains = monitor.app.officejs.com
# XXX Hard coded parameter
collector-db = /srv/slapgrid/var/data-log/collector.db
# Credentials
password = ${monitor-htpasswd:passwd}
username = admin
# XXX: type key value
# ex raw monitor-password resqdsdsd34
instance-configuration =
configuration-file-path = ${monitor-directory:etc}/monitor_knowledge0.cfg
interface-url = https://monitor.app.officejs.com
[monitor-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Monitor Frontend ${monitor-instance-parameter:monitor-title}
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = ${monitor-httpd-conf-parameter:url}
config-https-only = true
#software-type = custom-personal
return = domain secure_access
[monitor-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/monitor-http-frontend
url = ${monitor-frontend:connection-secure_access}
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
check-secure = 1
[monitor-bootstrap-promise]
recipe = collective.recipe.template
file = ${monitor-conf-parameters:promise-output-file}
error-log-file = ${buildout:directory}/.${slap-connection:partition-id}_${start-monitor:name}.log
input = inline:#!{{ dash_executable_location }}
pidfile=${monitor-conf-parameters:pid-file}
if [ -s $pidfile ]; then
COUNTER=0
# Wait until max 20 seconds, the limit promise timeout
while [ $COUNTER -lt 40 ]; do
if [ -n "$(ps -p $(cat $pidfile) -o pid=)" ]; then
((COUNTER=COUNTER+1))
sleep 0.5
else
break
fi
done
fi
if [ ! -f "${:file}" ]; then
echo "Monitor bootstrap exited with error."
log_file="${:error-log-file}"
if [ -s "$log_file" ]; then
echo " ---- Latest monitor-boostrap.log ----"
echo ""
tail -n 3 $log_file
fi
exit 2
else
echo "Bootstrap OK";
fi
output = ${directory:promises}/monitor-bootstrap-status
mode = 700
[promise-check-slapgrid]
recipe = collective.recipe.template
error-log-file = ${buildout:directory}/.slapgrid-${slap-connection:partition-id}-error.log
input = inline:#!/bin/sh
if [ -f "${:error-log-file}" ]; then
>&2 cat ${:error-log-file}
exit 1
fi
output = ${monitor-directory:promises}/buildout-${slap-connection:partition-id}-status
mode = 700
[promise-check-free-disk-space]
recipe = slapos.recipe.template:jinja2
template = {{ template_check_disk_space }}
rendered = ${monitor-directory:promises}/check-free-disk-space
mode = 0700
context =
key config_file :config-file
raw home_path ${buildout:directory}
raw python_bin {{ python_with_eggs }}
config-file = ${directory:etc}/min-free-disk-size
[monitor-base]
# create dependencies between required monitor parts
recipe = plone.recipe.command
command = true
update-command =
base-url = ${monitor-conf-parameters:base-url}
depends =
${monitor-globalstate-cron-entry:name}
${monitor-configurator-cron-entry:name}
${cron-entry-logrotate:name}
${logrotate-entry-cron:name}
${certificate-authority:wrapper}
${monitor-conf:rendered}
${start-monitor:wrapper-path}
${ca-monitor-httpd:wrapper}
${monitor-httpd-promise:filename}
${monitor-bootstrap-promise:file}
${promise-check-slapgrid:output}
${promise-monitor-httpd-is-process-older-than-dependency-set:wrapper-path}
[monitor-publish]
monitor-base-url = ${monitor-publish-parameters:monitor-base-url}
monitor-setup-url = ${monitor-instance-parameter:interface-url}/#page=settings_configurator&url=${monitor-publish-parameters:monitor-url}&username=${monitor-publish-parameters:monitor-user}&password=${monitor-publish-parameters:monitor-password}
[buildout]
extends =
{{ template_logrotate_base }}
slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates/ 0000775 0000000 0000000 00000000000 13173525115 0027611 5 ustar 00root root 0000000 0000000 check_free_disk.in 0000664 0000000 0000000 00000004311 13173525115 0033151 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates #!{{ python_bin }}
import os
import sys
def free_space(path, fn):
while True:
try:
disk = os.statvfs(path)
return fn(disk)
except OSError:
pass
if os.sep not in path:
break
path = os.path.split(path)[0]
def user_free_space(path):
return free_space(path, lambda d: d.f_bsize * d.f_bavail)
def check_inode_usage(path):
max_inode_usage = 97.99 # < 98% usage
st = os.statvfs(path)
usage_output = ""
total_inode = st.f_files
free_inode = st.f_ffree
usage = round((float(total_inode - free_inode) / total_inode), 4) * 100
if usage > max_inode_usage:
return "Disk Inodes are widely used: %s%%" % usage
elif os.path.exists('/tmp'):
# check if /tmp is mounted on another disk than path
tmp_st = os.statvfs('/tmp')
if tmp_st.f_blocks != st.f_blocks:
tmp_usage = round((float(tmp_st.f_files - tmp_st.f_ffree) / tmp_st.f_files), 4) * 100
if tmp_usage > max_inode_usage:
return "Disk Inodes are widely used: %s%%" % tmp_usage
return ""
if __name__ == '__main__':
home_path = '{{ home_path }}'
config_file = '{{ config_file }}'
min_free_size = 1024*1024*1024*2 # 2G by default
if os.path.exists(config_file):
with open(config_file) as f:
min_size_str = f.read().strip()
if min_size_str == '0':
# disable check
print "Free disk space check is disabled\n set a number up to 0 to enable!"
exit(0)
if min_size_str.isdigit():
value = int(min_size_str)
if value >= 200:
# Minimum value is 200Mb, it's already low
min_free_size = int(min_size_str)*1024*1024
else:
with open(config_file, 'w') as f:
f.write(str(min_free_size/(1024*1024)))
real_free_space = user_free_space(home_path)
if real_free_space > min_free_size:
inode_usage = check_inode_usage(home_path)
if inode_usage:
print inode_usage
exit(2)
print "Disk usage: OK"
exit(0)
real_space_g = round(real_free_space/(1024.0*1024*1024), 2)
min_space_g = round(min_free_size/(1024.0*1024*1024), 2)
print 'Free disk space low: remaning %s G (threshold: %s G)' % (
real_space_g, min_space_g)
print 'You can modify minimum value on your monitor interface.'
exit(1)
httpd-cors.cfg.in 0000664 0000000 0000000 00000001435 13173525115 0032712 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates {% if domain -%}
{% set allow_domain = '|'.join(domain.replace('.', '\.').split()) -%}
SetEnvIf Origin "^http(s)?://(.+\.)?({{ allow_domain }})$" ORIGIN_DOMAIN=$0
Header always set Access-Control-Allow-Origin "%{ORIGIN_DOMAIN}e" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Credentials "true" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Methods "PROPFIND, PROPPATCH, COPY, MOVE, DELETE, MKCOL, LOCK, UNLOCK, PUT, GETLIB, VERSION-CONTROL, CHECKIN, CHECKOUT, UNCHECKOUT, REPORT, UPDATE, CANCELUPLOAD, HEAD, OPTIONS, GET, POST" env=ORIGIN_DOMAIN
Header always set Access-Control-Allow-Headers "Overwrite, Destination, Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control, Authorization" env=ORIGIN_DOMAIN
{% endif -%}
instance-info.conf.in 0000664 0000000 0000000 00000000556 13173525115 0033551 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates [instance]
name = {{ instance_dict['name'] }}
root-name = {{ instance_dict['root-name'] }}
computer = {{ instance_dict['computer-id'] }}
ipv4 = {{ instance_dict['ipv4'] }}
ipv6 = {{ instance_dict['ipv6'] }}
software-release = {{ instance_dict['software-release'] }}
software-type = {{ instance_dict['software-type'] }}
partition = {{ instance_dict['partition-id'] }} monitor-httpd.conf.in 0000664 0000000 0000000 00000006533 13173525115 0033625 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates PidFile "{{ parameter_dict.get('pid-file') }}"
StartServers 1
ServerLimit 1
ThreadLimit 4
ThreadsPerChild 4
ServerName example.com
ServerAdmin someone@email
Listen [{{ parameter_dict.get('listening-ip') }}]:{{ parameter_dict.get('port') }}
Define MonitorPort
DocumentRoot "{{ directory.get('private') }}"
ErrorLog "{{ parameter_dict.get('error-log') }}"
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule mime_module modules/mod_mime.so
LoadModule dir_module modules/mod_dir.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule alias_module modules/mod_alias.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule headers_module modules/mod_headers.so
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule env_module modules/mod_env.so
LoadModule setenvif_module modules/mod_setenvif.so
# SSL Configuration
Define SSLConfigured
SSLCertificateFile {{ parameter_dict.get('cert-file') }}
SSLCertificateKeyFile {{ parameter_dict.get('key-file') }}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
AddType application/hal+json .haljson
SSLEngine On
Include {{ parameter_dict.get('httpd-cors-config-file') }}
Header set Vary Origin
Header set Cache-Control "private, max-age=40"
Header set Access-Control-Max-Age "40"
DavLockDB {{ directory.get('monitor-var') }}/DavLock
Alias /share {{ directory.get('webdav') }}
DirectoryIndex disabled
DAV On
Options Indexes FollowSymLinks
AuthType Basic
AuthName "webdav"
AuthUserFile "{{ parameter_dict.get('htpasswd-file') }}"
Require valid-user
Allow from all
Satisfy any
Alias /private {{ directory.get('private') }}/
Order Deny,Allow
Deny from env=AUTHREQUIRED
Order Allow,Deny
Deny from all
AuthType Basic
AuthName "Private access"
AuthUserFile "{{ parameter_dict.get('htpasswd-file') }}"
Require valid-user
Options Indexes FollowSymLinks
Satisfy all
Alias /public {{ directory.get('public') }}/
Options Indexes FollowSymLinks
Order Allow,Deny
Allow from all
{% if parameter_dict.get('httpd-include-file', '') -%}
Include {{ parameter_dict.get('httpd-include-file') }}
{% endif -%}
monitor.conf.in 0000664 0000000 0000000 00000000474 13173525115 0032502 0 ustar 00root root 0000000 0000000 slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates [monitor]
{% for key, value in parameter_dict.items() -%}
{% if key == "monitor-url-list" and monitor_base_urls is defined -%}
{{ key }} =
{% for key, value in monitor_base_urls.items() -%}
{{ ' ' ~ value }}
{% endfor -%}
{% else -%}
{{ key }} = {{ value.strip().replace("\n", "\n ") }}
{% endif -%}
{% endfor -%}
slapos-56c2a61eb200aefc1ec3f8ea4c4d5b3943f396c4-stack-monitor/stack/monitor/templates/wrapper.in 0000664 0000000 0000000 00000000041 13173525115 0031614 0 ustar 00root root 0000000 0000000 #!{{ dash_binary }}
{{ content }}