pax_global_header 0000666 0000000 0000000 00000000064 12732772745 0014532 g ustar 00root root 0000000 0000000 52 comment=3f1a8392d2a2eaf3812dd762e08e40956b17007d
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/ 0000775 0000000 0000000 00000000000 12732772745 0023705 5 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/ 0000775 0000000 0000000 00000000000 12732772745 0025537 5 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/ 0000775 0000000 0000000 00000000000 12732772745 0027730 5 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/README.txt 0000664 0000000 0000000 00000017527 12732772745 0031442 0 ustar 00root root 0000000 0000000 slaprunner
==========
Introduction
------------
This software release is used to deploy Slaprunner instances.
Slaprunner is an all-in-one IDE used to develop and test profiles and recipes for SlapOS.
You can learn how to use it here :
http://community.slapos.org/wiki/slapos-Wiki.Home/developer-Lecture.Web.Runner.Extended
Parameters
----------
user-authorized-key
~~~~~~~~~~~~~~
You need to specify your SSH public key in order to connect to the SSH server of this instance.
Example of parameter XML::
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdNp7qZMVAzNc07opHshiIFDmJpYXQpetfcSgUj39a409d42PpsJElp7WsAE/x0nN6gUIoWIl7UiAlMzf6bKEJGJVSOZEPTmiJVlgK1Gp+kE0x9yNcncYg7p38Jny0daVA/NkkpAFyRsAm5kLGzyLtaCcktSvy0cJuy7WSSHU05pd1f8Y8thofE9g5t+/JA2VZvipxPkRfkFAG3aOAGLULlTImTSDFSDFGSDFG5F6mMnl7yvY2d6vEHVBu+K+aKmAwZVfCUwtSpa/tq3i2Lppjrw3UfrxbQSFHZCkzefr+u+l4YYe+tJrX7rYJYXD7LIfZfdSeFTlHFaN/yI1 user@host.local
instance-amount
~~~~~~~~~~~~~~~
Optional parameter.
Default: 10
Public-directory
----------------
You can serve static files with the webrunner. For that, just put your data in "srv/runner/public". All these files will be served throught the url of the webrunner + "/public/". Useful for developping your own static website.
AUTO-DEPLOYMENT
---------------
for software
~~~~~~~~~~~~
You can automatically deploy a software release while deploying the webrunner itself, using the paramater XML.
To do this, you only need to pass as a parameter named "slapos-software" : "AAA/BBB", where AAA is the folder of slapos.git where is located your BBB software.
For example, to deploy the hello-world software, you need to pass : software/helloworld
This is possible because the slapos.git is automatically downloaded when the webrunner is deployed.
It is also possible to download you own git repository, by providing the url in the "slapos-repository" parameter.
Last but not least, it is also possible to switch the branch with the parameter "slapos-reference" (by default pointing on master)
for instance
~~~~~~~~~~~~
The parameter "auto-deploy-instance" can be explicitly set to allow or prevent the runner to deploy the instance at START TIME (if you manually restart the runner, or if the server reboots). Values : "true" or "false". Default value is "true", except for the instances of import (while type is resilient or test) which is "false"
There also exists the parameter "autorun", which will build&run your software if set to true. For this, you need "auto_deploy" to true, and set the parameter "slapos-software" to the software you want to deploy. Do not hesitate to clone a different repo than "slapos", or to change the tag/branch to use your custom Software Release. (see "slapos-repository" and "slapos-reference" in previous section).
To deploy the instance with some parameters, just give to the runner parameters starting with "parameter-", they will be correctly forwarded to the instance, which will use them for its configuration. For example, if you want to send to the sofware helloworld the parameter "name" with the value "nicolas", here is how to configure the parameter.xml of the webrunner for auto-depolyment :
software/helloworld
true
true
nicolas
Resilience :
------------
To order a resilient runner, you have to request a runner with the option: "--type resilient"
You can then decide on which node you want to deploy each instance, with the parameters.xml :
COMP-XXXX
COMP-XXXX
COMP-XXXX
COMP-XXXX
COMP-XXXX
If you want to check by yourself that the pull-backup instances do their job, you can change directory to the slappart of runner0, and run ./bin/exporter (after creating your account, using the given backend_url or url): it would fill ./srv/backup/runner with data. If you then go to an import instance (runner1 or runner2) on the port 50005, you should be able to sign in the runner.
Tips :
~~~~~
To find in which partition the instance has been deployed, you can open the page of this specific instance, and look for "slappartXX" on the page.
Tests :
-------
To test the runner, start by ordering a runner with default type. Then READ the important note below :
/!\ IMPORTANT NOTE ON THE TESTS /!\ : in order to make them work, you have to comment the last line of the file __init__.py in the runner module (which is just : "run()"). Indeed, this line is used to load the config, when importing this module throught Gunicorn (the wsgi server). But the test suite loads itself the configuration, in fonction of the tested scenarios, etc... (You can find the module in the SR folder directory, under ./eggs/slapos.toolbox, or under parts/slapos.toolbox if you have a development version)
Then, su in the concerned slapuserX and run "./bin/runTestSuite".
For the tests on the resiliency of the webrunner, please refer to the README in slapos.toolbox.git/slapos/resiliencytest
Request custom frontend :
-------------------------
While deploying a server in your instance, you may need to ask for a custom ipv4 frontend.
The way to do that is to send a new parameter to your runner instance, using the parameters XML. The name of it is "custom-frontend-backend-url".
To create the frontend, you now have to wait the slaprunner to be processed.
You can define the type of your backend using "custom-frontend-backend-type". eg: zope
If you deploy a server, which uses basic auth, you also have to declare the parameter "custom-frontend-basic-auth" as true, or your slaprunner instance won't show as correctly instanciated.
Example :
http(s)://[THE_IPV6_ADDRESS]:PORT
zope
true
Git repositories :
------------------
It is easy to give access to your git repository/ies to everyone, or to clone it on your own computer. For this, fetch the git_url parameter returned by your webrunner instance. It should look like that : https://[IPV6]:PORT/git/YourRepo.git/
To create the repo, go to the folder srv/runner/project and initiate a new git repo (git init/clone --bare YourRepo). Then, to clone the repo, just append its name to the previous URL.
Please notice that your repositories are in public access for reading. However, an account is needed to push to the repository (use your webrunner account, or create a new user in the parameters section of the webrunner).
For the moment, the PORT is the port of monitoring, which is 9685.
Things to notice for the nex developer :
----------------------------------------
As you can see in instance-runner-*.cfg, the buildout section extends a hard-coded template file. If one day you need to modify the filename, do not forget to modify it in instance.cfg, but also in these files ! (the problem is that the content of instance.cfg is not known by buildout while the deployment of the software release)
List of ports used by the webrunner:
------------------------------------
8602 : slapproxy, while running tests
9684 : apache (monitoring of slaprunner, git access)
22222 : dropbear
39986 : supervisord
50000 : slapproxy
50005 : webrunner (flask app), webdav access
Tips:
-----
You can use shellinabox in fullscreen, by accessing : https://[IPV6]:50005/shellinabox/
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/TODO.txt 0000664 0000000 0000000 00000000277 12732772745 0031244 0 ustar 00root root 0000000 0000000 - resilient sr: Cloned instances should not launch slapgrid-sr if it was not launched on export instance
- add test for parameter auto-deploy-instance
- Add download facility in file browser
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/common.cfg 0000664 0000000 0000000 00000013665 12732772745 0031714 0 ustar 00root root 0000000 0000000 [buildout]
extends =
../../component/bash/buildout.cfg
../../component/busybox/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/dropbear/buildout.cfg
../../component/git/buildout.cfg
../../component/tig/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/nano/buildout.cfg
../../component/nginx/buildout.cfg
../../component/rsync/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/screen/buildout.cfg
../../component/shellinabox/buildout.cfg
../../component/vim/buildout.cfg
../../component/zip/buildout.cfg
../../stack/slapos.cfg
../../stack/flask.cfg
../../stack/resilient/buildout.cfg
../../stack/monitor/buildout.cfg
# stacks are listed from most generic to most specific,
# to avoid versioning issues
common-parts =
template
eggs
instance-runner-import
instance-runner-export
template-slapos-cfg
# XXX: we have to manually add this for resilience
rdiff-backup
collective.recipe.template-egg
parts =
${:common-parts}
# Use shellinabox from github with AF_UNIX support
[shellinabox]
<= shellinabox-github
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
md5sum = bb7e0bf9959c4437ff1e23e645315ccf
mode = 0644
[template-runner]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner.cfg
output = ${buildout:directory}/template-runner.cfg.in
md5sum = f2d704c269244c4eb842aefbc9ff9201
mode = 0644
[template-runner-import-script]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/runner-import.sh.jinja2
download-only = true
md5sum = d7f88b58b2508ce5af42ea7d7241626e
filename = runner-import.sh.jinja2
mode = 0644
[instance-runner-import]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-import.cfg.in
output = ${buildout:directory}/instance-runner-import.cfg
md5sum = 673c30e5e7f9b7bb543f79465a56e43d
mode = 0644
[template-runner-export-script]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/runner-export.sh.jinja2
download-only = true
md5sum = 9dc934fe5015ff53869830a833266192
filename = runner-export.sh.jinja2
mode = 0644
[instance-runner-export]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-export.cfg.in
output = ${buildout:directory}/instance-runner-export.cfg
md5sum = 4b0ab39bc655ae0b865207147cb2e5bf
mode = 0644
[template-resilient]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-resilient.cfg.jinja2
md5sum = 7b3f74b6ab491e907bbdef2ff40a7791
filename = instance-resilient.cfg.jinja2
mode = 0644
[template-resilient-test]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/instance-resilient-test.cfg.jinja2
download-only = true
md5sum = fb8c45e5c35548331fb06c4633ec592a
filename = instance-resilient-test.cfg.jinja2
mode = 0644
[template_nginx_conf]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/nginx_conf.in
download-only = true
md5sum = e6c2e1dd2153afefa0805c4065066e4e
filename = nginx_conf.in
mode = 0644
[template_httpd_conf]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/httpd_conf.in
download-only = true
md5sum = 8067164fa87c7ca2c59d1625a34c5850
filename = httpd_conf.in
mode = 0644
[template_launcher]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/launcher.in
md5sum = 525e37ea8b2acf6209869999b15071a6
filename = launcher.in
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[template-slapos-cfg]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/${:filename}
md5sum = f406b91c5b0261b198a1c1930c195071
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = slapos.cfg.in
download-only = true
mode = 0644
[template-parameters]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/${:filename}
md5sum = f8446fcf254b4929eb828a9a1d7e5f62
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = parameters.xml.in
download-only = true
mode = 0644
[template-bash-profile]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/${:filename}
md5sum = 1c88cbca0c1e705eeb4b544ef4616097
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = bash_profile.in
download-only = true
mode = 0644
[template-supervisord]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/${:filename}
md5sum = d294d0dafd265048399de6da8c96345f
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = supervisord.conf.in
download-only = true
mode = 0644
[template-listener-slapgrid]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/${:filename}
md5sum = 49d50410cf7467175a841eb7cd0d93d4
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = listener_slapgrid.py.in
download-only = true
mode = 0644
[monitor-check-webrunner-internal-instance]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/template/${:filename}
download-only = true
#md5sum = 4c44d617d5bfd1db8695200e896480c0
destination = ${buildout:parts-directory}/${:filename}
filename = monitor-check-webrunner-internal-instances.py
mode = 0644
[eggs]
recipe = zc.recipe.egg
eggs =
collective.recipe.environment
collective.recipe.template
cns.recipe.symlink
erp5.util
lock-file
plone.recipe.command
slapos.recipe.build
slapos.toolbox[flask_auth]
Gunicorn
${slapos-cookbook:eggs}
[extra-eggs]
recipe = zc.recipe.egg
interpreter = pythonwitheggs
eggs +=
supervisor
development.cfg 0000664 0000000 0000000 00000003322 12732772745 0032654 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner # Development profile of slaprunner.
# Exactly the same as software.cfg, but fetch the slapos.cookbook and
# slapos.toolbox from git repository instead of fetching stable version,
# allowing to play with bleeding edge environment.
[buildout]
extends = common.cfg
parts =
# Development eggs
slapos.toolbox-dev
slapos.cookbook-dev
slapos.core-dev
erp5.util-dev
slapos-cookbook
${:common-parts}
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/slapos.toolbox.git
branch = master
git-executable = ${git:location}/bin/git
develop = true
[slapos.toolbox-dev]
recipe = zc.recipe.egg:develop
egg = slapos.toolbox
setup = ${slapos.toolbox-repository:location}
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/slapos.git
branch = master
git-executable = ${git:location}/bin/git
develop = true
[slapos.cookbook-dev]
recipe = zc.recipe.egg:develop
egg = slapos.cookbook
setup = ${slapos.cookbook-repository:location}
# Used for resiliency tests only
[erp5.util-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/erp5.git
branch = master
git-executable = ${git:location}/bin/git
develop = true
[erp5.util-dev]
recipe = zc.recipe.egg:develop
egg = erp5.util
setup = ${erp5.util-repository:location}
[slapos.core-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/slapos.core.git
branch = master
git-executable = ${git:location}/bin/git
develop = true
[slapos.core-dev]
recipe = zc.recipe.egg:develop
egg = slapos.core
setup = ${slapos.core-repository:location}
[versions]
slapos.cookbook =
slapos.core =
slapos.toolbox =
lockfile =
httpd_conf.in 0000664 0000000 0000000 00000010343 12732772745 0032332 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner PidFile "{{ parameters.path_pid }}"
ServerName example.com
ServerAdmin someone@email
Listen [{{ parameters.global_ip }}]:{{ parameters.global_port }}
Define HTTPDPort
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule mime_module modules/mod_mime.so
#LoadModule cgid_module modules/mod_cgid.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule alias_module modules/mod_alias.so
LoadModule env_module modules/mod_env.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule headers_module modules/mod_headers.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule cache_module modules/mod_cache.so
LoadModule file_cache_module modules/mod_file_cache.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule dir_module modules/mod_dir.so
LoadModule cgid_module modules/mod_cgid.so
LoadModule autoindex_module modules/mod_autoindex.so
ErrorLog "{{ parameters.path_error_log }}"
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog "{{ parameters.path_access_log }}" common
# SSL Configuration
Define SSLConfigured
SSLCertificateFile {{ parameters.cert_file }}
SSLCertificateKeyFile {{ parameters.key_file }}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
SSLEngine On
Include {{ parameters.httpd_cors_file }}
Header set Access-Control-Allow-Credentials "true"
Header set Access-Control-Allow-Methods "PROPFIND, PROPPATCH, COPY, MOVE, DELETE, MKCOL, LOCK, UNLOCK, PUT, GETLIB, VERSION-CONTROL, CHECKIN, CHECKOUT, UNCHECKOUT, REPORT, UPDATE, CANCELUPLOAD, HEAD, OPTIONS, GET, POST"
Header set Access-Control-Allow-Headers "Overwrite, Destination, Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control, Authorization"
DocumentRoot {{ parameters.runner_home }}/public
# Directory protection
Options FollowSymLinks
AllowOverride None
Require all denied
Alias /public {{ parameters.runner_home }}/public
Order Allow,Deny
Allow from all
AllowOverride All
Satisfy Any
Options Indexes FollowSymLinks
DirectoryIndex index.html
order allow,deny
deny from all
DavLockDB {{ parameters.dav_lock }}
Alias /share {{ parameters.runner_home }}
DirectoryIndex disabled
DAV On
Options Indexes FollowSymLinks
AuthType Basic
AuthName "Webrunner Dav"
AuthUserFile "{{ parameters.htpasswd_file }}"
Require valid-user
SetEnv GIT_HTTP_EXPORT_ALL
ScriptSock {{ parameters.cgid_sock }}
ScriptAlias /git/ {{ parameters.git_http_backend }}/
ScriptAlias /git-public/ {{ parameters.git_http_backend }}/
RewriteCond %{QUERY_STRING} service=git-receive-pack [OR]
RewriteCond %{REQUEST_URI} /git-receive-pack$
SetEnv GIT_PROJECT_ROOT:{{ parameters.project_private_folder }}
Order Deny,Allow
Deny from env=AUTHREQUIRED
AuthType Basic
AuthName "Git Access"
AuthUserFile "{{ parameters.htpasswd_file }}"
Require valid-user
SetEnv GIT_PROJECT_ROOT:{{ parameters.project_public_folder }}
Order Deny,Allow
Deny from env=AUTHREQUIRED
AuthType Basic
AuthName "Git Access"
AuthUserFile "{{ parameters.htpasswd_file }}"
Require valid-user
Satisfy any
instance-for-erp5testnode.cfg.in 0000664 0000000 0000000 00000000123 12732772745 0035742 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
extends = ${template:output}
[switch_softwaretype]
default = $${:test}
instance-resilient-test.cfg.jinja2 0000664 0000000 0000000 00000006013 12732772745 0036263 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
parts =
request-resilient-instance
deploy-unit-test
deploy-scalability-test
deploy-standalone-resiliency-test
[directory]
recipe = slapos.cookbook:mkdirectory
home = ${buildout:directory}
etc = ${:home}/etc/
var = ${:home}/var/
srv = ${:home}/srv/
bin = ${:home}/bin/
tmp = ${:home}/tmp/
log = ${:var}/log/
services = ${:etc}/service/
scripts = ${:etc}/run/
[deploy-scalability-test]
recipe = slapos.cookbook:wrapper
testnode-parameters = --test-result-path={{ slapparameter_dict.get('test-result-path') }} --revision={{ slapparameter_dict.get('test-suite-revision') }} --node-title={{ slapparameter_dict.get('scalability-launcher-title') }} --test-suite={{ slapparameter_dict.get('test-suite') }} --test-suite-master-url={{ slapparameter_dict.get('test-suite-master-url') }} --log-path=${directory:log}
test-parameters = server_url=${slap-connection:server-url} key_file=${slap-connection:key-file} cert_file=${slap-connection:cert-file} computer_id=${slap-connection:computer-id} partition_id=${slap-connection:partition-id} software=${slap-connection:software-release-url} namebase=runner root_instance_name='${request-resilient-instance:name}'
command-line = {{ bin_directory }}/runResiliencyScalabilityTestNode ${:testnode-parameters} ${:test-parameters}
wrapper-path = ${directory:scripts}/runResiliencyTestSuite
[deploy-unit-test]
recipe = collective.recipe.template
test-parameters = server_url=${slap-connection:server-url} key_file=${slap-connection:key-file} cert_file=${slap-connection:cert-file} computer_id=${slap-connection:computer-id} partition_id=${slap-connection:partition-id} software=${slap-connection:software-release-url} namebase=runner root_instance_name='${request-resilient-instance:name}'
input = inline:
#!/bin/sh
exec {{ bin_directory }}/runResiliencyUnitTestTestNode $@ ${:test-parameters}
output = ${directory:bin}/runTestSuite
mode = 755
[deploy-standalone-resiliency-test]
# Used to manually run the resilient test if we don't have a running testnode.
recipe = slapos.cookbook:wrapper
test-suite-title = slaprunner
command-line = {{ bin_directory }}/runStandaloneResiliencyTestSuite --test-suite-title=${:test-suite-title} ${deploy-scalability-test:test-parameters}
wrapper-path = ${directory:bin}/runStandaloneResiliencyTestSuite
[request-resilient-instance]
<= slap-connection
recipe = slapos.cookbook:request
software-url = ${slap-connection:software-release-url}
software-type = resilient
name = Resilient Instance (Root Instance)
{% for key, value in slapparameter_dict.get('cluster', {}).iteritems() -%}
config-{{ key }} = {{ dumps(value) }}
{% endfor -%}
config-resiliency-backup-periodicity = */10 * * * *
config-resilient-clone-number = 1
config-ignore-known-hosts-file = true
# XXX hardcoded
#config-frontend-domain = google.com
# XXX Hack to deploy Root Instance on the same computer as the type-test Instance
sla-computer_guid = ${slap-connection:computer-id}
return = backend_url
[slap-parameter]
instance-resilient.cfg.jinja2 0000664 0000000 0000000 00000010113 12732772745 0035302 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner # vim: set ft=cfg:
# Default parameters
# NOTE: in case of resilient instance, auto-deploy-instance
# set to false (else it could be dangerous)
{% do slapparameter_dict.__setitem__('auto-deploy', slapparameter_dict.get('auto-deploy', 'true')) %}
{% do slapparameter_dict.__setitem__('auto-deploy-instance', slapparameter_dict.get('auto-deploy-instance', 'false')) %}
{% do slapparameter_dict.__setitem__('instance-type', 'resilient') %}
# let decide the number of clones we want. We can't have more than 2 clones
{% set number_of_instances = slapparameter_dict.get('resilient-clone-number', 1)|int %}
{% if number_of_instances > 2 %}
{% set number_of_instances = 2 %}
{% endif %}
{% set slaprunner_return = ['url', 'ssh-public-key', 'ssh-url', 'notification-id', 'ip', 'backend_url', 'url', 'ssh_command', 'access_url', '1_info', '2_info', 'webdav_url', 'public_url', 'git_public_url', 'git_private_url'] -%}
{% set monitor_return = ['monitor-base-url', 'monitor-url', 'monitor-user', 'monitor-password'] -%}
{% set monitor_parameter = {'monitor-cors-domains': slapparameter_dict.pop('monitor-cors-domains', "monitor.app.officejs.com")} -%}
{% set monitor_dict = {'parameter': monitor_parameter, 'return': monitor_return, 'set-monitor-url': True} -%}
{% set monitor_interface_url = slapparameter_dict.pop('monitor-interface-url', 'https://monitor.app.officejs.com') -%}
{% import 'parts' as parts %}
{% import 'replicated' as replicated %}
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
# += because we need to take up parts (like instance-custom, slapmonitor etc) from the profile we extended
parts +=
{{ parts.replicate("runner", number_of_instances + 1) }}
publish-connection-information
[monitor-htpasswd]
recipe = slapos.cookbook:generate.password
storage-path = ${directory:etc}/.monitor_user
bytes = 8
username = admin
{% do monitor_parameter.__setitem__('monitor-username', slapparameter_dict.get('monitor-username', 'admin'))%}
{% do monitor_parameter.__setitem__('monitor-password', slapparameter_dict.get('monitor-password', '${monitor-htpasswd:passwd}'))%}
{{ replicated.replicate("runner", number_of_instances + 1, "runner-export", "runner-import", slapparameter_dict=slapparameter_dict, monitor_parameter_dict=monitor_dict) }}
[directory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
# Bubble up the parameters
[request-runner]
{% if slapparameter_dict.get('custom-frontend-backend-url') -%}
{% do slaprunner_return.append('custom-frontend-url') -%}
{% endif -%}
return = {{ slaprunner_return | join(' ')}} {{ monitor_return | join(' ') }}
[publish-connection-information]
recipe = slapos.cookbook:publish
1_info = ${request-runner:connection-1_info}
2_info = ${request-runner:connection-2_info}
backend_url = ${request-runner:connection-backend_url}
access_url = ${request-runner:connection-access_url}
url = ${request-runner:connection-url}
ssh_command = ${request-runner:connection-ssh_command}
webdav_url = ${request-runner:connection-webdav_url}
public_url = ${request-runner:connection-public_url}
git_public_url = ${request-runner:connection-git_public_url}
git_private_url = ${request-runner:connection-git_private_url}
{% if slapparameter_dict.get('custom-frontend-backend-url') -%}
custom-frontend-url = ${request-runner:connection-custom-frontend-url}
{% endif %}
monitor-base-url = ${request-runner:connection-monitor-base-url}
monitor_setup_url = {{ monitor_interface_url }}/#page=settings_configurator&url=${request-runner:connection-monitor-url}&username=${request-runner:connection-monitor-user}&password=${request-runner:connection-monitor-password}
[slap-parameter]
# Default parameters for distributed deployment
# I.e state "backup1 of maria should go there, ..."
# XXX-Cedric: Hardcoded number of backups. Should be dynamically generated.
{% for nb in range(1,number_of_instances+1) %}
runner{{nb}}-computer-guid =
pbs-runner{{nb}}-computer-guid =
{% endfor %}
# XXX-Cedric: Hardcoded parameters. Should be dynamically generated.
domain =
authorized-key =
instance-amount = 10
debug = false
resilient-clone-number = 1
instance-runner-export.cfg.in 0000664 0000000 0000000 00000004337 12732772745 0035400 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
extends = template-runner.cfg
${pbsready-export:output}
parts +=
nginx_conf
nginx-launcher
certificate-authority
ca-nginx
gunicorn-launcher
gunicorn-graceful
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
publish-connection-information
slaprunner-promise
slaprunner-frontend-promise
apache-httpd-promise
httpd-frontend-promise
slaprunner-supervisord-wrapper
dropbear-promise
runtestsuite
symlinks
shellinabox
slapos-cfg
slapos-repo
cron-entry-prepare-software
deploy-instance-parameters
instance-software
instance-software-type
minishell-cwd
bash-profile
supervisord-wrapper
supervisord-promise
httpd-graceful-wrapper
## Monitoring part
## Monitor for runner
monitor-base
monitor-check-resilient-feed-file
monitor-check-webrunner-internal-instance
[exporter]
recipe = slapos.recipe.template:jinja2
template = ${template-runner-export-script:location}/${template-runner-export-script:filename}
rendered = $${directory:bin}/$${slap-parameter:namebase}-exporter
# backward compatibility for resilient stack
wrapper = $${:rendered}
mode = 700
context =
section directory directory
raw shell_binary ${dash:location}/bin/dash
raw rsync_binary ${rsync:location}/bin/rsync
[monitor-instance-parameter]
monitor-httpd-port = 8437
# Pass some parameter to dispay in monitoring interface
instance-configuration =
file recovery-code $${recovery-code:storage-path}
httpdcors cors-domain $${slaprunner-httpd-cors:location} $${httpd-graceful-wrapper:output}
raw webrunner-url https://$${request-frontend:connection-domain}
# Extends publish section with resilient parameters
[publish-connection-information]
<= resilient-publish-connection-parameter
[monitor-check-resilient-feed-file]
recipe = slapos.recipe.template:jinja2
template = ${template-monitor-check-resilient-feed:location}/${template-monitor-check-resilient-feed:filename}
rendered = $${monitor-directory:reports}/check-create-resilient-feed-files
mode = 700
context =
key input_feed_directory directory:notifier-feeds
key monitor_feed_directory monitor-directory:public
raw base_url http://[$${notifier:host}]:$${notifier:port}/get/
raw python_executable ${buildout:executable}
instance-runner-import.cfg.in 0000664 0000000 0000000 00000006132 12732772745 0035364 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
extends = template-runner.cfg
${pbsready-import:output}
parts +=
nginx_conf
nginx-launcher
certificate-authority
ca-nginx
gunicorn-launcher
gunicorn-graceful
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
slaprunner-promise
slaprunner-supervisord-wrapper
dropbear-promise
runtestsuite
shellinabox
symlinks
slapos-cfg
slapos-repo
cron-entry-prepare-software
deploy-instance-parameters
instance-software-type
bash-profile
supervisord-wrapper
importer-consistency-promise
# have to repeat the next one, as it's not inherited from pbsready-import
import-on-notification
monitor-base
# For the needs of importer, we run the full slaprunner
# In case both exporter and importer (aka main instance and clone instance)
# run with the same IP (usually for testing purposes),
# run slaprunner using different ports.
[slaprunner]
proxy_port = 50000
runner_port = 50005
[importer]
recipe = slapos.recipe.template:jinja2
template = ${template-runner-import-script:location}/${template-runner-import-script:filename}
rendered = $${directory:bin}/$${slap-parameter:namebase}-importer
# backward compatibility for resilient stack
wrapper = $${:rendered}
mode = 700
restore-exit-code-file=$${directory:srv}/importer-exit-code-file
context =
key backend_url slaprunner:access-url
key proxy_host slaprunner:ipv4
section directory directory
raw shell_binary ${dash:location}/bin/dash
raw rsync_binary ${rsync:location}/bin/rsync
raw restore_exit_code_file $${:restore-exit-code-file}
[importer-consistency-promise]
# Test that the importer script and "after-import" subscripts:
# 1/ Have been run in the last 24 hours
# 2/ Have succeeded
recipe = collective.recipe.template
input = inline: #!/bin/sh
EXIT_CODE_FILE=$(find "$${importer:restore-exit-code-file}" -mtime -1)
if [ -z "$EXIT_CODE_FILE" ]; then
echo "Consistency check is too old."
exit 1
fi
EXIT_CODE=$(cat $EXIT_CODE_FILE)
exit $EXIT_CODE
output = $${directory:promises}/importer-consistency-promise
mode = 755
[slap-parameter]
auto-deploy-instance = false
auto-deploy = true
name = Webrunner import
monitor-cors-domains =
monitor-username = $${monitor-htpasswd:username}
monitor-password = $${monitor-htpasswd:passwd}
[resilient-publish-connection-parameter]
monitor-base-url = $${publish:monitor-base-url}
monitor-url = $${publish:monitor-url}
monitor-user = $${publish:monitor-user}
monitor-password = $${publish:monitor-password}
[monitor-instance-parameter]
monitor-httpd-port = 8360
#monitor-title = $${slap-parameter:name}
#cors-domains = $${slap-parameter:monitor-cors-domains}
#username = $${slap-parameter:monitor-username}
#password = $${slap-parameter:monitor-password}
# Pass some parameter to dispay in monitoring interface
instance-configuration =
raw takeover-url http://[$${resilient-web-takeover-httpd-configuration-file:listening-ip}]:$${resilient-web-takeover-httpd-configuration-file:listening-port}/
raw takeover-password $${resilient-web-takeover-password:passwd}
[monitor-conf-parameters]
private-path-list +=
$${directory:logrotate-backup}
instance-runner-input-schema.json 0000664 0000000 0000000 00000015772 12732772745 0036266 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner {
"$schema": "http://json-schema.org/draft-04/schema#",
"properties": {
"user-authorized-key": {
"title": "User Authorized Key",
"description": "SSH public key in order to connect to the SSH server of this runner instance.",
"textarea": true,
"type": "string"
},
"instance-amount": {
"title": "Partition Amount",
"description": "Number of slappart to deploy inside the runner (default is 10). Needs instance to be restarted.",
"type": "integer",
"minimum": 1,
"maximum": 40
},
"slapos-software": {
"title": "Pre-selected Software Release",
"description": "a relative path from the slapos git repo to a folder containing a software release, which will be automaticaly deployed while the runner instanciation, and only if the parameter auto-deploy is set to 'true'. For example: 'software/helloworld",
"type": "string"
},
"auto-deploy": {
"title": "Automatically Deploy Software",
"description": "Authorizes the software declared with 'slapos-software' to be automatically deployed, or not. Needs instance to be restarted. (default is false)",
"type": "boolean"
},
"slapos-repository": {
"title": "SlapOS Git Repository URL",
"description": "url of the default git repository that will be download by the runner while its instanciation. Will be cloned in a directory named 'slapos' (default is https://lab.nexedi.com/nexedi/slapos.git)",
"type": "string",
"format": "uri",
"pattern": "^(http|https|ftp)://"
},
"slapos-reference": {
"title": "SlapOS Git Branch Name",
"description": "Branch or hash on which the default repository will checkout (default is master)",
"type": "string"
},
"auto-deploy-instance": {
"title": "Automatically Deploy Instances",
"description": "Prevent the runner from deploying and starting instances. Needs instance to be restarted. It is set to false for instances of type 'import' in resiliency in any case (default is false)",
"type": "boolean"
},
"autorun": {
"title": "Automatically Run Sofware/Instance",
"description": "Let automaticaly build and run a declared software with 'slapos-software'. Only works if 'slapos-software' is set, and 'auto-deploy' is true. Needs instance to be restarted. (default is false)",
"type": "boolean"
},
"slapos-software-type": {
"title": "Deployed Instance Software Type" ,
"description": "Software type of your instance inside the runner",
"type": "string"
},
"cpu-usage-ratio": {
"title": "CPU Usage Ratio",
"description": "Ratio of the CPU use for compilation, if value is set to n, compilation will use number-of-cpu/n of cpus (need instance restart)",
"type": "integer",
"default" : 4
},
"instance-name": {
"title": "Displayed instance name",
"description": "Name of the instance, to show in the window title",
"type": "string"
},
"custom-frontend-backend-url": {
"title": "Custom Frontend Backend URL",
"description": "return an ipv4 frontend of the given ipv6(+optional port)",
"type": "string",
"format": "uri"
},
"custom-frontend-backend-type": {
"title": "Custom Frontend Backend Type",
"description": "The type of the frontend slave instance to ask",
"type": "string",
"enum": ["zope"]
},
"custom-frontend-basic-auth": {
"title": "Custom Frontend Basic Auth",
"description": "if the ip given with 'custom-frontend-backend-url' is secure, set it to true for the promise do not fail",
"type": "boolean"
},
"custom-frontend-instance-guid": {
"title": "Custom Frontend Instance GUID",
"description": "Instance guid of the frontend you whish to use",
"type": "string"
},
"custom-frontend-software-type": {
"title": "Custom Frontend Software Type",
"description": "SoftwareType of the frontend you request (default is RootSoftwareInstance)",
"type": "string",
"enum": ["RootSoftwareInstance", "default"]
},
"custom-frontend-software-url": {
"title": "Custom Frontend Software URL",
"description": "Software Url of the frontend you request (ie.: http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg)",
"type": "string",
"format": "uri"
},
"check-custom-frontend-promise": {
"title": "Check Custom Frontend Promise",
"description": "Enable a promise to check that HTTP frontend created from custom-frontend-backend-url is available",
"type": "string",
"enum": ["true", "false"],
"default": "false"
},
"monitor-port": {
"title": "Monitor Port",
"description": "Allow to manually change the port on wich the apache server running monitoring interface is listening. The default value for the webrunner is different from the default value of the standalone stack-monitor server (default 9684)",
"type": "integer",
"minimum": 9683,
"exclusiveMinimum": true
},
"monitor-interface-url": {
"title": "Monitor Web Interface URL",
"description": "Give Url of HTML web interface that will be used to render this monitor instance.",
"type": "string",
"format": "uri",
"default": "https://monitor.app.officejs.com"
},
"monitor-cors-domains": {
"title": "Monitor CORS domains",
"description": "List of cors domains separated with space. Needed for ajax query on this monitor instance from a different domain.",
"type": "string",
"default": "monitor.app.officejs.com"
},
"cpu-usage-ratio": {
"title": "CPU Usage Ratio",
"description": "Ratio of the CPU use for compilation, if value is set to n, compilation will use number-of-cpu/n of cpus (need instance restart)",
"type": "integer",
"default" : 4
},
"resilient-clone-number": {
"title": "Number of Resilient Clones",
"description": "Number of clones to be deployed when a resilient runner, if missing a single clone is provided. Its value can be 1 or 2",
"type": "integer",
"enum": [1, 2]
},
"instance-name": {
"title": "Instance Name",
"description": "Name of the instance, to show in the window title",
"type": "string"
}
}
}
instance-runner-output-schema.json 0000664 0000000 0000000 00000002627 12732772745 0036462 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner {
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Values returned by Runner instanciation",
"properties": {
"backend_url": {
"description": "ipv6 url to directly access your runner",
"type": "string"
},
"access_url": {
"description": "Url to set up your account and then login into your webrunner",
"type": "string"
},
"url": {
"description": "Url to access your runner once you set your account",
"type": "string"
},
"ssh_command": {
"description": "SSH command used to access your runner in ssh when you provided a ssh public key",
"type": "string"
},
"monitor_url": {
"description": "Url to access the monitoring interface of your instance. Use this url to get the recovery code needed to set up your runner",
"type": "string"
},
"webdav_url": {
"description": "Url for webdav access of your runner it access the root of your runner. Same users as the one for the runner",
"type": "string"
},
"public_url": {
"description": "Url of the public folder of your runner",
"type": "string"
},
"git_public_url": {
"description": "Public url of your repositories.",
"type": "string"
},
"git_private_url": {
"description": "Private url for your repositories. You can use it to push.",
"type": "string"
}
},
"type": "object"
} instance-runner-resilient-input-schema.json 0000664 0000000 0000000 00000006063 12732772745 0040253 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner {
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema",
"allOf": [
{
"$ref": "instance-runner-input-schema.json#/"
},
{
"properties": {
"resilient-clone-number": {
"title": "Amount of backup(s) to create",
"description": "Amount of backup(s) to create. Each backup consists of a Pull Backup Server and a clone.",
"type": "integer",
"default": 1,
"enum": [1, 2],
"optional": true
},
"-sla-runner0-computer_guid": {
"title": "Target computer for main instance",
"description": "Target computer GUID for main instance.",
"type": "string",
"optional": true
},
"-sla-runner1-computer_guid": {
"title": "Target computer for first clone",
"description": "Target computer for first clone instance.",
"type": "string",
"optional": true
},
"-sla-pbs1-computer_guid": {
"title": "Target computer for first PBS",
"description": "Target computer for first PBS instance.",
"type": "string",
"optional": true
},
"-sla-runner2-computer_guid": {
"title": "Target computer for second clone",
"description": "Target computer for second clone instance.",
"type": "string",
"optional": true
},
"-sla-pbs2-computer_guid": {
"title": "Target computer for second PBS",
"description": "Target computer for second PBS instance.",
"type": "string",
"optional": true
},
"resiliency-backup-periodicity": {
"title": "Periodicity of backup",
"description": "Periodicity of backup, in cron format.",
"type": "string",
"optional": true
},
"remove-backup-older-than": {
"title": "Remove backups older than...",
"description": "Remove all the backups in PBS that are older than specified value. It should be rdiff-backup-compatible.",
"type": "string",
"default": "2W",
"optional": true
},
"ignore-known-hosts-file": {
"title": "Ignore known_hosts file",
"description": "Set either to fill known_hosts file for ssh or not. Useful if main instance and PBS are using the same IP (slapos proxy, webrunner).",
"type": "boolean",
"default": false,
"optional": true
}
}
}
]
} instance-runner.cfg 0000664 0000000 0000000 00000064507 12732772745 0033461 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
parts =
nginx_conf
nginx-launcher
certificate-authority
ca-nginx
gunicorn-launcher
gunicorn-graceful
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
publish-connection-information
slaprunner-promise
slaprunner-frontend-promise
apache-httpd-promise
httpd-frontend-promise
slaprunner-supervisord-wrapper
dropbear-promise
runtestsuite
symlinks
shellinabox
slapos-cfg
slapos-repo
cron-entry-prepare-software
deploy-instance-parameters
instance-software
instance-software-type
minishell-cwd
bash-profile
supervisord-wrapper
supervisord-promise
httpd-graceful-wrapper
{% if slapparameter_dict.get('custom-frontend-backend-url') and slapparameter_dict.get('check-custom-frontend-promise', 'false') == 'true' %}
custom-frontend-promise
{% endif %}
## Monitoring part
monitor-base
monitor-check-webrunner-internal-instance
extends = ${monitor2-template:rendered}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
{% if slapparameter_dict.get('custom-frontend-backend-url') -%}
[request-custom-frontend]
recipe = slapos.cookbook:requestoptional
software-url = {{ slapparameter_dict.get('custom-frontend-software-url', 'http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg') }}
software-type = {{ slapparameter_dict.get('custom-frontend-software-type', 'RootSoftwareInstance') }}
slave = true
name = Custom Web Frontend
server-url = $${slap-connection:server-url}
key-file = $${slap-connection:key-file}
cert-file = $${slap-connection:cert-file}
computer-id = $${slap-connection:computer-id}
partition-id = $${slap-connection:partition-id}
{% if slapparameter_dict.get('custom-frontend-instance-guid') -%}
sla-instance_guid = $${slap-parameter:frontend-instance-guid}
{% endif -%}
{% set custom_frontend_backend_type = slapparameter_dict.get('custom-frontend-backend-type') -%}
{% if custom_frontend_backend_type -%}
config-type = {{ custom_frontend_backend_type }}
{% endif -%}
config-url = {{ slapparameter_dict.get('custom-frontend-backend-url') }}
return = site_url domain
[custom-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/custom_frontend_promise
url = https://$${request-custom-frontend:connection-domain}
{% if slapparameter_dict.get('custom-frontend-basic-auth') -%}
check-secure = 1
{% endif -%}
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
[publish-connection-information]
custom-frontend-url = https://$${request-custom-frontend:connection-domain}
{% endif %}
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
home = $${buildout:directory}
etc = $${:home}/etc/
var = $${:home}/var/
srv = $${:home}/srv/
bin = $${:home}/bin/
tmp = $${:home}/tmp/
sshkeys = $${:srv}/sshkeys
services = $${:etc}/service/
scripts = $${:etc}/run/
ssh = $${:etc}/ssh/
log = $${:var}/log/
run = $${:var}/run/
backup = $${:srv}/backup/
promises = $${:etc}/promise/
test = $${:etc}/test/
nginx-data = $${:srv}/nginx
ca-dir = $${:srv}/ssl
project = $${:srv}/runner/project
[runnerdirectory]
recipe = slapos.cookbook:mkdirectory
home = $${directory:srv}/runner/
test = $${directory:srv}/test/
project = $${:home}/project
public = $${:home}/public
software-root = $${:home}/software
instance-root = $${:home}/instance
project-test = $${:test}/project
software-test = $${:test}/software
instance-test = $${:test}/instance
sessions = $${buildout:directory}/.sessions
private-project = $${:home}/.git-private
public-project = $${:home}/.git-public
#Create password recovery code for slaprunner
[recovery-code]
recipe = slapos.cookbook:generate.password
storage-path = $${directory:etc}/.rcode
bytes = 8
[slaprunner]
slaprunner = ${buildout:directory}/bin/slaprunner
slapos = ${buildout:directory}/bin/slapos
slapproxy = ${buildout:directory}/bin/slapproxy
supervisor = ${buildout:directory}/bin/slapgrid-supervisorctl
git-binary = ${git:location}/bin/git
root_check = false
slapos.cfg = $${directory:etc}/slapos.cfg
working-directory = $${runnerdirectory:home}
project-directory = $${runnerdirectory:project}
instance_root = $${runnerdirectory:instance-root}
software_root = $${runnerdirectory:software-root}
instance-monitor-url = https://[$${:ipv6}]:$${monitor-parameters:port}
etc_dir = $${directory:etc}
log_dir = $${directory:log}
run_dir = $${directory:run}
ssh_client = $${sshkeys-dropbear-runner:wrapper}
public_key = $${sshkeys-dropbear-runner:public-key}
private_key = $${sshkeys-dropbear-runner:private-key}
ipv4 = $${slap-network-information:local-ipv4}
ipv6 = $${slap-network-information:global-ipv6}
instance_root = $${runnerdirectory:instance-root}
proxy_port = 50000
runner_port = 50005
partition-amount = $${slap-parameter:instance-amount}
wrapper = $${directory:services}/slaprunner
debug = $${slap-parameter:debug}
access-url = https://[$${:ipv6}]:$${:runner_port}
supervisord_config = $${directory:etc}/supervisord.conf
supervisord_server = http://$${supervisord:server}
proxy_database = $${slaprunner:working-directory}/proxy.db
console = False
verbose = False
debug = False
auto_deploy = $${slap-parameter:auto-deploy}
auto_deploy_instance = $${slap-parameter:auto-deploy-instance}
autorun = $${slap-parameter:autorun}
knowledge0_file = $${monitor-instance-parameter:configuration-file-path}
minishell_cwd_file = $${directory:etc}/.minishell-cwd
minishell_history_file = $${directory:etc}/.minishell_history
software_info_json = $${runnerdirectory:home}/software_info.json
instance_info_json = $${runnerdirectory:home}/instance_info.json
path = $${shell:path}
instance_name = $${slap-parameter:instance-name}
#---------------------------
#--
#-- supervisord managing slaprunner instance processes
[slaprunner-supervisord-wrapper]
recipe = slapos.cookbook:wrapper
# XXX hardcoded locations
command-line = $${buildout:directory}/bin/slapos node supervisord --cfg $${directory:etc}/slapos.cfg -n
wrapper-path = $${directory:services}/slaprunner-supervisord
[test-runner]
<= slaprunner
slapos.cfg = $${directory:etc}/slapos-test.cfg
working-directory = $${runnerdirectory:test}
project-directory = $${runnerdirectory:project-test}
software_root = $${runnerdirectory:software-test}
instance_root = $${runnerdirectory:instance-test}
proxy_port = 8602
etc_dir = $${directory:test}
autorun = False
auto_deploy = True
[runtestsuite]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:directory}/bin/slaprunnertest
wrapper-path = $${directory:bin}/runTestSuite
environment = RUNNER_CONFIG=$${slapos-cfg:rendered}
# Deploy dropbear (minimalist SSH server)
[sshkeys-directory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:sshkeys}/requests/
keys = $${directory:sshkeys}/keys/
[sshkeys-authority]
recipe = slapos.cookbook:sshkeys_authority
request-directory = $${sshkeys-directory:requests}
keys-directory = $${sshkeys-directory:keys}
wrapper = $${directory:services}/sshkeys_authority
keygen-binary = ${dropbear:location}/bin/dropbearkey
[dropbear-runner-server]
recipe = slapos.cookbook:dropbear
host = $${slap-network-information:global-ipv6}
port = 22222
home = $${buildout:directory}
wrapper = $${directory:bin}/runner_sshd
shell = ${bash:location}/bin/bash
rsa-keyfile = $${directory:ssh}/server_key.rsa
allow-port-forwarding = true
dropbear-binary = ${dropbear:location}/sbin/dropbear
[sshkeys-dropbear-runner]
<= sshkeys-authority
recipe = slapos.cookbook:sshkeys_authority.request
name = dropbear
type = rsa
executable = $${dropbear-runner-server:wrapper}
public-key = $${dropbear-runner-server:rsa-keyfile}.pub
private-key = $${dropbear-runner-server:rsa-keyfile}
wrapper = $${directory:services}/runner_sshd
[dropbear-server-add-authorized-key]
<= dropbear-runner-server
recipe = slapos.cookbook:dropbear.add_authorized_key
key = $${slap-parameter:user-authorized-key}
#---------------------------
#--
#-- Set nginx frontend
[tempdirectory]
recipe = slapos.cookbook:mkdirectory
client_body_temp_path = $${directory:tmp}/client_body_temp_path
proxy_temp_path = $${directory:tmp}/proxy_temp_path
fastcgi_temp_path = $${directory:tmp}/fastcgi_temp_path
uwsgi_temp_path = $${directory:tmp}/uwsgi_temp_path
scgi_temp_path = $${directory:tmp}/scgi_temp_path
[nginx-frontend]
# Options
nb_workers = 5
# Network
local-ip = $${slap-network-information:local-ipv4}
global-ip = $${slap-network-information:global-ipv6}
global-port = $${slaprunner:runner_port}
# Backend
runner-ip = $${slaprunner:ipv4}
runner-port = $${slaprunner:runner_port}
# SSL
ssl-certificate = $${ca-nginx:cert-file}
ssl-key = $${ca-nginx:key-file}
# Log
path_pid = $${directory:run}/nginx.pid
path_log = $${directory:log}/nginx.log
path_access_log = $${directory:log}/nginx.access.log
path_error_log = $${directory:log}/nginx.error.log
path_tmp = $${directory:tmp}/
nginx_prefix = $${buildout:directory}
# Config files
path_nginx_conf = $${directory:etc}/nginx.conf
# Executables
bin_nginx = ${nginx-webdav:location}/sbin/nginx
bin_launcher = $${directory:bin}/launcher
# Utils
path_shell = ${dash:location}/bin/dash
# Misc.
etc_dir = $${directory:etc}
work_dir = $${slaprunner:working-directory}
[nginx_conf]
recipe = slapos.recipe.template:jinja2
template = ${template_nginx_conf:location}/${template_nginx_conf:filename}
rendered = $${nginx-frontend:path_nginx_conf}
context =
key shellinabox_socket shellinabox:socket
key socket gunicorn:socket
section param_nginx_frontend nginx-frontend
section param_tempdir tempdirectory
[nginx-launcher]
recipe = slapos.recipe.template:jinja2
template = ${template_launcher:location}/${template_launcher:filename}
rendered = $${nginx-frontend:bin_launcher}
mode = 700
context =
section param_nginx_frontend nginx-frontend
[httpd-parameters]
path_pid = $${directory:run}/httpd.pid
path_error_log = $${directory:log}/httpd-error.log
path_access_log = $${directory:log}/httpd-access.log
# XXX Use ca-nginx, no need to regenerate certificate
cert_file = $${ca-nginx:cert-file}
key_file = $${ca-nginx:key-file}
global_ip = $${slap-network-information:global-ipv6}
global_port = $${slap-parameter:slaprunner-httpd-port}
working_directory = $${slaprunner:working-directory}
dav_lock = $${directory:var}/WebDavLock
htpasswd_file = $${monitor-httpd-conf-parameter:htpasswd-file}
etc_dir = $${directory:etc}
var_dir = $${directory:var}
project_folder = $${directory:project}
project_private_folder = $${runnerdirectory:private-project}
project_public_folder = $${runnerdirectory:public-project}
runner_home = $${runnerdirectory:home}
git_http_backend = ${git:location}/libexec/git-core/git-http-backend
cgid_sock = $${directory:run}/cgid.sock
#cgi_httpd_conf = $${monitor-httpd-configuration-file:rendered}
httpd_cors_file = $${slaprunner-httpd-cors:location}
[httpd-conf]
recipe = slapos.recipe.template:jinja2
template = ${template_httpd_conf:location}/${template_httpd_conf:filename}
rendered = $${directory:etc}/httpd.conf
context =
section parameters httpd-parameters
[apache-httpd]
recipe = slapos.cookbook:wrapper
apache-executable = ${apache:location}/bin/httpd
wrapper-path = $${directory:services}/slaprunner-httpd
command-line = $${:apache-executable} -f $${httpd-conf:rendered} -DFOREGROUND
access-url = https://[$${httpd-parameters:global_ip}]:$${httpd-parameters:global_port}
wait-for-files =
$${ca-nginx:cert-file}
$${ca-nginx:key-file}
[httpd-graceful-wrapper]
recipe = collective.recipe.template
input = inline:
#!/bin/sh
exec kill -USR1 $(cat $${httpd-parameters:path_pid})
output = $${directory:scripts}/slaprunner-httpd-graceful
mode = 700
[apache-httpd-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/$${:filename}
filename = apache-httpd-listening-on-tcp
url = $${apache-httpd:access-url}
check-secure = 1
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
[slaprunner-httpd-cors]
recipe = plone.recipe.command
command = if [ ! -f $${:location} ]; then touch $${:location}; fi
location = $${directory:etc}/$${:filename}
filename = slaprunner-httpd-cors.cfg
stop-on-error = true
#--------------------
#--
#-- WSGI
[gunicorn]
bin_gunicorn = $${directory:bin}/gunicorn
bin_launcher = $${directory:services}/gunicorn
path_shell = ${dash:location}/bin/dash
socket = $${directory:tmp}/flaskserver.sock
path_pid = $${directory:run}/gunicorn.pid
[gunicorn-launcher]
recipe = slapos.cookbook:wrapper
command-line = $${gunicorn:bin_gunicorn} slapos.runner.run:app -p $${gunicorn:path_pid} -b unix:$${gunicorn:socket} -e RUNNER_CONFIG=$${slaprunner:slapos.cfg} --error-logfile $${directory:log}/$${:error-log-file} --log-level error --preload
error-log-file = gunicorn-error.log
wrapper-path = $${gunicorn:bin_launcher}
environment = PATH=$${environ:PATH}:${git:location}/bin/
RUNNER_CONFIG=$${slaprunner:slapos.cfg}
LANG=en_GB.UTF-8
[gunicorn-graceful]
recipe = slapos.cookbook:wrapper
command-line = $${directory:bin}/killpidfromfile $${gunicorn:path_pid} SIGHUP
wrapper-path = $${directory:scripts}/gunicorn-graceful
#--------------------
#--
#-- ssl certificates
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:services}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[ca-nginx]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/nginx_frontend.key
cert-file = $${cadirectory:certs}/nginx_frontend.crt
executable = $${nginx-launcher:rendered}
wrapper = $${directory:services}/nginx-frontend
# Put domain name
name = example.com
#--------------------
#--
#-- Request frontend
[request-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = SlapRunner Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = $${slaprunner:access-url}
config-domain = $${slap-parameter:frontend-domain}
return = site_url domain
[request-httpd-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
# XXX - Unfortunately, we still call webrunner httpd frontend "Monitor Frontend" otherwise
# buildout will ignore previous frontend that was created and create a new one (in case of upgrade)
name = Monitor Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = $${apache-httpd:access-url}
config-domain =
return = secure_access domain
[httpd-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/slaprunner-apache-http-frontend
url = $${request-httpd-frontend:connection-secure_access}
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
check-secure = 1
#--------------------------------------
#--
#-- Send information to SlapOS Master
[publish-connection-information]
recipe = slapos.cookbook:publish
1_info = On your first run, Use "access_url" to setup you account. Then you can use both "url" or "access_url". Or "backend_url" if you want to use ipv6. Set up your account in the webrunner in order to use webdav, and being able to clone your git repositories from the runner.
2_info = In order to set up your account, get the recovery-code from the monitoring interface. Use "monitor_setup_url" to configure monitor instance.
backend_url = $${slaprunner:access-url}
access_url = $${:url}/login
url = https://$${request-frontend:connection-domain}
ssh_command = ssh $${dropbear-runner-server:host} -p $${dropbear-runner-server:port}
webdav_url = $${request-httpd-frontend:connection-secure_access}/share/
public_url = $${request-httpd-frontend:connection-secure_access}/public/
git_public_url = https://[$${httpd-parameters:global_ip}]:$${httpd-parameters:global_port}/git-public/
git_private_url = https://[$${httpd-parameters:global_ip}]:$${httpd-parameters:global_port}/git/
monitor-base-url = $${publish:monitor-base-url}
{% if slapparameter_dict.get('instance-type', '') != 'resilient' -%}
{% set monitor_interface_url = slapparameter_dict.get('monitor-interface-url', 'https://monitor.app.officejs.com') -%}
monitor_setup_url = {{ monitor_interface_url }}/#page=settings_configurator&url=$${publish:monitor-url}&username=$${publish:monitor-user}&password=$${publish:monitor-password}
{% else -%}
monitor-url = $${publish:monitor-url}
monitor-user = $${publish:monitor-user}
monitor-password = $${publish:monitor-password}
{% endif -%}
#---------------------------
#--
#-- Deploy promises scripts
[slaprunner-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/slaprunner
hostname = $${slaprunner:ipv6}
port = $${slaprunner:runner_port}
[slaprunner-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/slaprunner_frontend
url = https://$${request-frontend:connection-domain}/login
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
check-secure = 1
[dropbear-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/dropbear
hostname = $${dropbear-runner-server:host}
port = $${dropbear-runner-server:port}
[symlinks]
recipe = cns.recipe.symlink
symlink_target = $${directory:bin}
symlink_base = ${buildout:directory}/bin
[slap-parameter]
# Default value if no ssh key is specified
user-authorized-key =
# Default value of instances number in slaprunner
instance-amount = 10
debug = false
frontend-domain =
slapos-repository = https://lab.nexedi.com/nexedi/slapos.git
slapos-software =
slapos-software-type =
slapos-reference = master
auto-deploy = false
auto-deploy-instance = true
autorun = false
slaprunner-httpd-port = $${:monitor-port}
# XXX - for backward compatibility, monitor-port was for slaprunner httpd server
monitor-port = 9684
instance-name =
monitor-cors-domains =
monitor-interface-url =
# XXX - define a new port for monitor here and use monitor-port for backward compatibility
monitor-httpd-port = 8386
[monitor-parameters]
port = $${slap-parameter:monitor-port}
[slapos-cfg]
recipe = slapos.recipe.template:jinja2
template = ${template-slapos-cfg:location}/${template-slapos-cfg:filename}
rendered = $${slaprunner:slapos.cfg}
mode = 700
context =
section slaprunner slaprunner
[slapos-test-cfg]
recipe = slapos.recipe.template:jinja2
template = ${template-slapos-cfg:location}/${template-slapos-cfg:filename}
rendered = $${test-runner:slapos.cfg}
mode = 700
context =
section slaprunner test-runner
[shellinabox]
recipe = slapos.recipe.template:jinja2
# We cannot use slapos.cookbook:wrapper here because this recipe escapes too much
socket = $${directory:run}/siab.sock
mode = 0700
rendered = $${directory:services}/shellinaboxd
template = inline:
#!/bin/sh
exec ${shellinabox:location}/bin/shellinaboxd \
--disable-ssl \
--disable-ssl-menu \
--unixdomain-only=$${:socket}:$(id -u):$(id -g):0600 \
--service "/:$(id -u):$(id -g):HOME:$${shell:wrapper} -l"
[shell]
recipe = slapos.cookbook:shell
wrapper = $${directory:bin}/bash
shell = ${bash:location}/bin/bash
home = $${buildout:directory}
path = $${environ:PATH}:/usr/bin:/bin/:${nano:location}/bin:${vim:location}/bin:${screen:location}/bin:${git:location}/bin:${tig:location}/bin
ps1 = "\\w> "
[environ]
recipe = collective.recipe.environment
[slapos-repo]
recipe = slapos.recipe.build:gitclone
repository = $${slap-parameter:slapos-repository}
git-executable = ${git:location}/bin/git
develop = true
location = $${directory:project}/slapos
branch = $${slap-parameter:slapos-reference}
[prepare-software]
recipe = slapos.cookbook:wrapper
command-line = ${curl:location}/bin/curl -g https://[$${slaprunner:ipv6}]:$${slaprunner:runner_port}/isSRReady --max-time 1 --insecure
wrapper-path = $${directory:scripts}/prepareSoftware
[cron-entry-prepare-software]
<= cron
recipe = slapos.cookbook:cron.d
name = prepare-software
frequency = */2 * * * *
command = $${prepare-software:wrapper-path}
[instance-parameters]
recipe = slapos.recipe.template:jinja2
extensions = jinja2.ext.do
template = ${template-parameters:location}/${template-parameters:filename}
rendered = $${directory:etc}/.parameter.xml.default
mode = 0644
context =
key slapparameter_dict slap-configuration:configuration
[deploy-instance-parameters]
recipe = plone.recipe.command
stop-on-error = true
parameter-xml = $${directory:etc}/.parameter.xml
command = if [ ! -f $${:parameter-xml} ]; then cp $${instance-parameters:rendered} $${:parameter-xml}; fi
[instance-software-type]
recipe = plone.recipe.command
stop-on-error = true
# XXX It should not be named with .xml as it is not xml
software-type-path = $${directory:etc}/.software_type.xml
command = if [ ! -f $${:software-type-path} -a "$${slap-parameter:slapos-software-type}" != "" ]; then echo "$${slap-parameter:slapos-software-type}" > $${:software-type-path}; fi
[instance-software]
recipe = plone.recipe.command
stop-on-error = true
command = SR=$${slap-parameter:slapos-software} && if [ -n "$SR" ] && [ ! -f "$${directory:etc}/.project" ]; then echo workspace/slapos/$${slap-parameter:slapos-software}/ > $${directory:etc}/.project; fi
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[minishell-cwd]
recipe = plone.recipe.command
command = if [ ! -f $${slaprunner:minishell_cwd_file} ]; then echo $${runnerdirectory:home} > $${slaprunner:minishell_cwd_file}; fi
location = $${slaprunner:minishell_cwd_file}
stop-on-error = true
[bash-profile]
recipe = slapos.recipe.template:jinja2
template = ${template-bash-profile:location}/${template-bash-profile:filename}
rendered = $${buildout:directory}/.bash_profile
context =
raw path $PATH:${nano:location}/bin:${vim:location}/bin:${screen:location}/bin:${git:location}/bin:${curl:location}/bin:${python2.7:location}/bin:${tig:location}/bin:${zip:location}/bin
key workdir runnerdirectory:home
#---------------------------
#--
#-- supervisord managing slaprunner automation features
[supervisord]
autorestart = false
autostart = false
directory = $${buildout:directory}
exitcodes = 0
logfile = $${directory:log}/supervisord.log
no_logfile = NONE
numprocs = 1
path = $${shell:path}
pidfile = $${directory:run}/supervisord.pid
server = $${slaprunner:ipv4}:$${:port}
port = 39986
slapgrid-cp = slapgrid-cp
slapgrid-cp-command = $${slaprunner:slapos} node instance --all --cfg $${:slapos-cfg} --pidfile $${:slapgrid-cp-pid} --verbose --logfile $${:slapgrid-cp-log}
slapgrid-cp-log = $${runnerdirectory:home}/instance.log
slapgrid-cp-pid = $${directory:run}/slapgrid-cp.pid
slapgrid-cp-startretries = 0
slapgrid-sr = slapgrid-sr
slapgrid-sr-command = $${slaprunner:slapos} node software --all --cfg $${:slapos-cfg} --pidfile $${:slapgrid-sr-pid} --verbose --logfile $${:slapgrid-sr-log}
slapgrid-sr-log = $${runnerdirectory:home}/software.log
slapgrid-sr-pid = $${directory:run}/slapgrid-sr.pid
slapgrid-sr-startretries = 0
slapproxy = slapproxy
slapproxy-autorestart = true
slapproxy-autostart = true
slapproxy-startsecs = 1
slapproxy-command = $${slaprunner:slapos} proxy start --logfile $${:slapproxy-log} --cfg $${:slapos-cfg}
slapproxy-log = $${directory:log}/slapproxy.log
socket_name = unix://$${:socket_path}
socket_path = $${directory:tmp}/supervisord.sock
startsecs = 0
# This file logs errors from listeners. Supervisord has its own logfile.
# Processes should handle their logs by themselves
stderr_logfile = $${directory:log}/supervisord-errors.log
slapos-cfg = $${slaprunner:slapos.cfg}
[supervisord-conf]
recipe = slapos.recipe.template:jinja2
template = ${template-supervisord:location}/${template-supervisord:filename}
rendered = $${directory:etc}/supervisord.conf
context =
import multiprocessing multiprocessing
import builtin __builtin__
section supervisord supervisord
key slapparameter_dict slap-configuration:configuration
key listener_slapgrid listener-slapgrid-bin:rendered
[listener-slapgrid-bin]
recipe = slapos.recipe.template:jinja2
template = ${template-listener-slapgrid:location}/${template-listener-slapgrid:filename}
rendered = $${directory:bin}/listener_slapgrid.py
mode = 0744
context =
section supervisord supervisord
section slaprunner slaprunner
raw python_executable ${buildout:directory}/bin/${extra-eggs:interpreter}
[supervisord-wrapper]
recipe = slapos.cookbook:wrapper
command-line = $${buildout:directory}/bin/supervisord -c $${supervisord-conf:rendered} --nodaemon
wrapper-path = $${directory:services}/supervisord
[supervisord-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/supervisord
hostname = $${slaprunner:ipv4}
port = $${supervisord:port}
# XXX Monitor
[monitor-instance-parameter]
monitor-httpd-port = $${slap-parameter:monitor-httpd-port}
{% if slapparameter_dict.get('name', '') -%}
monitor-title = {{ slapparameter_dict['name'] }}
{% endif -%}
cors-domains = {{ slapparameter_dict.get('monitor-cors-domains', 'monitor.app.officejs.com') }}
{% if slapparameter_dict.get('monitor-username', '') -%}
username = {{ slapparameter_dict['monitor-username'] }}
{% endif -%}
{% if slapparameter_dict.get('monitor-password', '') -%}
password = {{ slapparameter_dict['monitor-password'] }}
{% endif -%}
{% if slapparameter_dict.get('monitor-url-list', '') -%}
monitor-url-list = {{ slapparameter_dict['monitor-url-list'] }}
{% endif -%}
# check if not resilient runner
{% if not slapparameter_dict.get('authorized-key', '') -%}
# Pass some parameter to dispay in monitoring interface
instance-configuration =
file recovery-code $${recovery-code:storage-path}
httpdcors cors-domain $${slaprunner-httpd-cors:location} $${httpd-graceful-wrapper:output}
raw webrunner-url https://$${request-frontend:connection-domain}
{% endif -%}
configuration-file-path = $${buildout:directory}/knowledge0.cfg
[monitor-check-webrunner-internal-instance]
recipe = slapos.recipe.template:jinja2
template = ${monitor-check-webrunner-internal-instance:location}/${monitor-check-webrunner-internal-instance:filename}
rendered = $${monitor-directory:reports}/$${:filename}
filename = monitor-check-webrunner-internal-instance
mode = 0744
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/instance.cfg0000664 0000000 0000000 00000005104 12732772745 0032215 0 ustar 00root root 0000000 0000000 [buildout]
parts =
switch_softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
[switch_softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${instance-base-runner:rendered}
resilient = $${instance-resilient:rendered}
test = $${instance-resilient-test:rendered}
runner = $${instance-base-runner:rendered}
runner-import = ${instance-runner-import:output}
runner-export = ${instance-runner-export:output}
frozen = ${instance-frozen:output}
pull-backup = ${template-pull-backup:output}
[instance-base-runner]
recipe = slapos.recipe.template:jinja2
template = ${template-runner:output}
rendered = $${buildout:directory}/template-runner.cfg
extensions = jinja2.ext.do
context = key buildout buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
raw curl_executable_location ${curl:location}/bin/curl
raw dash_executable_location ${dash:location}/bin/dash
raw dcron_executable_location ${dcron:location}/sbin/crond
mode = 0644
[instance-resilient]
recipe = slapos.recipe.template:jinja2
template = ${template-resilient:target}
rendered = $${buildout:directory}/instance-resilient.cfg
extensions = jinja2.ext.do
context = key buildout buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
template-parts-destination = ${template-parts:destination}
template-replicated-destination = ${template-replicated:destination}
import-list = file parts :template-parts-destination
file replicated :template-replicated-destination
mode = 0644
[instance-resilient-test]
recipe = slapos.recipe.template:jinja2
template = ${template-resilient-test:location}/instance-resilient-test.cfg.jinja2
rendered = $${buildout:directory}/template-resilient-test.cfg
bin-directory = ${buildout:bin-directory}
context =
key bin_directory instance-resilient-test:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
mode = 0644
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/launcher.in 0000664 0000000 0000000 00000000454 12732772745 0032064 0 ustar 00root root 0000000 0000000 #! {{ param_nginx_frontend['path_shell'] }}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
# Run nginx
exec {{ param_nginx_frontend['bin_nginx'] }} \
-p {{ param_nginx_frontend['nginx_prefix'] }} \
-c {{ param_nginx_frontend['path_nginx_conf'] }}
nginx_conf.in 0000664 0000000 0000000 00000007257 12732772745 0032344 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner worker_processes {{ param_nginx_frontend['nb_workers'] }};
pid {{ param_nginx_frontend['path_pid'] }};
error_log {{ param_nginx_frontend['path_error_log'] }};
daemon off;
events {
worker_connections 1024;
accept_mutex off;
}
http {
default_type application/octet-stream;
access_log {{ param_nginx_frontend['path_access_log'] }} combined;
client_max_body_size 10M;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen [{{ param_nginx_frontend['global-ip'] }}]:{{ param_nginx_frontend['global-port'] }} ssl;
server_name _;
ssl_certificate {{ param_nginx_frontend['ssl-certificate'] }};
ssl_certificate_key {{ param_nginx_frontend['ssl-key'] }};
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
keepalive_timeout 90s;
client_body_temp_path {{ param_tempdir['client_body_temp_path'] }};
proxy_temp_path {{ param_tempdir['proxy_temp_path'] }};
fastcgi_temp_path {{ param_tempdir['fastcgi_temp_path'] }};
uwsgi_temp_path {{ param_tempdir['uwsgi_temp_path'] }};
scgi_temp_path {{ param_tempdir['scgi_temp_path'] }};
location / {
# When no .htpasswd exist, redirect the user to account creation page
if ( !-f {{ param_nginx_frontend['etc_dir'] }}/.htpasswd ) {
# redirect URL is different wether nginx is accessed directly or behind apache.
# nginx does not support nested if or multiple conditions, so we use this well known hack.
set $test no_htpasswd;
}
if ( $host = [{{ param_nginx_frontend['global-ip'] }}] ) {
set $test "${test}_backend_access";
}
if ( $test = no_htpasswd) {
return 301 $scheme://$host/setAccount ;
}
if ( $test = no_htpasswd_backend_access) {
return 301 /setAccount ;
}
auth_basic "Restricted";
auth_basic_user_file {{ param_nginx_frontend['etc_dir'] }}/.htpasswd;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Accel-Mapping /private/;
proxy_pass http://unix:{{ socket }};
}
location ~ ^(/login|/doLogin|/static|/setAccount|/configAccount|/slapgridResult|/isSRReady) {
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Accel-Mapping /private/;
proxy_pass http://unix:{{ socket }};
}
location /shellinabox {
proxy_pass http://unix:{{ shellinabox_socket }}:/;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
auth_basic "Restricted";
auth_basic_user_file {{ param_nginx_frontend['etc_dir'] }}/.htpasswd;
proxy_redirect off;
proxy_buffering off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $http_host;
}
}
}
parameters.xml.in 0000664 0000000 0000000 00000001102 12732772745 0033135 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner {% set inst_parameter_dict = {} -%}
{% if slapparameter_dict is defined -%}
{% for key in slapparameter_dict.keys() -%}
{% if key.startswith('parameter-') -%}
{% do inst_parameter_dict.__setitem__(key[10:], slapparameter_dict.pop(key)) -%}
{% endif -%}
{% endfor -%}
{% endif -%}
{% if slapparameter_dict is defined %}
{% for parameter_name in inst_parameter_dict.keys() %}
{{ inst_parameter_dict[parameter_name] }}
{% endfor %}
{% endif %}
software-for-erp5testnode.cfg 0000664 0000000 0000000 00000000673 12732772745 0035375 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner [buildout]
extends = development.cfg
parts += template-erp5testnode
# Change default software-type to be "test", so that it can be run using erp5testnode.
[template-erp5testnode]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-for-erp5testnode.cfg.in
md5sum = 59a13145d3df30f38442ce172330dfb3
output = ${buildout:directory}/template.cfg
mode = 0644
[template]
output = ${buildout:directory}/template-original.cfg
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/software.cfg0000664 0000000 0000000 00000001453 12732772745 0032246 0 ustar 00root root 0000000 0000000 # Production profile of slaprunner.
# Exactly the same as common.cfg, but:
# 1/ Use a defined set of Python eggs instead of using the latest available
# ones from Pypi, to ensure stability;
[buildout]
extends = common.cfg
[versions]
Flask-Auth = 0.85
PyRSS2Gen = 1.1
apache-libcloud = 0.20.1
cns.recipe.symlink = 0.2.3
collective.recipe.environment = 0.2.0
ecdsa = 0.13
erp5.util = 0.4.44
gitdb = 0.6.4
gunicorn = 19.5.0
prettytable = 0.7.2
pycrypto = 2.6.1
slapos.recipe.template = 2.9
slapos.toolbox = 0.55
smmap = 0.9.0
# Required by:
# slapos.toolbox==0.55
GitPython = 2.0.5
# Required by:
# slapos.toolbox==0.55
atomize = 0.2.0
# Required by:
# slapos.toolbox==0.55
feedparser = 5.2.1
# Required by:
# slapos.toolbox==0.55
lockfile = 0.12.2
# Required by:
# slapos.toolbox==0.55
paramiko = 2.0.0
software.cfg.json 0000664 0000000 0000000 00000001061 12732772745 0033132 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner {
"name": "Web Runner",
"description": "Web Runner",
"serialisation": "xml",
"software-type": {
"default": {
"title": "Default",
"description": "Standalone Runner",
"request": "instance-runner-input-schema.json",
"response": "instance-runner-output-schema.json",
"index": 0
},
"resilient": {
"title": "Resilient",
"description": "Resilient Runner",
"request": "instance-runner-resilient-input-schema.json",
"response": "instance-runner-output-schema.json",
"index": 1
}
}
}
slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/template/ 0000775 0000000 0000000 00000000000 12732772745 0031543 5 ustar 00root root 0000000 0000000 bash_profile.in 0000664 0000000 0000000 00000002013 12732772745 0034445 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/template # Beware, this file is automatically processed by slapgrid
# Do not modify it, your changes will be lost
# If you want to load your custom bash configuration, please use a .bashrc file
cd {{ workdir }}
export PATH={{- path }}
export PS1="$ "
if [ -f "$HOME/.bashrc" ] ; then
source $HOME/.bashrc
fi
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
# enable color support
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
fi
# some more ls aliases
alias ll='ls -l'
alias la='ls -A'
alias l='ls -CF'
echo "Welcome to SlapOS slaprunner shell"
echo
# TODO: how to run slapos ?
#echo "You can use the following slapos CLI commands:"
#echo "slapos node software"
#echo "slapos node instance"
#echo "slapos request"
#echo ""
# XXX for now we can use:
#echo ~/bin/supervisorctl -c ~/etc/supervisorctl.conf
#echo ~/bin/supervisorctl -c ~/srv/runner/instance/etc/supervisorctl.conf
cors-domain.jinja 0000664 0000000 0000000 00000004144 12732772745 0034717 0 ustar 00root root 0000000 0000000 slapos-3f1a8392d2a2eaf3812dd762e08e40956b17007d-software-slaprunner/software/slaprunner/template #!{{ python_executable }}
import cgi
import cgitb
import ConfigParser
import os
import re
import subprocess
cgitb.enable(display=0, logdir="/tmp/cgi.log")
form = cgi.FieldStorage()
apache_file = "{{ apache_file }}"
config_file = "{{ config_cfg }}"
if not os.path.exists(config_file):
print "Your software does not embed 0-knowledge. \
This interface is useless in this case