Skip to content

slapos
slapos
Commit 88fa3070 authored by Xavier Thompson's avatar Xavier Thompson
Browse files
Options

software/theia: Add resilience

parent eb5d675c
Show whitespace changes
Inline Side-by-side
Showing with 672 additions and 508 deletions
software/theia/buildout.hash.cfg
View file @ 88fa3070
...@@ -13,9 +13,25 @@ ...@@ -13,9 +13,25 @@
# section inheritance (< = ...) are NOT supported (but you should really # section inheritance (< = ...) are NOT supported (but you should really
# not need these here). # not need these here).
[instance] [instance.cfg]
filename = instance.cfg.in filename = instance.cfg.in
md5sum = 4f119df2b2d71e5ec1fd04051fd8fff8 md5sum = df9548360119b698568694a037ec9c76
[instance-theia.cfg.jinja2.in]
filename = instance-theia.cfg.jinja2.in
md5sum = 3ce9e1c54943840b0e363764bf28111e
[instance-theia-import.cfg.in]
filename = instance-theia-import.cfg.in
md5sum = 45ef2827041459f897f9e9c45eb44832
[instance-theia-export.cfg.in]
filename = instance-theia-export.cfg.in
md5sum = 9feb1a143fe70b70dda222d752315e13
[instance-theia-resilient.cfg.jinja2]
filename = instance-theia-resilient.cfg.jinja2
md5sum = 388ef606ccbc6d543685a53b18eb7369
[yarn.lock] [yarn.lock]
filename = yarn.lock filename = yarn.lock
......
software/theia/instance-theia-export.cfg.in 0 → 100644
View file @ 88fa3070
[buildout]
extends = ${theia:rendered}
${template-pbsready-export:rendered}
parts += $${:theia-parts}
[slap-parameter]
namebase = ${slap-configuration:configuration.namebase}
[exporter]
recipe = slapos.cookbook:wrapper
command-line = echo "hello"
wrapper-path = $${directory:bin}/$${slap-parameter:namebase}-exporter
# wrapper parameter is needed by resilience stack
wrapper = $${:wrapper-path}
# Extends publish section with resilient parameters
[publish-connection-parameter]
<= resilient-publish-connection-parameter
software/theia/instance-theia-import.cfg.in 0 → 100644
View file @ 88fa3070
[buildout]
extends = ${theia:rendered}
${template-pbsready-import:rendered}
parts += $${:theia-parts}
[slap-parameter]
namebase = ${slap-configuration:configuration.namebase}
[importer]
recipe = slapos.cookbook:wrapper
command-line = echo "goodbye"
wrapper-path = $${directory:bin}/$${slap-parameter:namebase}-exporter
# wrapper parameter is needed by resilience stack
wrapper = $${:wrapper-path}
software/theia/instance-theia-resilient.cfg.jinja2 0 → 100644
View file @ 88fa3070
{% import 'parts' as parts %}
{% import 'replicated' as replicated with context %}
{% set number_of_instances = slapparameter_dict.get('resilient-clone-number', 1)|int %}
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
extends =
{{ monitor_template }}
parts +=
# Generate the parts to request theia-export, pull-backup and theia-import
# See stack/resilient/template-parts.cfg.in and stack/resilient/template-replicated.cfg.in
# See below for the generation of the sections corresponding to the parts generated here
{{ parts.replicate("theia", number_of_instances + 1) }}
# Also publish some connection parameters
publish-connection-parameter
[ArgLeader]
[ArgBackup]
# Generate sections to request theia-export, pull-backup and theia-import
# See stack/resilient/template-replicated.cfg.in
# In particular:
#
# [request-theia]
# <= ArgLeader
# ...
#
# [request-theia-pseudo-replicating-1]
# <= ArgBackup
# ...
#
# [request-pbs-theia-1]
# ...
#
{{ replicated.replicate("theia", number_of_instances + 1,
"theia-export", "theia-import",
"ArgLeader", "ArgBackup",
slapparameter_dict=slapparameter_dict) }}
# Extend the list of return parameters for the theia-export request
[request-theia]
return += url username password
# Publish some parameters from the theia-export instance
[publish-connection-parameter]
recipe = slapos.cookbook:publish
url = ${request-theia:connection-url}
username = ${request-theia:connection-username}
password = ${request-theia:connection-password}
software/theia/instance-theia.cfg.jinja2.in 0 → 100644
View file @ 88fa3070
[buildout]
theia-parts =
monitor-base
promises
frontend-reload
tasks.json
publish-connection-parameter
slapos-repository
runner-link
settings.json
parts = $${:theia-parts}
extends = ${monitor-template:rendered}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[frontend-instance-password]
recipe = slapos.cookbook:generate.password
username = node
bytes = 12
[frontend-instance-certificate]
recipe = plone.recipe.command
command =
if [ ! -e $${:key-file} ]
then
${openssl-output:openssl} req -x509 -nodes -days 3650 \
-subj "/C=AA/ST=X/L=X/O=Dis/CN=$${:common-name}" \
-newkey rsa:1024 -keyout $${:key-file} \
-out $${:cert-file}
fi
update-command = $${:command}
key-file = $${directory:etc}/$${:_buildout_section_name_}.key
cert-file = $${directory:etc}/$${:_buildout_section_name_}.crt
common-name = $${frontend-instance-config:ip}
location =
$${:key-file}
$${:cert-file}
[frontend-instance-config]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:etc}/$${:_buildout_section_name_}
template = inline:
:$${:port} {
bind $${:ip}
tls $${frontend-instance-certificate:cert-file} $${frontend-instance-certificate:key-file}
log stdout
errors stderr
gzip
# because caddy does not support upgrade http2 to websocket
# https://tools.ietf.org/html/rfc8441
tls {
alpn http/1.1
}
root $${directory:frontend-static}
browse
proxy / $${theia-instance:base-url} {
except $${frontend-instance-fonts:folder-name} $${frontend-instance-slapos.css:folder-name} public $${favicon.ico:filename} $${frontend-instance-logo:filename}
}
proxy /services $${theia-instance:base-url} {
websocket
}
proxy /file-upload $${theia-instance:base-url} {
websocket
}
basicauth $${frontend-instance-password:username} $${frontend-instance-password:passwd} {
realm "Theia"
/
}
}
ip = $${instance-parameter:ipv6-random}
hostname = [$${:ip}]
port = 3001
[frontend-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line =
${caddy:output} -conf $${frontend-instance-config:rendered} -pidfile $${:pidfile}
ip = $${frontend-instance-config:ip}
hostname = $${frontend-instance-config:hostname}
port = $${frontend-instance-config:port}
pidfile = $${directory:pidfiles}/$${:_buildout_section_name_}.pid
url = https://$${:hostname}:$${:port}/
[frontend-instance-fonts]
; XXX caddy 1 does not seem to serve different folders at different locations
; so we link fonts in static folder
recipe = plone.recipe.command
location = $${directory:frontend-static}/$${:folder-name}
folder-name = fonts
command =
mkdir $${:location}
ln -s ${source-code-pro-fonts:location} $${:location}/source-code-pro
ln -s ${jetbrains-mono-fonts:location} $${:location}/jetbrains-mono
stop-on-error = true
[frontend-instance-logo]
recipe = plone.recipe.command
filename = logo.png
full-path = $${directory:frontend-static}/$${:filename}
command =
if [ ! -e $${:full-path} ]
then
ln -s ${logo.png:output} $${:full-path}
fi
stop-on-error = true
[frontend-instance-slapos.css]
recipe = slapos.recipe.template:jinja2
template = ${slapos.css.in:output}
rendered = $${directory:frontend-static}/$${:folder-name}/slapos.css
folder-name = css
context =
key logo_image frontend-instance-logo:filename
[frontend-reload]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line =
${bash:location}/bin/bash -c
"kill -s USR1 $$(${coreutils:location}/bin/cat $${frontend-instance:pidfile}) \
&& ${coreutils:location}/bin/sleep infinity"
hash-files =
$${frontend-instance-config:rendered}
$${frontend-instance:wrapper-path}
wait-for-files = $${frontend-instance:pidfile}
[favicon.ico]
# generate a pseudo random favicon, different for each instance name.
recipe = slapos.recipe.build
install =
import hashlib, shutil
buildout_offline = self.buildout['buildout']['offline']
self.buildout['buildout']['offline'] = 'false'
try:
gravatar_url = "https://www.gravatar.com/avatar/" + hashlib.md5(
'''$${slap-configuration:root-instance-title}'''
).hexdigest() + "?s=256&d=retro"
shutil.copy(self.download(gravatar_url), '''$${:location}''')
except Exception:
# Because installation should work offline, if we can't download a favicon,
# just ignore this step.
self.logger.exception("Error while downloading favicon, using empty one")
open('''$${:location}''', 'w').close()
finally:
self.buildout['buildout']['offline'] = buildout_offline
location = $${directory:frontend-static}/$${:filename}
filename = $${:_buildout_section_name_}
[tasks.json]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:dot-theia}/tasks.json
template =
inline:
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
"version": "2.0.0",
"tasks": [
{
"label": "slapos node software",
"detail": "Build all software supplied to the node",
"type": "shell",
"command": "${buildout:bin-directory}/slapos",
"args": [
"node",
"software",
// debug mode can be enabled by commenting out this line:
// "--buildout-debug",
"--all"
],
"options": {
"env": {
"SLAPOS_CONFIGURATION": "$${slapos-standalone-config:slapos-configuration}",
"GIT_EXEC_PATH": ""
}
},
"group": {
"kind": "build",
"isDefault": true
},
"problemMatcher": []
},
{
"label": "slapos node instance",
"detail": "Create all instances requested on the node",
"type": "shell",
"command": "${buildout:bin-directory}/slapos",
"args": [
"node",
"instance",
// debug mode can be enabled by commenting out this line:
// "--buildout-debug",
"--all"
],
"options": {
"env": {
"SLAPOS_CONFIGURATION": "$${slapos-standalone-config:slapos-configuration}",
"GIT_EXEC_PATH": ""
}
},
"problemMatcher": [],
"group": {
"kind": "build",
"isDefault": true
}
}
]
}
[theia-service]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
{% raw %}
export THEIA_WEBVIEW_EXTERNAL_ENDPOINT='{{"{{"}}hostname{{"}}"}}'
{% endraw %}
export THEIA_OPEN_EDITOR_TOKEN=$(${openssl:location}/bin/openssl rand -hex 32)
export THEIA_URL=$${:base-url}
export THEIA_SHELL=$${theia-shell:rendered}
export HOME=$${buildout:directory}
export TMP=$${directory:tmp}
export TEMP=$TMP
export LC_ALL=C.UTF-8
export TERMINFO=${ncurses:location}/lib/terminfo/
export EDITOR="${python-language-server:location}/bin/python -m theia_open --wait"
exec ${theia-wrapper:rendered} $@
ip = $${instance-parameter:ipv4-random}
port = 3000
base-url = http://$${:ip}:$${:port}/
[theia-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line = $${theia-service:rendered} --hostname=$${:hostname} --port=$${:port} $${directory:project}
hash-existing-files =
${yarn.lock:output}
${theia-wrapper:rendered}
ip = $${instance-parameter:ipv4-random}
hostname = $${:ip}
port = $${theia-service:port}
base-url = $${theia-service:base-url}
[theia-shell]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template = inline:
#!${python:location}/bin/python
import os
import sys
import time
args = sys.argv[1:]
# when running interactively, activate slapos configuration and reset GIT_EXEC_PATH to workaround https://github.com/eclipse-theia/theia/issues/7555
if not args: args = ["-c", ". $${slapos-standalone-activate:rendered} && exec env GIT_EXEC_PATH= ${bash:location}/bin/bash --rcfile $${theia-bashrc:rendered}", ]
# otherwise, assume this shell is running task and add an artificial delay to workaround https://github.com/eclipse-theia/theia/issues/2961
else: time.sleep(1)
os.execv('${bash:location}/bin/bash', ['${bash:location}/bin/bash'] + args)
[theia-bashrc]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:etc}/$${:_buildout_section_name_}
template =
inline:
# enable bash completion
. ${bash-completion:location}/etc/profile.d/bash_completion.sh
# source user's .bashrc
[ -f ~/.bashrc ] && . ~/.bashrc
depends =
$${shell-setup-completion:recipe}
[shell-setup-completion]
recipe = plone.recipe.command
stop-on-error = true
command =
${buildout:bin-directory}/slapos complete > $${directory:bash-completions}/slapos
${buildout:bin-directory}/slapos complete --shell fish > $${directory:fish-completions}/slapos.fish
[slapos-standalone-config]
ipv4 = $${instance-parameter:ipv4-random}
ipv6 = $${instance-parameter:ipv6-random}
port = 4000
slapos-configuration = $${directory:runner}/etc/slapos.cfg
computer-id = slaprunner
[slapos-standalone-activate]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
export PATH=${buildout:bin-directory}:$PATH
export SLAPOS_CONFIGURATION=$${slapos-standalone-config:slapos-configuration}
export SLAPOS_CLIENT_CONFIGURATION=$SLAPOS_CONFIGURATION
echo 'Standalone SlapOS for computer `$${slapos-standalone-config:computer-id}` activated'
[slapos-standalone]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
export PATH=${buildout:bin-directory}:$PATH
exec ${slapos-standalone:script-path} \
$${directory:runner} \
$${slapos-standalone-config:ipv4} \
$${slapos-standalone-config:ipv6} \
$${slapos-standalone-config:port} \
$${slapos-standalone-config:computer-id} \
--sr='$${instance-parameter:configuration.embedded-sr}' \
--srtype='$${instance-parameter:configuration.embedded-sr-type}' \
$${slap-connection:server-url} \
$${slap-connection:computer-id} \
$${slap-connection:partition-id} \
--key='$${slap-connection:key-file}' \
--cert='$${slap-connection:cert-file}'
[slapos-standalone-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line = $${slapos-standalone:rendered}
hash-files =
$${slapos-standalone:rendered}
hostname = $${slapos-standalone-config:ipv4}
port = $${slapos-standalone-config:port}
[promises]
recipe =
instance-promises =
$${theia-listen-promise:name}
$${frontend-listen-promise:name}
$${apache-frontend-url-available-promise:name}
$${slapos-standalone-listen-promise:name}
$${slapos-autorun-promise:name}
[theia-listen-promise]
<= monitor-promise-base
module = check_port_listening
name = $${:_buildout_section_name_}.py
config-hostname = $${theia-instance:ip}
config-port = $${theia-instance:port}
[frontend-listen-promise]
<= monitor-promise-base
module = check_port_listening
name = $${:_buildout_section_name_}.py
config-hostname = $${frontend-instance:ip}
config-port = $${frontend-instance:port}
[apache-frontend-url-available-promise]
<= monitor-promise-base
module = check_url_available
name = $${:_buildout_section_name_}.py
config-url = $${apache-frontend:connection-secure_access}
config-check-secure = 1
[slapos-standalone-listen-promise]
<= monitor-promise-base
module = check_port_listening
# XXX promise plugins can not contain "slapos" in their names
name = standalone-listen-promise.py
config-hostname = $${slapos-standalone-instance:hostname}
config-port = $${slapos-standalone-instance:port}
[slapos-autorun-promise]
<= monitor-promise-base
module = check_service_state
# XXX promise plugins can not contain "slapos" in their names
name = autorun-state-promise.py
config-service = $${slapos-autorun:service-name}
config-expect = $${slapos-autorun:autorun}
[apache-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Theia Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = $${frontend-instance:url}
config-https-only = true
config-type = websocket
config-websocket-path-list = /services /file-upload
return = domain secure_access
[publish-connection-parameter]
<= monitor-publish
recipe = slapos.cookbook:publish
url = $${apache-frontend:connection-secure_access}
username = $${frontend-instance-password:username}
password = $${frontend-instance-password:passwd}
[instance-parameter]
{% for k, v in slapconfiguration_section.items() -%}
{{ k }} = {{ v }}
{% endfor -%}
[slapos-autorun]
recipe = plone.recipe.command
command =
case $${:autorun} in
( running ) ${buildout:bin-directory}/supervisorctl -c $${:supervisor-conf} start $${:service-name};;
( stopped ) ${buildout:bin-directory}/supervisorctl -c $${:supervisor-conf} stop $${:service-name};;
esac
update-command = $${:command}
service-name = slapos-node-auto
supervisor-conf = $${directory:runner}/etc/supervisord.conf
autorun = $${instance-parameter:configuration.autorun}
[slapos-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/slapos.git
location = $${directory:project}/slapos
branch = 1.0
develop = true
git-executable = ${git:location}/bin/git
[settings.json]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:dot-theia}$${:_buildout_section_name_}
once = $${:rendered}
template =
inline:
{
"files.watcherExclude": {
"**/.eggs/**": true,
"**/.env/**": true,
"**/.git/**": true,
"**/node_modules/**": true,
"$${directory:runner}/**":true,
"$${directory:project}/runner/**":true
}
}
[runner-link]
recipe = slapos.cookbook:symbolic.link
target-directory = $${directory:project}
link-binary = $${directory:runner}
[directory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc
var = $${buildout:directory}/var
srv = $${buildout:directory}/srv
bin = $${buildout:directory}/bin
tmp = $${buildout:directory}/tmp
dot-theia = $${buildout:directory}/.theia/
pidfiles = $${:var}/run
services = $${:etc}/service
runner = $${:srv}/runner
project = $${:srv}/project
frontend-static = $${:srv}/frontend-static
frontend-static-public = $${:frontend-static}/public
frontend-static-css = $${:frontend-static}/css
bash-completions = $${buildout:directory}/.local/share/bash-completion/completions/
fish-completions = $${buildout:directory}/.config/fish/completions/
software/theia/instance.cfg.in
View file @ 88fa3070
[buildout] [buildout]
extends =
${monitor-template:rendered}
theia-environment-parts =
tasks.json
slapos-repository
runner-link
settings.json
theia-parts =
frontend-reload
promises
parts = parts =
monitor-base switch_softwaretype
$${:theia-parts}
$${:theia-environment-parts} extends =
publish-connection-parameter ${template-resilient-templates:output}
eggs-directory = ${buildout:eggs-directory} eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory} develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[instance-parameter] [slap-configuration]
recipe = slapos.cookbook:slapconfiguration recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id} computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id} partition = $${slap-connection:partition-id}
...@@ -34,493 +19,51 @@ configuration.autorun = running ...@@ -34,493 +19,51 @@ configuration.autorun = running
configuration.embedded-sr = configuration.embedded-sr =
configuration.embedded-sr-type = configuration.embedded-sr-type =
[switch_softwaretype]
[publish-connection-parameter] recipe = slapos.cookbook:switch-softwaretype
<= monitor-publish default = $${:theia}
recipe = slapos.cookbook:publish theia = theia:rendered
url = $${apache-frontend:connection-secure_access} theia-import = theia-import:output
username = $${frontend-instance-password:username} theia-export = theia-export:output
password = $${frontend-instance-password:passwd} pull-backup = template-pull-backup:rendered
theia-resilient = theia-resilient:rendered
RootSoftwareInstance = $${:default}
[directory]
recipe = slapos.cookbook:mkdirectory [template-base]
etc = $${buildout:directory}/etc recipe = slapos.recipe.template
var = $${buildout:directory}/var output = $${buildout:directory}/$${:filename}
srv = $${buildout:directory}/srv mode = 0644
bin = $${buildout:directory}/bin
tmp = $${buildout:directory}/tmp [theia]
dot-theia = $${buildout:directory}/.theia/
pidfiles = $${:var}/run
services = $${:etc}/service
runner = $${:srv}/runner
project = $${:srv}/project
frontend-static = $${:srv}/frontend-static
frontend-static-public = $${:frontend-static}/public
frontend-static-css = $${:frontend-static}/css
bash-completions = $${buildout:directory}/.local/share/bash-completion/completions/
fish-completions = $${buildout:directory}/.config/fish/completions/
# Promises
# --------
[promises]
recipe =
instance-promises =
$${theia-listen-promise:name}
$${frontend-listen-promise:name}
$${apache-frontend-url-available-promise:name}
$${slapos-standalone-listen-promise:name}
$${slapos-autorun-promise:name}
[theia-listen-promise]
<= monitor-promise-base
module = check_port_listening
name = $${:_buildout_section_name_}.py
config-hostname = $${theia-instance:ip}
config-port = $${theia-instance:port}
[frontend-listen-promise]
<= monitor-promise-base
module = check_port_listening
name = $${:_buildout_section_name_}.py
config-hostname = $${frontend-instance:ip}
config-port = $${frontend-instance:port}
[apache-frontend-url-available-promise]
<= monitor-promise-base
module = check_url_available
name = $${:_buildout_section_name_}.py
config-url = $${apache-frontend:connection-secure_access}
config-check-secure = 1
[slapos-standalone-listen-promise]
<= monitor-promise-base
module = check_port_listening
# XXX promise plugins can not contain "slapos" in their names
name = standalone-listen-promise.py
config-hostname = $${slapos-standalone-instance:hostname}
config-port = $${slapos-standalone-instance:port}
[slapos-autorun-promise]
<= monitor-promise-base
module = check_service_state
# XXX promise plugins can not contain "slapos" in their names
name = autorun-state-promise.py
config-service = $${slapos-autorun:service-name}
config-expect = $${slapos-autorun:autorun}
# Remote Apache Frontend
# ----------------------
[apache-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Theia Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config-url = $${frontend-instance:url}
config-https-only = true
config-type = websocket
config-websocket-path-list = /services /file-upload
return = domain secure_access
# Local Caddy Frontend
# --------------------
[frontend-instance-password]
recipe = slapos.cookbook:generate.password
username = node
bytes = 12
[frontend-instance-port]
recipe = slapos.cookbook:free_port
minimum = 3000
maximum = 3100
ip = $${instance-parameter:ipv6-random}
[frontend-instance-certificate]
recipe = plone.recipe.command
command =
if [ ! -e $${:key-file} ]
then
${openssl-output:openssl} req -x509 -nodes -days 3650 \
-subj "/C=AA/ST=X/L=X/O=Dis/CN=$${:common-name}" \
-newkey rsa:1024 -keyout $${:key-file} \
-out $${:cert-file}
fi
update-command = $${:command}
key-file = $${directory:etc}/$${:_buildout_section_name_}.key
cert-file = $${directory:etc}/$${:_buildout_section_name_}.crt
common-name = $${frontend-instance-config:ip}
location =
$${:key-file}
$${:cert-file}
[frontend-instance-config]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:etc}/$${:_buildout_section_name_}
template = inline:
:$${:port} {
bind $${:ip}
tls $${frontend-instance-certificate:cert-file} $${frontend-instance-certificate:key-file}
log stdout
errors stderr
gzip
# because caddy does not support upgrade http2 to websocket
# https://tools.ietf.org/html/rfc8441
tls {
alpn http/1.1
}
root $${directory:frontend-static}
browse
proxy / $${theia-instance:base-url} {
except $${frontend-instance-fonts:folder-name} $${frontend-instance-slapos.css:folder-name} public $${favicon.ico:filename} $${frontend-instance-logo:filename}
}
proxy /services $${theia-instance:base-url} {
websocket
}
proxy /file-upload $${theia-instance:base-url} {
websocket
}
basicauth $${frontend-instance-password:username} $${frontend-instance-password:passwd} {
realm "Theia"
/
}
}
ip = $${frontend-instance-port:ip}
hostname = [$${:ip}]
port = $${frontend-instance-port:port}
[frontend-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line =
${caddy:output} -conf $${frontend-instance-config:rendered} -pidfile $${:pidfile}
ip = $${frontend-instance-config:ip}
hostname = $${frontend-instance-config:hostname}
port = $${frontend-instance-config:port}
pidfile = $${directory:pidfiles}/$${:_buildout_section_name_}.pid
url = https://$${:hostname}:$${:port}/
[frontend-instance-fonts]
; XXX caddy 1 does not seem to serve different folders at different locations
; so we link fonts in static folder
recipe = plone.recipe.command
location = $${directory:frontend-static}/$${:folder-name}
folder-name = fonts
command =
mkdir $${:location}
ln -s ${source-code-pro-fonts:location} $${:location}/source-code-pro
ln -s ${jetbrains-mono-fonts:location} $${:location}/jetbrains-mono
stop-on-error = true
[frontend-instance-logo]
recipe = plone.recipe.command
filename = logo.png
full-path = $${directory:frontend-static}/$${:filename}
command =
if [ ! -e $${:full-path} ]
then
ln -s ${logo.png:output} $${:full-path}
fi
stop-on-error = true
[frontend-instance-slapos.css]
recipe = slapos.recipe.template:jinja2 recipe = slapos.recipe.template:jinja2
template = ${slapos.css.in:output} template = ${instance-theia.cfg.jinja2.in:output}
rendered = $${directory:frontend-static}/$${:folder-name}/slapos.css rendered = $${buildout:directory}/instance-theia.cfg
folder-name = css
context = context =
key logo_image frontend-instance-logo:filename section slapconfiguration_section slap-configuration
[frontend-reload]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line =
${bash:location}/bin/bash -c
"kill -s USR1 $$(${coreutils:location}/bin/cat $${frontend-instance:pidfile}) \
&& ${coreutils:location}/bin/sleep infinity"
hash-files =
$${frontend-instance-config:rendered}
$${frontend-instance:wrapper-path}
wait-for-files = $${frontend-instance:pidfile}
[favicon.ico]
# generate a pseudo random favicon, different for each instance name.
recipe = slapos.recipe.build
install =
import hashlib, shutil
buildout_offline = self.buildout['buildout']['offline']
self.buildout['buildout']['offline'] = 'false'
try:
gravatar_url = "https://www.gravatar.com/avatar/" + hashlib.md5(
'''$${slap-configuration:root-instance-title}'''
).hexdigest() + "?s=256&d=retro"
shutil.copy(self.download(gravatar_url), '''$${:location}''')
except Exception:
# Because installation should work offline, if we can't download a favicon,
# just ignore this step.
self.logger.exception("Error while downloading favicon, using empty one")
open('''$${:location}''', 'w').close()
finally:
self.buildout['buildout']['offline'] = buildout_offline
location = $${directory:frontend-static}/$${:filename}
filename = $${:_buildout_section_name_}
[theia-import]
<= template-base
url = ${instance-theia-import.cfg.in:output}
filename = instance-theia-import.cfg
# Theia Backend [theia-export]
# ------------- <= template-base
url = ${instance-theia-export.cfg.in:output}
[theia-service-port] filename = instance-theia-export.cfg
recipe = slapos.cookbook:free_port
minimum = 3500
maximum = 3600
ip = $${instance-parameter:ipv4-random}
[theia-service]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
{% raw %}
export THEIA_WEBVIEW_EXTERNAL_ENDPOINT='{{hostname}}'
{% endraw %}
export THEIA_OPEN_EDITOR_TOKEN=$(${openssl:location}/bin/openssl rand -hex 32)
export THEIA_URL=$${:base-url}
export THEIA_SHELL=$${theia-shell:rendered}
export HOME=$${buildout:directory}
export TMP=$${directory:tmp}
export TEMP=$TMP
export LC_ALL=C.UTF-8
export TERMINFO=${ncurses:location}/lib/terminfo/
export EDITOR="${python-language-server:location}/bin/python -m theia_open --wait"
exec ${theia-wrapper:rendered} $@
ip = $${theia-service-port:ip}
port = $${theia-service-port:port}
base-url = http://$${:ip}:$${:port}/
[theia-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line = $${theia-service:rendered} --hostname=$${:hostname} --port=$${:port} $${directory:project}
hash-existing-files =
${yarn.lock:output}
${theia-wrapper:rendered}
ip = $${instance-parameter:ipv4-random}
hostname = $${:ip}
port = $${theia-service:port}
base-url = $${theia-service:base-url}
[theia-shell] [theia-resilient]
recipe = slapos.recipe.template:jinja2 recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_} template = ${instance-theia-resilient.cfg.jinja2:output}
mode = 0700 rendered = $${buildout:directory}/instance-theia-resilient.cfg
template = inline: extensions = jinja2.ext.do
#!${python:location}/bin/python context =
import os key buildout buildout:bin-directory
import sys key develop_eggs_directory buildout:develop-eggs-directory
import time key eggs_directory buildout:eggs-directory
args = sys.argv[1:] key slapparameter_dict slap-configuration:configuration
# when running interactively, activate slapos configuration and reset GIT_EXEC_PATH to workaround https://github.com/eclipse-theia/theia/issues/7555 raw monitor_template ${monitor-template:rendered}
if not args: args = ["-c", ". $${slapos-standalone-activate:rendered} && exec env GIT_EXEC_PATH= ${bash:location}/bin/bash --rcfile $${theia-bashrc:rendered}", ] template-parts-destination = ${template-parts:target}
# otherwise, assume this shell is running task and add an artificial delay to workaround https://github.com/eclipse-theia/theia/issues/2961 template-replicated-destination = ${template-replicated:target}
else: time.sleep(1) import-list = file parts :template-parts-destination
os.execv('${bash:location}/bin/bash', ['${bash:location}/bin/bash'] + args) file replicated :template-replicated-destination
mode = 0644
[theia-bashrc]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:etc}/$${:_buildout_section_name_}
template =
inline:
# enable bash completion
. ${bash-completion:location}/etc/profile.d/bash_completion.sh
# source user's .bashrc
[ -f ~/.bashrc ] && . ~/.bashrc
depends =
$${shell-setup-completion:recipe}
[shell-setup-completion]
recipe = plone.recipe.command
stop-on-error = true
command =
${buildout:bin-directory}/slapos complete > $${directory:bash-completions}/slapos
${buildout:bin-directory}/slapos complete --shell fish > $${directory:fish-completions}/slapos.fish
# SlapOS Standalone
# -----------------
[slapos-standalone-port]
recipe = slapos.cookbook:free_port
minimum = 4000
maximum = 4100
ip = $${instance-parameter:ipv4-random}
[slapos-standalone-config]
ipv4 = $${instance-parameter:ipv4-random}
ipv6 = $${instance-parameter:ipv6-random}
port = $${slapos-standalone-port:port}
slapos-configuration = $${directory:runner}/etc/slapos.cfg
computer-id = slaprunner
[slapos-standalone-activate]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
export PATH=${buildout:bin-directory}:$PATH
export SLAPOS_CONFIGURATION=$${slapos-standalone-config:slapos-configuration}
export SLAPOS_CLIENT_CONFIGURATION=$SLAPOS_CONFIGURATION
echo 'Standalone SlapOS for computer `$${slapos-standalone-config:computer-id}` activated'
[slapos-standalone]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:bin}/$${:_buildout_section_name_}
mode = 0700
template =
inline:#!/bin/sh
export PATH=${buildout:bin-directory}:$PATH
exec ${slapos-standalone:script-path} \
$${directory:runner} \
$${slapos-standalone-config:ipv4} \
$${slapos-standalone-config:ipv6} \
$${slapos-standalone-config:port} \
$${slapos-standalone-config:computer-id} \
--sr='$${instance-parameter:configuration.embedded-sr}' \
--srtype='$${instance-parameter:configuration.embedded-sr-type}' \
$${slap-connection:server-url} \
$${slap-connection:computer-id} \
$${slap-connection:partition-id} \
--key='$${slap-connection:key-file}' \
--cert='$${slap-connection:cert-file}'
[slapos-standalone-instance]
recipe = slapos.cookbook:wrapper
wrapper-path = $${directory:services}/$${:_buildout_section_name_}
command-line = $${slapos-standalone:rendered}
hash-files =
$${slapos-standalone:rendered}
hostname = $${slapos-standalone-config:ipv4}
port = $${slapos-standalone-config:port}
# Slapos Standalone Autoprocessing
# --------------------------------
[slapos-autorun]
recipe = plone.recipe.command
command =
case $${:autorun} in
( running ) ${buildout:bin-directory}/supervisorctl -c $${:supervisor-conf} start $${:service-name};;
( stopped ) ${buildout:bin-directory}/supervisorctl -c $${:supervisor-conf} stop $${:service-name};;
esac
update-command = $${:command}
service-name = slapos-node-auto
supervisor-conf = $${directory:runner}/etc/supervisord.conf
autorun = $${instance-parameter:configuration.autorun}
# Theia Local Environment Setup
# -----------------------------
[tasks.json]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:dot-theia}/tasks.json
template =
inline:
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
"version": "2.0.0",
"tasks": [
{
"label": "slapos node software",
"detail": "Build all software supplied to the node",
"type": "shell",
"command": "${buildout:bin-directory}/slapos",
"args": [
"node",
"software",
// debug mode can be enabled by commenting out this line:
// "--buildout-debug",
"--all"
],
"options": {
"env": {
"SLAPOS_CONFIGURATION": "$${slapos-standalone-config:slapos-configuration}",
"GIT_EXEC_PATH": ""
}
},
"group": {
"kind": "build",
"isDefault": true
},
"problemMatcher": []
},
{
"label": "slapos node instance",
"detail": "Create all instances requested on the node",
"type": "shell",
"command": "${buildout:bin-directory}/slapos",
"args": [
"node",
"instance",
// debug mode can be enabled by commenting out this line:
// "--buildout-debug",
"--all"
],
"options": {
"env": {
"SLAPOS_CONFIGURATION": "$${slapos-standalone-config:slapos-configuration}",
"GIT_EXEC_PATH": ""
}
},
"problemMatcher": [],
"group": {
"kind": "build",
"isDefault": true
}
}
]
}
[slapos-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/slapos.git
location = $${directory:project}/slapos
branch = 1.0
develop = true
git-executable = ${git:location}/bin/git
[settings.json]
recipe = slapos.recipe.template:jinja2
rendered = $${directory:dot-theia}$${:_buildout_section_name_}
once = $${:rendered}
template =
inline:
{
"files.watcherExclude": {
"**/.eggs/**": true,
"**/.env/**": true,
"**/.git/**": true,
"**/node_modules/**": true,
"$${directory:runner}/**":true,
"$${directory:project}/runner/**":true
}
}
[runner-link]
recipe = slapos.cookbook:symbolic.link
target-directory = $${directory:project}
link-binary = $${directory:runner}
software/theia/software.cfg
View file @ 88fa3070
...@@ -15,6 +15,7 @@ extends = ...@@ -15,6 +15,7 @@ extends =
../../component/fonts/buildout.cfg ../../component/fonts/buildout.cfg
../../stack/slapos.cfg ../../stack/slapos.cfg
../../stack/monitor/buildout.cfg ../../stack/monitor/buildout.cfg
../../stack/resilient/buildout.cfg
../../component/defaults.cfg ../../component/defaults.cfg
./download-plugins.cfg ./download-plugins.cfg
./buildout.hash.cfg ./buildout.hash.cfg
...@@ -22,7 +23,14 @@ extends = ...@@ -22,7 +23,14 @@ extends =
parts = parts =
theia-wrapper theia-wrapper
slapos-cookbook slapos-cookbook
instance instance-theia.cfg.jinja2.in
instance-theia-import.cfg.in
instance-theia-export.cfg.in
instance-theia-resilient.cfg.jinja2
instance.cfg
# XXX: we have to manually add this for resilience
rdiff-backup
# pbs-recipe-egg
# default for slapos-standalone # default for slapos-standalone
shared-part-list = shared-part-list =
...@@ -157,6 +165,15 @@ url = ${:_profile_base_location_}/${:filename} ...@@ -157,6 +165,15 @@ url = ${:_profile_base_location_}/${:filename}
output = ${buildout:parts-directory}/${:_buildout_section_name_} output = ${buildout:parts-directory}/${:_buildout_section_name_}
mode = 0644 mode = 0644
[download-base]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
destination = ${buildout:directory}
download-only = true
ignore-existing = true
output = ${:destination}/${:filename}
mode = 0644
[python-language-server] [python-language-server]
version = 0.19.0 version = 0.19.0
recipe = plone.recipe.command recipe = plone.recipe.command
...@@ -326,6 +343,19 @@ template = ...@@ -326,6 +343,19 @@ template =
exec ${yarn:location}/bin/yarn theia start $@ exec ${yarn:location}/bin/yarn theia start $@
[instance] [instance.cfg]
<= template-base <= template-base
output = ${buildout:directory}/instance.cfg output = ${buildout:directory}/instance.cfg
[instance-theia.cfg.jinja2.in]
<= template-base
output = ${buildout:directory}/instance-theia.cfg.jinja2
[instance-theia-import.cfg.in]
<= download-base
[instance-theia-export.cfg.in]
<= download-base
[instance-theia-resilient.cfg.jinja2]
<= download-base
stack/resilient/README.rst
View file @ 88fa3070
...@@ -253,3 +253,21 @@ We can find 2 kinds of magic keys: ...@@ -253,3 +253,21 @@ We can find 2 kinds of magic keys:
* id : example, in "-sla-2-foo" 2 is the magic key and the parameter will be used for each request with id 2 (in case of kvm: kvm2 and PBS 2) * id : example, in "-sla-2-foo" 2 is the magic key and the parameter will be used for each request with id 2 (in case of kvm: kvm2 and PBS 2)
* nameid : example, in "-sla-kvm2-foo", foo will be used for kvm2 request. Name for pbs is "pbs" -> "-sla-pbs2-foo". * nameid : example, in "-sla-kvm2-foo", foo will be used for kvm2 request. Name for pbs is "pbs" -> "-sla-pbs2-foo".
IMPORTANT NOTE: in case the same foo parameter is asked for the group, the nameid key prevail IMPORTANT NOTE: in case the same foo parameter is asked for the group, the nameid key prevail
Documentation Edits
-------------------
- slap-parameter required by import and export
- export / import template names have changed: template-pbsready-import / template-pbsready-export
- slap-configuration required by switch_softwareype
- resilient jinja required importing parts and replicate
- resilient jinja template requires ext.do
- exporter:wrapper and importer:wrapper required by resilient stack (path of wrapper)
- rdiff-backup
- pbs-recipe-egg ??
Changes
-------
- template-replicated.cfg.in: Add optional leaderReturn option ??? => need to access the parameter after anyway so no
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7