slapos:2f461da8e99b3443d54c12469084584d7080293a commitshttps://lab.nexedi.com/nexedi/slapos/-/commits/2f461da8e99b3443d54c12469084584d7080293a2021-01-11T17:42:17+01:00https://lab.nexedi.com/nexedi/slapos/-/commit/2f461da8e99b3443d54c12469084584d7080293asoftware/galene: use new insecure option2021-01-11T17:42:17+01:00Thomas Gambierthomas.gambier@nexedi.com
Since Galene is behind our frontend, no need to take care of the HTTPS certificates.https://lab.nexedi.com/nexedi/slapos/-/commit/15388d59c22935eaeb25f9426fd3da01f195448asoftware/galene: increase to version 0.22021-01-11T14:58:13+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/d1a489d186ea1a66e9f53d5d7aef997f131d4465UsePrivilegeSeparation is deprecated2021-01-08T16:37:09+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/010f66ea68a0fd063b5a291e26093586269592f9stack/resilient: fix sshd-graceful after 1e514399b7c3e9e7428935d97e022822ea73...2021-01-08T12:16:43+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/09022d0ad335b02e7651d17ab1f687fc15be0c25Jupyter with Cython+2021-01-07T19:30:24+01:00Xavier Thompsonxavier.thompson@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/880" data-original="nexedi/slapos!880" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4455" data-project-path="nexedi/slapos" data-iid="880" data-mr-title="Jupyter with Cython+" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!880</a>https://lab.nexedi.com/nexedi/slapos/-/commit/78b23c0dea1bd3e9fa769ddd88a66f1f542462e0stack/erp5: Fix LD_PRELOAD extension.2021-01-05T17:46:11+09:00Vincent Pelletiervincent@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/263214b9010c3ab6dd086923678e28711c0c4a16neotest: v↑ NEO, Go1232021-01-03T21:06:46+03:00Kirill Smelkovkirr@nexedi.com
This also pulls in crawshaw.io/sqlite because it is used by lonet.https://lab.nexedi.com/nexedi/slapos/-/commit/732202db1fdd19ab5f58ca5cb5399e10f4573c4fcaddy-frontend: Check how stale-if-error is respected2020-12-31T13:26:43+01:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/34aafd151ca1d6de46164bdef8f8183b51912048caddy-frontend: Improve in-test comment2020-12-31T13:26:43+01:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/7a8951188f3b94082f3f8a7cb715f1e31a250d1dcaddy-frontend: Workaround lost connections to haproxy2020-12-31T13:22:27+01:00Łukasz Nowakluke@nexedi.com
Due to unknown yet reason ATS and Caddy are loosing connections to haproxy,
which results with increased amount of 5xx (about 0.5% more cases).
No known solution was found yet on haproxy side, but this kind of workaround
stabilises frontend and definitely drops amount of in-frontend 5xx. The
observed situation with this workaround is no more 5xx sourcing in frontend
itself.https://lab.nexedi.com/nexedi/slapos/-/commit/a26a440931c2bf00e2e5b011f52df67a017f0733software/cython-test: Remove cython-test SR2020-12-28T21:45:58+01:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/824a1ffaaeec3add5fb6063c831fc0fa9af94648software/jupyter: Add cython+ to the environment2020-12-28T21:45:58+01:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/ab8c0ee5cdd24a65184cd21bf8269fcdd55dcfcasoftware/cythonplus-dev: Add software release for cython+ development2020-12-24T16:59:23+01:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/9d810ee998472e84fa310f246c790f74e29cb4c3Galene (formerly known as SFU)2020-12-24T10:56:47+01:00Thomas Gambierthomas.gambier@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/878" data-original="nexedi/slapos!878" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4445" data-project-path="nexedi/slapos" data-iid="878" data-mr-title="Galene (formerly known as SFU)" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!878</a>https://lab.nexedi.com/nexedi/slapos/-/commit/4692cf40b33c271d75b20b099d57ffbfda0f9410software/galene: Add Galene software in slapos2020-12-24T07:23:47+01:00Alain Takoudjoualain.takoudjou@nexedi.com
See <a href="https://galene.org" rel="nofollow noreferrer noopener" target="_blank">https://galene.org</a>
Use first official version (0.1)
Co-authored-by: <span data-trailer="Co-authored-by:" data-user="737"><a href="https://lab.nexedi.com/tomo" title="thomas.gambier@nexedi.com"><img alt="Thomas Gambier's avatar" src="https://lab.nexedi.com/uploads/-/system/user/avatar/737/avatar.png?width=16" class="avatar s16 avatar-inline" title="Thomas Gambier"></a><a href="https://lab.nexedi.com/tomo" title="thomas.gambier@nexedi.com">Thomas Gambier</a> <<a href="mailto:thomas.gambier@nexedi.com" title="thomas.gambier@nexedi.com">thomas.gambier@nexedi.com</a>></span>https://lab.nexedi.com/nexedi/slapos/-/commit/212eaf048e820754f35e4d27a8c7dddaba5ff774increase buildout version to 2.7.1+slapos0102020-12-24T07:19:21+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/6f9cbbe9b1ae833e828af8d3f2bf251cdb308472component/gcc: Add gcc-10.22020-12-23T15:55:47+01:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/b7b73a33b1f429b33c2acf1562a51075e9b2bcd4software/jupyter: move to python 32020-12-22T11:05:04+01:00Thomas Gambierthomas.gambier@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/868" data-original="nexedi/slapos!868" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4409" data-project-path="nexedi/slapos" data-iid="868" data-mr-title="software/jupyter: move to python 3" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!868</a>https://lab.nexedi.com/nexedi/slapos/-/commit/6acab77dabf21fa1bae0cfea48da77b458d4cd7aERP5 Software Release upgrade test + restart zopes on conf change2020-12-22T06:11:24+01:00Jérome Perrinjerome@nexedi.com
Introduce a new "software release upgrade test", which tests a scenario of keating
an ERP5 instance with an old version of software release and then update it to
current version.
To support this test and because we are using this technique for most of slapos
softwares nowadays, configure zopes to restart automatically when their
configuration change.
See merge request <a href="/nexedi/slapos/-/merge_requests/875" data-original="nexedi/slapos!875" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4430" data-project-path="nexedi/slapos" data-iid="875" data-mr-title="ERP5 Software Release upgrade test + restart zopes on conf change" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!875</a>https://lab.nexedi.com/nexedi/slapos/-/commit/cafa2ec5c244617e8b969df4ec61d44fe8cc65a9software/slapos-sr-testing: run erp5 upgrade test2020-12-21T09:51:46+01:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/8a61ff90e73b82c855728cb2263f7f19a31c8bd6software/erp5: simple SlapOS upgrade test2020-12-21T09:51:45+01:00Jérome Perrinjerome@nexedi.com
Confirms SlapOS ERP5 software release can be updated from
a reference version to current versionhttps://lab.nexedi.com/nexedi/slapos/-/commit/a8271791126afea88a23aba133519c5d3ae33e73stack/erp5: new promise to check software release URL of zope instances2020-12-21T09:51:42+01:00Jérome Perrinjerome@nexedi.com
When root instance is updated to a new software release URL, it will re-request
all the instances with the new software release URL.
To make sure the new root instance does not appear has ready when it is
re-requested with new software release URL, introduce a promise that will check
that the instances requested by the root instance have the same software
release URL. For now we do this only for Zope instances, because they are
stateless and restart automatically on configuration changes, unlike stateful
instances like mariadb or ZEO that we don't restart automatically (yet ?).https://lab.nexedi.com/nexedi/slapos/-/commit/597453227355104863b7d91ca5d804beadd0db8astack/erp5: restart zopes on configuration changes2020-12-21T09:51:27+01:00Jérome Perrinjerome@nexedi.com
We are using this pattern for most of our services since several
months without any issue, so let's also use it for zopes. This
makes automatic upgrade possible.
Also remove "zope running current products" promise, since we
restart we no longer need to check this.https://lab.nexedi.com/nexedi/slapos/-/commit/b15cac5fd8c5eb82041e8b895009d6685a402075software/jupyter: move to python 32020-12-17T10:53:30+01:00Leo-Paul Geneauleo-paul.geneau@nexedi.com
moves python2 release to component/jupyter-py2 because it is required by ERP5
but not maintained anymore
moves to python 3 :
- test/test.py
- jupyter_notebook_config.py.jinja
- ERP5kernel.py
modifies custom.js to create a workaround for events issue
see (<a href="https://github.com/jupyter/notebook/issues/2499" rel="nofollow noreferrer noopener" target="_blank">https://github.com/jupyter/notebook/issues/2499</a>)
upgrades slapos.cookbook version to 1.0.171 to use zero_knowledge recipe
with python 3
adds jupyter partition check in software/erp5/test/test_erp5.pyhttps://lab.nexedi.com/nexedi/slapos/-/commit/fa6bd1770c5aed0941fe36a42a9c19a893e29d88component/userhost: Version up to a05fe5a3a5cb7005351ef4ec41460089f3ce4d0a2020-12-17T12:28:18+09:00Vincent Pelletiervincent@nexedi.com
Fixes compatibility with glibc >= 2.30 .
Update users to new usage pattern.https://lab.nexedi.com/nexedi/slapos/-/commit/d2a9bf5cae3052200103f9e8fc56b4c46e0fa266software/turnserver: fix tests2020-12-16T09:31:45+01:00Thomas Gambierthomas.gambier@gmail.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/bceebc5980dbbd165bc9792d23c323ab217ce368caddy-frontend: Transmit slave reference for rejection report2020-12-15T10:28:03+01:00Łukasz Nowakluke@nexedi.com
It's much easier to find the slave by it's reference than by it's title for
the cluster administrator, so use the reference in this case.
Fixes "caddy-frontend: Add promise for rejected slaves", even if title seemed
more human readable, but it's unusable.https://lab.nexedi.com/nexedi/slapos/-/commit/06382dc4409995e27176099faafb5832f645a9dccaddy-frontend: Silence not important Broken Pipe errors in tests2020-12-15T10:28:00+01:00Łukasz Nowakluke@nexedi.com
Since haproxy does non-HTTP style checks to the python provided backend it
started to emit a lot of logs about broken pipe, which are not important
in context of testing and shall be silenced.https://lab.nexedi.com/nexedi/slapos/-/commit/fca55801a72031cc7362474f8d20fa8205d545ee*: typo instanciate -> instantiate2020-12-14T02:22:28+01:00Jérome Perrinjerome@nexedi.com
See <a href="https://en.wiktionary.org/wiki/instanciated" rel="nofollow noreferrer noopener" target="_blank">https://en.wiktionary.org/wiki/instanciated</a> :
> Misspelling of instantiated.https://lab.nexedi.com/nexedi/slapos/-/commit/bd3c2b184509cafed8d425cb284c580095f9071cRemove httpd from ERP5 and use haproxy instead2020-12-14T02:08:13+01:00Jérome Perrinjerome@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/858" data-original="nexedi/slapos!858" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4365" data-project-path="nexedi/slapos" data-iid="858" data-mr-title="Remove httpd from ERP5 and use haproxy instead" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!858</a>https://lab.nexedi.com/nexedi/slapos/-/commit/aeece80c7c2af7c8afe7620055ec036af25c4fe5version up OpenSSL 1.1.1i, including bug and security fixes.2020-12-10T14:44:17+01:00Kazuhiko SHIOZAKIkazuhiko@nexedi.comhttps://lab.nexedi.com/nexedi/slapos/-/commit/5f2ba34462d9c59a3e605e220547bcbe020f92e3software/powerdns: add multidomain support2020-12-09T20:33:34+01:00Thomas Gambierthomas.gambier@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/863" data-original="nexedi/slapos!863" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4391" data-project-path="nexedi/slapos" data-iid="863" data-mr-title="software/powerdns: add multidomain support" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!863</a>https://lab.nexedi.com/nexedi/slapos/-/commit/362114207a5d4e0b6f1d4387abccfa29ab809e80software/powerdns: add multidomain support2020-12-09T16:41:32+01:00Leo-Paul Geneauleo-paul.geneau@nexedi.com
Allow each instance to support a zone list instead of a single zone.
Each slave is set for only one defined zone.https://lab.nexedi.com/nexedi/slapos/-/commit/4d6ce0238472ad901bf409a3e6c449143592624acaddy-frontend: Fix aibcc and aikc caucase-updater2020-12-09T08:25:58+01:00Łukasz Nowakluke@nexedi.com
While caucase-updater aibcc was added, it was copy & pasted from aikc, and
by mistake it shared data_dir, which results with not working update of
certificates.
This fixes this issue introduced in "caddy-frontend: Setup backend client auth"https://lab.nexedi.com/nexedi/slapos/-/commit/7f87762177cea9c6d89687f07a50eb0fbe79cde0Move wendelin.core from Wendelin to ERP52020-12-07T19:15:47+03:00Kirill Smelkovkirr@nexedi.com
We are starting to use wendelin.core not only in Wendelin context.
So it makes sense to have it installed in base ERP5, like we already do for
example for NumPy and SciPy.
/reviewed-by <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a>, <a href="/jp" data-user="2" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jean-Paul Smets">@jp</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/874" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/874" data-link="false" data-link-reference="true" data-project="15" data-merge-request="4426" data-project-path="nexedi/slapos" data-iid="874" data-mr-title="Move wendelin.core from Wendelin to ERP5" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!874</a>https://lab.nexedi.com/nexedi/slapos/-/commit/ad34ff4acc4c04a239d482fb4dd834864f1716acwendelin.core: Always use git checkout for both release and development version2020-12-07T19:15:42+03:00Kirill Smelkovkirr@nexedi.com
Having only one section [wendelin.core] instead of [wendelin.core] and
[wendelin.core-dev] is easier to handle in "we want to use such and such
particular version" scenarious without deciding in advance whether an SR needs
to inherit from wendelin/software.cfg or wendelin/software-dev.cfg
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/874" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/874" data-link="false" data-link-reference="true" data-project="15" data-merge-request="4426" data-project-path="nexedi/slapos" data-iid="874" data-mr-title="Move wendelin.core from Wendelin to ERP5" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!874</a>https://lab.nexedi.com/nexedi/slapos/-/commit/82a249b6031c719126d996a240d7e655c1f27313stack/erp5: socat wrapper to get haproxy stats2020-12-07T02:20:14+01:00Jérome Perrinjerome@nexedi.com
haproxy can be controlled with this socket, so it might be useful
to "expose" it - it's not really expose because we only use a
UNIX socket.https://lab.nexedi.com/nexedi/slapos/-/commit/06ce6b17dd70f73ff5c7346e9791f1e83f3e8362stack/erp5: leave apachedex reports with wrong microsecond timing for now2020-12-07T02:20:14+01:00Jérome Perrinjerome@nexedi.com
The apachedex reports when produced on backend will be wrong, because haproxy
logs timings in milliseconds and apachedex parses as microsecond, but as far as
I know we produce reports from frontend logs, so it should not really affect
our operations.https://lab.nexedi.com/nexedi/slapos/-/commit/6a8f58c533b25047319d98aef38c3b9ff006b138stack/erp5: remove httpd and use haproxy instead2020-12-07T02:20:13+01:00Jérome Perrinjerome@nexedi.com
Two main differences of haproxy are file format for certificates and logs.
HAProxy also uses certificates in PEM format, but it expect its own server
certificate and the key to be in the same file (although recent version seems
to accept separate files, we don't use this now) and the CRL and CA certificates
also all together in the same file.
We change to use the same file for certificate and key and for CA and CRL, in
the updater script we we build PEM files by containing all CA certificates and
all CRL together.
Also, since haproxy needs to be reloaded when certificate change, we run it in
master-worker mode, with a pid file so that we can signal it to reload.
For the logs, since haproxy does not log to file, we introduce a rsyslogd to
log to a file. The log format is same as with httpd, except that timing are not
in microseconds but in milliseconds - this did not seem to be configurable.
This is a problem for apachedex reports on log, for that we plan to use an
updated version of apachedex with support for `%{ms}T` for durations.
HAProxy is configured with same timeouts, except:
- "connect" timeout has been increased a bit (from 5 to 10s), because the
comment "The connection should be immediate on LAN" was no longer true, now
that haproxy is accessed from frontend.
- the server entries for testrunner are a very long timeout (8h) because some
ERP5 functional tests exceeed the 305s timeout.
The SSL configuration is with current "modern" config from <a href="https://ssl-config.mozilla.org/" rel="nofollow noreferrer noopener" target="_blank">https://ssl-config.mozilla.org/</a>
Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
like httpd was doing several haproxy features (keep alive and gzip
compression) are only available when backend uses HTTP/1.1, so we adjusted
tests to use a 1.1 backend.
There was also differences with logs, because of the time being in milliseconds.
TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
was working by chance because when accessed behind httpd->haproxy->zope, zope
was producing a redirect URL that was the URL of haproxy, which could be
resolved by chance. This test was updated to access zope with a path that
contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
type) is supposed to set.
This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.https://lab.nexedi.com/nexedi/slapos/-/commit/ec79e38490388ca9e7e3c853a5e25e3be65c6dd6software/erp5/test: remove PYTHONPATH hack2020-12-04T05:54:57+01:00Jérome Perrinjerome@nexedi.com
we no longer run tests with `python setup.py test`, so this hack is no
longer necessary.
See merge request <a href="/nexedi/slapos/-/merge_requests/871" data-original="nexedi/slapos!871" data-link="false" data-link-reference="false" data-project="15" data-merge-request="4416" data-project-path="nexedi/slapos" data-iid="871" data-mr-title="software/erp5/test: remove PYTHONPATH hack" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!871</a>