slapos:3522a7da9c6fa73ce0428b76cdaa06e73e9e6dfc commitshttps://lab.nexedi.com/tomo/slapos/-/commits/3522a7da9c6fa73ce0428b76cdaa06e73e9e6dfc2021-12-13T06:10:41+01:00https://lab.nexedi.com/tomo/slapos/-/commit/3522a7da9c6fa73ce0428b76cdaa06e73e9e6dfcpre-commit hook: version up and python3 support2021-12-13T06:10:41+01:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/700655a2a03e1b4129b0eca4609075a743495edbupdate-hash: python3 support2021-12-12T22:53:41+09:00Jérome Perrinjerome@nexedi.com
also mention automatic installation methodhttps://lab.nexedi.com/tomo/slapos/-/commit/ede3f1a160866b155aeadc6083e16038fd558218format-json: python3 support2021-12-12T22:38:34+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/5442c76f01800ab39247cd6614e566a8959607e9software/theia: use nodejs 14, remove explicit python2.72021-12-12T22:37:42+09:00Jérome Perrinjerome@nexedi.com
Since <a href="https://github.com/eclipse-theia/theia/pull/10343" rel="nofollow noreferrer noopener" target="_blank">https://github.com/eclipse-theia/theia/pull/10343</a> theia depend on
nodejs >= 12 and they test on 12 and 14. Use 14 to have an officially
tested version.
This removes the strong requirement to have python2.7 in theia, so we
just use the software python, which might be python3https://lab.nexedi.com/tomo/slapos/-/commit/7fdbbebb95b7095ccc89488f038b0a39ad6883b9software/metabase: version up metabase 0.41.42021-12-12T22:36:19+09:00Jérome Perrinjerome@nexedi.com
<a href="https://github.com/metabase/metabase/releases/tag/v0.41.4" rel="nofollow noreferrer noopener" target="_blank">https://github.com/metabase/metabase/releases/tag/v0.41.4</a>https://lab.nexedi.com/tomo/slapos/-/commit/7477ed7a33c9f7c042e4ba39207702361b7270e9stack/slapos.cfg: version up Werkzeug 2.0.2 for python3 only2021-12-12T21:59:00+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/6ad7022a7430a4c745cc3c9364bad2e7122a107astack/slapos: version up slapos.core 1.7.12021-12-12T21:59:00+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/97f4a5fb2fee1c97b32581899d8f2b13b264d935software/proftpd/test: relax assertion on TestBanLog2021-12-12T21:59:00+09:00Jérome Perrinjerome@nexedi.com
We observed some test failures like for example 1 where this assertion
was failing because message is `Error reading SSH protocol banner`.
Because the purpose of this specific test is to check that log rotation
of ban log work as expected and because we test more thoroughly the
ban itself in TestBan.test_client_are_banned_after_5_wrong_passwords, we
can simplify this test by just asserting that connection was refused, by
expecting a general exception. We don't care about the details of the
exception here.
After a ban, the first connection attemps seem to always be refused with
"Connection reset by peer" and that's why we did not observed failures
with TestBan.test_client_are_banned_after_5_wrong_passwords
[1]: <a href="https://nexedijs.erp5.net/#/test_result_module/20211208-1520AC26C/21" rel="nofollow noreferrer noopener" target="_blank">https://nexedijs.erp5.net/#/test_result_module/20211208-1520AC26C/21</a>https://lab.nexedi.com/tomo/slapos/-/commit/aa6f42e2075867682a0869e5ab110598e4ac42a9fixup! component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/m...2021-12-10T16:08:00+01:00Kirill Smelkovkirr@nexedi.com
Fix the following build failure in nxdtest's own test:
[2021-12-10 15:57:15,142] INFO While:
[2021-12-10 15:57:15,142] INFO Installing.
[2021-12-10 15:57:15,142] INFO Getting section python-interpreter.
[2021-12-10 15:57:15,142] INFO Initializing section python-interpreter.
[2021-12-10 15:57:15,142] INFO Getting option python-interpreter:eggs.
[2021-12-10 15:57:15,142] INFO Getting section nxdtest.
[2021-12-10 15:57:15,142] INFO Initializing section nxdtest.
[2021-12-10 15:57:15,142] INFO Getting option nxdtest:eggs.
[2021-12-10 15:57:15,142] INFO Getting section .nxdtest.pyexe.
[2021-12-10 15:57:15,142] INFO Initializing section .nxdtest.pyexe.
[2021-12-10 15:57:15,142] INFO Getting option .nxdtest.pyexe:eggs.
[2021-12-10 15:57:15,142] INFO Getting option nxdtest:eggs.
[2021-12-10 15:57:15,142] INFO Getting option .nxdtest.pyexe:eggs.
[2021-12-10 15:57:15,142] INFO Error: Circular reference in substitutions.
This is hot, not a proper, fix to recover nxdtest.UnitTest-Master status.
I will think more calmly what to properly do.https://lab.nexedi.com/tomo/slapos/-/commit/71ced145fc6789cf85a59046de3678d6304b41e4golang += patches to fix tests under user namespaces2021-12-10T14:22:48+01:00Kirill Smelkovkirr@nexedi.com
If we enter user namespace via regular unshare without help from SUID
newuidmap/newgidmap, all supplementary groups are mapped to -1. As the result
when Go test tries to chown to a supplementary group, it gets EINVAL:
<a href="https://github.com/golang/go/issues/42525" rel="nofollow noreferrer noopener" target="_blank">https://github.com/golang/go/issues/42525</a>
-> work it around with patch to skip this chown tests.
A more proper, longer-term fix would be to fix Linux kernel to allow writes to
/proc/self/gid_map to setup mapping not only to original gid, but to all
original supplementary groups as well here:
<a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/user_namespace.c?id=v5.16-rc4-0-g0fcfb00b28c0#n1143" rel="nofollow noreferrer noopener" target="_blank">https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/user_namespace.c?id=v5.16-rc4-0-g0fcfb00b28c0#n1143</a>
this fix, even if accepted by upstream, would be long to be waited for to
propagate to distribution kernels that we currently use. So we go with this
workaround for now.
--------
Another patch is to fix the following TestSCMCredentials failure:
=== RUN TestSCMCredentials
creds_test.go:81: WriteMsgUnix failed with invalid argument, want EPERM
--- FAIL: TestSCMCredentials (0.00s)
There the code tries to send uid0/gid0 credentials from non-zero uid and
expects EPERM reject from kernel. However under `unshare -Umc` uid0/gid0 are
not mapped to anywhere and so implicitly map to -1 and are rejected with EINVAL
by the kernel.
/reviewed-by <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-link="false" data-link-reference="true" data-project="15" data-merge-request="5273" data-project-path="nexedi/slapos" data-iid="1095" data-mr-title="component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!1095</a>https://lab.nexedi.com/tomo/slapos/-/commit/0fcadfbd9b8b793ed63d21432359c02e7fc0d490component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_re...2021-12-10T14:22:41+01:00Kirill Smelkovkirr@nexedi.com
4) Hook in python-prctl, as it becomes nxdtest dependency: see
<a href="https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13/diffs?commit_id=79d13eff8820cc3797e0a186025f7f1e7a0132fa" data-original="https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13/diffs?commit_id=79d13eff8820cc3797e0a186025f7f1e7a0132fa" data-link="false" data-link-reference="true" data-project="1269" data-merge-request="5271" data-project-path="nexedi/nxdtest" data-iid="13" data-mr-title="Run each testcase with its own /tmp and /dev/shm" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="trun: Deactivate most capabilities before spawning user test" class="gfm gfm-merge_request">nxdtest!13 (<span class="gfm gfm-commit">79d13eff</span>)</a>
/reviewed-by <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-link="false" data-link-reference="true" data-project="15" data-merge-request="5273" data-project-path="nexedi/slapos" data-iid="1095" data-mr-title="component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!1095</a>https://lab.nexedi.com/tomo/slapos/-/commit/e328aa491f15b161d88ab022dd7282091fe55565component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_re...2021-12-10T14:22:09+01:00Kirill Smelkovkirr@nexedi.com
Prepare for upcoming nxdtest changes to run each testcase with its own
/tmp and /dev/shm:
1) put unshare from SlapOS component into $PATH, so that our version is
used even if OS provides /bin/unshare. As <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a> explains we need
features that were added relatively recently and are missing in unshare
on Debian 10: <a href="https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13#note_146752" data-original="https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13#note_146752" data-link="false" data-link-reference="true" data-project="1269" data-merge-request="5271" data-project-path="nexedi/nxdtest" data-iid="13" data-mr-title="Run each testcase with its own /tmp and /dev/shm" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nxdtest!13 (comment 146752)</a>
It is anyway better to "isolate" from OS by using our own component
instead of system-provided one.
Correspondingly adjust util-linux to enable unshare in its build.
2) similarly to "1" adjust util-linux to enable mount so that our version
is used instead of /bin/mount. For example on Debian 9, even if we
successfully enter user/mount namespace with `unshare -Umc`
/bin/mount -t tmpfs none /tmp
complains that
mount: only root can use "--types" option
-> Fix it the same way as with unshare by forcing usage of
SlapOS-provided mount.
3) rework how nxdtest script is generated and split it into .nxdtest.pyexe and
nxdtest itself. .nxdtest.pyexe is python interpreter via which nxdtest is run.
This interpreter has all eggs required by nxdtest in sys.path, so that
nxdtest could spawn its trun.py via sys.executable. If we don't care to have
properly setup sys.executable, trun.py will fail when importing any module that
nxdtest.py could already successfully import.
Initially I tried to workaround this issue via adjusting $PYTHONPATH <-
sys.path in main nxdtest script, but <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a> points out that, $PYTHONPATH,
if set, also affects processes that trun.py spawns, which is not good:
<a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095#note_146799" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095#note_146799" data-link="false" data-link-reference="true" data-project="15" data-merge-request="5273" data-project-path="nexedi/slapos" data-iid="1095" data-mr-title="component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!1095 (comment 146799)</a>
-> so fix this via running nxdtest via environment where sys.executable is
properly setup python interpreter with path for all eggs that nxdtest has
access to.
Because we already have half-way workarounds for similar problem in several
places, and because running a script with correctly setup sys.executable is
generally better, I would say it should be a good idea to rework
zc.recipe.egg:scripts to generate all scripts to work this way, but I do not
want to fight about it.
So let's leave this scheme nxdtest-specific for now.
/cc <a href="/tomo" data-user="737" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Thomas Gambier">@tomo</a>
/helped-and-reviewed-by <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/1095" data-link="false" data-link-reference="true" data-project="15" data-merge-request="5273" data-project-path="nexedi/slapos" data-iid="1095" data-mr-title="component/nxdtest: Prepare for https://lab.nexedi.com/nexedi/nxdtest/merge_requests/13" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!1095</a>https://lab.nexedi.com/tomo/slapos/-/commit/07439e2e6a8c39430d5be84ff3d77d2aae4d99c6software/kvm: fix netconfig.sh and ipv6_config.sh scripts2021-12-10T11:52:37+01:00Thomas Gambierthomas.gambier@nexedi.com
When adding a route to an interface not up yet, "ip route" command fails
with:
Error: Device for nexthop is not up.
So we need to up the device before adding the routes.https://lab.nexedi.com/tomo/slapos/-/commit/3836a13feef43adf566635574d931f310254a47cversion up: slapos.recipe.build 0.522021-12-10T10:27:52+01:00Julien Muchembledjm@nexedi.com
This is a regression with 0.48 that prevented parts to be shared depending
on which Python is used to run Python (whether it's version 2 or 3).https://lab.nexedi.com/tomo/slapos/-/commit/3f75a3ff96216d73b0df93ce804644126576fca3ncurses: clean up2021-12-10T10:27:29+01:00Julien Muchembledjm@nexedi.com
The patch was useless since version 6.0:
<a href="http://ncurses.scripts.mit.edu/?p=ncurses.git;h=6a530b46563470c2ca73579d1994a0c8e275dd98" rel="nofollow noreferrer noopener" target="_blank">http://ncurses.scripts.mit.edu/?p=ncurses.git;h=6a530b46563470c2ca73579d1994a0c8e275dd98</a>https://lab.nexedi.com/tomo/slapos/-/commit/7162005a39a936f8b68dc82ab7337ddf360c3f62ruby: drop versions 2.1 & 2.22021-12-10T10:27:29+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/aa44579ff567c44d5174812c0796eb1137f6d692openssh: use newer openssl2021-12-10T10:27:29+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/b9d4836c0a7e5471fe8d47c6ff8ff03d8f4cb883version up: bash 5.1.122021-12-10T10:27:29+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/236b98089859e7206067202cb340148a8beb1f24*: prepare for next slapos.core version2021-12-10T08:51:31+09:00Jérome Perrinjerome@nexedi.com
slapos.core now depends on PyYAML and jsonschemahttps://lab.nexedi.com/tomo/slapos/-/commit/19fe94366b631aa9ba54283011ad75b5674f6367pygolang: v↑ (0.0.8 -> 0.0.9)2021-12-09T04:29:14+03:00Kirill Smelkovkirr@nexedi.com
This upgrade fixes deadlock when new context is created from
already-canceled parent[1]. It also comes with other minor enhancements
and fixes needed e.g. for `zodb restore`[2].
[1] <a href="https://lab.nexedi.com/nexedi/pygolang/commit/58d4cbfe" data-original="https://lab.nexedi.com/nexedi/pygolang/commit/58d4cbfe" data-link="false" data-link-reference="true" data-project="1156" data-commit="58d4cbfe738fd94f1ea3dd7a228300a01971e102" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="context: Fix deadlock when new context is created from already-canceled parent" class="gfm gfm-commit has-tooltip">pygolang@58d4cbfe</a>
[2] <a href="https://pypi.org/project/pygolang/#pygolang-change-history" rel="nofollow noreferrer noopener" target="_blank">https://pypi.org/project/pygolang/#pygolang-change-history</a>https://lab.nexedi.com/tomo/slapos/-/commit/03c5b3116f08ece26d32492e1cf6817267f442ebStop using hexagonit.recipe.download2021-12-08T17:11:33+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/d4a89f53ca33cb1126e425acc27711b215a5c2a7xz-utils: skip redirection to HTTPS2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/5746b12b234e03a40b0a1c7d2e9200f997f20a43Release slapos.cookbook (1.0.220)2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/da8d534cecb4f8a354c0db7140db453dcec6d2a2ocropy: make [ocropy-eng-traineddata] shared2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/191256c56255fc640d122d47494286c47e5d459ahadoop: move stack to unstable2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/1ebb34dab1db38681731b886d33f4ee6d3acbb94Remove obsolete [subversion-1.9] and 'neon' component2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/1691811a6021488d2772527dfae61a13c4438972Move components to unstable: gateone, kerberos, libatlas, python-kerberos2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/d1c7e7577c7c605bfcce29980f6ce4f00ef2cb9anosqltestbed: remove unused recipe, move related components to unstable2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/9f2f397b4a542c8850257a38276778734119b218Remove obsolete components: boa, json-c, memcached, ocropus, oood, xpdf2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.com
json-c, memcached and xpdf are still maintained but current
buildout.cfg are not useful if we want to ship a new version.https://lab.nexedi.com/tomo/slapos/-/commit/ce837c84afb23c58f64debb1c1185436c3044dd8lxc: move component to unstable, remove recipe2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/17b6b7bd5e551bea4492dd3dc977c4bd1394a234sphinx: move component to unstable, remove recipe2021-12-08T17:09:22+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/c1794c82b6f867f5529280b471da05a2644e1074accords: move component to unstable, remove recipe2021-12-08T17:09:21+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/3a4e5da9ca816ae86acefb7ecfe0e0c75aba33a2condor: move component to unstable, remove recipe2021-12-08T17:09:21+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/b30820410d7e19d402dc7e3f92a38de6350a4f77Remove unused slapos.recipe.lampgeneric2021-12-08T17:09:21+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/6240cef3fd78fddf1a5678571bccecc120172fe9hellorina: fix build of [file] on Debian 82021-12-08T16:05:35+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/2f87888dbb18cb317cc06bd2f7123ddfac337ad8vm-img: fix partially Let's Encrypt certificate on old OS due to expired root CA2021-12-08T16:05:35+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/59dff29dfa0940e3dbf264cb10d884ce5771d800vm-img: drop Debian 72021-12-08T16:05:35+01:00Julien Muchembledjm@nexedi.comhttps://lab.nexedi.com/tomo/slapos/-/commit/11535a19c832e5ed91844203b5df07d00cb57f2asoftware/htmlvalidatorserver: restore the default software type2021-12-08T14:58:31+01:00Romain Courteaudromain@nexedi.com
Keep compatibility with the default webrunner configuration.https://lab.nexedi.com/tomo/slapos/-/commit/4fdd40d9628787652af092abcc76b0fee0d633ccrecipe/request: do not propagate the request destroyed state2021-12-08T14:56:07+01:00Romain Courteaudromain@nexedi.com
Prevent unexpected instance deletion.https://lab.nexedi.com/tomo/slapos/-/commit/ef5585ab74ba808c144003500ea83ff46c3accf4KVM promises2021-12-07T12:07:09+01:00Jérome Perrinjerome@nexedi.com
See merge request <a href="/nexedi/slapos/-/merge_requests/1094" data-original="nexedi/slapos!1094" data-link="false" data-link-reference="false" data-project="15" data-merge-request="5266" data-project-path="nexedi/slapos" data-iid="1094" data-mr-title="KVM promises" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!1094</a>