slapos.toolbox:0.124 commitshttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commits/0.1242021-08-25T14:56:19+02:00https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/983b810f933dccc6884ce95b5245f2389d3c425cRelease 0.1242021-08-25T14:56:19+02:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/e734bd5995d2d68837f8da55c643509131940c9apubsub: Fix Python3 incompatibilities2021-08-24T16:56:41+02:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/95726826cdab4b2f022bff2d71696fbb68b3b217Release 0.1232021-07-23T18:17:53+02:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/a0466ad7ec38350807e144f8bce1f29f75bb2d47plugin/check_url_available: rewrite options2021-07-23T18:10:25+02:00Eric Zhengeric.zheng@nexedi.com
The check_url_available promise has been improved with the following
changes:
- The `http_code` parameter has been renamed `http-code` for consistency.
- The `check-secure` parameter has been removed. Users should specify
`http-code = 401` instead.
- The `username` and `password` parameters have been added to optionally
enable HTTP basic authentication.
- The logging output has been changed slightly.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/831b32a641d1a636120fde916925bbebd334922aRelease 0.1222021-06-29T17:30:21+02:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/c7062ceeee1983941dbce6511de22a01e8c89af7promise/plugin: Fix check_socket_listening promise2021-06-29T17:24:34+02:00Xavier Thompsonxavier.thompson@nexedi.com
Accept unicode value for `port` parameterhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/9c8b7a698c7529f04418ebade9a31e76511a849dRelease 0.1212021-06-24T15:03:09+02:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/3b2ca6a52283148707bb3c7853d8ebb5599d8ef3qemuqmpclient: Support qemu 5.x+2021-06-24T15:01:34+02:00Łukasz Nowakluke@nexedi.com
pc.ram key is used for the initial memory, and it can't be removed.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/597ad67a40df72eeaa01d534d7193f483cfeb838Release 0.1202021-06-23T10:32:30+02:00Xavier Thompsonxavier.thompson@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/428870e358a0f7543f59e70ecf3cdb5ab7d12ae3promise/plugin: Add check_socket_listening promise2021-06-22T18:11:33+02:00Xavier Thompsonxavier.thompson@nexedi.com
This promise generalizes the check_port_listening promise to include
AF_UNIX sockets in addition to AF_INET and AF_INET6 host/port sockets.
The socket family and address is deduced from the set of arguments:
- `host` & `port`: AF_INET or AF_INET6
- `pathname`: filesystem pathname address for AF_UNIX
- `abstract`: '\0' is prefixed to build an abstract AF_UNIX address
All other combinations of arguments are forbidden.
The promise creates a socket accordingly and attempts to connect to the
provided address. The promise fails if the connection fails.
The check_port_listening promise has been removed.
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/97" data-original="nexedi/slapos.toolbox!97" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4880" data-project-path="nexedi/slapos.toolbox" data-iid="97" data-mr-title="promise/plugin: Add check_socket_listening promise" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos.toolbox!97</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/b0c5cb3f6258e3d90968a36823be350a133801c0Revert "monitor: delegate argument checking to argparse"2021-06-04T11:08:56+09:00Nicolas Wavrantnicolas.wavrant@nexedi.com
This reverts commit <a href="/nexedi/slapos.toolbox/-/commit/4a992aaaafec3f14627021626be98d55dd999ec1" data-original="4a992aaaafec3f14627021626be98d55dd999ec1" data-link="false" data-link-reference="false" data-project="78" data-commit="4a992aaaafec3f14627021626be98d55dd999ec1" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="monitor: delegate argument checking to argparse" class="gfm gfm-commit has-tooltip">4a992aaa</a>.
runpromise takes arguments from command line, but also read some from
a config file, so indeed it is mandatory to know partition_folder's
value, but it is not mandatory to pass it as an argument from the
command linehttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/a28fa444aa917b4267887f351305c168cd2c6732monitor: do not try to remove pid file if script is run without defining one2021-05-07T16:39:13+09:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/4a992aaaafec3f14627021626be98d55dd999ec1monitor: delegate argument checking to argparse2021-05-07T16:35:14+09:00Nicolas Wavrantnicolas.wavrant@nexedi.com
So instead of a traceback we get a better error message
written by argparse. Also , it will print a better help
messagehttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/ea09102fa990538055f008f06e1349a860ac90e5Release 0.1192021-04-21T10:06:22+09:00Jérome Perrinperrinjerome@gmail.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/07a18130e99b81bda13ee7151573c421c95624c5slaprunner: fix popen on python32021-04-21T02:55:08+02:00Jérome Perrinjerome@nexedi.com
On python3 several views were not functional, because subprocess output was
retrieved as bytes, but manipulated as a string.
This adds a few tests for views, not complete because it would be too much time
consuming for now.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/c284b91953a7533495e413bc6f6f9191e2f656d2Release 0.1182021-03-29T10:26:27+02:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/325f7993e8869653f6efb46c785596b039df39d3Feature/domain2ip2021-03-29T10:22:53+02:00Łukasz Nowakluke@nexedi.com
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/94" data-original="nexedi/slapos.toolbox!94" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4625" data-project-path="nexedi/slapos.toolbox" data-iid="94" data-mr-title="Feature/domain2ip" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!94</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/44c18a7d999b27ac4d8c4e7b4da45a7572cab719dnsresolver: Introduce new tool2021-03-26T11:40:03+01:00Łukasz Nowakluke@nexedi.com
Simple and predictable tool to resolve massively domain names to IPs.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/408aec2b7bdc0fb04a50919d28e7f0a50b156708test_check_service_state: fix a data race2021-02-04T14:45:38+01:00Xavier Thompsonxavier.thompson@nexedi.com
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/93" data-original="nexedi/slapos.toolbox!93" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4540" data-project-path="nexedi/slapos.toolbox" data-iid="93" data-mr-title="test_check_service_state: fix a data race" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!93</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/6fb40ef9d273f2d63e9a6acd211018eb42c0730cRelease 0.1172021-02-03T19:15:43+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/5d5c98b95d628dd16db778f1e3b2f2da0f38cd16promise/plugin: Implement check_service_state promise plugin2021-02-03T10:58:49+01:00Xavier Thompsonxavier.thompson@nexedi.com
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/90" data-original="nexedi/slapos.toolbox!90" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4510" data-project-path="nexedi/slapos.toolbox" data-iid="90" data-mr-title="promise/plugin: Implement check_service_state promise plugin" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!90</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/1eeef169aa6a2bde9b1a83a91685c7e5063971d1test_networkbench: update after ip changed2021-02-02T01:55:12+01:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/72d01e706ae5f25b21b3035f6fe77b1da007608fRelease 0.1162021-02-01T10:29:48+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/dd5a118be161726565fe28da15fdb2976fb07e7erunner: import werkzeug.middleware.proxy_fix.ProxyFix from its new location2021-01-29T10:59:51+01:00Jérome Perrinjerome@nexedi.com
from
<a href="https://werkzeug.palletsprojects.com/en/0.14.x/contrib/fixers/#werkzeug.contrib.fixers.ProxyFix" rel="nofollow noreferrer noopener" target="_blank">https://werkzeug.palletsprojects.com/en/0.14.x/contrib/fixers/#werkzeug.contrib.fixers.ProxyFix</a>
to
<a href="https://werkzeug.palletsprojects.com/en/1.0.x/middleware/proxy_fix/#werkzeug.middleware.proxy_fix.ProxyFix" rel="nofollow noreferrer noopener" target="_blank">https://werkzeug.palletsprojects.com/en/1.0.x/middleware/proxy_fix/#werkzeug.middleware.proxy_fix.ProxyFix</a>
the behavior seems the same, default config in both cases is to trust one
level of proxy.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/bd63a31a3a9036a1f159986533a67ce9103fb3acRelease 0.1152021-01-27T13:54:40+01:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/587c0792a36912fee761a355afb1649901ce1784check_surykatka_json: Minimise message variation2021-01-27T13:52:13+01:00Łukasz Nowakluke@nexedi.com
Monitoring is about states, not exact values, so report the states, as
otherwise it results with bloating databases, very long accesses to results
and long synchronisations.
Instead of many entries like:
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: ERROR IP IP replied in 5.02s which is longer than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: ERROR IP IP replied in 5.03s which is longer than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: ERROR IP IP replied in 5.04s which is longer than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: ERROR IP IP replied in 5.01s which is longer than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in 0.08s which is shorter than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in 1.37s which is shorter than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in 0.09s which is shorter than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in 1.37s which is shorter than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in 0.09s which is shorter than maximum 5.00s
It's minimised to:
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: ERROR IP IP replied in more time than maximum 5.00s
<a href="http://url/" rel="nofollow noreferrer noopener" target="_blank">http://url/</a> : http_query: OK status code 302 on IPs IP elapsed_time: OK IP IP replied in less time than maximum 5.00s
This affects:
* bot_status
* http_query
Obviously some information are removed from the monitor, but anyway they are
just text entries and not parsable. So other approaches shall be used to make
graphs from those values, which is not covered by pure promise status.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/68e5bd44c6d6c07605893524d49a6bbcde304f3fRelease 0.1142021-01-22T14:24:01+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/cc153a6ea14fd2c5d35570fd94b6b0fdc565e374apachedex: expect apachedex options to be passed as a file2021-01-20T06:12:09+01:00Jérome Perrinjerome@nexedi.com
Arguments passed to apachedex are complex, they contain quotes, backslashes,
spaces and they are propagated through several levels of buildout profiles
and scripts.
Simplify all this by relying on argparse ability of passing arguments from
a file. Users of this tool are now expected to generate a config file and
specify the path of this config file as an arguments.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/e1b77dae38b9e028eb5f959a86c2ed206181879dcheck_free_disk_space: Btrfs has no fixed inode limit2021-01-11T19:11:40+01:00Julien Muchembledjm@nexedi.com
This fixes division by 0.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/70293e956af0427340bb1a47c3c68eda12a7b254Release 0.1132020-11-11T09:43:51+09:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/faaf0cd0eedcac326dbdf67a6fc54ab50bab4ea5Fix apachedex wrapper2020-11-11T01:35:04+01:00Jérome Perrinjerome@nexedi.com
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/86" data-original="nexedi/slapos.toolbox!86" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4361" data-project-path="nexedi/slapos.toolbox" data-iid="86" data-mr-title="Fix apachedex wrapper" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos.toolbox!86</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0a1fd4b698762e155925a114782f6f2392563f24apachedex: small cleanups2020-11-10T02:24:48+01:00Jérome Perrinjerome@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0420fe31a418f3941bed0f79f3b029576cd951d8apachedex: fix argument parsing2020-11-10T02:24:48+01:00Jérome Perrinjerome@nexedi.com
Use shlex from standard library which splits arguments properly instead of
implementing our own buggy version.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/840e40bbc01aa7e558ed6b1a6732aa51aa2bbaf5Release 0.1122020-10-20T12:42:34+02:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/1a3485265c76193c22459f80e2a5e6104fcf347dfixup! runner: Fix symlinks on softwareLink2020-10-13T16:14:40+02:00Thomas Gambierthomas.gambier@nexedi.com
fixup for <a href="/eric.zheng/slapos.toolbox/-/commit/ecf144ee4267ece11a6772be309f679d3fd04fee" data-original="ecf144ee4267ece11a6772be309f679d3fd04fee" data-link="false" data-link-reference="false" data-project="1367" data-commit="ecf144ee4267ece11a6772be309f679d3fd04fee" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="runner: Fix symlinks on softwareLink" class="gfm gfm-commit has-tooltip">ecf144ee</a>
current_project variable is mandatoryhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/2699c3bd639f67683f6e5b2a4eac9d1e49012fe3Fix random test failures2020-09-18T07:39:49+02:00Jérome Perrinjerome@nexedi.com
"test_networkbench: increase resolution time tolerance" addresses [this](<a href="https://erp5.nexedi.net/test_result_module/20200911-8C18D791/14" rel="nofollow noreferrer noopener" target="_blank">https://erp5.nexedi.net/test_result_module/20200911-8C18D791/14</a>):
```
======================================================================
FAIL: test_dnsbench_fail (slapos.test.test_networkbench.TestDNSBench)
Test dns failure resolution
----------------------------------------------------------------------
Traceback (most recent call last):
File "/srv/slapgrid/slappart20/srv/testnode/cum/inst/test0-0/parts/slapos.toolbox/slapos/test/test_networkbench.py", line 66, in test_dnsbench_fail
self.assertLess(info[3], 1)
AssertionError: 2.0736539363861084 not less than 1
```
"cpu load plugin: fix test and minor cleanups" addresses [this](<a href="https://erp5.nexedi.net/test_result_module/20200911-C444DB47/13" rel="nofollow noreferrer noopener" target="_blank">https://erp5.nexedi.net/test_result_module/20200911-C444DB47/13</a>):
```
======================================================================
ERROR: test_check_cpu_load_run (slapos.test.promise.plugin.test_check_server_cpu_load.TestCheckServerCPULoad)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/srv/slapgrid/slappart9/srv/testnode/bvi/inst/test0-0/parts/slapos.toolbox/slapos/test/promise/plugin/test_check_server_cpu_load.py", line 48, in test_check_cpu_load_run
self.launcher.run()
File "/srv/slapgrid/slappart9/srv/testnode/bvi/soft/65dbbd3d11daa99cf78c9566a0a4798e/parts/slapos.core-repository/slapos/grid/promise/__init__.py", line 884, in run
failed_promise_output))
PromiseError: Promise 'server-cpu-load-promise.py' failed with output: Error: Promise timed out after 0.5 seconds
```
Some more fixes are in <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/809" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/809" data-link="false" data-link-reference="true" data-project="15" data-merge-request="4203" data-project-path="nexedi/slapos" data-iid="809" data-mr-title="software/slapos-testing: include rsync for slapos.toolbox tests" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">slapos!809</a>
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/85" data-original="nexedi/slapos.toolbox!85" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4204" data-project-path="nexedi/slapos.toolbox" data-iid="85" data-mr-title="Fix random test failures" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!85</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/ca67bbc000012850ba9789f83e51c48a33f6d832resiliencytest: wait longer before some operations2020-09-15T03:33:01+02:00Jérome Perrinjerome@nexedi.com
Sometimes it's not enough, as we can guess from the tests failures.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/a015a2e672eb6ebbd0de0ceab1fb2c968171ab21Release 0.1112020-09-14T12:55:58+02:00Łukasz Nowakluke@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/69bf449fe6b275925366d1eec10a3edbc915cfa2support checking the http response headers2020-09-14T10:19:02+02:00Łukasz Nowakluke@nexedi.com
See merge request <a href="/nexedi/slapos.toolbox/-/merge_requests/84" data-original="nexedi/slapos.toolbox!84" data-link="false" data-link-reference="false" data-project="78" data-merge-request="4188" data-project-path="nexedi/slapos.toolbox" data-iid="84" data-mr-title="support checking the http response headers" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!84</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/35b56408e8676e65ecf79c3ec1c67e08ba565c25test_networkbench: increase resolution time tolerance2020-09-14T02:49:45+02:00Jérome Perrinjerome@nexedi.com
We don't really care how long DNS resolution took here, since we see
cases where on test machines a name resolution takes more than one second,
increase the minimal time to something higher to prevent such failures.
Also remove not used commented out code.