slapos.toolbox:28019db4ebbd6391a0e2606fc8d68e57745d0777 commitshttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commits/28019db4ebbd6391a0e2606fc8d68e57745d07772019-03-08T06:30:16+01:00https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/28019db4ebbd6391a0e2606fc8d68e57745d0777runner: API-fy inspectInstance2019-03-08T06:30:16+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0d084fb6d0662e0222012b7b7a2dd89ad6e75c1cpromise.plugin: fix test ipv6_is_faster failing when ping outside is not poss...2019-03-07T11:58:19+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/766937d4e4b6a7c8a4b7ceae35d2b75dfe365f53Release 0.882019-03-06T11:18:40+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/76ad5861c11cdda26fe02ef3c414d92b0400f10b[runner] add logging when we remove all instances inside a webrunner2019-03-06T10:33:59+01:00Thomas Gambierthomas.gambier@nexedi.com
This shall be helpful when debugging why the data disappeared from a webrunner.
/cc <a href="/cedric.leninivin" data-user="4" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Cédric Le Ninivin">@cedric.leninivin</a> <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a> <a href="/Nicolas" data-user="34" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Nicolas Wavrant">@Nicolas</a> <a href="/alain.takoudjou" data-user="3" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Alain Takoudjou">@alain.takoudjou</a> <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/52" data-original="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/52" data-link="false" data-link-reference="true" data-project="78" data-merge-request="2806" data-project-path="nexedi/slapos.toolbox" data-iid="52" data-mr-title="[runner] add logging when we remove all instances inside a webrunner" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!52</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/d7496eaf1301c78fea9c026af60d76aba5d91085[runner] add logging when we remove all instances inside a webrunner.2019-03-05T11:41:40+01:00Thomas Gambierthomas.gambier@nexedi.com
This shall be helpful when debugging why the data disappeared from a
webrunner.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/7344877947ccd7997d4981cc442b657b36714e08[runner] cleanup: do not use list as a variable2019-03-05T11:15:07+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/eeed319f160745a2b2d445c8dd24447f1e392491Release 0.872019-02-28T12:57:14+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/69bd6500ecceb717f1220bb6f2e6561b43bdbe7cmonitor.runpromise: exit with non zero code if a promise has failed2019-02-28T11:45:58+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/35979a0510af6cdb69c39bc87e20e519f70eabfdpromise check_port_listening: port parameter should be int or str, unicode is...2019-02-15T15:34:01+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/9b4a52876650929a40510cee3def416e8224bef5Release 0.862019-02-06T09:54:20+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/7b1cd1364400d105ef7c76c2c86e2e36b9ea42a6promise: disable test check for some promises2019-02-05T18:17:07+01:00Alain Takoudjoualain.takoudjou@nexedi.com
This promises are used in caddy-frontend to check slave frontend status. We don't want to run them everytime buildout will deploy frontend because:
- there is too many slaves, check all slave frontend promises while deploying frontend will slow down slave frontend deployment.
- we don't want to reprocess frontend if one of slave frontend promise is failinghttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/55ad3276db5ffaf4c71d12b3012cc43d123d16a3promise: remove old version of promises used for caddy-frontend2019-02-05T18:17:07+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/98bd30c9d078ea1b0cfeef9176e709c408362c57migrates old promises used in caddy-frontend SR to new format2019-02-05T18:17:07+01:00Alain Takoudjoualain.takoudjou@nexedi.com
First start of promise migration to new python promises format. Promises migrated here are
used in caddy-frontend and monitor stack.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/47b0ebc0978b438725ce57c79693c2726129b3c9release 0.852019-01-09T16:49:29+01:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/f60a1f0f3dd4a49a96a736c19c02ce0fe47c4a57[webrunner] when displaying connection parameters, sort them in alphabetical...2019-01-09T16:48:45+01:00Thomas Gambierthomas.gambier@nexedi.com[webrunner] when displaying connection parameters, sort them in alphabetical order (as done on slapos master)
<a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a> <a href="/jm" data-user="30" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Julien Muchembled">@jm</a> <a href="/vpelletier" data-user="23" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Vincent Pelletier">@vpelletier</a> <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a> <a href="/Nicolas" data-user="34" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Nicolas Wavrant">@Nicolas</a> <a href="/guillaume.hervier" data-user="550" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Guillaume Hervier">@guillaume.hervier</a> <a href="/alain.takoudjou" data-user="3" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Alain Takoudjou">@alain.takoudjou</a>
I tested in my webrunner
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/48" data-original="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/48" data-link="false" data-link-reference="true" data-project="78" data-merge-request="2596" data-project-path="nexedi/slapos.toolbox" data-iid="48" data-mr-title="[webrunner] when displaying connection parameters, sort them in alphabetical order (as done on slapos master)" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!48</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0dc99fa17c654cc4ba8982ec8ba0fa1282a38943longrequest_promise: change to use assertEqual2019-01-09T13:48:42+01:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/5aadeec41316f18c2a19477b36e922cff4023231longrequest_promise: more comprehensible message2019-01-09T13:48:42+01:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/d5b915b660be6b85fa71ee29fe4795833475edf2promise: add check zope longrequest log promise2019-01-09T13:48:42+01:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/cd9a7718ec0bfafa918227fc0dd70b932ee87a1brelease 0.842018-12-13T15:08:45+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0b0a053684b35add43c38065faaf8977d7951e0c[backupserver promise] The cron date is actually in the system's timezone2018-12-11T16:08:34+01:00Thomas Gambierthomas.gambier@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/90cad43dfa6fa3b6996a3814257d856a63c86d5eresiliencytestsuite: try to find a stable state to test promises2018-12-06T02:48:59+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/dc9acdcc7277374bf2fc8caa8079b23260f51208resiliencytestsuite: fix path of promise folders and try to test promises at ...2018-12-05T08:21:50+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/b1fadc16097e008033bb1a2cc39b6b660110930dfixes "resiliencytest: resiliency process isn't OK if some promises fail"2018-12-04T03:42:06+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
Function should return True if everything is OK.
Also add some logginghttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/88e4967756dfe734cb8f77cac18af51ea7bddc67monitor: fix missing timezone in promise tests2018-12-03T17:39:05+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/d9d8dbc36cd10f15006c7635680ca45f2d95550fresiliencytest: resiliency process isn't OK if some promises fail2018-12-03T10:40:58+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0dc7caf568570e25b1cd34ecff313bc193a788beresilient: exclude files using fnmatch instead of regexp2018-11-28T06:31:43+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
As mentionned by <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a> in <a href="/luke/slapos.toolbox/-/commit/2848a965b0380b1fd3d7942ec572b99c0cfec7d8" data-original="2848a965b0380b1fd3d7942ec572b99c0cfec7d8" data-link="false" data-link-reference="false" data-project="956" data-commit="2848a965b0380b1fd3d7942ec572b99c0cfec7d8" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="resilient: exclude files from signature calculation using regex" class="gfm gfm-commit has-tooltip">2848a965</a>
fnmatch allows more natural definitions for path manipulationshttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/47d9bf42f1fabd85673a1ebdcebb91b7e505466cFix the promise for backupserver2018-11-28T05:51:00+01:00Thomas Gambierthomas.gambier@nexedi.com
The promise should not fail if the backup was never launched because the instance just deployed.
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/47" data-original="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/47" data-link="false" data-link-reference="true" data-project="78" data-merge-request="2533" data-project-path="nexedi/slapos.toolbox" data-iid="47" data-mr-title="Fix the promise for backupserver" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos.toolbox!47</a>https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/2848a965b0380b1fd3d7942ec572b99c0cfec7d8resilient: exclude files from signature calculation using regex2018-11-28T03:38:32+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
Thus we have more possibilities in filtering than just excluding
a full directory.https://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/732472b11e7d3d808be07b22fd346a1bbd91dc3fmonitor: add UTC offset on date2018-11-26T17:42:59+01:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/cd545748480c9c16f8a45fbeddbbc459c8982110resilient: writing signatures in the file should be done outside of the loop2018-11-20T07:14:24+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/4037453cef300cbd4ba769f391ba68018ac19c67resilient: fix module name : s/system/sys2018-11-20T07:10:50+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/6cff48df9acf6212d97d3d0bd92a91e1be54c8eerelease 0.832018-11-14T08:29:17+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/a2b8dbbd13c8569d724a5f152393d554c04a846bresilient: do not compute the signature for same file several times2018-11-14T07:22:09+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/56895d5e6340f74a495ef9cbf3b55de3ca562ae8resilient: add script to be used as a backup_identity_file2018-11-14T06:31:36+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
It can exclude directories from the hash computation, which may
be handy for more than one software release.
Its first application will be to exclude logrotated logs from the signature
file, and the zodb backup, as they integrate their own integrity mechanismehttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/c97ea9e7f3266484152e8bf4e991bc984cfe5b68resilient: reproduce better the folder tree of a webrunner2018-11-14T06:31:36+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/32ef27cb30d8b5065f9014aeb80508603c172ebafixup! resilient: create a "utils" library from runner_exporter2018-11-14T06:31:36+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/00b26ae9a411cff60d8b6cbd039bbee682c19f43fixup! resilient: port the export script of the webrunner to python2018-11-14T06:31:36+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/0191b85701d3085375d467cda8733c233bfae0b5resilient: new entry point for slaprunner's post-notification-run2018-11-13T03:50:49+01:00Nicolas Wavrantnicolas.wavrant@nexedi.comhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/2adbc5c13153f5187f28581e2b00df5ba640ab6eresilient: create a "utils" library from runner_exporter2018-11-13T03:38:23+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
To provide uselful functions that can be reused in other scripts
used in the context of webrunner's resiliencyhttps://lab.nexedi.com/nexedi/slapos.toolbox/-/commit/de334909464470e95cc5557740bf32e2065ee831resilient: port the export script of the webrunner to python2018-11-13T03:27:57+01:00Nicolas Wavrantnicolas.wavrant@nexedi.com
Rewrite the [runner-exporter bash script](<a href="https://lab.nexedi.com/nexedi/slapos/blob/2a01586bfe17f46e2ef26ec6e45bef692435ed28/software/slaprunner/template/runner-export.sh.jinja2">https://lab.nexedi.com/nexedi/slapos/blob/2a01586bfe17f46e2ef26ec6e45bef692435ed28/software/slaprunner/template/runner-export.sh.jinja2</a>) in python.
Also parametarize the signature function to not use only sha256sum which can create issues on some systems. This should be a separated commit, but that shouldn't be a blocker.
Write unit tests for the main functions of this exporter script. As every trick in the original bash script were not obvious / understood / found by me, I may have missed some.
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/44" data-original="https://lab.nexedi.com/nexedi/slapos.toolbox/merge_requests/44" data-link="false" data-link-reference="true" data-project="78" data-merge-request="2441" data-project-path="nexedi/slapos.toolbox" data-iid="44" data-mr-title="resilient: port the export script of the webrunner to python" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!44</a>