nexedi / slapos.core

Typo fix of commit 7467ef4c

/reviewed-on !87

If there is not changes in slapformat.xml, current slapformat will not dump computer
resources (as well as partition resources).

For compatibility with olders version of slapformat, dump partition resources
if it's not created yet event if the file slapformat.xml didn't change.

/reviewed-on !88

https://docs.python.org/dev/whatsnew/3.2.html python3 warns about not unclosed files and sockets.

Use context managers with files to close them.

Use context manager to close socket to supervisor on python 3. On python 2 these sockets remain not closed as before.

There are a few warnings left, but output starts to be readable again.

/reviewed-on !78

tap_ipv6 option let you chose if you want IPv6 on the taps or not. Some
customers may decide to have only IPv4 inside their VMs.

We put only 1 address on the tap itself (e.g.
2001:67c:1254:e:89:69b0:ffff:ffff/128 instead of
2001:67c:1254:e:89:69b0:ffff:ffff/96) otherwise, when we assign the
address 2001:67c:1254:e:89:69b0::/96 inside the VM, the address has
"dadfailed" status (dad = "Duplicate Address Detection"). The problem
appears only with address "0" (ending in ::) but not with others
addresses. We had 2 solutions:
 * put only one address on the tap (the solution we choose)
 * forbid address "0" inside the VM

The chosen solution has the advantage that the host machine won't
answer the ping on behalf of the VM since the tap has only 1 address
(its own).

/cc @alain.takoudjou

/reviewed-on !86

reorganise a bit the code so that resources that we cleanup in finally:
happens just before the try:

Also, remove the handler in finally case, partitions that where failing
again and again apparently added handler again and again.

by using it getSupervisorRPC in a context manager which on python 3
automatically closes.
This needs fix from https://github.com/Supervisor/supervisor/issues/1184
we assume that this fix is only on python 3.
On python2, we keep this same behavior of not closing socket explicitly
and leaving it to destructors.

Also re-raise a few error cases that were ignored.

They will be closed in __del__, probably soon enough, but it's cleaner
to close explicity. This also stops the ResourceWarning on python3

assertNotEquals is deprecated since 3.2 (see https://docs.python.org/3.7/library/unittest.html#deprecated-aliases)
This is a followup of 175b5788

/reviewed-on !81

`test_one_failing_daemon_in_service_will_bang_with_watchdog` often fail with:

```
======================================================================
FAIL: test_one_failing_daemon_in_service_will_bang_with_watchdog (slapos.tests.slapgrid.TestSlapgridCPWithMasterWatchdog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/slapgrid/slappart9/srv/testnode/bpy/inst/test0-0/parts/slapos.core/slapos/tests/slapgrid.py", line 907, in test_one_failing_daemon_in_service_will_bang_with_watchdog
    'etc', 'software_release', 'worked', '.slapos-retention-lock-delay'])
  File "/srv/slapgrid/slappart9/srv/testnode/bpy/soft/5082e1741ad09c0910ec59bf9feae300/eggs/six-1.11.0-py2.7.egg/six.py", line 673, in assertCountEqual
    return getattr(self, _assertCountEqual)(*args, **kwargs)
AssertionError: Element counts were not equal:
First has 1, Second has 0:  'crashed'
First has 1, Second has 0:  'launched'
```

This test uses a service that will create files `launched` and `crashed` files
when running, and just after telling supervisor to start the service test
inspects the directory content.
If service had time to start, then files are created, otherwise they are not.

Change the service to wait for a delay before creating the files to reduce the
change of race condition here.

/reviewed-on !82

Slapos proxy was not really deleting the software, just removing it from its database but not telling the node that requester requested deletion.

As a result, node was not destroying the software and leaving the software folder around.

These changes are about telling the node that requester wanted destruction of software, so that node really delete the folder.

/reviewed-on !76

Tests are sometimes failing with:

```
======================================================================
FAIL: testForwardToMasterInList (slapos.tests.slapproxy.TestMultiMasterSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/slapgrid/slappart9/srv/testnode/byq/inst/test0-0/parts/slapos.core/slapos/tests/slapproxy/__init__.py", line 1003, in setUp
    self.startExternalProxy()
  File "/srv/slapgrid/slappart9/srv/testnode/byq/inst/test0-0/parts/slapos.core/slapos/tests/slapproxy/__init__.py", line 1050, in startExternalProxy
    self.fail('Could not start external proxy.')
AssertionError: Could not start external proxy.

```

especially on `COMP-2203-3Nodes-ERP5PROJECT2`. I logged on that machine and check the tests, it was really a problem that this slapproxy needs more than the `(20 * .1) = 2` seconds this tests waits before concluding that slap proxy could not start.

Change the waiting loop to wait more each time. With this approach the max waiting time is 20 seconds.

/reviewed-on !83

/reviewed-on !85

this XXX comment no longer apply

now that we are using version 12
Also, rename the dump as "current", so that it's more obvious that
unlike the database_dump_version_10.sql file from the same folder that's
supposed to be a fixed version, this one is for the current version and
needs to be updated each time.

proxy was just removing the software from database when destruction was
requested. Instead, keep it as requested_state = "destroyed" and return
this information to the node, so that nodes properly destroy the
software.

Use logger which is only enabled in debug mode.
Also don't print tips when executing this file, describe it in the
docstring instead.

Especially, don't use None which cause output on the console.

to keep tests silent.

On python2, this uses subprocess32, which backports DEVNULL (and
default setting of closing file descriptors).

it's now configured in tests/__init__.py

When running tests we don't want to clutter the output with logs.
Environment variable SLAPOS_TEST_DEBUG can be set to restore the
verbosity

Instead of printing when passed log=True, always log at DEBUG level,
application can decide what to do with these logs.

/reviewed-on !71

/reviewed-on !70

After c931d60c, default options were changed and we didn't want that.

Rewrite the code so that we have a clearer scheme:
 * all options of format are listed as FormatConfig attributes with default values
 * all values present in configuration file (.cfg) erase the previous options
 * all values present in command line erase the previous options

We use the special default argparse.SUPPRESS that removes totally the arg from arg dict so that only PRESENT options erase something

fix after !62

@rafael currently slapos node boot is broken on all machines with ipv6_interface different from interface_name (e.g. rapid space machines)

/reviewed-on !68