Keeping buildout.cfg and .installed.cfg can help diagnosing test
failures.

Directories were ignored by mistake, we found out that several files
that would help us diagnosing problems were missing, because we only
snapshot etc/* which did not snapshot recursive directories such as
etc/promise or etc/service.

Symlinks were copied (the target was copied), because they matched the
isfile check, but if the symlink target was not existent, this caused
errors.

Also rename _snapshot_instance_file to _copySnapshot as it is no longer
treating files only

The leading spaces in the XML will lead the XML to become invalid

/reviewed-on nexedi/slapos.core!153

It is the most used value in most scenarios.

This reverts commit 01e612bf.

This reverts commit 767475e9.

This reverts commit 7a4b44d1.

Let slapos.grid.slapgrid module control the default cli setting for
promise-timeout

Despite PROMISE_TIMEOUT is configurable on module level, the hardcoded value
took precedence. In order to honour it, just pass None while instantiating
the class.

This is the default configuration of most of the deployed slapos.core cases,
so follow it here.

   Save global and public state for further use by monitor. Those files are
   extended by the monitor with extra parameters (like frontends urls), to
   compose the final files.

   This is a port of the code from slapos.toolbox for future
   replacement.

/reviewed-on nexedi/slapos.core!150

This operation is done by the instance itself, updating the timestamp
here will create an infinity call, where the instance is always edited
and it is forced to run.

  At the ERP5 implementation of slapos master this api don't modify the
timestamp (which is not defined by modification date but by an specific
 workflow.

/cc @jerome

/reviewed-on nexedi/slapos.core!151

Now that slapos node instance outputs summary of problems, no need to
pollute the logs so much

This will improve the summary of problems at the end of slapos node
instance.

/reviewed-on nexedi/slapos.core!144

[Test result](https://nexedijs.erp5.net/#/test_result_module?extended_search=%20%22Boxiang-SLAPOS-CORE-FIXUP%22&page=form&view=view)

/reviewed-on nexedi/slapos.core!149

Used to remove shared parts that are no longer referenced.

With shared part installed for example in `/opt/slapgrd/shared/name/HASH`
the heuristic is to assume that if `HASH` is present in a software's
`.installed.cfg`, then this software is using this shared part.
Similar, if `HASH` is present in
`/opt/slapgrid/shared/another/ANOTHER_HASH/.slapos*signature` then share
part another for hash `ANOTHER_HASH` is using the shared part.

This also have some support to recursively look in instances and if they
contain a `slapos.cfg` file, look for software and shared parts to see if
they use the shared part.

In other cases, this shared part can be deleted.

/reviewed-on nexedi/slapos.core!139

python3-wheels and python3-setuptools are installed when we installed
python3-pip only if recommended packages are installed. It's better to
force the installation of what we need.

  This normalizes the result on python2 and python3, and as well, makes the
  result more accurate.

When reinstalling a computer, 'node software' may run before
'node format' has created users, then it fails because it can't set
ownership of the directory it created. The directory already exists
on next runs and before this commit, it didn't fix the ownership.

/reviewed-on nexedi/slapos.core!132

The most important are:
 * sometimes instance was not properly destroyed, apparently we need to retry `slapos node report`
 * if instance is not properly destroy, remove the files ourself and mark test failed
 * we need to see log files and config files from the instance

there's also a few small commits not directly related.

/reviewed-on nexedi/slapos.core!145

/reviewed-on nexedi/slapos.core!143

  Introduce a new command for run promises only, from all partitions. This helps to perform quick
  checks.

  On the tests for ConsumptionReportBase, it contains a port from a test
   on slapos.toolbox done by Alain Takoudjou.

  This may case more memory usage and more 'live' data to handle,
  for this reason, I'm making it configurable

  This new report class is responsable for generate a file (serealising it per pertition),
  instead rely on concurrent calls to the sqlite.

  This is just to clean the methods a bit

  Extend monitor to grab this information, this allow us use slapos node collect more efficiently, and
  prevent others softwares to access directly the database.