See merge request nexedi/cloudooo!31

instead of re-implementing our own. Updating to zope.interface 5.4
caused test failures:

      File "cloudooo/handler/ooo/tests/testOooInterface.py", line 90, in testIOdfDocument
        self.assertEquals(IOdfDocument.names(), method_list)
    AssertionError: Lists differ: ['getFile', 'source_format', '... != ['getContentXml', 'parsed_cont...

    First differing element 0:
    'getFile'
    'getContentXml'

    - ['getFile', 'source_format', 'getContentXml', 'parsed_content']
    + ['getContentXml', 'parsed_content', 'source_format', 'getFile']

This is because Interface.name() order is unspecified.

As we could see in the XXX comments in the test, hardcoding the method
names from the interface was not ideal, using the built-in verifyClass
is better because it performs more checks and because error reporting
is also much more detailed.

cf. https://zopeinterface.readthedocs.io/en/latest/verify.html

Over the years (for example commits 89d754f2, 58a72658, a6fc7dfe, etc),
the signature of the implementation loadSettings evolved and the
interface became outdated.

This updates the interface with current implementation, so that we can
use zope.interface.verify.verifyClass in the tests

cloudooo includes its own mimes.types, so that it does not depend on
system configuration and behave the same regardless of the underlying
system, but the embedded mime.types have to be loaded explicitly by
calling utils.loadMimetypeList() by each python process before using
mimetypes module.

This was not done for ooo and x2t handlers, so in practice they were
depending on the system mime.types and the tests were written to expect
the mimetypes from debian 10, but with debian 11 some mimetypes became
different ( for example .bmp extension was guessed as image/x-ms-bmp [1]
on debian 10 and image/bmp on debian 11 [2]),

Theses changes:
 - include mime.types from debian 10 [3], but keeping the extra mimetypes
   for  only-office documents (docy, ppty and xlsy) that were added in
   0bb5fbdc (x2t: add handler, 2016-09-22)
 - change the handlers to call utils.loadMimetypeList(), it was only
   strictly necessary for ooo handler, but do it as well in x2t for
   consistency.
 - adjust some tests in testX2tHandler, because now that we have
   loaded mimetypes database the mimetype of xlsy is returned in
   metadata, so we have "application/x-asc-spreadsheet" and not just "xlsy"
 - add a few more test for xlsy and docy, because the magic based test
   just verify that they are zip files, these new tests also make a few
   assertions on the content of the zip files

[1]: https://salsa.debian.org/debian/mime-support/-/blob/debian/3.62/mime.types#L677
[2]: https://salsa.debian.org/debian/media-types/-/blob/4.0.0/mime.types#L1804
[3]: https://salsa.debian.org/debian/mime-support/-/blob/debian/3.62/mime.types

zope.interface does not use self argument.

Also set a pylint directive not to report false positive regarding
the lack of self argument.

This uses psutil.process_iter to iterate on all processes, but in the meantime
some process might terminate.

As we could observe in the logs:

    2021-04-26 12:53:02 - Cloudooo - DEBUG - Stop Pid - 15816
    Exception in thread Thread-1:
    Traceback (most recent call last):
      File "python2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
        self.run()
      File "cloudooo-repository/cloudooo/handler/ooo/monitor/request.py", line 60, in run
        self.openoffice.restart()
      File "cloudooo-repository/cloudooo/handler/ooo/application/application.py", line 79, in restart
        self.stop()
      File "cloudooo-repository/cloudooo/handler/ooo/application/openoffice.py", line 169, in stop
        self._releaseOpenOfficePort()
      File "cloudooo-repository/cloudooo/handler/ooo/application/openoffice.py", line 122, in _releaseOpenOfficePort
        if process.exe() == join(self.office_binary_path, self._bin_soffice):
      File "psutil-5.8.0-py2.7-linux-x86_64.egg/psutil/__init__.py", line 660, in exe
        exe = self._proc.exe()
      File "psutil-5.8.0-py2.7-linux-x86_64.egg/psutil/_pslinux.py", line 1688, in exe
        raise NoSuchProcess(self.pid, self._name)
    NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=19382)

Update to catch and ignore such exceptions, and minor refactorings such as:
 - Not calculate the path of openoffice binary in the loop, this is not
   supposed to change.
 - Remove reference to lsof in logged message, this does not use lsof.
 - Catch all exceptions.

In python2, mimetypes module was not deterministic and this snippet:

    import mimetypes; print(mimetypes.guess_extension("text/html"))

use to be ".html" on python 2.7.14, but is ".htm" on python 2.7.18

Similarly:

    import mimetypes; print(mimetypes.guess_extension("application/msword"))

was ".doc" on 2.7.14 and ".dot" on 2.7.18 (this was in my observations, this
does not look deterministic as it is iterating on a dict, maybe this behaviour
is not always same)

For html conversion engine, ERP5 expect that the extension for text/html is
.html, not .htm. Some tests also expect that the conversion for word is .doc so
to keep compatibility with extensions used in ERP5 compatibility mode, define
explicitly these two extensions instead of depending on python standard library.

See merge request !25

See merge request !24

Tests passing at
https://nexedi.erp5.net/test_result_module/20200102-6DAB14B9

/reviewed-on !23

/reviewed-on !22

Due to a5157949 cloudooo will retry 10 times before raising thus take this into account
Raise rather than swallow.

Test do pass at
https://nexedi.erp5.net/test_result_module/20191227-20A35E0F

/reviewed-on !21

/reviewed-on !18

using python csv module

latin1 is still supported as a fallback when input csv does not decode
as utf8

Split into several test classes for easier maintainance

no need for test_suite everywhere

The cloudooo test on erp5-master is not able to finish after last erp5.util release: https://nexedijs.erp5.net/#/test_result_module/20180719-86977F42

After fix, there is a cloned test suite and the test is completed: https://nexedijs.erp5.net/#/test_result_module/20180719-781978C2 , but some test fails.

I don't know these particular tests neither if they are failing due to the egg release, but the change in this MR solves the problem with test execution and it is ready to merge.

/reviewed-on !17

(cherry picked from commit e0068547)

(cherry picked from commit 50329148)

/reviewed-on !14

`cloudooo.handler.yformat.handler` was mainly contributed by
Boris Kocherov. Commit 0bb5fbdc
should have been credited to him.

fix backward compatibily + update test