Manaully, because there is no automatic dependency tracking in
setuptools...

Dependency tracking is needed to avoid miscompilation after incremental
update under SlapOS/buildout/testnode/... when e.g. only .h was changed.

Tests inside wcfs/ care to do this, but e.g. test.py/fs-wcfs autospawns
wcfs servers during regular bigfile tests. If we don't stop spawned
wcfs, those processes will leak, and also they keep `nxdtest
test.py/*-wcfs` in "hung" state, because nxdtest is waiting for wcfs to
stop as wcfs stdout is connected to nxdtest input.

Currently kills wcfs in abrupt way, because graceful pinner shutdown is
not yet implemented there.

tWCFS is responsible for starting/mounting/unmounting/stopping wcfs
tDB uses tWCFS and provides commit/test service on top.

We'll use tWCFS in the next patch to unmount/stop WCFS processes that
are automatically spawned during test.py

Once WCFS instance is created, use wc.mountpoint to refer to where this
wcfs is mounted. It does not change anything right now, but in a
follow-up patches we'll reuse the code from wcfs_test to work on any wc,
not neccessarily mounted on testmntpt.

Else, when runing tests intree `import wcfs` and `import wendelin.wcfs`
will give two different modules, and inspecting e.g. wendelin.wcfs at
teardown will see fresh module state (_wcregistry) because it was wcfs
which was used.

Also just `import wcfs` will raise ImportError when run out of tree.

Starting from 3f83469c Conn and WatchLink started to inherit from
interface, which made them to use virtual functions, which, without
destructor being also virtual emits the following warnings:

    wcfs/client/wcfs.cpp: In member function ‘virtual void wcfs::_Conn::decref()’:
    wcfs/client/wcfs.cpp:1531:16: warning: deleting object of polymorphic class type ‘wcfs::_Conn’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
             delete this;
                    ^~~~

    wcfs/client/wcfs_watchlink.cpp: In member function ‘virtual void wcfs::_WatchLink::decref()’:
    wcfs/client/wcfs_watchlink.cpp:514:16: warning: deleting object of polymorphic class type ‘wcfs::_WatchLink’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
             delete this;
                    ^~~~

FUSE puts X as st_dev's minor, which, for minors <= 255 is the same as st_dev.
However when there are many connections, and minor goes after 255, minor becomes != st_dev:

    In [2]: os.makedev(0, 254)
    Out[2]: 254

    In [3]: os.makedev(0, 255)
    Out[3]: 255

    In [5]: os.makedev(0, 256)
    Out[5]: 1048576

As a result we were constructing wrong patch, and if wcfs was failing we were
also failing to kill it with something like:

    t = <wcfs.wcfs_test.tDB object at 0x7fef78043260>

        @func
        def __init__(t):
            t.root = testdb.dbopen()
            def _(): # close/unlock db if __init__ fails
                exc = sys.exc_info()[1]
                if exc is not None:
                    dbclose(t.root)
            defer(_)

            assert not os.path.exists(testmntpt)
            t.wc = wcfs.join(testzurl, autostart=True)
            assert os.path.exists(testmntpt)
            assert is_mountpoint(testmntpt)

            # force-unmount wcfs on timeout to unstuck current test and let it fail.
            # Force-unmount can be done reliably only by writing into
            # /sys/fs/fuse/connections/<X>/abort. For everything else there are
            # cases, when wcfs, even after receiving `kill -9`, will be stuck in kernel.
            # ( git.kernel.org/linus/a131de0a482a makes in-kernel FUSE client to
            #   still wait for request completion even after fatal signal )
    >       t._wcfuseabort   = open("/sys/fs/fuse/connections/%d/abort" % os.stat(testmntpt).st_dev, "w")
    E       IOError: [Errno 2] No such file or directory: '/sys/fs/fuse/connections/2097264/abort'

    wcfs/wcfs_test.py:236: IOError

In the above failure st_dev=2097264 corresponds to X=624:

    In [6]: os.minor(2097264)
    Out[6]: 624

Without special care a forked child may interfere in parent-wcfs
exchange via Python GC -> PyFileH.__del__ -> FileH.close -> message to
WCFS sent from the child. This actually happens for real when running
test.py/neo-wcfs because NEO test cluster spawns master and storage
nodes with just fork without exec.

-> detach from wcfs in child right after fork and deactivate all
mappings in order not to provide stale data. See top-level comments
added to wcfs/client/wcfs.cpp for details.

Currently in wcfs_test.py there is only waiting for a proc
(subprocess.Popen instance) to become ready. However in the next patch
we'll need to wait via polling for another condition.

-> Generalize the pollwait code into waitfor* variants, and make
procwait* use waitfor* internally.

Currently the code to convert `int err` or errno into string is usde
only in _pathError, but in the next patches we'll need it to also handle
error from pthread_atfork. -> Factor-out to separate function.

ZEO4 can have ._addr as either ("host", port) or "path".

People don't want to raise up soft defaults
nexedi/slapos.package!130 (comment 118373)
nexedi/slapos.package!130 (comment 118405)