This way on plain ZODB4 the following non-wcfs tests will continue to
pass

   test.py/fs-!wcfs
   test.py/zeo-!wcfs
   test.py/neo-!wcfs

instead of failing as e.g. in here:

   https://nexedijs.erp5.net/#/test_result_module/20211116-123A66706

On plain ZODB4 WCFS-related functionality - which uses zconn_at - will
continue to raise corresponding assertion in WCFS-related tests, as e.g. in

https://nexedijs.erp5.net/#/test_result_module/20211116-123A66706/6

With NEO we were creating test database on /tmp but we were not deleting
it in the end. As the result many /tmp/neo_XXXXXX non-empty directories
were being leaked.

-> Fix it by creating testdb directory outselves and removing it at the
end, similarly to FileStorage and ZEO.

Fixes: 7fc4ec66 (tests: Allow to test with ZEO & NEO ZODB storages)

Fix how unpatched ZODB4 is reported to lack required patch:

Before:

    Traceback (most recent call last):
      File "/home/kirr/src/wendelin/wendelin.core/lib/tests/test_zodb.py", line 251, in test_zconn_at
        assert zconn_at(conn1) == at0
      File "/home/kirr/src/wendelin/wendelin.core/lib/zodb.py", line 162, in zconn_at
        assert 'conn:MVCC-via-loadBefore-only' in ZODB.nxd_patches, \
    AttributeError: 'module' object has no attribute 'nxd_patches'

After:

    Traceback (most recent call last):
      File "/home/kirr/src/wendelin/wendelin.core/lib/tests/test_zodb.py", line 251, in test_zconn_at
        assert zconn_at(conn1) == at0
      File "/home/kirr/src/wendelin/wendelin.core/lib/zodb.py", line 163, in zconn_at
        "ZODB!1")
      File "/home/kirr/src/wendelin/wendelin.core/lib/zodb.py", line 191, in _zassertHasNXDPatch
        (zmajor, patch, details_link))
    AssertionError: ZODB4 is not patched with required Nexedi patch 'conn:MVCC-via-loadBefore-only'
            See ZODB!1 for details

Fixes 1f866c00 (lib/zodb: Teach zconn_at to work on ZODB4).

It is not possible for WCFS to access data of in-RAM storage of another
process. But without explicit explanation the error message is confusing
- it was something like:

    NotImplementedError: don't know how to extract zurl from <ZODB.MappingStorage.MappingStorage object at 0x7f28f04cea10>

which suggests it was just not implemented.

In 3bd82127 (lib/zodb: Add zconn_at draft (ZODB5 only)) we added
zconn_at function to find out as of which state a ZODB connection is
viewing the database. That was ZODB5-only however.

Let's add support for ZODB4 now - by requiring ZODB4-wc2 - a version of
ZODB4 with MVCC backported from ZODB5: ZODB!1

This makes wendelin.core to work on either ZODB5 or ZODB4-wc2, but not
plain ZODB4. However as zconn_at will be used only for WCFS-integration,
non-wcfs mode will continue to work on all ZODB5, ZODB4-wc2 and plain
ZODB4.

ZBigFile + WCFS client integration will use zconn_at to open WCFS
connection that corresponds to ZODB connection.

Preliminary history:

kirr/wendelin.core@1c3b7750    X zconn_at for ZODB4

Add patch to ZODB.Connection to support callback on after database is
closed. ZBigFile + WCFS client integration will use this callback to
close WCFS connection when corresponding ZODB.DB is closed.

Preliminary history:

kirr/wendelin.core@a26d9659    X lib/zodb: Connection += onShutdownCallback

In 959ae2d0 (lib/zodb: Add patch to ZODB.Connection to support callback
on connection DB view change) we added patch for ZODB.Connection to
support callback when database view of the connection changes. At that
time the patch was working for ZODB5 and ZODB4 was TODO.
Let's add support for ZODB4 (both ZODB4 and ZODB4-wc2) now.

As a reminder: ZBigFile + WCFS client integration will use this callback
to keep WCFS connection in sync with ZODB connection.

Preliminary history:

kirr/wendelin.core@533a4cfa     X onResyncCallback for ZODB4

In 6637d216 (lib/zodb: Add zstor_2zurl - way to convert a ZODB storage
into URL to access it) we added zstor_2zurl function to convert a ZODB
storage client object into an URL to access the storage. At that time
the function knew how to understand FileStorage only. Let's add support
for other storages that WCFS will need to support now.

NEO URI scheme matches the one currently used on ZODB/go side. It
semantically needs neoppod!18
to be also applied to NEO/py side, but we do not care for now that that
patch is not merged (yet, or forever) because extracted ZURL is used
only with WCFS which uses NEO/go.

NEO support also depends on custom patch to remember SSL credentials on
NEO Client:

kirr/neo@a2f192cb

Some preliminary history:

kirr/wendelin.core@5cb39463    fixup! X wcfs/zeo started to work locally
kirr/wendelin.core@1cf3b228    X zstor_2zurl += NEO
kirr/wendelin.core@7f8fa32a    X lib/zodb: zstor_2zurl += NEO/SSL support
kirr/wendelin.core@e26524df    X wcfs, lib/zodb: DemoStorage support

lib/tests/testprog/zloadrace.py:90:1 'ZODB.FileStorage.FileStorage' imported but unused

This amends commit c37a989d.

Do what we can do without gdb and then tail to regular segmentation
fault. With core file gdb can still be used, but it is handy if we
already can get traceback of the crash into the log automatically.

TODO better use https://github.com/ianlancetaylor/libbacktrace because
backtrace_symbols often does not provide symbolic information.

We do not do this now because libbacktrace is not always automatically
installed.

Else, e.g. after a failing test, that closed its storage and DB, but not
all Connections, another test, just by starting new transaction, would
invoke synchronization on that unclosed connection, which will try to
access closed storage and likely fail.

Fixes e.g. https://nexedijs.erp5.net/#/test_result_module/20210401-31B27B3D/5

Crash scenariou is the same as described in 5a5ed2c7 (tests: Force-close
ZODB connections in teardown, that testing code forgot to explicitly
close). Only now we try to isolate tests from each other not only for
different modules, but also for tests inside the same module.

The tests verify that there is no concurrency bugs around load,
Connection.open and invalidations. See e.g.

https://github.com/zopefoundation/ZODB/issues/290
https://github.com/zopefoundation/ZEO/issues/155

By including the tests into wendelin.core, we will have CI coverage for
all supported storages (FileStorage, ZEO, NEO), and for all supported
ZODB (currently ZODB4, ZODB4-wc2 and ZODB5).

ZEO5 is know to currently fail zloadrace.
However, even though ZODB#290 was fixed, ZEO5 turned out to also fail on zopenrace:

        def test_zodb_zopenrace():
            # exercises ZODB.Connection + particular storage implementation
    >       zopenrace.main()

    lib/tests/test_zodb.py:382:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    <decorator-gen-1>:2: in main
        ???
    ../../tools/go/pygolang/golang/__init__.py:103: in _
        return f(*argv, **kw)
    lib/tests/testprog/zopenrace.py:115: in main
        test(zstor)
    <decorator-gen-2>:2: in test
        ???
    ../../tools/go/pygolang/golang/__init__.py:103: in _
        return f(*argv, **kw)
    lib/tests/testprog/zopenrace.py:201: in test
        wg.wait()
    golang/_sync.pyx:246: in golang._sync.PyWorkGroup.wait
        ???
    golang/_sync.pyx:226: in golang._sync.PyWorkGroup.go.pyrunf
        ???
    lib/tests/testprog/zopenrace.py:165: in T1
        t1()
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

        def t1():
            transaction.begin()
            zconn = db.open()

            root = zconn.root()
            obj1 = root['obj1']
            obj2 = root['obj2']

            # obj1 - reload it from zstor
            # obj2 - get it from zconn cache
            obj1._p_invalidate()

            # both objects must have the same values
            i1 = obj1.i
            i2 = obj2.i
            if i1 != i2:
    >           raise AssertionError("T1: obj1.i (%d)  !=  obj2.i (%d)" % (i1, i2))
    E           AssertionError: T1: obj1.i (3)  !=  obj2.i (2)

    lib/tests/testprog/zopenrace.py:156: AssertionError

After switching to ZODB >= 4 in the previous commit, we can safely
require zodbtools, because there is now no conflict in between
ZODB3/ZODB eggs.

It's been a while since last ZODB3 3.10.7 release in 2016 and the last
commit in upstream ZODB3 repository (3.10 branch) is from 2017. The
world switched since then to ZODB4 and to ZODB5 after that.

We were still requiring ZODB3, because ZODB3 3.11 egg was just a
dependency on newer ZODB, ZEO, BTrees and persistent; and this way we
could be supporting all ZODB3.10.x and  ZODB4 and ZODB5 via ZODB3.11.

However upcoming Wendelin.core 2, for its proper working, needs MVCC
semantic as implemented in ZODB5. This forces us, even for ZODB4, to
backport non-trivial bits from ZODB5 (see [1]). Maintaining ZODB3
support at this point becomes non-practical, because, to our knowledge,
there is no wendelin.core user that plans to continue using ZODB3
without switching to at least ZODB4 in the near future.

So goodbye ZODB3. Even though ZODB still stays with us, it gives a
feeling similar to [2], because in 2014, when I was myself learning
ZODB, it was through ZODB3 - still at the time when all ZODB bits were
living together in one place.

[1] ZODB!1
[2] https://lists.osuosl.org/pipermail/darcs-users/2008-September/014095.html

DB.close() does `del self.storage`.

https://github.com/zopefoundation/ZODB/blob/5.6.0-14-g0eae10cd0/src/ZODB/DB.py#L646

This way if DB was closed, but some conn(s) were not, it will crash in
teardown as e.g. below:

    _____________ ERROR at teardown of test_bigfile_zblk1_zdata_reuse ______________

        def teardown_module():
    >       testdb.teardown()

    bigfile/tests/test_filezodb.py:58:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    self = <wendelin.lib.testing.TestDB_ZEO object at 0x7fb9c0216350>

        def teardown(self):
            # close connections that test code forgot to close
            for connref, tb in self.connv:
                conn = connref()
                if conn is None:
                    continue
                if not conn.opened:
                    continue    # still alive, but closed
                print("W: testdb: teardown: %s left not closed by test code"
                      "; opened by:\n%s" % (conn, tb), file=sys.stderr)

                db = conn.db()
    >           stor = db.storage
    E           AttributeError: 'DB' object has no attribute 'storage'

    lib/testing.py:217: AttributeError

The fix is simple - don't use db.storage at all, because it is not actually used in that code.

It is hard for people to understand current wording, so let's expand
zconn_at description to additionally explain what it is providing with
second set of words, which, hopefully, lowers potential ambiguity a bit.

/reported-by @jwolf083

Wendelin.core 2 will need to spawn WCFS filesystem server that accesses
the same ZODB database as the program that spawns it. The database
argument passed to WCFS is passed in the form of URL[1,2].  Even though
zodburi provides way to convert an URL into ZODB storage instance, there
is currently no way for reverse operation - to convert ZODB storage
instance into URL to access it(*). So we have to build it by our own.

Provide zstor_2zurl stub that currently works for FileStorage only.
ZEO and NEO support is TODO.

In the future we might want to move this functionality into
zodbtools/py.

[1] https://lab.nexedi.com/nexedi/zodbtools/blob/a2e4dd23/zodbtools/help.py#L27-53
[2] https://lab.nexedi.com/kirr/neo/blob/3d909114/go/zodb/zodbtools/help.go#L25-51

(*) contrary to ZODB/go where this functionality is provided out of the box:
    https://godoc.org/lab.nexedi.com/kirr/neo/go/zodb#IStorage

Wendelin.core 2 will need to hook into when client ZODB.Connection
changes its database view and readjust WCFS-level client connection
accordingly.

ZODB.Connection can change its view on either connection reopen, or even
without reopen on start of new transaction.

This patch implements ZODB.Connection.onResyncCallback for ZODB5 only.

ZODB4 and ZODB3 support is TODO.

For wendelin.core v2 we need a way to know at which particular database
state application-level ZODB connection is viewing the database. Knowing
that state, WCFS client library will interact with WCFS filesystem server
and, in simple terms, request the server to provide data as of that
particular database state.

Contrary to ZODB/go[1] ZODB/py does not provide the functionality to
obtain DB state of connection view, so we have to build it ourselves.
Let us call the function that for a client ZODB connection returns
database state corresponding to its database view as zconn_at.

It is relatively easy to implement zconn_at for ZODB5, since ZODB5
adopted MVCC uniformly and this patch does just that. However even with
ZODB5 currently all released ZODB5 versions have race in
Connection.open() vs invalidations[2], and so the first ZODB5 release
with which zconn_at implemented here will work reliable should be
upcoming ZODB 5.5.2

It is TODO to implement zconn_at for ZODB4 and ZODB3, which organize
things differently.

Please note what would happen if zconn_at gives, even a bit, incorrect
answer: wcfs client will ask wcfs server to provide array data as of
different database state compared to current on-client ZODB connection.
This will result in that data accessed via ZBigArray will _not_
correspond to all other data accessed via regular ZODB mechanism.
It is, in other words, would be a data corruptions.

[1] https://godoc.org/lab.nexedi.com/kirr/neo/go/zodb#Connection
[2] https://github.com/zopefoundation/ZODB/issues/290

This will be needed in the following patches to know how to inject
zconn_at or zconn resync functionality into particular ZODB version.

It was from long-ago marked as "XXX move to common place".

Add package-level documentation to

  - bigfile/file_zodb.py,
  - bigarray/array_zodb.py, and
  - lib/zodb.py

The most interesting read is file_zodb.py .

Slightly improve documenation for functions in a couple of places.

Improving documentation was long overdue and it is improved only slightly by this commit.

We already keep FileStorage test database on /tmp/ and NEO itself (via
neo.tests.functional.NEOCluster) also keeps test data on tmpfs. However
test database for ZEO was created in current directory and was wearing
out SSD unnecessarily.

FIXME zeo_forker currently does not provide API to keep all server files
in particular place. This way server conf and log are still emitted in
current directory, but at least we move data.fs away. Since conf and log
are uniquely named, e.g. server-<ΧΧΧ>.conf and tmpYYY.log, and it was
only that Data.fs was named non-uniquely, by moving Data.fs into unique
per-server place, this also helps with-ZEO tests to execute correctly in
parallel with `tox -p`.

For tests this makes sure that if one test fails, it won't make following
tests fail just because the next test will fail trying to lock test database.

For regular code (demo_zbigarray.py) this is also a good thing to do -
to always close the database irregardless of whether an exception was
raised before program reached end of main.

Pygolang becomes regular - not test only - dependency. Being regular
dependency is currently required only by demo_zbigarray.py, but it will
be also used in upcoming wcfs, so adding pygolang into wendelin.core
dependencies aligns with the plan.

dbclose now uses defer almost everywhere - there are still few places in
tests, where one test function is opening/closing test database multiple
times - those were not (yet ?) converted.

Structured creates view of the array interpreting its minor axis as fully covered by a dtype.

It is similar to arr.view(dtype) + corresponding reshape, but does
not have limitations of ndarray.view(). For example:

  In [1]: a = np.arange(3*3, dtype=np.int32).reshape((3,3))

  In [2]: a
  Out[2]:
  array([[0, 1, 2],
         [3, 4, 5],
         [6, 7, 8]], dtype=int32)

  In [3]: b = a[:2,:2]

  In [4]: b
  Out[4]:
  array([[0, 1],
         [3, 4]], dtype=int32)

  In [5]: dtxy = np.dtype([('x', np.int32), ('y', np.int32)])

  In [6]: dtxy
  Out[6]: dtype([('x', '<i4'), ('y', '<i4')])

  In [7]: b.view(dtxy)
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  <ipython-input-66-af98529aa150> in <module>()
  ----> 1 b.view(dtxy)

  ValueError: To change to a dtype of a different size, the array must be C-contiguous

  In [8]: structured(b, dtxy)
  Out[8]: array([(0, 1), (3, 4)], dtype=[('x', '<i4'), ('y', '<i4')])

Structured always creates view and never copies data.

Here is original context where separately playing with .shape and .dtype
was not enough, since it was creating array copy and OOM'ing the machine:

klaus/wendelin@cbe4938b

We are going to use this code in another place, so move this out to
dommon place as a preparatory step first.

On a related note: Since ArrayRef is generic and quite independent from
BigArray (it only supports it, but equally it supports just other - e.g.
plain arrays), the proper place for it might be also to be lib/xnumpy.py .
We might get to this topic a bit later.

bigfile/tests/test_filezodb.py ........W: testdb: teardown: <Connection at 7f8fe2b43b90> left not closed by test code; opened by:
  ...
  File "/home/kirr/src/wendelin/wendelin.core/bigfile/tests/test_filezodb.py", line 754, in test_bigfile_zblk1_zdata_reuse
    _test_bigfile_zblk1_zdata_reuse()
  File "/home/kirr/src/wendelin/wendelin.core/bigfile/tests/test_filezodb.py", line 759, in _test_bigfile_zblk1_zdata_reuse
    root = dbopen()
  File "/home/kirr/src/wendelin/wendelin.core/bigfile/tests/test_filezodb.py", line 47, in dbopen
    return testdb.dbopen()
  File "/home/kirr/src/wendelin/wendelin.core/lib/testing.py", line 188, in dbopen
    self.connv.append( (weakref.ref(conn), ''.join(traceback.format_stack())) )

lib/tests/test_zodb.py .W: testdb: teardown: <Connection at 7f8fe26f13d0> left not closed by test code; opened by:
  ...
  File "/home/kirr/src/wendelin/wendelin.core/lib/tests/test_zodb.py", line 49, in test_deactivate_btree
    root = dbopen()
  File "/home/kirr/src/wendelin/wendelin.core/lib/tests/test_zodb.py", line 30, in dbopen
    return testdb.dbopen()
  File "/home/kirr/src/wendelin/wendelin.core/lib/testing.py", line 188, in dbopen
    self.connv.append( (weakref.ref(conn), ''.join(traceback.format_stack())) )

If a test forgets to explicitly close ZODB connection it was using, this
connection stays alive in transaction synchronizers (it is a weakset),
and continues to be used on e.g. transaction.commit() when all
synchronizers are invoked. This could lead to crashes like below when
underlying ZODB storage was closed by test module teardown and testing
moved on to another test module:

    $ WENDELIN_CORE_TEST_DB="<neo>" py.test  bigfile/tests/test_filezodb.py::test_bigfile_zblk1_zdata_reuse lib/tests/test_zodb.py
    ======= test session starts ========
    platform linux2 -- Python 2.7.14+, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
    rootdir: /home/kirr/src/wendelin/wendelin.core, inifile:
    collected 2 items

    bigfile/tests/test_filezodb.py .                                     [ 50%]
    lib/tests/test_zodb.py F                                             [100%]

    ______ test_deactivate_btree _______

        def test_deactivate_btree():
            root = dbopen()
            # init btree with many leaf nodes
            leafv = []
            root['btree'] = B = IOBTree()
            for i in range(10000):
                B[i] = xi = XInt(i)
                leafv.append(xi)
    >       transaction.commit()

    lib/tests/test_zodb.py:56:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    ../venv/z5/local/lib/python2.7/site-packages/transaction/_manager.py:131: in commit
        return self.get().commit()
    ../venv/z5/local/lib/python2.7/site-packages/transaction/_transaction.py:316: in commit
        self._synchronizers.map(lambda s: s.afterCompletion(self))
    ../venv/z5/local/lib/python2.7/site-packages/transaction/weakset.py:62: in map
        f(elt)
    ../venv/z5/local/lib/python2.7/site-packages/transaction/_transaction.py:316: in <lambda>
        self._synchronizers.map(lambda s: s.afterCompletion(self))
    ../venv/z5/local/lib/python2.7/site-packages/ZODB/Connection.py:757: in afterCompletion
        self.newTransaction(transaction, False)
    ../venv/z5/local/lib/python2.7/site-packages/ZODB/Connection.py:737: in newTransaction
        invalidated = self._storage.poll_invalidations()
    ../venv/z5/local/lib/python2.7/site-packages/ZODB/mvccadapter.py:131: in poll_invalidations
        self._start = p64(u64(self._storage.lastTransaction()) + 1)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    self = <neo.client.Storage.Storage object at 0x7ffa1be8d410>

        def lastTransaction(self):
            # Used in ZODB unit tests
    >       return self.app.last_tid
    E       AttributeError: 'NoneType' object has no attribute 'last_tid'

    ../../neo/src/lab.nexedi.com/kirr/neo/neo/client/Storage.py:181: AttributeError

where NEO's Storage.app is None because the storage was closed.

----

To avoid such kind of failures make sure TestDB.teardown() always closes
all ZODB connections that were ever opened via TestDB.dbopen().

Add a warning about such force-closing with information about corresponding
connection and code place that created it, so that it is easy to
understand which test needs a fix.

/suggested-by @jm

Since 7fc4ec66 (tests: Allow to test with ZEO & NEO ZODB storages) we
can run the tests with either FileStorage, ZEO or NEO. But ZEO test
adapter started to fail with ZEO5:

	self = <wendelin.lib.testing.TestDB_ZEO object at 0x7f1feb5091d0>

	    def setup(self):
	        port  = self.zeo_forker.get_port()
	        zconf = self.zeo_forker.ZEOConfig(('', port))
	        self.addr, self.adminaddr, self.pid, self.path = \
	>               self.zeo_forker.start_zeo_server(zeo_conf=zconf, port=port)
	E       ValueError: need more than 2 values to unpack

This is because in ZEO5 forker.start_zeo_server() was reworked to return only
addr and stop closure instead of returning all details and relying on caller to
implement stop itself.

Adapt the test to detect ZEO5 and use new calling convention.

This allows e.g. to open `neo://cluster@master?compress=false` - in
other words with using options, which our current simplified opening
code does not support.

Keep old dbstoropen around as the fallback to work when
zodbtools/zodburi are not available, since we still want to try to
support ZODB 3.10.

Relicense to GPLv3+ with wide exception for all Free Software / Open Source projects + Business options.

Nexedi stack is licensed under Free Software licenses with various exceptions
that cover three business cases:

- Free Software
- Proprietary Software
- Rebranding

As long as one intends to develop Free Software based on Nexedi stack, no
license cost is involved. Developing proprietary software based on Nexedi stack
may require a proprietary exception license. Rebranding Nexedi stack is
prohibited unless rebranding license is acquired.

Through this licensing approach, Nexedi expects to encourage Free Software
development without restrictions and at the same time create a framework for
proprietary software to contribute to the long term sustainability of the
Nexedi stack.

Please see https://www.nexedi.com/licensing for details, rationale and options.

13c0c17c (bigfile/zodb: Format #1 which is optimized for small changes)
used BTree to organize ZBlk1 block's chunks and for loadblkdata() added
"TODO we are missing to free internal BTree structures on data load".

#3 besides other
things showed that even when we deactivate ZData objects, we are still
keeping them as ghosts occupying memory and the same for IOBucket
objects.

This all happens because there is no proper way to deactivate whole
btree - including internal buckets objects. And since internal buckets
are not deactivated, they stay in picklecache and thus hold a reference
to ZData objects and ZData objects in turn, even if explicitly
deactivated, stay in memory.

We can fix this all via implementing whole-btree deactivation procedure.

To do so we need to iterate over all btree buckets recursively, but
unfortunately there is no BTree API to access/iterate btree's buckets.
We can however still get reference to first top-level buckets via
gc.get_referents(btree) and then scan buckets further without hacks.

gc.get_referents(btree) is a hack, but

- it works in O(1)  (we only get pointers from btree, not scanning all
  gcable objects and deducing them)
- it works reliable if we filter out non-interesting objects.

So in the end it works.

Before the patch loading more and more ZBlk1 data with objgraph
instrumentation was showing itself like

    #                                    Nobj        δ
    wendelin.bigfile.file_zodb.ZData     7168      +512
    BTrees.IOBTree.IOBucket               238       +17
    BTrees.IOBTree.IOBTree                 14        +1

and after this patch we now have

    BTrees.IOBTree.IOBTree                 14        +1

we cannot remove that "IOBTree + 1", since ZBlk1 is holding direct
reference on it (via .chunktab) and we have to keep ZBlk1 live with
._v_zfile and ._v_zblk set for invalidation to work. "+1 IOBtree" is
however small - 144 bytes per 2M (= 0.006%) so we can neglect that the
same way we neglect keeping ZBlk1 staying live for each block.

i.e. it is ok to copy smaller data into larger buffer.

- not only multiple of 8. We can do it by using uint8 typed arrays, and
it does not hurt performance:

In [1]: from wendelin.lib.mem import bzero, memset, memcpy
In [2]: A = bytearray(2*1024*1024)
In [3]: B = bytearray(2*1024*1024)

        memcpy(B, A)    bzero(A)        memset(A, 0xff)

old:    718 µs          227 µs / 1116   228 µs / 1055 (*)
new:    718 µs          176 µs / 1080   175 µs / 1048

    (*) the second number comes from e.g.

        In [8]: timeit bzero(A)
        The slowest run took 4.63 times longer than the fastest.
        This could mean that an intermediate result is being cached
        10000 loops, best of 3: 228 µs per loop

        so the second number is more realistic and says performance
        stays aproximately the same and only slightly improves.

Mutex lock/unlock should not fail if mutex was correctly initialized/used.

We factored out SIGSEGV block/restore from fileh_dirty_writeout() to all
functions in cb7a7055 (bigfile/virtmem: Block/restore SIGSEGV in
non-pagefault-handling function). The restoration however just sets
whole thread sigmask.

It could be possible that between block/restore calls procmask for other
signals could be changed, and this way - setting procmask directly - we
will overwrite them.

So be careful, and when restoring SIGSEGV mask, touch mask bit for only
that signal.

( we need xsigismember helper to get this done, which is also introduced
  in this patch )