This reduces compilation time, since these are merely linked with the
pre-built module, rather than recompiling it into the test (which
allows for fancy things like failtest).

This reduces the test compile time down from about 62 seconds to 45
seconds.  Since ccanlint compiles tests three times (once normally,
once with coverage, and once with reduced config.h) this makes a
difference: we go from 780 seconds to 729 seconds.

We track malloc and free, but we didn't catch the free() inside
external_agent, which means that our list of allocations keeps
growing.  Particularly under valgrind, which re-uses memory less than
the glibc allocator.

The TDB1 code's tdb1_find() returns 0 on error; the callers should
not assume that the error means that the entry wasn't found, but use
last_error to determine it.

This was found by looking at how long the failure path testing for
test/run-10-simple-store.c was taking under valgrind, ie:

	valgrind -q ./run-10-simple-store --show-slowest

This change dropped the time for that test from 53 seconds to 19
seconds.

There are some minor changes required, in particular:

1) Make sure lockcheck understands tdb1 allrecord lock upgrades.
2) Handle tdb1 sequence number jumps: various operations increment the
   sequence number twice, especually tdb_append.
3) Don't test fail on unlock, since it gets triggered with traversal on the
   tdb1 backend (we didn't actually ever test this case for tdb2).
4) Move clear_if_first to offset 4, to match tdb1. 

The TDB2 tests are stricter about this; they want every error logged.

We're always allowed to unlock after a fork; by setting the count to 0
before calling the generic unlock function we don't trigger the pid
check.

This reports errors if we fork() while holding a lock, or misuse a tdb
which we have dual-opened.

Rather than rturning TDB_ERR_EINVAL, return TDB_ERR_RDONLY, and log
when we fail due to nesting.

It's actually quite a good fit; we use compare_wrong_bucket for dead
records, which is kind of correct (they should be in the free list).

The tdb1 backend simply returns "true" if there's an error determining
if a tdb needs recovery.  But this leads failtest down a rabbit hole;
it's better to return the error at this case (and makes for better for
diagnostics, since they will come from the first fault, not later in
tdb1_transaction_recover().

This allows us to simulate the old "volatile" flag for tdb1.  It's not
necessary for tdb2.

As this is the last function in tdb1.h, we remove that file.

It's only used in two files, make it static in one and do it manually
in the other.

Move the tdb1_repack() code into the core, make it general, rename to
tdb_repack().

It's generic code: copy database into temporary, wipe it, copy back.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.  Also, change tdb1_check's checkfn argument
to return an error code (and set tdb->last_error accordingly).

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag; we now only do a tdb1_traverse_read on a
read-only database, as there is no tdb2 equivalent.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Switch on the TDB_VERSION1 flag.

Finally, we gut tdb1_open() to the tdb1-specific parts, and call it
from tdb_open if they specify the TDB_VERSION1 flag or the version is
a TDB1.

We also unify tdb_close(), based on the TDB_VERSION1 flag.

Note that tdb_open(TDB_VERSION1) will fail on an existing tdb if it's
a TDB2.

It also only makes sense with O_CREAT.

This replaces the tdb1_open "hash_size" argument.  It will only be
valid when you call tdb_open() with O_CREAT in open_flags, and
TDB_VERSION1 in tdb_flags.

This brings it closer to tdb_open(), so we can unify more easily.

We're going to need access to it from tdb1_open, so expose it now.
It's better in hash.c anyway.

This is a percursor to unifying with tdb_open() from tdb2, which handles
this itself.

Finally, we split out the tdb2-specific parts of tdb_context, and put
them into a "tdb2" sub-struct; the tdb1 parts go into a "tdb1"
sub-struct.  We get rido of tdb1_context and use tdb_context
everywhere.

There's also a semantic change here: for tdb1, being read-only meant no
locking, and it was an error to try to lock a r/o database.  For TDB2, you'd
need to specify TDB_NOLOCK, which suppresses locking silently.

This means converting the tdb1 inbuilt hash functions to the
tdb2-style, so they return 64 bit.  We truncate to 32 bit everywhere
but in tdb_check() which needs to do so explicitly.

In TDB2, the user can override locking functions, so they may
deliberarely fail (eg. be non-blocking) expecting to retry.

For this reason, the TDB2 API requires the caller to cancel the
transaction if tdb_transaction_prepare_commit() fails.

The TDB1 code logs multiple times on errors; we must prevent that in
the limited case where locking fails.  With TDB2, this can happen due
to the lock function attribute, where the user supplies replacement
locking functions which are allowed to return with errno EAGAIN or
EINTR for various special-effects.  Flooding the logs for this is
unfriendly.