This happens in SAMBA with the TDB_VERSION1, presumably due to a
read-only traverse nested inside a normal traverse (since it doesn't
occur without TDB_VERSION1).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This clarifies the code a little, but also provides a more explicit
mechanism which can be used to debug error handling (by introducing
tdb_err_t and making it a pointer type).

Measuring ccanlint -d ccan/tdb2 on my dual core hyperthread laptop gives:
Before: 5:30
After: 3:31

Lazy cut & paste of tdb names causes failure when we run in parallel.

Measuring ccanlint -d ccan/tdb2 on my dual core hyperthread laptop gives:
Before: 5:59
After: 5:30

Measuring ccanlint -d ccan/tdb2 on my dual core hyperthread laptop gives:
Before: 6:47
After: 5:59

We use the lbalance module to figure out how many to run in parallel.

Reallocing smaller can move the pointer; certainly valgrind detects
it.  Found while chasing intermittant errors on "make fastcheck".

Meets all of the requirements of ccanlint.
Code coverage of the tests covers everything except for a malloc() failure.
Added some xml files for the tests to work on.

Removed the last references of the old buffering method. Should be
stable now. Time to write some proper testing.

tdb_repack() returns an enum TDB_ERROR, whereas
tdb1_transaction_commit is expected to return 0 or -1.

A left-over -1 return; should be returning ecode (probably TDB_ERR_IO).

Since enums and ints are compatible in C, the compiler didn't warn
that our prototypes for these functions disagreed with the
definitions.

These two tdb1-specific tests have been superceded by the TDB_VERSION1
flags used in the main tests.

Don't run tdb_check with failtest on, since it is very slow.  Do the
tdb_check w/ failtest in a new test, but skip MMAP suppression which
doesn't add much and slows down valgrind a lot.

Before this change run-01-new_database took 40 seconds (under
valgrind), after it takes 8 seconds, and run-12-check takes 3
seconds).

The time to run tests under valgrind has become excessive; particularly
the failure tests which fork().   Thus we cut down testing:

1) api-94-repack: reduce from 234 seconds to 2 seconds by cutting
   iterations, despite adding TDB_VERSION1 tests.

2) api-missing-entries: reduce from 17 seconds to under 1 second by
   not checking db inside loop, but at end.

This reduces the total ccanlint time from 729 to 489 seconds.

This reduces compilation time, since these are merely linked with the
pre-built module, rather than recompiling it into the test (which
allows for fancy things like failtest).

This reduces the test compile time down from about 62 seconds to 45
seconds.  Since ccanlint compiles tests three times (once normally,
once with coverage, and once with reduced config.h) this makes a
difference: we go from 780 seconds to 729 seconds.

We track malloc and free, but we didn't catch the free() inside
external_agent, which means that our list of allocations keeps
growing.  Particularly under valgrind, which re-uses memory less than
the glibc allocator.

The TDB1 code's tdb1_find() returns 0 on error; the callers should
not assume that the error means that the entry wasn't found, but use
last_error to determine it.

This was found by looking at how long the failure path testing for
test/run-10-simple-store.c was taking under valgrind, ie:

	valgrind -q ./run-10-simple-store --show-slowest

This change dropped the time for that test from 53 seconds to 19
seconds.

There are some minor changes required, in particular:

1) Make sure lockcheck understands tdb1 allrecord lock upgrades.
2) Handle tdb1 sequence number jumps: various operations increment the
   sequence number twice, especually tdb_append.
3) Don't test fail on unlock, since it gets triggered with traversal on the
   tdb1 backend (we didn't actually ever test this case for tdb2).
4) Move clear_if_first to offset 4, to match tdb1. 

The TDB2 tests are stricter about this; they want every error logged.

We're always allowed to unlock after a fork; by setting the count to 0
before calling the generic unlock function we don't trigger the pid
check.

This reports errors if we fork() while holding a lock, or misuse a tdb
which we have dual-opened.

Rather than rturning TDB_ERR_EINVAL, return TDB_ERR_RDONLY, and log
when we fail due to nesting.

It's actually quite a good fit; we use compare_wrong_bucket for dead
records, which is kind of correct (they should be in the free list).

The tdb1 backend simply returns "true" if there's an error determining
if a tdb needs recovery.  But this leads failtest down a rabbit hole;
it's better to return the error at this case (and makes for better for
diagnostics, since they will come from the first fault, not later in
tdb1_transaction_recover().

This allows us to simulate the old "volatile" flag for tdb1.  It's not
necessary for tdb2.

As this is the last function in tdb1.h, we remove that file.

It's only used in two files, make it static in one and do it manually
in the other.

Move the tdb1_repack() code into the core, make it general, rename to
tdb_repack().

It's generic code: copy database into temporary, wipe it, copy back.

Switch on the TDB_VERSION1 flag.