This adds transactions to tdb2; the code is taken from tdb1 with minimal
modifications, as are the unit 

If we have a write lock and ask for a read lock, that's OK, but not the
other way around.  tdb_nest_lock() allowed both, tdb_allrecord_lock() allowed
neither.

This wasn't fixed when we converted to ccan/opt in 8d706678.

Unfortunately, unistd.h defines optarg, so the compiler didn't catch
this.

This adds chains of free tables: we choose one at random at the start and
we iterate through them when they are exhausted.

Currently there is no code to actually add a new free table, but we test
that we can handle it if we add one in future.

Zones were a bad idea.  They mean we can't simply add stuff to the end
of the file (which transactions relied upon), and there's no good heuristic
in how to size them.

This patch reverts us to a single free table.

We were previously jumping straight from the first bucket to the end.

We were adding 50% to datalen twice, so move it out of adjust_size and
make the callers do it.

We also add a test that the heuristic is working at all.

We don't actually need it.

We can't enlarge the lock without risking deadlock, so tdb_chainlock() can't
simply grab a one-byte lock; it needs to grab the lock we'd need to protect
the hash.

In theory, tdb_chainlock_read() could use a one-byte lock though.

When we're coalescing, we need to drop the lock on the current free list, as
we've enlarged the block and it may now belong in a different list.

Unfortunately (as shown by repeated tdbtorture -n 8) another coalescing run
can do the coalescing while we've dropped the lock.  So for this case, we
use the TDB_COALESCING_MAGIC flag so it doesn't look free.

A new special record marker to indicate coalescing is in progress.

When we find a free block, we need to mark it as used before we drop the
free lock, even though we've removed it from the list.  Otherwise the
coalescing code can find it.

This means handing the information we need down to lock_and_alloc, which
also means we know when we're grabbing a "growing" entry, and can relax
the heuristics about finding a good-sized block in that case.

When coalescing, we check the adjacent entry then lock its free list: we
need to *recheck* after locking, to make sure it's still in that free list.

We actually only need the bottom 5 bits of the hash value, so don't waste
8 bytes passing it.

We're going to want it in get_free() in the next patch, so move it upwards.
Trivial changes, too: add to size before min length check, and rename growing
to want_extra.

This supersedes the previous Fails: section, into a more general set of
lines of form:

      <testname> <option>...

With the special <option> "FAIL" to mean we know we fail this test.
We accept options to valgrind-tests; in particular tdb2 wants
--partial-loads-ok=yes passed to valgrind.

I wanted to see what happened with tdb2's valgrind test (suppressed in the
_info file).

This allows tests (like in tdb2) to run the simpler tests first and build up,
making it easier to spot the root cause of errors.

ccanlint complained about it; it's trivial but fixing it adds one point to 
our score.

ccanlint did this automatically.

Also, add Author and License lines to _info (ccanlint complained).
Now ccanlint can build it as a module.

This makes it closer to compiling under ccanlint.

This means we can pass various compulation tests, but deduct points if
they get warnings.

We want to distinguish between warnings and errors: the first step is to
return the output even if the command doesn't fail.

Test for inserting idempotent header was wrong way around, and we check
all headers at once, rather than finishing after one.

Also, turn - into _ rather than removing.

This means we can see messages even if we don't fail; ie. for compiler warnings
once they are no longer fatal.

This means that various tests have to be more careful in not setting
score->error.

We only grabbed one character at a time from stdin, so "y<return>" answered
the next question as well.

Michael Ellerman noted that ccan/opt has no symlink, so add ccanlint test.

I am told that CentOS 5.3 doesn't like isblank (it's ISO C).

Delio Brignoli points out that check_type fails when HAVE_TYPEOF is 0,
so turn that off here and see what else breaks...

"C has fairly weak typing:" from check_type/_info is not a new section heading!
Enforce that each word in multi-word sections must be caps.