Thanks to Zoltán Lajos Kis for the bug report and test case!

This breaks when rlimit is less.  Unfortunately, valgrind (32 bit x86,
3.7.0.SVN, Ubuntu) fails to set the file limit properly on the test:
reducing it to the obvious getrlimit/setrlimit/getrlimit works fine,
so leaving diagnostics for another day.

This was an early experiment in putting code into CCAN, but TDB is a public
library distributed as part of the Samba sources; there is no reason to
publish it here (especially now the unit tests are also in Samba).

An interesting case came up with the tdb2 code in SAMBA recently.  It's a
public library, and it doesn't want to be dependent on CCAN, but the header
uses cast and typesafe_cb for extra type safety.

A good solution to this is to put dummy versions under !HAVE_CCAN.  Of course,
any CCAN config.h should define HAVE_CCAN.

TDB2 uses the same expansion logic as TDB1, which got factored out
recently.  So update TDB2 to match.

If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

(Syncs with tdb1 SAMBA patch 3a2a755e3380a8f81374009d463cd06161352507)

(Syncs tdb2 with SAMBA tdb1's 664add17757836c5ee98618aef11371f412b6e44)

It's much harder to wrap a 64-bit tdb2 than a 32-bit tdb1, but we should still
take care against bugs.

Also, we should *not* cast the length to a size_t when comparing it to
the stat result, in case size_t is 32 bit.

I came across a tdb which had wrapped to 4G + 4K, and the contents had been
destroyed by processes which thought it only 4k long.  Fix this by checking
on open, and making tdb_oob() check for wrap itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Ported from tdb1 SAMBA commit b64494535dc62f4073fc6302847593ed6e6ec38b)

We allocate a new recovery area by expanding the file.  But if the
recovery area is already at the end of file (as shown in at least one
client case), we can simply expand the record, rather than freeing it
and creating a new one.

If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

Recent real usage case showed this leak when we call opt_usage_and_exit:
we don't bother freeing before exit.  With valgrind, it matters.

See https://github.com/rustyrussell/ccan/pull/1#issuecomment-2856561
for details.
(Reworked with minor cleanups -- Rusty)

Based on patch by Andrey Smirnov <andrew.smirnov@gmail.com>:
    
    There is are certain use-cases when it is necessary to know the offset
    of the member in a structure's memory layout. One such use-case can be
    seen in `ccan/list/list.h' in macros `list_for_each' and
    `list_for_each_safe'. This commit implements said functionality with
    `container_of_var_off' macro.

Renamed container_of_var_off -> container_off_var now we have container_off.

It's convenient to check that the member is the given type, but we can leave
that to the callers.

Just adding `-g' to list of CFLAGS doesn't make gcc to generate debug
information required for macro expansion during debugging. Replacing
it with `-g3 -ggdb' rectifies this.

(configurator.c commit folded by Rusty)

Nice for distributed option declaration.

Also add #include to AUTODATA_TYPE example, so ccanlint doesn't try to
put it inside a function (which works, for this, but gives a warning
about unused variables).

Makes things a bit neater.

eg. "See also" can match "see-also".

Gets rid of generated file.

This is a more portable variant of the ELF section trick.

Very slow, but sometimes you need to know without crashing.

It's much easier to simply say "keep all", and it simplifies the code
quite a bit.

I use --target far more than --timeout.

Rather than using a separate index of tetst to skip, make sure tests
we don't want to run are removed from the graph of tests to start
with.

This means that the "skip" field is only set when we fail a test (and
thus must be reset when we test the next module).

Rather than walking the list of tests manually, use
dgraph_traverse_to() to run the tests.