If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

(Syncs with tdb1 SAMBA patch 3a2a755e3380a8f81374009d463cd06161352507)

(Syncs tdb2 with SAMBA tdb1's 664add17757836c5ee98618aef11371f412b6e44)

It's much harder to wrap a 64-bit tdb2 than a 32-bit tdb1, but we should still
take care against bugs.

Also, we should *not* cast the length to a size_t when comparing it to
the stat result, in case size_t is 32 bit.

I came across a tdb which had wrapped to 4G + 4K, and the contents had been
destroyed by processes which thought it only 4k long.  Fix this by checking
on open, and making tdb_oob() check for wrap itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Ported from tdb1 SAMBA commit b64494535dc62f4073fc6302847593ed6e6ec38b)

We allocate a new recovery area by expanding the file.  But if the
recovery area is already at the end of file (as shown in at least one
client case), we can simply expand the record, rather than freeing it
and creating a new one.

If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

Recent real usage case showed this leak when we call opt_usage_and_exit:
we don't bother freeing before exit.  With valgrind, it matters.

See https://github.com/rustyrussell/ccan/pull/1#issuecomment-2856561
for details.
(Reworked with minor cleanups -- Rusty)

Based on patch by Andrey Smirnov <andrew.smirnov@gmail.com>:
    
    There is are certain use-cases when it is necessary to know the offset
    of the member in a structure's memory layout. One such use-case can be
    seen in `ccan/list/list.h' in macros `list_for_each' and
    `list_for_each_safe'. This commit implements said functionality with
    `container_of_var_off' macro.

Renamed container_of_var_off -> container_off_var now we have container_off.

It's convenient to check that the member is the given type, but we can leave
that to the callers.

Just adding `-g' to list of CFLAGS doesn't make gcc to generate debug
information required for macro expansion during debugging. Replacing
it with `-g3 -ggdb' rectifies this.

(configurator.c commit folded by Rusty)

Nice for distributed option declaration.

Also add #include to AUTODATA_TYPE example, so ccanlint doesn't try to
put it inside a function (which works, for this, but gives a warning
about unused variables).

Makes things a bit neater.

eg. "See also" can match "see-also".

Gets rid of generated file.

This is a more portable variant of the ELF section trick.

Very slow, but sometimes you need to know without crashing.

It's much easier to simply say "keep all", and it simplifies the code
quite a bit.

I use --target far more than --timeout.

Rather than using a separate index of tetst to skip, make sure tests
we don't want to run are removed from the graph of tests to start
with.

This means that the "skip" field is only set when we fail a test (and
thus must be reset when we test the next module).

Rather than walking the list of tests manually, use
dgraph_traverse_to() to run the tests.

Instead of a linked list of tests, we use dgraph and strmap.

Make info_exists() call add_info_options() directly, rather than detecting
that we've found _info inside the loop of tests.

It means targeting is now a global, which is a bit sucky.

This way, we can debug really early failures (eg. during option
parsing).

It's reduced to a flag which means we stop processing the module.  We have
to enhance our dependencies a bit, but they've always been quite good.

This simplifies things quite a bit.

Parsing options like --verbose and --debug can be a pain.  You need to
have everything set up before invoking parse_args(), but that may be a
significant amount of work, for which you may want verbose or
debugging enabled.

Thus the concept of "early" args: you can nominate arguments to be
parse before anything else, using opt_early_parse().

Make a false return abort the iteration, not true.

The old way makes sense for search functions (true == I found it), but
other kinds of iteration are more common (brute force search is
probably dumb).

strset_set -> strset_add
strset_clear -> strset_del
strset_test -> strset_get
strset_destroy -> strset_clear