This is a percursor to unifying with tdb_open() from tdb2, which handles
this itself.

Finally, we split out the tdb2-specific parts of tdb_context, and put
them into a "tdb2" sub-struct; the tdb1 parts go into a "tdb1"
sub-struct.  We get rido of tdb1_context and use tdb_context
everywhere.

There's also a semantic change here: for tdb1, being read-only meant no
locking, and it was an error to try to lock a r/o database.  For TDB2, you'd
need to specify TDB_NOLOCK, which suppresses locking silently.

This means converting the tdb1 inbuilt hash functions to the
tdb2-style, so they return 64 bit.  We truncate to 32 bit everywhere
but in tdb_check() which needs to do so explicitly.

In TDB2, the user can override locking functions, so they may
deliberarely fail (eg. be non-blocking) expecting to retry.

For this reason, the TDB2 API requires the caller to cancel the
transaction if tdb_transaction_prepare_commit() fails.

The TDB1 code logs multiple times on errors; we must prevent that in
the limited case where locking fails.  With TDB2, this can happen due
to the lock function attribute, where the user supplies replacement
locking functions which are allowed to return with errno EAGAIN or
EINTR for various special-effects.  Flooding the logs for this is
unfriendly.

This means they will use the TDB_ATTRIBUTE_FLOCK functions, and get automatic
reopen support.

Because tdb2 allows multiple opens of the same TDB, we separate out
the file information from the struct tdb_context.  Do the same for
tdb1.

This will be set for old TDBs; we can start distinguishing paths based
on it now.

This means getting rid of TDB_VOLATILE (perhaps we should use an
attribute for that?), TDB_INCOMPATIBLE_HASH (use the
tdb_attribute_hash for that) and TDB_CLEAR_IF_FIRST (use the
tdb_attribute_openhook for that).

We also get rid of TDB_DISALLOW_NESTING: that's the default for TDB2.

Rather than leak TDB_INCOMPATIBLE_HASH through to the TDB2 API, we make
it that if they use the tdb1_incompatible_hash function as their hash,
then we treat it as if they had specified the TDB_INCOMPATIBLE_HASH
flag (ie. we mark the header so it's unusable by tdb < 1.2.6).

This precludes the possibility of using TDB_INCOMPATIBLE_HASH with a
custom hash function: that used to allow the user to ensure that old TDB
versions couldn't open the TDB file (and recent ones check the header to
ensure they're using the right hash).  But that's a small loss.

To do this, we make sure that the first few fields of tdb1_context and
tdb_context are the same.

This sweep also fixes up a few paths where we weren't setting
tdb->ecode before our old logging function.

They're about to start sharing TDB2's definitions.

By moving all the parts of struct tdb_context which logging and
locking use to the beginning of the structure, we can make
tdb1_context match, and thus pass that directly to the tdb_logerr()
function, and later the locking functions.

This is only necessary until we unify tdb1_context and tdb_context.

The caller should do this: the SAMBA compatibility later does.

We do this using hooks in tdb2.

They'll all be accessed via the tdb2 API.

We're going to use TDB2's API, so some TDB1 APIs are obviously unnecessary.  
We also get rid of USE_RIGHT_MERGES and TRACE code.

The main change is to s/tdb/tdb1_/ everywhere.

Since we've renamed everything in tdb1, they won't clash.

We import the entire codebase, putting a "tdb1_" prefix on the files
and changing the "tdb_" prefix to "tdb1_" everywhere.

The next patches will gradually merge it with the TDB2 code where
necessary.

This lets us add the new files to a single place.

Create an iterator over every open tdb (not internal TDBs).  This is
useful for re-establishing the tdb1-style active lock for
CLEAR_IF_FIRST.

Instead of a per-file linked list, use a per-tdb list.  This is needed
for tdb_foreach().

This makes it easy to call it again after a fork(), such as for
re-establishing the CLEAR_IF_FIRST files locks.

You can only unset it if the TDB was originally opened O_RDWR.

Also, cleaned up error handling in tdb_allrecord_lock() so we only get
one log message on a r/o database.

This is more accurate than returning TDB_ERR_EINVAL.

It's more consistent with what tdb1 does, and slightly more encapsulated.

Since we've deprecated tdb_error() function (and it didn't work right
from inside the logging function anyway, since we didn't set
tdb->ecode yet) we need to hand it to the log function.

Make it the same offset as TDB1.  This isn't strictly necessary, but
it would allow for total unification later, since TDB1 and TDB2's
transaction code is otherwise completely compatible.

If the caller actually specifies TDB_CONVERT, fail if the TDB does not
need conversion.  It's unusual for the user to specify TDB_CONVERT
(it's auto-detected) but better to be strict.

The tdb_oob() function can fail due to errors, as well as because the length
asked for is greater than the size of the file.  Clean that up:

(1) If probe is true, only fail if there's an error, not if the length is
    too great.
(2) Exit tdb_open() if it tdb_oob() probe fails; this helps cut down
    test time for failtest.
(3) Don't set probe to true in tdb_direct() fail; a minor issue, but it means
    we log failure.

It can be 64 bits, for huge databases.

As noted, failtest was taking a long time, because a failure injected here
was not detected.

We use off_t in the tdb2 interface (for tdb_attribute_flock); we need
to make sure that all callers agree on the size.

This also causes a problem in the tests: it's not enough to include
config.h first, we need the _FILE_OFFSET_BITS define from private.h.
Otherwise, we can disagree about the definitions of F_SETLK, F_SETLKW
and off_t, causing strange problems.

Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Wed Aug 10 21:12:59 CEST 2011 on sn-devel-104

(Imported from SAMBA commit f7b820c3590ac78dd0dea67ac882f53ae6f550d9)

(Imported from SAMBA commit fed977514019e130322fb21ad5726ee997965898)

(Imported from SAMBA commit 35550ba641d251a2dc3762b08eda92d0f7ce6bed)

(Imported from SAMBA commit eac2f44cde39d16b827b7e2d360f7a7fb1d940e5)