be done as late as possible (if ever)

DB._closeConnection() sets the connection's _db member to None now, under
protection of the lock it holds anyway.  At a deeper level, it's unclear
why _db keeps getting set and unset to begin with, but no time for that
now (and there are already XXX comments about it in the code).

Alas, I wasn't able to write a test that provoked the original bug in
finite time (it's a tiny timing hole), except via calling sleep() between
two lines that don't exist anymore.  Good enough.

an object as a non-negative integer.  Code trying to pass addresses to
an %x format uses positive_id(), and this avoids a Python FutureWarning
about applying %x to negative ints.  The primary difference between this
and the last stab is that positive_id() should work OK on 64-bit boxes
too.  What we really want here is C's %p format code, but in Python we
can't even reliably know the width of native addresses.

negative ints *as* negative ints when fed into %x formats.  Python 2.3
still renders them as positive ints, but spews

FutureWarning: %u/%o/%x/%X of negative int will return a signed string
               in Python 2.4 and up

to warn about the impending change.  Jim reported two instances of that
warning when running the tests on a box where addresses happen to "be
negative".  So make the addresses look positive instead (2.3 and 2.4
treat those the same, so 2.3 doesn't warn about those).

Problem:  it occurs to me now that I'm assuming addresses fit in 32
bits here.

the former, remember it and raise it after calling _cleanup.  If
abort_sub() or tpc_abort() in _cleanup() also raise exceptions, log
them but don't propagate them.

This stops stray output produced by tests testExceptionInTpcAbort and
testExceptionInSubAbortSub.

Grrrrr:  I haven't yet been able to figure out how to get logging to work
in ZODB, so haven't yet been able to check that the new logging is
reasonable.

import here wasn't changed to track it.  Repaired.

get_transaction() is still stuffed into __builtin__ as a magical side
effect of importing ZODB.

testExceptionInTpcAbort().  The cause is clear now, but a solution isn't;
an exception in tpc_abort is nasty.

(something's not right in this test).

taught about the new "transaction" package.

This branch introduces a new transaction API.  The key features are:
  - top-level functions in transaction -- get(), commit(), abort()
  - explicit transaction manager objects
  - Transaction objects are used for exactly one transaction
  - support for transaction synchronizers

The changes here are still provisional, but we want to get them off an
obscure branch and onto the head for further development.

of pack in buffered mode, then switching to unbuffered mode to copy the
tail, actually broke pack completely on Windows:  we didn't close the
buffered file handle before opening the unbuffered one, and self.gc held
on to the still-open former handle.  This prevented the caller from
renaming Data.fs to Data.fs.old (a handle on Data.fs was still open).
The cure is simply to close a handle when we stop using it.

Only open the file for unbuffered I/O after finishing the first phase
of pack.  The first phase gets its end-of-file position from the main
thread, so there's no possibility of reading a 'c' record.

Timings on Linux are inconclusive, but it seems like using buffered
I/O for the initial phase should be faster.

Provide a way to shutdown the servers using an exit status.

can still be in progress, and they're written to the same physical file via
a different file object.  Using buffered I/O in the packer creates the
possiblity for the packer to see stale data from its file object's stdio
buffers.  This is now known to happen under Linux, Gentoo, OS X, Cygwin,
and Debian.  It apparently doesn't happen under native Windows, which is
why everyone except me has been seeing the new checkPackLotsWhileWriting
test fail.

This will probably need to be backported everywhere, but first I want to
see in which new way checkPackLotsWhileWriting fails on 48 Linux boxes
overnight <wink>.

(distutils mungs it anyway, so it's just relevant for developers)

    FileStorageError:  The database has already been packed to a
    later time or no changes have been made since the last pack

exception.  Instead that message is logged (at INFO level), and
the pack attempt simply returns then (no pack is performed).

Incidentally, this should repair frequent reports of failure in the
new checkPackLotsWhileWriting test.  On non-Windows systems, it
seems that the worker thread often didn't get enough cycles to commit
a change between the main thread's attempts to run pack() (and so
the exception above got raised then).

thread is interesting here, so removed the hair catering to the possiblity
of N > 1 threads.

failures in the ZEO FileStorage pack-while-writing tests, where they die
with
    CorruptedError: ... transaction with checkpoint flag set

By coincidence, the same kind of death during pack() was reported on
zodb-dev today(!).  The new checkPackLotsWhileWriting test provokes this
on my box reliably (failed each time it was run).

The problem appears to be that pack's idea of where a FileStorage
ends was obtained merely by seeking to the end of the file.  But if
a transaction is in progress, there can be an extra "not really there
yet" transaction at the end of the file, and when either buildPackIndex()
or findReachableFromFuture() bumped into a such a thing, the CorruptedError
above got raised.  This is always rare, and was exceedingly rare before
because only one pack per test was tried.  The new test tries packing
repeatedly while a thread keeps hammering the database, so is much more
likely to provoke the problem.

The fix amounts to passing FileStorage._pos to the pack code, telling
the latter directly where the legit transactions in the file end.

The only values _packt ever had were z64 and None.  The only thing _packt
was used for that appeared to make sense was to block undo while a pack
was in progress, and a bool works better for that.  tids were compared
to _packt in a few spots, but since the only values _packt could have
were in {z64, None}, the comparisons were doomed to (regardless of tid)
return "greater than" (z64) or to be undefined (None; although in recent
Pythons None is reliably less than objects of other types, so "greater
than" was the only outcome possible from these comparisons under 2.3.3).
Removed these comparisons, collapsing surrounding expressions to
equivalents taking into account that tid < self._packt was always false.

- remove old XXX comments: we have added information on schema, and
  no, we really don't want to talk about "handlers" -- they're evil

context

it, and the only value a subclass ever set it to was None.

FileStoragePacker's constructor mean.

  <key name="+">; keys need to be provided for each default value
- fix bug in converting an empty section that contained a
  <multikey name="+">
- document <default> element in reference manual and DTD
- more tests

a two-state flag, with possible values z64 or None -- eh?).

record's "status byte" are and mean.

Make _added_during_commit a regular instance variable.  Don't use
try/finally to reset it; just clear it at the start of a transaction.
XXX There was a test that needed to be removed, but it seemed to be
just a shallow test that try/finally was used.  Can't see any feature
that depends on specific of error handling: The txn is going to abort.

Remove unused _opened instance variable.
Split commit() into two smaller parts.
Get rid of extra manipulation of _creating.
Don't look for _p_serial of None; z64 is now required.
Undo local variable aliases in subtransaction methods.

Also, trivial change to pickle cache API -- get() works like dict get().

with

    CorruptedError:  ... transaction with checkpoint flag set

on my box when doing the ZEO flavor of FileStorage.  This is like the
other pack-while-writing tests, but uses only 1 client thread that does
twice as much, and packs repeatedly for so long as that thread runs.  I
usually see it fail before the fourth pack attempt.