Code and new test to ensure that making a savepoint triggers cache gc.

the new addBeforeCommitHook() method of transactions.

Minor, unrelated changes in release.py.

addBeforeCommitHook() is new, a generalization of the
now-deprecated beforeCommitHook() that allows influencing
the order in which commit hooks are called.

[Martijn Pieters]
Include INSTANCE_HOME in the runzeo and zeoctl startup scripts from
the get-go; this helps poor schmucks like me who have to make
app-level conflict resolution work in Products as well. Including
INSTANCE_HOME makes sure ZEO can find the product code in the first
place.

- References to headers now involve the package names.

- The zpkg support files have been moved to the top-level directory,
  and the releases/ tree has been removed.  The resource map and
  configuration file have been combined now that zpkg allows that.

- The scripts are now all located in src/scripts/ in a checkout.

- Scripts that were imported in the tests have been split into library
  and script components; the library portions share name of the
  script (in order to reduce changes).  The library portions may not
  be used as scripts directly.

More on collector 1847 (ZEO cache tracing).

simul.py hit NameErrors when startup cache verification
found data in the cache to invalidate.

The cache's loadBefore() implementation called _trace()
incorrectly, passing the tid where the version argument
belonged.

Part of Collector 1860 (the other part is in Zope).

There's no possiblity of rollback here, so no need to insist that the
data manager support rollbacks.

Plug leaks in the ZEO client cache.

ClientCache._evicted():  When deleting the last range of
non-current tids for an oid, remove the list from the
noncurrent dict instead of leaving an empty list sitting
there forever.  This also required adjusting the side-
effect dance in ClientCache._remove_noncurrent_revisions().

FileCache._makeroom():  This was the major leak -- it
neglected to remove an evicted object's Entry from the
key2entry dict.

  move things around so that zeo scripts are installed for
  standalone zodb releases but not for zope releases

Needs to import errno now.

Worm around suspected Windows socket bug in Windows trigger code.

See the thread starting at
 http://mail.zope.org/pipermail/zope/2005-July/160433.html
for gory details.

makeroom():  Simplify retrieve+del from filemap.

Removed the `key2size` OIBTree.

Maintaining this is a significant expense; the code didn't
make real use of it; and every time I've tried to exploit it
I've ended up with grossly too-large simulated hit rates.
Would probably take another day to figure out exactly why that
is, and the simulated rates are "good enough" now without it.

More words; update examples w/ new output.

The cache simulation seems good enough to be useful now,
although the toy app I wrote to generate a 500MB trace
file doesn't use MVCC in an essential way (some of the
MVCC simulation code is nevertheless exercised, since
an invalidation of current data in the presence of MVCC
always creates a cache record for the newly non-current
revision).

Still puzzling over what to do when the trace file records
a load hit but the simulated cache gets a miss.  The
old simulation code seemed to assume that a store for the same
oid would show up in the trace file next, and it could get
the info it needed about the missing object from the store
trace.  But that isn't true:  precisely because the load was
a hit in the trace file, the object isn't going to be stored
again "soon" in the trace file.

Here are some actual-vs-simulated hit rate results, for a
20MB cache, with a trace file covering about 9 million
loads, over 3 ZEO client (re)starts:

actual   simulated
------   ---------
  93.1        92.7
  79.8        79.0
  68.0        69.1
      
  81.4        81.1  overall

Since the simulated hit rates are both higher and lower
than the actual hit rates, that argues against a gross
systematic bias in the simulation (although there may
be several systematic biases in opposite directions).

Massive rewrite to make simul.py's circular cache aware of
MVCC.  It isn't blowing up, but it thinks the hit rate is
100% -- might have missed something there <wink>.

More steps on the way toward having useful cache simulation.

I think the primary hangup now is that simul.py doesn't know
anything about MVCC.  As a result, it thinks there's significatly
more free space in the cache than there really is, and that
probably accounts for it predicting significantly higher hit
rates than I actually see.

Lots of fiddling to work w/ current trace file format.

Output isn't obviously wholly insane, but isn't a reasonably
close enough match to stats.py output either yet.

stats.py:  cut the # of file reads by 1/3, and the # of
struct.unpack() calls by 1/2.

Various bugfixes elsewhere.

Many little bugfixes and improvements in stats.py.
This has survived several 100 MB of trace files
I generated over the last few days, so it's solid
now if not necessarily perfect.

Replaced simul.py with the much broader-ranging code
Jeremy and I were working on a couple years ago,
although it can't work with the current trace file
format (no real loss there -- the simul.py it's
replacing can't work with the current format either).

Stitch in ZConfig's docs from ZConfig.

Remove this -- it's a way out-of-date copy of the ZConfig
docs.  I'll stitch them back in from the ZConfig repo next,
via svn:externals.  Ideally, ZODB shouldn't have this as
part of its checkout tree at all, but that also has implications
for the way ZRS gets packaged, etc.

A large number of changes so that cache tracing isn't
100% broken -- I can get a trace file now, and stats.py
doesn't blow up anymore.  Don't yet know whether it's
reporting sensible things, but it's not obviously insane.
simul.py doesn't know about the current cache implementation
either yet.

ClientCache:  made it a new-style class.

FileCache:  Removed the reuse= argument.  IIRC, it made some sense
when different cache schemes were first being tried, but complicated
the logic now to no good end.

Nuked the maddening log warnings about reuse=True.  They were never
helpful and were often confusing.

Added info-level log messages to record the path of the client cache
file, whether it's persistent or temporary, and if it's persistent
whether we're creating or reusing it.

Minor changes to comments.

Massive rewrite of cache.txt, to match current reality.

Collector 1843:  IISet.keys: bad exception handling.

Bucket_rangeSearch():  De-obfuscated the calls to
Bucket_findRangeEnd(), so that they stop ignoring
the latter's error returns (a mind-bending combination
of embedded assignment nested in an UNLESS macro,
seemingly copy+paste'd so that the error occurred twice).

Port from ZODB 3.2.

Collector #1846:  If an uncommitted transaction was found, fsrecover.py
fell into an infinite loop.  Fixed that, and added a new test
(testUncommittedAtEnd) to ensure this stays fixed.