Fix bug indexing BTreeItems with negative values. Previous code assumed that idexes passed to BTreeItem_item would be negative, in practice this was not the case, and an index of -1 actually passed len(items)-1 as the index argument. Consequently, it was possible to access each item using two negative indexes (i.e., -1 and -len(items)-1). This made each item appear twice in a reverse iteration.

Code in BTreeItems_seek attempted to match the sign of the pseudoindex kept in the BTreeItems obj and the incoming index. Since this incoming index was never legitimately negative as it assumed, actually negatives passed in due to "overshooting" the first item would repeat the items again from the end. Removal of this code corrects the problem.

Unittests which provoke the error in the original code have also been added.

Removing the subtree rooted at ZODB/src/ZODB/zodb4/.

execute permissions.  This deals with the ZODB and ZEO ones
enumerated there.

of the most recent full backup, repozo.py failed to find the appropriate
files, erroneously claiming

    No files in repository before <specified time>

Repaired that.  Also made it much more robust against "junk files" the
user may create, or leave behind, in the backup directory.  Added test.

but nothing in scripts/tests/ does.  It failed before the
earlier (today) fixes to repozo.py, and passes now.
Better Than Nothing -- which is what we had before.

Under exceedingly rare conditions, a timing hole made it
possible for a second open() call on a database to block for an
arbitrarily long time.  This accounts for the intermittent
failure of a thread to make any progress after 5 minutes in
checkConcurrentUpdates1Storage.

We intend to get rid of most of this delicate lock
business, but before then the test failures are still
hurting me.

Collector #1330:  repozo.py -R can create corrupt .fs.
When looking for the backup files needed to recreate a Data.fs file,
repozo could (unintentionally) include its meta .dat files in the list,
or random files of any kind created by the user in the backup directory.
These would then get copied verbatim into the reconstructed file, filling
parts with junk.  Repaired by filtering the file list to include only
files with the data extensions repozo.py creates (.fs, .fsz, .deltafs,
and .deltafsz).  Thanks to James Henderson for the diagnosis.

This fixes it, based on an approach suggested by Toby
Dickenson.  The triggering condition wasn't entirely
sane, so was very rare:  a persistent object with a
__del__ method that referenced an attribute of self.

scan_gc_items():  Look at persistent objects accessed
while this is running at most once.

New test checkMinimizeTerminates():  This spawns a thread
that will in fact run forever if you don't recompile
cPickleCache.c.  The test suite will keep running, but
checkMinimizeTerminates will fail, and all future calls
to cacheMinimize() will be effectively ignored (so other
bad things may happen as a result).

why the sleeps are needed.  The explanation (and the
sleeps) didn't make sense -- must be left over
from a cache implementation before my time.

Added long-requested docs about mutating BTrees while iterating over them.  Will regen the .pdf and .html later (I want to add more .tex docs first).

BTree.  This came up on the zodb-dev list last week, and Steve Alexander
pointed out that the simple fix is virtually unknown in the community.
Then again, I hope the problem is virtually unknown in the wild now too.

The refcount reported for a ghost was one too small, because the code for
ghosts and non-ghosts was the same, and presumably "it's a feature" that
the code for non-ghosts deliberately lies about the true Python refcount,
in order (guessing here) to report what the count would have been if the
cPickleCache didn't exist.  But while the cPickleCache holds on to a real
reference to non-ghost objects, it does not hold a real reference to ghost
objects, so subtracting "an extra" count for all objects made it appear
that non-referenced ghosts exist in the cache (which doesn't actually
happen).

What a tangled web we weave ...

- fix typo in extension module name

testNoDamage():  Verify that the recovered .fs is in fact identical
to the original .fs.

that the number of bytes removed ends with "0".

Other tests:  if the output doesn't contain 'error', show the actual
output.

fsrecover.py:  the call to restore() was still using .serial, which no
longer exists.  Changed to use .tid.

testRecover.py:  the tests here passed despite that fsrecover wasn't
working at all.  Changed RecoverTest.recover() to eliminate its unused
arguments, and to return all the output fsrecover.recover() produces (the
output was being thrown away, unlooked at).  Added new testNoDamage(),
which feeds recovery a healthy .fs, and then looks at the output of
recovery, to verify that no errors were reported and no bytes were
removed by recovery.  Changed all the other tests to stop passing unused
arguments to RecoverTest.recover(), and to verify that error output was
produced by recovery.

XXX I'm not sure that every call to RecoverTest.recover() will actually
XXX produce an error in fsrecover.recover(), so we may get some spurious
XXX test failures here.  I'll keep my open for those.  (The kinds of
XXX damage tested are randomized, so one run doesn't prove anything.)

'pack' default value from 0 to None (this looks like the clear intent
of the code).

Note that fsrecover.py is totally broken, but testRecover.py isn't
detecting it.  I'm working on repairing both.

the first function defn.

tried to print the module docstring, but in an unreachable line.  In
case of more than 2 positional args, the code died with a mysterious
"unpack tuple of wrong size" exception; check for the correct # of args
directly instead.  Print all error msgs to stderr instead of stdout.