It's possible to get OOV wids here due to words the lexicon knows
about that the index has no current instances of.

in the reindexing text.

*any* words in common across the versions.  Helped Will along by adding
a pragmatic comment to his "knocking indeed" rant.  Reworked to use
the inscrutable magic of dict.setdefault.

take longer to construct now; both indexers' _get_frequencies routines
were fiddled to return the same kind of stuff again, and I had
previously fiddled the cosine indexer's _get_frequencies to do something
weirder but (probably) faster than this.

code left in it!

get_words, and in calling contexts nothing but a list of wids could
possibly make sense.

I need a break.

indexers.

CAUTION:  I'm sure I don't understand how persistency needs to be spelled.
Is it enough to say just that the base class derives from Persistent, or
does that need to be duplicated (or done instead exclusively) in the
derived classes?

Is there a point to keeping "import ZODB" in the derived-class files?  Is
there a point to keeping it anywhere <wink>?

logic to deal with all cases.  All the tests pass again.

globToWordIds():  This was building a list of words and then throwing
it away without referencing it.  Deleted the code.

If we update a document and reindex it, ZCTextIndex is currently
broken.  The test passes py virtue of calling unindex_object() after
each update, then calling index_object() again.  We need to fix our
code, and then remove the calls to unindex_object() from the test.

XXX This code causes OkapiIndex to fail because it doesn't expect to
have no wordinfo for a wid.  I tried to fix this in CosineIndex, but I
want to Tim think more about it and try to fix OkapiIndex.

This case can arise when the last occurence of a word is removed, or
when a lexicon is shared across multiple indexes.

XXX Not sure this code is correct, but it might be and the tests pass.
If it's wrong, we need more tests.

internals.

testStopWords().  That isn't the real point.  The real point is to
fiddle the scaffolding enough to make it possible to run a white box
test of the Okapi indexer too.  That isn't here yet.

with the docid is greater than 0 too.

and the Okapi indexer.  testZCTextIndex was, and remains, strongly tied
to the cosine indexer.