HBTreeFolder2 fixes (!112) · Merge Requests · nexedi / erp5

HBTreeFolder2 fixes

There's no magic in this patch series: it is known that HBTreeFolder2 has limitations about the ids that can be set without conflict, and this can't be fixed without causing compatibility issues with existing data.

What I suggest here is:

some optimization
bug fixes
detect id conflicts before causing data loss

I also started to study this product when trying to use a newer version of ZODB. Recent BTrees fails on the following line of _setOb:

        if len(id_list) == 1 and not htree.has_key(None):

(None is not valid key for comparison)

One of the commit removes _tree_list. I hope this does not make getTreeIdList too slow.

The long story about id conflicts is that a HBTreeFolder2 can't store 2 objects <A> and <A>-<B> where <A> does not contain '-', and that's the rule followed by _getOb/_setOb/_delOb. However:

Conflicts are detected by testing the type of the value, which means HBTreeFolder2 can't store values of the same type as the one it uses internally (i.e. OOBTree).
For performance reasons, _htree_iteritems and getTreeIdList use a stricter rule: they assume there can't be 2 objects <A> and <A>-<B>, regardless of the presence of a separator in <A>. Maybe this rule should be enforced in _setOb.

What I suggest here is:
- some optimization
- bug fixes
- detect id conflicts before causing data loss

I also started to study this product when trying to use a newer version of ZODB. Recent BTrees fails on the following line of `_setOb`:
```python
        if len(id_list) == 1 and not htree.has_key(None):
```
([None is not valid key for comparison](https://github.com/zopefoundation/ZODB/commit/bb5aac21277f43333d6450064dc6670c8c280e40))

One of the commit removes `_tree_list`. I hope this does not make `getTreeIdList` too slow.

The long story about id conflicts is that a HBTreeFolder2 can't store 2 objects \<A> and \<A>-\<B> where \<A> does not contain '-', and that's the rule followed by _getOb/_setOb/_delOb. However:
- Conflicts are detected by testing the type of the value, which means HBTreeFolder2 can't store values of the same type as the one it uses internally (i.e. OOBTree).
- For performance reasons, _htree_iteritems and getTreeIdList use a stricter rule: they assume there can't be 2 objects \<A> and \<A>-\<B>, regardless of the presence of a separator in \<A>. Maybe this rule should be enforced in _setOb.