GitLab

Hello,

these patches aim to support manual garbage collection of ZBigArrays, so that they can be packed. The idea is to support GC of ZBigArray before a more general implementation within NEO exists.

I tested this approach with NEO and FileStorage (with deactivated GC). With FileStorage I could see that the size of Data.fs shrank after calling .pack(). With NEO I could see that the row count of data shrank after calling .pack().

What still seems non-ideal in this patch to me is how the history of a given object is retrieved. The .history() function may be fine, but with NEO we can't be sure that indeed all revisions of an object are removed. I see two approaches:

1. We support float('inf') as a size argument in NEO. The mysql query could skip the second LIMIT argument in case size == float('inf'). Maybe we would need a mechanism to avoid too big packages (or it already exists).
2. We accept the current approximation as sufficient, at least for refreshed Data Array documents there shouldn't be that many transactions.

Any other ideas?

The test is a sketch. It succeeds, but coding style test fails (useless .assertTrue).

If we can agree on the general direction of the underlying mechanism, I'll add patches that apply the ZBigArray GC when the refresh workflow is called.

Best, Levin

/cc @jm @klaus