• Carlos Ramos Carreño's avatar
    wcfs: tests: xbtree.py: Execute `zip` eagerly when we need list. · d014045b
    Carlos Ramos Carreño authored
    The builtin `zip` in Python 3 returns an iterator, not a list.
    Thus, one cannot directly use the `len` method on the object returned
    by `zip`, or we will have errors like the following one:
    
    ```python
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 617, in <module>
        main()
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 613, in main
        cmd(argv)
      File "/srv/slapgrid/slappart66/venvs/wendelin.core/lib/python3.9/site-packages/decorator.py", line 232, in fun
        return caller(func, *(extras + args), **kw)
      File "/srv/slapgrid/slappart66/git/pygolang/golang/__init__.py", line 125, in _
        return f(*argv, **kw)
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 589, in cmd_trees
        TreesSrv(zstor, r)
      File "/srv/slapgrid/slappart66/venvs/wendelin.core/lib/python3.9/site-packages/decorator.py", line 232, in fun
        return caller(func, *(extras + args), **kw)
      File "/srv/slapgrid/slappart66/git/pygolang/golang/__init__.py", line 125, in _
        return f(*argv, **kw)
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 234, in TreesSrv
        treetxtPrev = zctx.ztreetxt(ztree)
      File "/srv/slapgrid/slappart66/venvs/wendelin.core/lib/python3.9/site-packages/decorator.py", line 232, in fun
        return caller(func, *(extras + args), **kw)
      File "/srv/slapgrid/slappart66/git/pygolang/golang/__init__.py", line 125, in _
        return f(*argv, **kw)
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 536, in ztreetxt
        return zctx.TopoEncode(xbtree.StructureOf(ztree))
      File "/srv/slapgrid/slappart66/venvs/wendelin.core/lib/python3.9/site-packages/decorator.py", line 232, in fun
        return caller(func, *(extras + args), **kw)
      File "/srv/slapgrid/slappart66/git/pygolang/golang/__init__.py", line 125, in _
        return f(*argv, **kw)
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree/xbtreetest/treegen.py", line 542, in TopoEncode
        return xbtree.TopoEncode(tree, zctx.vencode)
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree.py", line 797, in TopoEncode
        for nodev in _walkBFS(tree):
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree.py", line 701, in _walkBFS
        for level in __walkBFS(tree):
      File "/srv/slapgrid/slappart66/git/wendelin.core/wcfs/internal/xbtree.py", line 724, in __walkBFS
        assert len(rv) == len(rn.node.children)
    TypeError: object of type 'zip' has no len()
    ```
    
    Thus, we have to create a list from the result of `zip` before calling
    `len` on it.
    
    --------
    kirr:
    
    There were only two places where zip was used to build a list. All other
    places where zip is used - both in wcfs/xbtree and in other packages -
    are calling zip to iterate over zip result:
    
        (py39.venv) kirr@deca:~/src/wendelin/wendelin.core$ git grep -w zip
        bigarray/__init__.py:        for n, s in zip(self.shape, self.stridev):
        bigarray/__init__.py:        for n, s in zip(a.shape, a.strides):
        bigarray/array_zodb.py:BigArray_defaults = dict(zip(reversed(_.args), reversed(_.defaults)))
        wcfs/internal/xbtree.py:            for i, (klo, khi) in enumerate(zip(v[:-1], v[1:])): # (klo, khi) = [] of (k_i, k_{i+1})
        wcfs/internal/xbtree.py:                kvv = ['%s:%s' % (k,v) for (k,v) in zip(b.keyv, b.valuev)]
        wcfs/internal/xbtree.py:        for (j,i) in zip(jv, iv):
        wcfs/internal/xbtree.py:                    for (child, k) in zip(node.children[1:], node.keyv):
        wcfs/internal/xbtree.py:                    for (k,v) in zip(node.keyv, node.valuev):
        wcfs/internal/xbtree.py:            for (xlo, xhi) in zip(ksplitv[:-1], ksplitv[1:]): # (klo, s1), (s1, s2), ..., (sN, khi)
        wcfs/internal/xbtree.py:            for (xlo, xhi) in zip(ksplitv[:-1], ksplitv[1:]): # (klo, s1), (s1, s2), ..., (sN, khi)
        wcfs/internal/xbtree.py:                                    for (k,vtxt) in zip(node.keyv, vtxtv)])
        wcfs/internal/xbtree/xbtreetest/treegen.py:                    for (k,v) in zip(node.keyv, node.valuev):
        wcfs/internal/xbtree_test.py:    for (child, childOK) in zip(kids, children):
        wcfs/internal/xbtree_test.py:        for (i,(k,v)) in enumerate(zip(keys, values)):
    
        # handled in hereby patch
        wcfs/internal/xbtree.py:                rv = list(zip(v[:-1], v[1:]))  # (klo,k1), (k1,k2), ..., (kN,khi)
        wcfs/internal/xbtree.py:                rv = list(zip(v[:-1], v[1:]))  # (klo,k1), (k1,k2), ..., (kN,khi)
    
    /reviewed-by @kirr
    /reviewed-on nexedi/wendelin.core!27
    d014045b
xbtree.py 39.1 KB