1. 29 Oct, 2018 2 commits
    • Kirill Smelkov's avatar
      lib.xnumpy.structured: New utility to create structured view of an array · 32ca80e2
      Kirill Smelkov authored
      Structured creates view of the array interpreting its minor axis as fully covered by a dtype.
      
      It is similar to arr.view(dtype) + corresponding reshape, but does
      not have limitations of ndarray.view(). For example:
      
        In [1]: a = np.arange(3*3, dtype=np.int32).reshape((3,3))
      
        In [2]: a
        Out[2]:
        array([[0, 1, 2],
               [3, 4, 5],
               [6, 7, 8]], dtype=int32)
      
        In [3]: b = a[:2,:2]
      
        In [4]: b
        Out[4]:
        array([[0, 1],
               [3, 4]], dtype=int32)
      
        In [5]: dtxy = np.dtype([('x', np.int32), ('y', np.int32)])
      
        In [6]: dtxy
        Out[6]: dtype([('x', '<i4'), ('y', '<i4')])
      
        In [7]: b.view(dtxy)
        ---------------------------------------------------------------------------
        ValueError                                Traceback (most recent call last)
        <ipython-input-66-af98529aa150> in <module>()
        ----> 1 b.view(dtxy)
      
        ValueError: To change to a dtype of a different size, the array must be C-contiguous
      
        In [8]: structured(b, dtxy)
        Out[8]: array([(0, 1), (3, 4)], dtype=[('x', '<i4'), ('y', '<i4')])
      
      Structured always creates view and never copies data.
      
      Here is original context where separately playing with .shape and .dtype
      was not enough, since it was creating array copy and OOM'ing the machine:
      
      klaus/wendelin@cbe4938b
      32ca80e2
    • Kirill Smelkov's avatar
      bigarray: Factor-out our custom numpy.lib.stride_tricks.as_strided-alike into lib/xnumpy.py · 6a5dfefa
      Kirill Smelkov authored
      We are going to use this code in another place, so move this out to
      dommon place as a preparatory step first.
      
      On a related note: Since ArrayRef is generic and quite independent from
      BigArray (it only supports it, but equally it supports just other - e.g.
      plain arrays), the proper place for it might be also to be lib/xnumpy.py .
      We might get to this topic a bit later.
      6a5dfefa