• Kirill Smelkov's avatar
    bigarray: Explicitly reject dtypes with object inside · e44bd761
    Kirill Smelkov authored
    From time to time people keep trying to use wendelin.core with
    dtype=object arrays and get segfaults without anything in logs or
    whatever else.
    
    Wendelin.core does not support it, because in case of dtype=object elements are
    really pointers and data for each object is stored in separate place in RAM
    with different per-object size.
    
    As we are memory-mapping arrays this won't work. It also does not
    essentially work for numpy.memmap for the same reason:
    
        (z4+numpy) kirr@mini:~/src/wendelin$ dd if=/dev/zero of=zero.dat bs=128 count=1
        1+0 records in
        1+0 records out
        128 bytes copied, 0.000209873 s, 610 kB/s
        (z4+numpy) kirr@mini:~/src/wendelin$ dd if=/dev/urandom of=random.dat bs=128 count=1
        1+0 records in
        1+0 records out
        128 bytes copied, 0.000225726 s, 567 kB/s
        (z4+numpy) kirr@mini:~/src/wendelin$ ipython
        ...
    
        In [1]: import numpy as np
    
        In [2]: np.memmap('zero.dat', dtype=np.object)
        Out[2]:
        memmap([None, None, None, None, None, None, None, None, None, None, None,
               None, None, None, None, None], dtype=object)
    
        In [3]: np.memmap('random.dat', dtype=np.object)
        Out[3]: Segmentation fault
    
    So let's clarify this to users via explicitly raising exception when
    BigArray with non-appropriate dtype is trying to be created with
    descriptive explanation also logged.
    
    /reviewed-on nexedi/wendelin.core!4
    e44bd761
test_basic.py 17 KB