GitLab

From time to time people keep trying to use wendelin.core with dtype=object arrays and get segfaults without anything in logs or whatever else.

Wendelin.core does not support it, because in case of dtype=object elements are really pointers and data for each object is stored in separate place in RAM with different per-object size.

As we are memory-mapping arrays this won't work. It also does not essentially work for numpy.memmap for the same reason:

(z4+numpy) kirr@mini:~/src/wendelin$ dd if=/dev/zero of=zero.dat bs=128 count=1
1+0 records in
1+0 records out
128 bytes copied, 0.000209873 s, 610 kB/s
(z4+numpy) kirr@mini:~/src/wendelin$ dd if=/dev/urandom of=random.dat bs=128 count=1
1+0 records in
1+0 records out
128 bytes copied, 0.000225726 s, 567 kB/s
(z4+numpy) kirr@mini:~/src/wendelin$ ipython
...

In [1]: import numpy as np

In [2]: np.memmap('zero.dat', dtype=np.object)
Out[2]:
memmap([None, None, None, None, None, None, None, None, None, None, None,
       None, None, None, None, None], dtype=object)

In [3]: np.memmap('random.dat', dtype=np.object)
Out[3]: Segmentation fault

So let's clarify this to users via explicitly raising exception when BigArray with non-appropriate dtype is trying to be created with descriptive explanation also logged.