GitLab

(reported by @Tyagov)

Hi, I let it run over night my quick feed into NEO w.c. array.

Now simply out of space and array shape is

(23379050506,)

Here's my script:

This script will consume as much memory as machine has an eventually it will be killed by kernel. So it's worth investigating why we have this leak and why garbage collection not working with w.c. array (my wild guess now)

from wendelin.bigarray.array_zodb import ZBigArray
from wendelin.lib.zodb import dbopen, dbclose
import transaction
import numpy as np

root = dbopen('neo://neo-standalone-comp-2477@[2001:67c:1254:YY::ZZZZ]:2051')
#root['B'] = A = ZBigArray((10,), np.float64)
#transaction.commit()


A = root['B']

slice_index = 10000
shape = A.shape[0]

print shape

# iterate over array
for i in xrange(0, shape, slice_index):
  np.average(A[i:i+slice_index])


#for i in range(20000):
#  A.append( np.random.rand(4*1024*1024)   ) #np.arange(4*1024*1024) )
#  print i, A.shape
#  transaction.commit()

To simulate:

1. ssh root@2001:67c:1254:XX::1

2. su ivan; cd /home/ivan

3. . test/bin/activate

4. ipython feed.py

Regards Ivan