Commit 1118ba3a authored by Kirill Smelkov's avatar Kirill Smelkov

test/gen_testdata: Retrieve object keys in predictable order independent of python version

gen_testdata.py picks up root keys randomly and shuffles extension dict
keys also randomly. But even with predictable PRNG if the input to e.g.
rand.choice is different, the result will be different as well.

So far we were lucky: we were running gen_testdata.py only via py2 and
the order of retrieved root keys was - by chance - the same each time.
That's why generated testdata databases were the same after each
gen_testdata.py run. But even on py2 there is no such guaranty and when
runnning gen_testdata.py via py3 the order of keys is really different:

    $ python2
    Python 2.7.18 (default, Jul 14 2021, 08:11:37)
    [GCC 10.2.1 20210110] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> d = {}
    >>> d['a'] = 1
    >>> d['b'] = 2
    >>> d['c'] = 3
    >>> d.keys()
    ['a', 'c', 'b']

    $ python3
    Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> d = {}
    >>> d['a'] = 1
    >>> d['b'] = 2
    >>> d['c'] = 3
    >>> list(d.keys())
    ['a', 'b', 'c']

So let's prepare for py3 generation beforehand by making sure that
keys input to PRNG is the same be it py2 or py3, thus, giving a chance
for generated py2/py3 databases to be really close to each other.

Since here we change the order of keys that are feed to PRNG, generated
test databases are shuffled a bit.
parent 667a0088
...@@ -105,6 +105,12 @@ class Object(Persistent): ...@@ -105,6 +105,12 @@ class Object(Persistent):
rand = random.Random() rand = random.Random()
del random del random
# keys returns list of obj.keys() in predictable order independent of python version.
def keys(obj):
vk = list(obj.keys())
vk.sort()
return vk
# prepare extension dictionary for subject # prepare extension dictionary for subject
alnum = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" alnum = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
def ext4subj(subj): def ext4subj(subj):
...@@ -120,7 +126,7 @@ def ext4subj(subj): ...@@ -120,7 +126,7 @@ def ext4subj(subj):
d[xcookie] = cookie d[xcookie] = cookie
# shufle extension dict randomly - to likely trigger different ordering on save # shufle extension dict randomly - to likely trigger different ordering on save
keyv = list(d.keys()) keyv = keys(d)
rand.shuffle(keyv) rand.shuffle(keyv)
ext = {} ext = {}
for key in keyv: for key in keyv:
...@@ -243,13 +249,13 @@ def _gen_testdb(outfs_path, zext): ...@@ -243,13 +249,13 @@ def _gen_testdb(outfs_path, zext):
# create a cyclic object -> object reference # create a cyclic object -> object reference
# pretty=zpickledis used not to handle this well because in ZODB pickle the reference # pretty=zpickledis used not to handle this well because in ZODB pickle the reference
# referes to referred type by GET that is prepared by PUT in class part of the pickle. # referes to referred type by GET that is prepared by PUT in class part of the pickle.
name = rand.choice(list(root.keys())) name = rand.choice(keys(root))
obj = root[name] obj = root[name]
obj.value = obj obj.value = obj
commit(u"user", u"cyclic reference", ext("cycle")) commit(u"user", u"cyclic reference", ext("cycle"))
# delete an object # delete an object
name = rand.choice(list(root.keys())) name = rand.choice(keys(root))
obj = root[name] obj = root[name]
root[name] = Object("%s%i*" % (name, i)) root[name] = Object("%s%i*" % (name, i))
# NOTE user/ext are kept empty on purpose - to also test this case # NOTE user/ext are kept empty on purpose - to also test this case
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment