tests/cluster: speedup waiting a bit
NEO functional tests use pdb.wait() in a few places, for example in NEOCluster .run(), .start() and .expectCondition(). The wait implementation uses polling with exponentially growing wait period. With the following instrumentation --- a/neo/tests/cluster.py +++ b/neo/tests/cluster.py @@ -236,6 +236,7 @@ def wait(self, test, timeout): return False finally: cluster_dict.release() + print 'next_sleep:', next_sleep sleep(next_sleep) return True during execution of functional tests it is not uncommon to see the following sleep periods next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.15 next_sleep: 0.225 next_sleep: 0.3375 next_sleep: 0.50625 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.15 next_sleep: 0.225 next_sleep: 0.3375 next_sleep: 0.50625 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.15 next_sleep: 0.225 next_sleep: 0.3375 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.1 next_sleep: 0.15 next_sleep: 0.225 next_sleep: 0.3375 next_sleep: 0.50625 . Without going into reworking the wait mechanism to use real notifications instead of polling, it was observed that the exponential progression tends to create too coarse sleeps. Initial 0.1s interval was found to be also too much. This patch remove the exponential period growth and reduces period by order of one magnitude. For functional tests timings on my computer it is thus: before patch: Functional tests 28 Tests, 0 Failed Title : TestRunner Date : 2018-04-04 Node : deco Machine : x86_64 System : Linux Python : 2.7.14 Directory : /tmp/neo_tests/1522868674.115798 Status : 100.000% NEO_TESTS_ADAPTER : SQLite NEO TESTS REPORT Test Module | run | unexpected | expected | skipped | time --------------------------+-------+------------+----------+---------+---------- Client | 6 | . | . | . | 8.51s Cluster | 7 | . | . | . | 9.84s Master | 4 | . | . | . | 9.68s Storage | 11 | . | . | . | 20.76s --------------------------+-------+------------+----------+---------+---------- neo.tests.functional | | | | | --------------------------+-------+------------+----------+---------+---------- Summary | 28 | . | . | . | 48.79s --------------------------+-------+------------+----------+---------+---------- after patch: Functional tests 28 Tests, 0 Failed Title : TestRunner Date : 2018-04-04 Node : deco Machine : x86_64 System : Linux Python : 2.7.14 Directory : /tmp/neo_tests/1522868527.624376 Status : 100.000% NEO_TESTS_ADAPTER : SQLite NEO TESTS REPORT Test Module | run | unexpected | expected | skipped | time --------------------------+-------+------------+----------+---------+---------- Client | 6 | . | . | . | 7.38s Cluster | 7 | . | . | . | 7.05s Master | 4 | . | . | . | 8.22s Storage | 11 | . | . | . | 19.22s --------------------------+-------+------------+----------+---------+---------- neo.tests.functional | | | | | --------------------------+-------+------------+----------+---------+---------- Summary | 28 | . | . | . | 41.87s --------------------------+-------+------------+----------+---------+---------- in other words ~ 10% improvement for the whole time to run functional tests. /reviewed-by @vpelletier, @jm /reviewed-on nexedi/neoppod!10
Showing
Please register or sign in to comment