GitLab

When using Sequences to run tests, we usually have multiple tests that look like this :

def test_01(self):
  sequence_list = SequenceList()
  sequence_string = self.base_sequence_string # Initialize the tests with a shared base sequence
  # Add more steps uniq to this test
  sequence_string += """
        stepXXX
        stepXXY
        ...
        stepZZZ
  """ 
  sequence_list.addSequenceString(sequence_string)
  sequence_list.play(self)

The idea is to run only once the steps defined in the sequence self.base_sequence_string, save the stte, and restore it for each state based on self.base_sequence_string. The state of ERP5 is stored in a Trash Bin in portal_trash. The state is composed of : all the objects contained in ERP5 and the content of Sequence._dict.

The code would become something like this :

# Register the base sequence_string
registerSequenceString("base", """
  step1
  step2
  ...
"""

def test_01(self):
  sequence_list = SequenceList()
  # Use a StoredSequence, and inform it to bootstrap
  # from the base sequence_string
  sequence = StoredSequence(self, "base")
  # Define the custom step sequence for this test
  sequence.setSequenceString("""
        stepXXX
        stepXXY
        ...
        stepZZZ
  """)
  sequence_list.addSequence(sequence)
  sequence_list.play(self)

I did a little benchmark using real tests from a customer project. The test has 22 steps in the base sequence, and 10 custom steps. I ran the 3 following cases 5 times, excluded the 2 most extrem values each time, and calculated the average time of the 3 remaining times.

• running the test with a Sequence, as usual : 29.642s
• running the test with a StoredSequence, but the iniatial state is not stored in portal_trash yet : 31.007s
• running the test with a StoredSequence, restoring the initial state from portal_trash : 20.878s

So we see an overhead of 4% when we need to store the state, compared to running the test "as usual". This overhead happens only once. Then the test runs 30% faster.

Of course this benchmark is really case dependant, as the more shared states, the more time could be saved. But it gives an idea.

Another advantage of the StoredSequence is that it is very easy to adapt existing code. There is no need to write any new logic, to split test cases, and can be adopted progressively.