Skip to content

G
gitlab-ce
Commit 42df87c8 authored by Mark Lapierre's avatar Mark Lapierre
Browse files
Options

Merge branch 'docs/third-iteration-on-writing-end-to-end-tests-doc' into 'master' 

Third iteration on writing end to end tests doc

Closes #62158

See merge request gitlab-org/gitlab-ce!28716
parents 15c05f5b 6d77a2f5
Show whitespace changes
Inline Side-by-side
Showing with 262 additions and 162 deletions
qa/docs/best_practices.md
View file @ 42df87c8
...@@ -4,7 +4,7 @@ The majority of the end-to-end tests require some state to be built in the appli ...@@ -4,7 +4,7 @@ The majority of the end-to-end tests require some state to be built in the appli
A good example is a user being logged in as a pre-condition for testing the feature. A good example is a user being logged in as a pre-condition for testing the feature.
But if the login feature is already covered with end-to-end tests through the GUI, there is no reason to perform such an expensive task to test the functionality of creating a project, or importing a repo, even if this features depend on a user being logged in. Let's see an example to make things clear. But if the login feature is already covered with end-to-end tests through the GUI, there is no reason to perform such an expensive task to test the functionality of creating a project, or importing a repo, even if these features depend on a user being logged in. Let's see an example to make things clear.
Let's say that, on average, the process to perform a successful login through the GUI takes 2 seconds. Let's say that, on average, the process to perform a successful login through the GUI takes 2 seconds.
...@@ -33,6 +33,6 @@ Finally, interacting with the application only by its GUI generates a higher rat ...@@ -33,6 +33,6 @@ Finally, interacting with the application only by its GUI generates a higher rat
**The takeaways here are:** **The takeaways here are:**
- Building state through the GUI is time consuming and it's not sustainable as the test suite grows. - Building state through the GUI is time consuming and it's not sustainable as the test suite grows.
- When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generates a higher rate of test flakiness. - When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness.
Now that we are aware of all of it, [let's go create some tests](writing_tests_from_scratch.md). Now that we are aware of all of it, [let's go create some tests](writing_tests_from_scratch.md#this-document-covers-the-following-items).
qa/docs/writing_tests_from_scratch.md
View file @ 42df87c8
...@@ -12,22 +12,21 @@ If you don't exactly understand what we mean by **not everything needs to happen ...@@ -12,22 +12,21 @@ If you don't exactly understand what we mean by **not everything needs to happen
## This document covers the following items: ## This document covers the following items:
0. Identifying if end-to-end tests are really needed - [0.](#0-are-end-to-end-tests-needed) Identifying if end-to-end tests are really needed
1. Identifying the [DevOps stage](https://about.gitlab.com/stages-devops-lifecycle/) of the feature that you are going to cover with end-to-end tests - [1.](#1-identifying-the-devops-stage) Identifying the [DevOps stage](https://about.gitlab.com/stages-devops-lifecycle/) of the feature that you are going to cover with end-to-end tests
2. Creating the skeleton of the test file (`*_spec.rb`) - [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`)
3. The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases logic - [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic
4. Extracting duplicated code into methods - [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods
5. Tests' pre-conditions (`before :all` and `before`) using resources and [Page Objects](./qa/page/README.md) - [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects](./qa/qa/page/README.md)
6. Optimizing the test suite - [6.](#6-optimization) Optimizing the test suite
7. Using and implementing resources - [7.](#7-resources) Using and implementing resources
8. Moving elements definitions and its methods to [Page Objects](./qa/page/README.md) - [8.](#8-page-objects) Moving element definitions and methods to [Page Objects](./qa/qa/page/README.md)
- Adding testability to the application
### 0. Are end-to-end tests needed? ### 0. Are end-to-end tests needed?
At GitLab we respect the [test pyramid](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/testing_guide/testing_levels.md), and so, we recommend to check the code coverage of a specific feature before writing end-to-end tests. At GitLab we respect the [test pyramid](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/testing_guide/testing_levels.md), and so, we recommend you check the code coverage of a specific feature before writing end-to-end tests, for both [CE](https://gitlab-org.gitlab.io/gitlab-ce/coverage-ruby/#_AllFiles) and [EE](https://gitlab-org.gitlab.io/gitlab-ee/coverage-ruby/#_AllFiles) projects.
Sometimes you may notice that there is already a good coverage in other test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests. Sometimes you may notice that there is already good coverage in other test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests.
If after this analysis you still think that end-to-end tests are needed, keep reading. If after this analysis you still think that end-to-end tests are needed, keep reading.
...@@ -35,7 +34,7 @@ If after this analysis you still think that end-to-end tests are needed, keep re ...@@ -35,7 +34,7 @@ If after this analysis you still think that end-to-end tests are needed, keep re
The GitLab QA end-to-end tests are organized by the different [stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/specs/features/browser_ui), and so, if you are creating tests for issue creation, for instance, you would locate the spec files under the `qa/qa/specs/features/browser_ui/2_plan/` directory since issue creation is part of the Plan stage. The GitLab QA end-to-end tests are organized by the different [stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/specs/features/browser_ui), and so, if you are creating tests for issue creation, for instance, you would locate the spec files under the `qa/qa/specs/features/browser_ui/2_plan/` directory since issue creation is part of the Plan stage.
In another case of a test for listing merged merge requests (MRs), the test should go under the `qa/qa/specs/features/browser_ui/3_create/` directory since merge request is a feature from the Create stage. In another case of a test for listing merged merge requests (MRs), the test should go under the `qa/qa/specs/features/browser_ui/3_create/` directory since merge requests are a feature from the Create stage.
> There may be sub-directories inside the stages directories, for different features. For example: `.../browser_ui/2_plan/ee_epics/` and `.../browser_ui/2_plan/issues/`. > There may be sub-directories inside the stages directories, for different features. For example: `.../browser_ui/2_plan/ee_epics/` and `.../browser_ui/2_plan/issues/`.
...@@ -60,7 +59,7 @@ Specs have an outer `context` that indicates the DevOps stage. The next level is ...@@ -60,7 +59,7 @@ Specs have an outer `context` that indicates the DevOps stage. The next level is
```ruby ```ruby
module QA module QA
context 'Plan' do context 'Plan' do
describe 'Editing scoped labels properties on issues' do describe 'Editing scoped labels on issues' do
end end
end end
end end
...@@ -68,12 +67,12 @@ end ...@@ -68,12 +67,12 @@ end
#### The `it` blocks #### The `it` blocks
Every test suite is composed by at least one `it` block, and a good way to start writing end-to-end tests is by typing test cases descriptions as `it` blocks. Take a look at the following example: Every test suite is composed of at least one `it` block, and a good way to start writing end-to-end tests is by writing test cases descriptions as `it` blocks. These might help you to think of different test scenarios. Take a look at the following example:
```ruby ```ruby
module QA module QA
context 'Plan' do context 'Plan' do
describe 'Editing scoped labels properties on issues' do describe 'Editing scoped labels on issues' do
it 'replaces an existing label if it has the same key' do it 'replaces an existing label if it has the same key' do
end end
...@@ -88,96 +87,104 @@ end ...@@ -88,96 +87,104 @@ end
For the [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of our test cases, let's say that we already have the application in the state needed for the tests, and then let's focus on the logic of the test cases only. For the [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of our test cases, let's say that we already have the application in the state needed for the tests, and then let's focus on the logic of the test cases only.
To evolve the test cases drafted on step 2, let's imagine that the user is already logged in a GitLab EE instance, they already have at least a Premium license in use, there is already a project created, there is already an issue opened in the project, the issue already has a scoped label (e.g. `foo::bar`), there are other scoped labels (for the same scope and for a different scope, e.g. `foo::baz` and `bar::bah`), and finally, the user is already on the issue's page. Let's also suppose that for every test case the application is in a clean state, meaning that one test case won't affect another. To evolve the test cases drafted on step 2, let's imagine that the user is already logged into a GitLab EE instance, they already have at least a Premium license in use, there is already a project created, there is already an issue opened in the project, the issue already has a scoped label (e.g. `animal::fox`), there are other scoped labels (for the same scope and for a different scope (e.g. `animal::dolphin` and `plant::orchid`), and finally, the user is already on the issue's page. Let's also suppose that for every test case the application is in a clean state, meaning that one test case won't affect another.
> Note: there are different approaches to create an application state for end-to-end tests. Some of them are very time consuming and subject to failures, such as when using the GUI for all the pre-conditions of the tests. On the other hand, other approaches are more efficient, such as using the public APIs. The latter is more efficient since it doesn't depend on the GUI. We won't focus on this part yet, but it's good to keep it in mind. > Note: there are different approaches to creating an application state for end-to-end tests. Some of them are very time consuming and subject to failures, such as when using the GUI for all the pre-conditions of the tests. On the other hand, other approaches are more efficient, such as using the public APIs. The latter is more efficient since it doesn't depend on the GUI. We won't focus on this part yet, but it's good to keep it in mind.
Let's now focus on the first test case. Let's now focus on the first test case.
```ruby ```ruby
it 'keeps the latest scoped label when adding a label with the same key of an existing one, but with a different value' do it 'replaces an existing label if it has the same key' do
# This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects. # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8).
page.find('.block.labels .edit-link').click page.find('.block.labels .edit-link').click
page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['foo::baz', :enter] page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['animal::dolphin', :enter]
page.find('#content-body').click page.find('#content-body').click
page.refresh page.refresh
scoped_label = page.find('.qa-labels-block .scoped-label-wrapper') labels_block = page.find('.qa-labels-block')
expect(scoped_label).to have_content('foo::baz') expect(labels_block).to have_content('animal::dolphin')
expect(scoped_label).not_to have_content('foo::bar') expect(labels_block).not_to have_content('animal::fox')
expect(page).to have_content('added foo::baz label and removed foo::bar') expect(page).to have_content('added animal::dolphin label and removed animal::fox')
end end
``` ```
> Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later.
> The exemplified test cases' MVC is not enough for the change to be submitted in an MR, but they help on building up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects](./qa/page/README.md) before they can be merged. > The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects](./qa/qa/page/README.md) before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language.
Below are the steps that the test covers: Below are the steps that the test covers:
1. The test finds the 'Edit' link for the labels and clicks on it 1. The test finds the 'Edit' link for the labels and clicks on it.
2. Then it fills in the 'Assign labels' input field with the value 'foo::baz' and press enter 2. Then it fills in the 'Assign labels' input field with the value 'animal::dolphin' and press enters.
3. Then it clicks in the content body to apply the label and refreshes the page 3. Then it clicks in the content body to apply the label and refreshes the page.
4. Finally the expectation that the previous scoped label was removed and that the new one was added happens 4. Finally, the expectations check that the previous scoped label was removed and that the new one was added.
Let's now see how the second test case would look like. Let's now see how the second test case would look.
```ruby ```ruby
it 'keeps both scoped labels when adding a label with a different key' do it 'keeps both scoped labels when adding a label with a different key' do
# This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects. # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8).
page.find('.block.labels .edit-link').click page.find('.block.labels .edit-link').click
page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['bar::bah', :enter] page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['plant::orchid', :enter]
page.find('#content-body').click page.find('#content-body').click
page.refresh page.refresh
scoped_labels = page.all('.qa-labels-block .scoped-label-wrapper') labels_block = page.find('.qa-labels-block')
expect(scoped_labels.first).to have_content('bar::bah') expect(labels_block).to have_content('animal::fox')
expect(scoped_labels.last).to have_content('foo::ba') expect(labels_block).to have_content('plant::orchid')
expect(page).to have_content('added bar::bah') expect(page).to have_content('added animal::fox')
expect(page).to have_content('added foo::ba') expect(page).to have_content('added plant::orchid')
end end
``` ```
> Note that elements are always located using CSS selectors, and a good practice is to add test specific attribute:value for elements (this is called adding testability to the application and we will talk more about it later.) > Note that elements are always located using CSS selectors, and a good practice is to add test-specific selectors (this is called adding testability to the application and we will talk more about it later.) For example, the `labels_block` element uses the selector `.qa-labels-block`, which was added specifically for testing purposes.
Below are the steps that the test covers: Below are the steps that the test covers:
1. The test finds the 'Edit' link for the labels and clicks on it 1. The test finds the 'Edit' link for the labels and clicks on it.
2. Then it fills in the 'Assign labels' input field with the value 'bar::bah' and press enter 2. Then it fills in the 'Assign labels' input field with the value 'plant::orchid' and press enters.
3. Then it clicks in the content body to apply the label and refreshes the page 3. Then it clicks in the content body to apply the label and refreshes the page.
4. Finally the expectation that the both scoped labels are present happens 4. Finally, the expectations check that both scoped labels are present.
> Similar to the previous test, this one is also very straight forward, but there is some code duplication. Let's address it. > Similar to the previous test, this one is also very straightforward, but there is some code duplication. Let's address it.
### 4. Extracting duplicated code ### 4. Extracting duplicated code
If we refactor the tests created on step 3 we could come up with something like this: If we refactor the tests created on step 3 we could come up with something like this:
```ruby ```ruby
it 'keeps the latest scoped label when adding a label with the same key of an existing one, but with a different value' do before do
select_label_and_refresh 'foo::baz' ...
@initial_label = 'animal::fox'
@new_label_same_scope = 'animal::dolphin'
@new_label_different_scope = 'plant::orchid'
...
end
expect(page).to have_content('added foo::baz') it 'replaces an existing label if it has the same key' do
expect(page).to have_content('and removed foo::bar') select_label_and_refresh @new_label_same_scope
scoped_label = page.find('.qa-labels-block .scoped-label-wrapper') labels_block = page.find('.qa-labels-block')
expect(scoped_label).to have_content('foo::baz') expect(labels_block).to have_content(@new_label_same_scope)
expect(scoped_label).not_to have_content('foo::bar') expect(labels_block).not_to have_content(@initial_label)
expect(page).to have_content("added #{@new_label_same_scope}")
expect(page).to have_content("and removed #{@initial_label}")
end end
it 'keeps both scoped label when adding a label with a different key' do it 'keeps both scoped label when adding a label with a different key' do
select_label_and_refresh 'bar::bah' select_label_and_refresh @new_label_different_scope
expect(page).to have_content('added bar::bah')
expect(page).to have_content('added foo::ba')
scoped_labels = page.all('.qa-labels-block .scoped-label-wrapper') labels_block = page.find('.qa-labels-block')
expect(scoped_labels.first).to have_content('bar::bah') expect(labels_blocks).to have_content(@new_label_different_scope)
expect(scoped_labels.last).to have_content('foo::ba') expect(labels_blocks).to have_content(@initial_label)
expect(page).to have_content("added #{@new_label_different_scope}")
expect(page).to have_content("added #{@initial_label}")
end end
def select_label_and_refresh(label) def select_label_and_refresh(label)
...@@ -188,72 +195,67 @@ def select_label_and_refresh(label) ...@@ -188,72 +195,67 @@ def select_label_and_refresh(label)
end end
``` ```
By creating a reusable `select_label_and_refresh` method, we remove the code duplication, and later we can move this method to a Page Object class that will be created for easier maintenance purposes. First, we remove the duplication of strings by defining the global variables `@initial_label`, `@new_label_same_scope` and `@new_label_different_scope` in the `before` block, and by using them in the expectations.
> Notice that the reusable method is created in the bottom of the file. The reason for that is that reading the code should be similar to reading a newspaper, where high-level information is at the top, like the title and summary of the news, while low level, or more specific information, is at the bottom. Then, by creating a reusable `select_label_and_refresh` method, we remove the code duplication of this action, and later we can move this method to a Page Object class that will be created for easier maintenance purposes.
> Notice that the reusable method is created at the bottom of the file. The reason for that is that reading the code should be similar to reading a newspaper, where high-level information is at the top, like the title and summary of the news, while low level, or more specific information, is at the bottom (this helps readability).
### 5. Tests' pre-conditions using resources and Page Objects ### 5. Tests' pre-conditions using resources and Page Objects
In this section, we will address the previously mentioned subject of creating the application state for the tests, using the `before :all` and `before` blocks, together with resources and Page Objects. In this section, we will address the previously mentioned subject of creating the application state for the tests, using the `before :context` and `before` blocks, together with resources and Page Objects.
#### `before :all` #### `before :context`
A pre-condition for the entire test suite is defined in the `before :all` block. A pre-condition for the entire test suite is defined in the `before :context` block.
For our test suite example, some things that could happen before the entire test suite starts are: > For our test suite, due to the need of the tests being completely independent of each other, we won't use the `before :context` block. The `before :context` block would make the tests dependent on each other because the first test changes the label of the issue, and the second one depends on the `'animal::fox'` label being set.
- The user logging in; > **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions.
- A premium license already being set up;
- A project being created with an issue and labels already setup.
> In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions.
#### `before` #### `before`
A pre-condition for each test case is defined in the `before` block. As the pre-conditions for our test suite, the things that needs to happen before each test starts are:
- The user logging in;
- A premium license already being set;
- A project being created with an issue and labels already set;
- The issue page being opened with only one scoped label applied to the it.
For our test cases samples, what we need is that for every test the issue page is opened, and there is only one scoped label applied to it. > When running end-to-end tests as part of the GitLab's continuous integration process [a license is already set as an environment variable](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a60d926740db10e3b5724713285780a4f470531/qa/qa/ee/strategy.rb#L20). For running tests locally you can set up such license by following the document [what tests can be run?](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-remote-grid-environment-variables), based on the [supported GitLab environment variables](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables).
#### Implementation #### Implementation
In the following code we will focus on the test suite and the test cases' pre-conditions only: In the following code we will focus only on the test suite's pre-conditions:
```ruby ```ruby
module QA module QA
context 'Plan' do context 'Plan' do
describe 'Editing scoped labels properties on issues' do describe 'Editing scoped labels on issues' do
before :all do before do
project = Resource::Project.fabricate_via_api! do |resource| Runtime::Browser.visit(:gitlab, Page::Main::Login)
resource.name = 'scoped-labels-project' Page::Main::Login.perform(&:sign_in_using_credentials)
end
@foo_bar_scoped_label = 'foo::bar' @initial_label = 'animal::fox'
@new_label_same_scope = 'animal::dolphin'
@new_label_different_scope = 'plant::orchid'
@issue = Resource::Issue.fabricate_via_api! do |issue| issue = Resource::Issue.fabricate_via_api! do |issue|
issue.project = project
issue.title = 'Issue to test the scoped labels' issue.title = 'Issue to test the scoped labels'
issue.labels = @foo_bar_scoped_label issue.labels = @initial_label
end end
@labels = ['foo::baz', 'bar::bah'] [@new_label_same_scope, @new_label_different_scope].each do |label|
@labels.each do |label|
Resource::Label.fabricate_via_api! do |l| Resource::Label.fabricate_via_api! do |l|
l.project = project.id l.project = issue.project.id
l.title = label l.title = label
end end
end end
Runtime::Browser.visit(:gitlab, Page::Main::Login) issue.visit!
Page::Main::Login.perform(&:sign_in_using_credentials)
end
before do
Page::Project::Issue::Show.perform do |issue_page|
@issue.visit!
end
end end
it 'keeps the latest scoped label when adding a label with the same key of an existing one, but with a different value' do it 'replaces an existing label if it has the same key' do
... ...
end end
...@@ -269,9 +271,11 @@ module QA ...@@ -269,9 +271,11 @@ module QA
end end
``` ```
In the `before :all` block we create all the application state needed for the tests to run. We do that by fabricating resources via APIs (`project`, `@issue`, and `@labels`), by using the `Runtime::Browser.visit` method to go to the login page, and by performing a `sign_in_using_credentials` from the `Login` Page Object. In the `before` block we create all the application state needed for the tests to run. We do that by using the `Runtime::Browser.visit` method to go to the login page, by performing a `sign_in_using_credentials` from the `Login` Page Object, by fabricating resources via APIs (`issue`, and `Resource::Label`), and by using the `issue.visit!` to visit the issue page.
> When creating the resources, notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `name` for the `project` resource; `project`, `title`, and `labels` for the `issue` resource; and `project`, and `title` for `label` resources. > A project is created in the background by creating the `issue` resource.
> When [creating the resources](./qa/qa/resource/README.md), notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource.
> What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage.
...@@ -281,58 +285,33 @@ In the `before :all` block we create all the application state needed for the te ...@@ -281,58 +285,33 @@ In the `before :all` block we create all the application state needed for the te
As already mentioned in the [best practices](./BEST_PRACTICES.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. As already mentioned in the [best practices](./BEST_PRACTICES.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible.
> Differently than unit tests, that exercise every little piece of the application in isolation, usually having only one assertion per test, and being very fast to run, end-to-end tests can have more actions and assertions in a single test to help on speeding up the test's feedback since they are much slower when comparing to unit tests. > Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test.
Some improvements that we could make in our test suite to optimize its time to run are: Some improvements that we could make in our test suite to optimize its time to run are:
1. Having a single test case (an `it` block) that exercise both scenarios to avoid "wasting" time in the tests' pre-conditions, instead of having two different test cases. 1. Having a single test case (an `it` block) that exercises both scenarios to avoid "wasting" time in the tests' pre-conditions, instead of having two different test cases.
2. Moving all the pre-conditions to the `before` block since there will be only one `it` block. 2. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method.
3. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method.
Let's look at a suggestion that addresses the above points, one by one: Let's look at a suggestion that addresses the above points, one by one:
```ruby ```ruby
module QA module QA
context 'Plan' do context 'Plan' do
describe 'Editing scoped labels properties on issues' do describe 'Editing scoped labels on issues' do
before do before do
project = Resource::Project.fabricate_via_api! do |resource| ...
resource.name = 'scoped-labels-project'
end
@foo_bar_scoped_label = 'foo::bar'
@issue = Resource::Issue.fabricate_via_api! do |issue|
issue.project = project
issue.title = 'Issue to test the scoped labels'
issue.labels = @foo_bar_scoped_label
end
@labels = ['foo::baz', 'bar::bah']
@labels.each do |label|
Resource::Label.fabricate_via_api! do |l|
l.project = project.id
l.title = label
end
end
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.perform(&:sign_in_using_credentials)
Page::Project::Issue::Show.perform do |issue_page|
@issue.visit!
end
end end
it 'correctly applies the scoped labels depending if they are from the same or a different scope' do it 'correctly applies scoped labels depending on if they are from the same or a different scope' do
select_labels_and_refresh @labels select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope]
scoped_labels = page.all('.qa-labels-block .scoped-label-wrapper') labels_block = page.all('.qa-labels-block')
expect(page).to have_content("added #{@foo_bar_scoped_label}") expect(labels_block).to have_content(@new_label_same_scope)
expect(page).to have_content("added #{@labels[1]} #{@labels[0]} labels and removed #{@foo_bar_scoped_label}") expect(labels_block).to have_content(@new_label_different_scope)
expect(scoped_labels.count).to eq(2) expect(labels_block).not_to have_content(@initial_label)
expect(scoped_labels.first).to have_content(@labels[1]) expect(page).to have_content("added #{@initial_label}")
expect(scoped_labels.last).to have_content(@labels[0]) expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}")
end end
def select_labels_and_refresh(labels) def select_labels_and_refresh(labels)
...@@ -348,20 +327,19 @@ module QA ...@@ -348,20 +327,19 @@ module QA
end end
``` ```
As you can see, now all the pre-conditions from the `before :all` block were moved to the `before` block, addressing point 2.
To address point 1, we changed the test implementation from two `it` blocks into a single one that exercises both scenarios. Now the new test description is: `'correctly applies the scoped labels depending if they are from the same or a different scope'`. It's a long description, but it describes well what the test does. To address point 1, we changed the test implementation from two `it` blocks into a single one that exercises both scenarios. Now the new test description is: `'correctly applies the scoped labels depending if they are from the same or a different scope'`. It's a long description, but it describes well what the test does.
> Notice that the implementation of the new and unique `it` block had to change a little bit. Below we describe in details what it does. > Notice that the implementation of the new and unique `it` block had to change a little bit. Below we describe in details what it does.
1. At the same time, it selects two scoped labels, one from the same scope of the one already applied in the issue during the setup phase (in the `before` block), and another one from a different scope. 1. It selects two scoped labels simultaneously, one from the same scope of the one already applied in the issue during the setup phase (in the `before` block), and another one from a different scope.
2. It runs the assertions that the labels where correctly added and removed; that only two labels are applied; and that those are the correct ones, and that they are shown in the right order. 2. It asserts that the correct labels are visible in the `labels_block`, and that the labels were correctly added and removed;
3. Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.)
Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.)
### 7. Resources ### 7. Resources
You can think of resources as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. **Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation.
You can think of [resources](qa/qa/resource/README.md) as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI.
With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc.
...@@ -369,21 +347,11 @@ As you saw in the tests' pre-conditions and the optimization sections, we're alr ...@@ -369,21 +347,11 @@ As you saw in the tests' pre-conditions and the optimization sections, we're alr
> We could be using the `fabricate!` method instead, which would use the `fabricate_via_api!` method if it exists, and fallback to GUI fabrication otherwise, but we recommend being explicit to make it clear what the test does. Also, we always recommend fabricating resources via API since this makes tests faster and more reliable. > We could be using the `fabricate!` method instead, which would use the `fabricate_via_api!` method if it exists, and fallback to GUI fabrication otherwise, but we recommend being explicit to make it clear what the test does. Also, we always recommend fabricating resources via API since this makes tests faster and more reliable.
For our test suite example, the [project resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/project.rb#L55) already had a `fabricate_via_api!` method available, while other resources don't have it, so we will have to create them, like for the issue and label resources. Also, we will have to make a small change in the project resource to expose its `id` attribute so that we can refer to it when fabricating the issue. For our test suite example, the resources that we need to create don't have the necessary code for the `fabricate_via_api!` method to correctly work (e.g., the issue and label resources), so we will have to create them.
#### Implementation #### Implementation
Following we describe the changes needed in every of the before-mentioned resource files. In the following we describe the changes needed in each of the resource files mentioned above.
**Project resource**
Let's start with the smallest change.
In the [project resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/project.rb), let's expose its `id` attribute.
Add the following `attribute :id` right below the [`attribute :description`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/project.rb#L11).
> This line is needed to allow for issues and labels to be automatically added to a project when fabricating them via API.
**Issue resource** **Issue resource**
...@@ -467,16 +435,148 @@ By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_ ...@@ -467,16 +435,148 @@ By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_
### 8. Page Objects ### 8. Page Objects
Page Objects are used in end-to-end tests for maintenance reasons, where a page's elements and methods are defined to be reused in any test.
> Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`). > Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`).
Page Objects are used in end-to-end tests for maintenance reasons, where page's elements and methods are defined to be reused in any test. Take a look at [this document for a detailed explanation about Page Objects](./qa/page/README.md).
Now, let's go back to our example.
As you may have noticed, we are defining elements with CSS selectors and the `select_labels_and_refresh` method directly in the test file, and this is an anti-pattern since we need to better separate the responsibilities.
To address this issue, we will move the implementation to Page Objects, and the test suite will only focus on the business rules that we are testing.
#### Updates in the test file
Take a look at [this document that specifically details the usage of Page Objects](./qa/page/README.md). As in a test-driven development approach, let's start changing the test file even before the Page Object implementation is in place.
Replace the code of the `it` block in the test file by the following:
```ruby
module QA
context 'Plan' do
describe 'Editing scoped labels on issues' do
before do
...
end
it 'correctly applies scoped labels depending on if they are from the same or a different scope' do
Page::Project::Issue::Show.perform do |issue_page|
issue_page.select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope]
expect(page).to have_content("added #{@initial_label}")
expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}")
expect(issue_page.text_of_labels_block).to have_content(@new_label_same_scope)
expect(issue_page.text_of_labels_block).to have_content(@new_label_different_scope)
expect(issue_page.text_of_labels_block).not_to have_content(@initial_label)
end
end
end
end
end
```
Notice that `select_labels_and_refresh` is now a method from the issue Page Object (which is not yet implemented), and that we verify the labels' text by using `text_of_labels_block`, instead of via the `labels_block` element. The `text_of_labels_block` method will also be implemented in the issue Page Object.
Let's now update the Issue Page Object.
#### Updates in the Issue Page Object
> Page Objects are located in the `qa/qa/page/` directory, and its sub-directories.
The file we will have to change is the [Issue Page Object](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/page/project/issue/show.rb).
First, add the following code right below the definition of an already implemented view:
```ruby
view 'app/views/shared/issuable/_sidebar.html.haml' do
element :labels_block
element :edit_link_labels
element :dropdown_menu_labels
end
view 'app/helpers/dropdowns_helper.rb' do
element :dropdown_input_field
end
```
Similarly to what we did before, let's first change the Page Object even without the elements being defined in the view (`_sidebar.html.haml`) and the `dropdowns_helper.rb` files, and later we will update them by adding the appropriate CSS selectors.
Now, let's implement the methods `select_labels_and_refresh` and `text_of_labels_block`.
Somewhere between the definition of the views and the private methods, add the following snippet of code:
```ruby
def select_labels_and_refresh(labels)
click_element(:edit_link_labels)
labels.each do |label|
within_element(:dropdown_menu_labels, text: label) do
send_keys_to_element(:dropdown_input_field, [label, :enter])
end
end
click_body
labels.each do |label|
has_element?(:labels_block, text: label)
end
refresh
end
def text_of_labels_block
find_element(:labels_block)
end
```
##### Details of `select_labels_and_refresh`
Notice that we have not only moved the `select_labels_and_refresh` method, but we have also changed its implementation to:
1. Click the `:edit_link_labels` element previously defined, instead of using `find('.block.labels .edit-link').click`
2. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]`
3. Use `click_body` after iterating on each label, instead of using `find('#content-body').click`
4. Iterate on every label again, and then we use `has_element?(:labels_block, text: label)` after clicking the page body (which applies the labels), and before refreshing the page, to avoid test flakiness due to refreshing too fast.
##### Details of `text_of_labels_block`
The `text_of_labels_block` method is a simple method that returns the `:labels_block` element (`find_element(:labels_block)`).
#### Updates in the view (*.html.haml) and `dropdowns_helper.rb` files
Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the Page Object.
In the [app/views/shared/issuable/_sidebar.html.haml](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/views/shared/issuable/_sidebar.html.haml) file, on [line 105 ](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L105), add an extra class `qa-edit-link-labels`.
The code should look like this: `= link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link qa-edit-link-labels float-right'`.
In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L121), add an extra class `.qa-dropdown-menu-labels`.
The code should look like this: `.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.qa-dropdown-menu-labels.dropdown-menu-selectable`.
In the [`dropdowns_helper.rb`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/helpers/dropdowns_helper.rb) file, on [line 94](https://gitlab.com/gitlab-org/gitlab-ee/blob/99e51a374f2c20bee0989cac802e4b5621f72714/app/helpers/dropdowns_helper.rb#L94), add an extra class `qa-dropdown-input-field`.
The code should look like this: `filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off'`.
> Classes starting with `qa-` are used for testing purposes only, and by defining such classes in the elements we add **testability** in the application.
> When defining a class like `qa-labels-block`, it is transformed into `:labels_block` for usage in the Page Objects. So, `qa-edit-link-labels` is tranformed into `:edit_link_labels`, `qa-dropdown-menu-labels` is transformed into `:dropdown_menu_labels`, and `qa-dropdown-input-field` is transformed into `:dropdown_input_field`. Also, we use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective `qa-` selectors in the specified views.
> We did not define the `qa-labels-block` class in the `app/views/shared/issuable/_sidebar.html.haml` file because it was already there to be used.
#### Updates in the `QA::Page::Base` class
The last thing that we have to do is to update `QA::Page::Base` class to add the `send_keys_to_element` method on it.
Add the following snippet of code somewhere where class methods are defined:
```ruby
def send_keys_to_element(name, keys)
find_element(name).send_keys(keys)
end
```
Now, let's go back to our examples. This method receives an element (`name`) and the `keys` that it will send to that element, and the keys are an array that can receive strings, or "special" keys, like `:enter`.
... As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`.
#### Adding testability ___
TBD. With that, you should be able to start writing end-to-end tests yourself. *Congratulations!*
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7