Commit eb066f64 authored by ddavison's avatar ddavison

Consolidate documentation written for e2e tests under dev docs

Previously the documentation was separated.  If we want to bake
quality into the product, how better than to include everything
we use directly in the development documentation
Signed-off-by: default avatarddavison <ddavison@gitlab.com>

Fix broken internal doc link

Move documentation for page objects to the e2e section
parent 13cd5b59
...@@ -35,4 +35,4 @@ Finally, interacting with the application only by its GUI generates a higher rat ...@@ -35,4 +35,4 @@ Finally, interacting with the application only by its GUI generates a higher rat
- Building state through the GUI is time consuming and it's not sustainable as the test suite grows. - Building state through the GUI is time consuming and it's not sustainable as the test suite grows.
- When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness. - When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness.
Now that we are aware of all of it, [let's go create some tests](writing_tests_from_scratch.md#this-document-covers-the-following-items). Now that we are aware of all of it, [let's go create some tests](quick_start_guide.md).
...@@ -65,7 +65,7 @@ Below you can read more about how to use it and how does it work. ...@@ -65,7 +65,7 @@ Below you can read more about how to use it and how does it work.
Currently, we are using _multi-project pipeline_-like approach to run QA Currently, we are using _multi-project pipeline_-like approach to run QA
pipelines. pipelines.
![QA on merge requests CI/CD architecture](img/qa_on_merge_requests_cicd_architecture.png) ![QA on merge requests CI/CD architecture](../img/qa_on_merge_requests_cicd_architecture.png)
<details> <details>
<summary>Show mermaid source</summary> <summary>Show mermaid source</summary>
...@@ -136,6 +136,12 @@ Once you decided where to put [test environment orchestration scenarios] and ...@@ -136,6 +136,12 @@ Once you decided where to put [test environment orchestration scenarios] and
the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing
instance-level scenarios][instance-level scenarios]. instance-level scenarios][instance-level scenarios].
Continued reading:
- [Quick Start Guide](quick_start_guide.md)
- [Style Guide](style_guide.md)
- [Best Practices](best_practices.md)
## Where can I ask for help? ## Where can I ask for help?
You can ask question in the `#quality` channel on Slack (GitLab internal) or You can ask question in the `#quality` channel on Slack (GitLab internal) or
...@@ -149,7 +155,7 @@ you can find an issue you would like to work on in ...@@ -149,7 +155,7 @@ you can find an issue you would like to work on in
[gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md
[quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines
[quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines
[review-apps]: ./review_apps.md [review-apps]: ../review_apps.md
[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md
[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario
[gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test
......
...@@ -17,10 +17,10 @@ If you don't exactly understand what we mean by **not everything needs to happen ...@@ -17,10 +17,10 @@ If you don't exactly understand what we mean by **not everything needs to happen
- [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`) - [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`)
- [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic - [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic
- [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods - [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods
- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects](./qa/qa/page/README.md) - [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects]
- [6.](#6-optimization) Optimizing the test suite - [6.](#6-optimization) Optimizing the test suite
- [7.](#7-resources) Using and implementing resources - [7.](#7-resources) Using and implementing resources
- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects](./qa/qa/page/README.md) - [8.](#8-page-objects) Moving element definitions and methods to [Page Objects]
### 0. Are end-to-end tests needed? ### 0. Are end-to-end tests needed?
...@@ -111,7 +111,7 @@ end ...@@ -111,7 +111,7 @@ end
> Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later.
> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects](./qa/qa/page/README.md) before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. > The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language.
Below are the steps that the test covers: Below are the steps that the test covers:
...@@ -275,7 +275,7 @@ In the `before` block we create all the application state needed for the tests t ...@@ -275,7 +275,7 @@ In the `before` block we create all the application state needed for the tests t
> A project is created in the background by creating the `issue` resource. > A project is created in the background by creating the `issue` resource.
> When [creating the resources](./qa/qa/resource/README.md), notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. > When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource.
> What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage.
...@@ -283,7 +283,7 @@ In the `before` block we create all the application state needed for the tests t ...@@ -283,7 +283,7 @@ In the `before` block we create all the application state needed for the tests t
### 6. Optimization ### 6. Optimization
As already mentioned in the [best practices](./BEST_PRACTICES.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. As already mentioned in the [best practices](best_practices.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible.
> Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test. > Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test.
...@@ -339,7 +339,7 @@ To address point 1, we changed the test implementation from two `it` blocks into ...@@ -339,7 +339,7 @@ To address point 1, we changed the test implementation from two `it` blocks into
**Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation. **Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation.
You can think of [resources](qa/qa/resource/README.md) as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. You can think of [Resources] as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI.
With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc.
...@@ -439,7 +439,7 @@ Page Objects are used in end-to-end tests for maintenance reasons, where a page' ...@@ -439,7 +439,7 @@ Page Objects are used in end-to-end tests for maintenance reasons, where a page'
> Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`). > Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`).
Take a look at [this document for a detailed explanation about Page Objects](./qa/page/README.md). Take a look at the [Page Objects] documentation.
Now, let's go back to our example. Now, let's go back to our example.
...@@ -580,3 +580,6 @@ As you might remember, in the Issue Page Object we call this method like this: ` ...@@ -580,3 +580,6 @@ As you might remember, in the Issue Page Object we call this method like this: `
___ ___
With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* With that, you should be able to start writing end-to-end tests yourself. *Congratulations!*
[Page Objects]: page_objects.md
[Resources]: resources.md
...@@ -5,11 +5,11 @@ be created via the API or the CLI. ...@@ -5,11 +5,11 @@ be created via the API or the CLI.
## How to properly implement a resource class? ## How to properly implement a resource class?
All resource classes should inherit from [`Resource::Base`](./base.rb). All resource classes should inherit from `Resource::Base`.
There is only one mandatory method to implement to define a resource class. There is only one mandatory method to implement to define a resource class.
This is the `#fabricate!` method, which is used to build the resource via the This is the `#fabricate!` method, which is used to build the resource via the
browser UI. Note that you should only use [Page objects](../page/README.md) to browser UI. Note that you should only use [Page objects](page_objects.md) to
interact with a Web page in this method. interact with a Web page in this method.
Here is an imaginary example: Here is an imaginary example:
...@@ -74,7 +74,7 @@ module QA ...@@ -74,7 +74,7 @@ module QA
end end
``` ```
The [`Project` resource](./project.rb) is a good real example of Browser The `Project` resource is a good real example of Browser
UI and API implementations. UI and API implementations.
#### Resource attributes #### Resource attributes
......
# Style guide for writing E2E tests # Style guide for writing end-to-end tests
This document describes the conventions used at GitLab for writing E2E tests using the GitLab QA project. This document describes the conventions used at GitLab for writing End-to-end (E2E) tests using the GitLab QA project.
## `click_` versus `go_to_` ## `click_` versus `go_to_`
...@@ -45,7 +45,7 @@ Notice that in the above example, before clicking the `:operations_environments_ ...@@ -45,7 +45,7 @@ Notice that in the above example, before clicking the `:operations_environments_
> We can create these methods as helpers to abstract multi-step navigation. > We can create these methods as helpers to abstract multi-step navigation.
### Element Naming Convention ### Element naming convention
When adding new elements to a page, it's important that we have a uniform element naming convention. When adding new elements to a page, it's important that we have a uniform element naming convention.
...@@ -70,6 +70,7 @@ We follow a simple formula roughly based on hungarian notation. ...@@ -70,6 +70,7 @@ We follow a simple formula roughly based on hungarian notation.
#### Examples #### Examples
**Good** **Good**
```ruby ```ruby
view '...' do view '...' do
element :edit_button element :edit_button
...@@ -81,6 +82,7 @@ end ...@@ -81,6 +82,7 @@ end
``` ```
**Bad** **Bad**
```ruby ```ruby
view '...' do view '...' do
# `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate. # `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate.
......
...@@ -71,7 +71,7 @@ Everything you should know about how to test Rake tasks. ...@@ -71,7 +71,7 @@ Everything you should know about how to test Rake tasks.
--- ---
## [End-to-end tests](end_to_end_tests.md) ## [End-to-end tests](end_to_end/index.md)
Everything you should know about how to run end-to-end tests using Everything you should know about how to run end-to-end tests using
[GitLab QA][gitlab-qa] testing framework. [GitLab QA][gitlab-qa] testing framework.
......
...@@ -17,7 +17,7 @@ Currently, our suite consists of this basic functionality coverage: ...@@ -17,7 +17,7 @@ Currently, our suite consists of this basic functionality coverage:
Smoke tests have the `:smoke` RSpec metadata. Smoke tests have the `:smoke` RSpec metadata.
See [End-to-end Testing](./end_to_end_tests.md) for more details about See [End-to-end Testing](end_to_end/index.md) for more details about
end-to-end tests. end-to-end tests.
--- ---
......
...@@ -178,7 +178,7 @@ Every new feature should come with a [test plan]. ...@@ -178,7 +178,7 @@ Every new feature should come with a [test plan].
| ---------- | -------------- | ----- | | ---------- | -------------- | ----- |
| `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] | | `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] |
> See [end-to-end tests](end_to_end_tests.md) for more information. > See [end-to-end tests](end_to_end/index.md) for more information.
Note that `qa/spec` contains unit tests of the QA framework itself, not to be Note that `qa/spec` contains unit tests of the QA framework itself, not to be
confused with the application's [unit tests](#unit-tests) or confused with the application's [unit tests](#unit-tests) or
......
# GitLab QA - End-to-end tests for GitLab # GitLab QA - End-to-end tests for GitLab
This directory contains [end-to-end tests](doc/development/testing_guide/end_to_end_tests.md) This directory contains [end-to-end tests](../../../doc/development/testing_guide/end_to_end/index.md)
for GitLab. It includes the test framework and the tests themselves. for GitLab. It includes the test framework and the tests themselves.
The tests can be found in `qa/specs/features` (not to be confused with the unit The tests can be found in `qa/specs/features` (not to be confused with the unit
...@@ -29,7 +29,7 @@ verify coupling between page objects implemented as a part of GitLab QA ...@@ -29,7 +29,7 @@ verify coupling between page objects implemented as a part of GitLab QA
and corresponding views / partials / selectors in CE / EE. and corresponding views / partials / selectors in CE / EE.
Whenever `qa:selectors` job fails in your merge request, you are supposed to Whenever `qa:selectors` job fails in your merge request, you are supposed to
fix [page objects](qa/page/README.md). You should also trigger end-to-end tests fix [page objects](../doc/development/testing_guide/end_to_end/page_objects.md). You should also trigger end-to-end tests
using `package-and-qa` manual action, to test if everything works fine. using `package-and-qa` manual action, to test if everything works fine.
## How can I use it? ## How can I use it?
...@@ -49,10 +49,10 @@ will need to [modify your GDK setup](https://gitlab.com/gitlab-org/gitlab-qa/blo ...@@ -49,10 +49,10 @@ will need to [modify your GDK setup](https://gitlab.com/gitlab-org/gitlab-qa/blo
### Writing tests ### Writing tests
- [Writing tests from scratch tutorial](docs/writing_tests_from_scratch.md) - [Writing tests from scratch tutorial](../doc/development/testing_guide/end_to_end/quick_start_guide.md)
- [Best practices](docs/best_practices.md) - [Best practices](../doc/development/testing_guide/best_practices.md)
- [Using page objects](qa/page/README.md) - [Using page objects](../doc/development/testing_guide/end_to_end/page_objects.md)
- [Guidelines](docs/guidelines.md) - [Guidelines](../doc/development/testing_guide/index.md)
### Running specific tests ### Running specific tests
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment