Commit 6f15d027 authored by Lin Jen-Shin's avatar Lin Jen-Shin

Merge branch 'docs-improve-end-to-ends-tests-to-mention-review-qa-all-job' into 'master'

Mention review-qa-all in the end-to-end tests documentation

See merge request gitlab-org/gitlab-ce!26541
parents 1f1e965b 4793b281
...@@ -27,10 +27,15 @@ Results are reported in the `#qa-staging` Slack channel. ...@@ -27,10 +27,15 @@ Results are reported in the `#qa-staging` Slack channel.
### Testing code in merge requests ### Testing code in merge requests
#### Using the `package-and-qa` job
It is possible to run end-to-end tests for a merge request, eventually being run in It is possible to run end-to-end tests for a merge request, eventually being run in
a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project, a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project,
by triggering the `package-and-qa` manual action in the `test` stage (which should by triggering the `package-and-qa` manual action in the `test` stage (not
be present in a merge request widget, unless the merge request comes from a fork). available for forks).
**This runs end-to-end tests against a custom Omnibus package built from your
merge request's changes.**
Manual action that starts end-to-end tests is also available in merge requests Manual action that starts end-to-end tests is also available in merge requests
in [Omnibus GitLab][omnibus-gitlab]. in [Omnibus GitLab][omnibus-gitlab].
...@@ -85,7 +90,25 @@ subgraph gitlab-qa pipeline ...@@ -85,7 +90,25 @@ subgraph gitlab-qa pipeline
1. The result of the [GitLab QA pipeline][gitlab-qa-pipelines] is being 1. The result of the [GitLab QA pipeline][gitlab-qa-pipelines] is being
propagated upstream, through Omnibus, back to the CE / EE merge request. propagated upstream, through Omnibus, back to the CE / EE merge request.
#### How do I write tests? #### Using the `review-qa-all` jobs
On every pipeline during the `test` stage, the `review-qa-smoke` job is
automatically started: it runs the QA smoke suite against the
[Review App][review-apps].
You can also manually start the `review-qa-all`: it runs the full QA suite
against the [Review App][review-apps].
**This runs end-to-end tests against a Review App based on [the official GitLab
Helm chart][helm-chart], itself deployed with custom
[Cloud Native components][cng] built from your merge request's changes.**
See [Review Apps][review-apps] for more details about Review Apps.
[helm-chart]: https://gitlab.com/charts/gitlab/
[cng]: https://gitlab.com/gitlab-org/build/CNG
## How do I write tests?
In order to write new tests, you first need to learn more about GitLab QA In order to write new tests, you first need to learn more about GitLab QA
architecture. See the [documentation about it][gitlab-qa-architecture]. architecture. See the [documentation about it][gitlab-qa-architecture].
...@@ -105,9 +128,11 @@ you can find an issue you would like to work on in ...@@ -105,9 +128,11 @@ you can find an issue you would like to work on in
[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab [omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab
[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa [gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines
[gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md
[quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines
[quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines
[review-apps]: ./review_apps.md
[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md
[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario
[gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test
......
...@@ -90,8 +90,8 @@ subgraph GCP `gitlab-review-apps` project ...@@ -90,8 +90,8 @@ subgraph GCP `gitlab-review-apps` project
## QA runs ## QA runs
On every [pipeline][gitlab-pipeline] during the `test` stage, the On every [pipeline][gitlab-pipeline] during the `test` stage, the
`review-qa-smoke` job is automatically started: it runs the smoke QA suite. `review-qa-smoke` job is automatically started: it runs the QA smoke suite.
You can also manually start the `review-qa-all`: it runs the full QA suite. You can also manually start the `review-qa-all`: it runs the QA full suite.
Note that both jobs first wait for the `review-deploy` job to be finished. Note that both jobs first wait for the `review-deploy` job to be finished.
......
...@@ -7,13 +7,19 @@ functionality is working. ...@@ -7,13 +7,19 @@ functionality is working.
Currently, our suite consists of this basic functionality coverage: Currently, our suite consists of this basic functionality coverage:
- User Login (Standard Auth) - User standard authentication
- Project Creation - SSH Key creation and addition to a user
- Issue Creation - Project simple creation
- Merge Request Creation - Project creation with Auto-DevOps enabled
- Issue creation
- Merge Request creation
- Snippet creation
Smoke tests have the `:smoke` RSpec metadata. Smoke tests have the `:smoke` RSpec metadata.
See [End-to-end Testing](./end_to_end_tests.md) for more details about
end-to-end tests.
--- ---
[Return to Testing documentation](index.md) [Return to Testing documentation](index.md)
...@@ -234,6 +234,8 @@ you should write an integration test using Jasmine. ...@@ -234,6 +234,8 @@ you should write an integration test using Jasmine.
[big]: https://twitter.com/timbray/status/822470746773409794 [big]: https://twitter.com/timbray/status/822470746773409794
[picture]: https://twitter.com/withzombies/status/829716565834752000 [picture]: https://twitter.com/withzombies/status/829716565834752000
[tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e [tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e
[RSpec]: https://github.com/rspec/rspec-rails#feature-specs
[Capybara]: https://github.com/teamcapybara/capybara
--- ---
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment