Commit 51830b66 authored by Kamil Trzciński's avatar Kamil Trzciński

Merge branch '22849-ci-build-ref-slug' into 'master'

Introduce $CI_BUILD_REF_SLUG

## What does this MR do?

Adds `$CI_BUILD_REF_SLUG` to the variables exposed to the runner. This is based on `$CI_BUILD_REF_NAME` but lowercased, shortened to 63 bytes maximum, and with characters invalid in URLs and domain names replaced with `-`. 

## Are there points in the code the reviewer needs to double check?

Slugs don't have a uniqueness guarantee. !7983 introduces an environment name slug which *is* unique, so I'm not as exercised about this as I was.

Should the slug be published in the API? It's available through the `variables` endpoint, but perhaps it should be part of `GET /project/:id/builds` ?

I've called it `ref_slug` rather than just `slug` as there are number of possibilities for slugification in a build (unlike an environment, where only the name makes sense to slugify).

## Why was this MR needed?

`$CI_BUILD_REF_NAME` is not suited for URLs and domain names, given the list of valid characters in a git ref. 

## Screenshots (if relevant)

## Does this MR meet the acceptance criteria?

- [x] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added
- [X] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
  - [X] Added for this feature/bug
  - [x] All builds are passing
- [X] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [X] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [X] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [X] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Closes #22849

See merge request !8072
parents 74e39274 1fdd5d68
...@@ -195,6 +195,17 @@ module Ci ...@@ -195,6 +195,17 @@ module Ci
project.build_timeout project.build_timeout
end end
# A slugified version of the build ref, suitable for inclusion in URLs and
# domain names. Rules:
#
# * Lowercased
# * Anything not matching [a-z0-9-] is replaced with a -
# * Maximum length is 63 bytes
def ref_slug
slugified = ref.to_s.downcase
slugified.gsub(/[^a-z0-9]/, '-')[0..62]
end
def variables def variables
variables = predefined_variables variables = predefined_variables
variables += project.predefined_variables variables += project.predefined_variables
...@@ -529,6 +540,7 @@ module Ci ...@@ -529,6 +540,7 @@ module Ci
{ key: 'CI_BUILD_REF', value: sha, public: true }, { key: 'CI_BUILD_REF', value: sha, public: true },
{ key: 'CI_BUILD_BEFORE_SHA', value: before_sha, public: true }, { key: 'CI_BUILD_BEFORE_SHA', value: before_sha, public: true },
{ key: 'CI_BUILD_REF_NAME', value: ref, public: true }, { key: 'CI_BUILD_REF_NAME', value: ref, public: true },
{ key: 'CI_BUILD_REF_SLUG', value: ref_slug, public: true },
{ key: 'CI_BUILD_NAME', value: name, public: true }, { key: 'CI_BUILD_NAME', value: name, public: true },
{ key: 'CI_BUILD_STAGE', value: stage, public: true }, { key: 'CI_BUILD_STAGE', value: stage, public: true },
{ key: 'CI_SERVER_NAME', value: 'GitLab', public: true }, { key: 'CI_SERVER_NAME', value: 'GitLab', public: true },
......
---
title: Introduce $CI_BUILD_REF_SLUG
merge_request: 8072
author:
...@@ -248,7 +248,7 @@ deploy_review: ...@@ -248,7 +248,7 @@ deploy_review:
- echo "Deploy a review app" - echo "Deploy a review app"
environment: environment:
name: review/$CI_BUILD_REF_NAME name: review/$CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_NAME.example.com url: https://$CI_BUILD_REF_SLUG.example.com
only: only:
- branches - branches
except: except:
...@@ -259,13 +259,14 @@ Let's break it down in pieces. The job's name is `deploy_review` and it runs ...@@ -259,13 +259,14 @@ Let's break it down in pieces. The job's name is `deploy_review` and it runs
on the `deploy` stage. The `script` at this point is fictional, you'd have to on the `deploy` stage. The `script` at this point is fictional, you'd have to
use your own based on your deployment. Then, we set the `environment` with the use your own based on your deployment. Then, we set the `environment` with the
`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting `environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting
one. Since the [environment name][env-name] can contain also slashes (`/`), we one. Since the [environment name][env-name] can contain slashes (`/`), we can
can use this pattern to distinguish between dynamic environments and the regular use this pattern to distinguish between dynamic environments and the regular
ones. ones.
So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME`
which takes the value of the branch name. We also use the same which takes the value of the branch name. Since `$CI_BUILD_REF_NAME` itself may
`$CI_BUILD_REF_NAME` value in the `environment:url` so that the environment also contain `/`, or other characters that would be invalid in a domain name or
URL, we use `$CI_BUILD_REF_SLUG` in the `environment:url` so that the environment
can get a specific and distinct URL for each branch. Again, the way you set up can get a specific and distinct URL for each branch. Again, the way you set up
the webserver to serve these requests is based on your setup. the webserver to serve these requests is based on your setup.
...@@ -299,7 +300,7 @@ deploy_review: ...@@ -299,7 +300,7 @@ deploy_review:
- echo "Deploy a review app" - echo "Deploy a review app"
environment: environment:
name: review/$CI_BUILD_REF_NAME name: review/$CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_NAME.example.com url: https://$CI_BUILD_REF_SLUG.example.com
only: only:
- branches - branches
except: except:
...@@ -329,16 +330,16 @@ deploy_prod: ...@@ -329,16 +330,16 @@ deploy_prod:
A more realistic example would include copying files to a location where a A more realistic example would include copying files to a location where a
webserver (NGINX) could then read and serve. The example below will copy the webserver (NGINX) could then read and serve. The example below will copy the
`public` directory to `/srv/nginx/$CI_BUILD_REF_NAME/public`: `public` directory to `/srv/nginx/$CI_BUILD_REF_SLUG/public`:
```yaml ```yaml
review_app: review_app:
stage: deploy stage: deploy
script: script:
- rsync -av --delete public /srv/nginx/$CI_BUILD_REF_NAME - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_SLUG
environment: environment:
name: review/$CI_BUILD_REF_NAME name: review/$CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_NAME.example.com url: https://$CI_BUILD_REF_SLUG.example.com
``` ```
It is assumed that the user has already setup NGINX and GitLab Runner in the It is assumed that the user has already setup NGINX and GitLab Runner in the
...@@ -346,7 +347,7 @@ server this job will run on. ...@@ -346,7 +347,7 @@ server this job will run on.
>**Note:** >**Note:**
Be sure to check out the [limitations](#limitations) section for some edge Be sure to check out the [limitations](#limitations) section for some edge
cases regarding naming of you branches and Review Apps. cases regarding naming of your branches and Review Apps.
--- ---
...@@ -418,7 +419,7 @@ deploy_review: ...@@ -418,7 +419,7 @@ deploy_review:
- echo "Deploy a review app" - echo "Deploy a review app"
environment: environment:
name: review/$CI_BUILD_REF_NAME name: review/$CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_NAME.example.com url: https://$CI_BUILD_REF_SLUG.example.com
on_stop: stop_review on_stop: stop_review
only: only:
- branches - branches
...@@ -480,9 +481,8 @@ exist, you should see something like: ...@@ -480,9 +481,8 @@ exist, you should see something like:
## Checkout deployments locally ## Checkout deployments locally
Since 8.13, a reference in the git repository is saved for each deployment. So Since 8.13, a reference in the git repository is saved for each deployment, so
knowing what the state is of your current environments is only a `git fetch` knowing the state of your current environments is only a `git fetch` away.
away.
In your git config, append the `[remote "<your-remote>"]` block with an extra In your git config, append the `[remote "<your-remote>"]` block with an extra
fetch line: fetch line:
...@@ -493,10 +493,10 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* ...@@ -493,10 +493,10 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/*
## Limitations ## Limitations
1. If the branch name contains special characters (`/`), and you use the 1. `$CI_BUILD_REF_SLUG` is not *guaranteed* to be unique, so there is a small
`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might chance of collisions between similarly-named branches (`fix-foo` would
be complications during your Review Apps deployment. Follow the conflict with `fix/foo`, for instance). Following a well-defined workflow
[issue 22849][ce-22849] for more information. such as [GitLab Flow][gitlab-flow] can keep this from being a problem.
1. You are limited to use only the [CI predefined variables][variables] in the 1. You are limited to use only the [CI predefined variables][variables] in the
`environment: name`. If you try to re-use variables defined inside `script` `environment: name`. If you try to re-use variables defined inside `script`
as part of the environment name, it will not work. as part of the environment name, it will not work.
...@@ -520,6 +520,6 @@ Below are some links you may find interesting: ...@@ -520,6 +520,6 @@ Below are some links you may find interesting:
[only]: yaml/README.md#only-and-except [only]: yaml/README.md#only-and-except
[onstop]: yaml/README.md#environment-on_stop [onstop]: yaml/README.md#environment-on_stop
[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 [ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015
[ce-22849]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22849 [gitlab-flow]: ../workflow/gitlab_flow.md
[gitlab runner]: https://docs.gitlab.com/runner/ [gitlab runner]: https://docs.gitlab.com/runner/
[git-strategy]: yaml/README.md#git-strategy [git-strategy]: yaml/README.md#git-strategy
...@@ -40,6 +40,7 @@ version of Runner required. ...@@ -40,6 +40,7 @@ version of Runner required.
| **CI_BUILD_NAME** | all | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | | **CI_BUILD_NAME** | all | 0.5 | The name of the build as defined in `.gitlab-ci.yml` |
| **CI_BUILD_STAGE** | all | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | **CI_BUILD_STAGE** | all | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` |
| **CI_BUILD_REF_NAME** | all | all | The branch or tag name for which project is built | | **CI_BUILD_REF_NAME** | all | all | The branch or tag name for which project is built |
| **CI_BUILD_REF_SLUG** | 8.15 | all | `$CI_BUILD_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. |
| **CI_BUILD_REPO** | all | all | The URL to clone the Git repository | | **CI_BUILD_REPO** | all | all | The URL to clone the Git repository |
| **CI_BUILD_TRIGGERED** | all | 0.5 | The flag to indicate that build was [triggered] | | **CI_BUILD_TRIGGERED** | all | 0.5 | The flag to indicate that build was [triggered] |
| **CI_BUILD_MANUAL** | 8.12 | all | The flag to indicate that build was manually started | | **CI_BUILD_MANUAL** | 8.12 | all | The flag to indicate that build was manually started |
......
...@@ -690,18 +690,12 @@ The `stop_review_app` job is **required** to have the following keywords defined ...@@ -690,18 +690,12 @@ The `stop_review_app` job is **required** to have the following keywords defined
#### dynamic environments #### dynamic environments
> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. > [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6.
`$CI_BUILD_REF_SLUG` was [introduced][ce-8072] in GitLab 8.15.
`environment` can also represent a configuration hash with `name` and `url`. `environment` can also represent a configuration hash with `name` and `url`.
These parameters can use any of the defined [CI variables](#variables) These parameters can use any of the defined [CI variables](#variables)
(including predefined, secure variables and `.gitlab-ci.yml` variables). (including predefined, secure variables and `.gitlab-ci.yml` variables).
>**Note:**
Be aware than if the branch name contains special characters and you use the
`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might
be complications during deployment. Follow the
[issue 22849](https://gitlab.com/gitlab-org/gitlab-ce/issues/22849) for more
information.
For example: For example:
``` ```
...@@ -710,7 +704,7 @@ deploy as review app: ...@@ -710,7 +704,7 @@ deploy as review app:
script: make deploy script: make deploy
environment: environment:
name: review-apps/$CI_BUILD_REF_NAME name: review-apps/$CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_NAME.review.example.com/ url: https://$CI_BUILD_REF_SLUG.review.example.com/
``` ```
The `deploy as review app` job will be marked as deployment to dynamically The `deploy as review app` job will be marked as deployment to dynamically
...@@ -726,6 +720,10 @@ The common use case is to create dynamic environments for branches and use them ...@@ -726,6 +720,10 @@ The common use case is to create dynamic environments for branches and use them
as Review Apps. You can see a simple example using Review Apps at as Review Apps. You can see a simple example using Review Apps at
https://gitlab.com/gitlab-examples/review-apps-nginx/. https://gitlab.com/gitlab-examples/review-apps-nginx/.
`$CI_BUILD_REF_SLUG` is another environment variable set by the runner, based on
`$CI_BUILD_REF_NAME` but lower-cased, and with some characters replaced with
`-`, making it suitable for use in URLs and domain names.
### artifacts ### artifacts
>**Notes:** >**Notes:**
...@@ -1245,4 +1243,5 @@ CI with various languages. ...@@ -1245,4 +1243,5 @@ CI with various languages.
[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 [ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323
[environment]: ../environments.md [environment]: ../environments.md
[ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 [ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669
[ce-8072]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/xxxx
[variables]: ../variables/README.md [variables]: ../variables/README.md
...@@ -254,6 +254,24 @@ describe Ci::Build, models: true do ...@@ -254,6 +254,24 @@ describe Ci::Build, models: true do
end end
end end
describe '#ref_slug' do
{
'master' => 'master',
'1-foo' => '1-foo',
'fix/1-foo' => 'fix-1-foo',
'fix-1-foo' => 'fix-1-foo',
'a' * 63 => 'a' * 63,
'a' * 64 => 'a' * 63,
'FOO' => 'foo',
}.each do |ref, slug|
it "transforms #{ref} to #{slug}" do
build.ref = ref
expect(build.ref_slug).to eq(slug)
end
end
end
describe '#variables' do describe '#variables' do
let(:container_registry_enabled) { false } let(:container_registry_enabled) { false }
let(:predefined_variables) do let(:predefined_variables) do
...@@ -265,6 +283,7 @@ describe Ci::Build, models: true do ...@@ -265,6 +283,7 @@ describe Ci::Build, models: true do
{ key: 'CI_BUILD_REF', value: build.sha, public: true }, { key: 'CI_BUILD_REF', value: build.sha, public: true },
{ key: 'CI_BUILD_BEFORE_SHA', value: build.before_sha, public: true }, { key: 'CI_BUILD_BEFORE_SHA', value: build.before_sha, public: true },
{ key: 'CI_BUILD_REF_NAME', value: 'master', public: true }, { key: 'CI_BUILD_REF_NAME', value: 'master', public: true },
{ key: 'CI_BUILD_REF_SLUG', value: 'master', public: true },
{ key: 'CI_BUILD_NAME', value: 'test', public: true }, { key: 'CI_BUILD_NAME', value: 'test', public: true },
{ key: 'CI_BUILD_STAGE', value: 'test', public: true }, { key: 'CI_BUILD_STAGE', value: 'test', public: true },
{ key: 'CI_SERVER_NAME', value: 'GitLab', public: true }, { key: 'CI_SERVER_NAME', value: 'GitLab', public: true },
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment