[Read more about using object storage with GitLab](../object_storage.md).
NOTE:
We recommend using the [consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). The following instructions apply to the original config format.
We recommend using the [consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). The following instructions apply to the original configuration format.
@@ -204,7 +204,7 @@ control over how the Pages daemon runs and serves content in your environment.
| `artifacts_server_url` | API URL to proxy artifact requests to. Defaults to GitLab `external URL` + `/api/v4`, for example `https://gitlab.com/api/v4`.
| `auth_redirect_uri` | Callback URL for authenticating with GitLab. Defaults to project's subdomain of `pages_external_url` + `/auth`.
| `auth_secret` | Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration.
| `dir` | Working directory for config and secrets files.
| `dir` | Working directory for configuration and secrets files.
| `enable` | Enable or disable GitLab Pages on the current system.
| `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`.
| `external_https` | Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_https`.
...
...
@@ -241,7 +241,7 @@ control over how the Pages daemon runs and serves content in your environment.
| `pages_path` | The directory on disk where pages are stored, defaults to `GITLAB-RAILS/shared/pages`.
| `pages_nginx[]` | |
| `enable` | Include a virtual host `server{}` block for Pages inside NGINX. Needed for NGINX to proxy traffic back to the Pages daemon. Set to `false` if the Pages daemon should directly receive all requests, for example, when using [custom domains](index.md#custom-domains).
| `FF_ENABLE_REDIRECTS` | Feature flag to disable redirects (enabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#disable-redirects) for more info. |
| `FF_ENABLE_REDIRECTS` | Feature flag to disable redirects (enabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#disable-redirects) for more information. |
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
### Scope: commits **(STARTER)**
...
...
@@ -675,7 +675,7 @@ Example response:
```
NOTE:
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
### Scope: commits **(STARTER)**
...
...
@@ -1072,7 +1072,7 @@ Example response:
```
NOTE:
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
### Scope: commits
...
...
@@ -1146,7 +1146,7 @@ Example response:
```
NOTE:
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
`filename` is deprecated in favor of `path`. Both return the full path of the file inside the repository, but in the future `filename` will be only the filename and not the full path. For details, see [issue 34521](https://gitlab.com/gitlab-org/gitlab/-/issues/34521).
your project, update commit statuses, and create a web hook to notify
GitLab of new commits.
1. In GitLab, go to the [new project page](../../gitlab-basics/create-project.md#create-a-project-in-gitlab), select the **CI/CD for external repo** tab, and then click
1. In GitLab, go to the [new project page](../../gitlab-basics/create-project.md#create-a-project-in-gitlab), select the **CI/CD for external repository** tab, and then click
**GitHub**.
1. Paste the token into the **Personal access token** field and click **List
In case you're interested in deploying Spring Boot applications to Kubernetes
using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/blog/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/).
@@ -12,7 +12,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla
## Initialize the module
1. Open a terminal and navigate to the project's repo
1. Open a terminal and navigate to the project's repository
1. Run `npm init`. Name the module according to [the Package Registry's naming conventions](../../user/packages/npm_registry/index.md#package-naming-convention). For example, if the project's path is `gitlab-examples/semantic-release-npm`, name the module `@gitlab-examples/semantic-release-npm`.
1. Install the following NPM packages:
...
...
@@ -97,7 +97,7 @@ As part of publishing a package, semantic-release increases the version number i
## Configure semantic-release
semantic-release pulls its configuration info from a `.releaserc.json` file in the project. Create a `.releaserc.json` at the root of the repository:
semantic-release pulls its configuration information from a `.releaserc.json` file in the project. Create a `.releaserc.json` at the root of the repository:
@@ -10,7 +10,7 @@ GitLab community members and their privileges/responsibilities.
| Roles | Responsibilities | Requirements |
|-------|------------------|--------------|
| Maintainer | Accepts merge requests on several GitLab projects | Added to the [team page](https://about.gitlab.com/company/team/). An expert on code reviews and knows the product/codebase |
| Maintainer | Accepts merge requests on several GitLab projects | Added to the [team page](https://about.gitlab.com/company/team/). An expert on code reviews and knows the product/codebase |
| Reviewer | Performs code reviews on MRs | Added to the [team page](https://about.gitlab.com/company/team/) |
| Developer |Has access to GitLab internal infrastructure & issues (e.g. HR-related) | GitLab employee or a Core Team member (with an NDA) |
| Contributor | Can make contributions to all GitLab public projects | Have a GitLab.com account |
@@ -153,7 +153,7 @@ test its execution using `CREATE INDEX CONCURRENTLY` in the `#database-lab` Slac
projects provide enough data to serve as a good example.
- That means that no query plan should return 0 records or less records than the provided limit (if a limit is included). If a query is used in batching, a proper example batch with adequate included results should be identified and provided.
- If your queries belong to a new feature in GitLab.com and thus they don't return data in production, it's suggested to analyze the query and to provide the plan from a local environment.
- More info on how to find the number of actual returned records in [Understanding EXPLAIN plans](understanding_explain_plans.md)
- More information on how to find the number of actual returned records in [Understanding EXPLAIN plans](understanding_explain_plans.md)
- For query changes, it is best to provide both the SQL queries along with the
plan _before_ and _after_ the change. This helps spot differences quickly.
- Include data that shows the performance improvement, preferably in
@@ -14,7 +14,14 @@ Currently we rely on different sources to present diffs, these include:
## Deep Dive
In January 2019, Oswaldo Ferreira hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/issues/1`) on GitLab's Diffs and Commenting on Diffs functionality to share his domain specific knowledge with anyone who may work in this part of the code base in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=K6G3gMcFyek), and the slides on [Google Slides](https://docs.google.com/presentation/d/1bGutFH2AT3bxOPZuLMGl1ANWHqFnrxwQwjiwAZkF-TU/edit) and in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/b5ad2f336e0afcfe0f99db0af0ccc71a/). Everything covered in this deep dive was accurate as of GitLab 11.7, and while specific details may have changed since then, it should still serve as a good introduction.
In January 2019, Oswaldo Ferreira hosted a Deep Dive (GitLab team members only:
`https://gitlab.com/gitlab-org/create-stage/issues/1`) on GitLab's Diffs and Commenting on Diffs
functionality to share his domain specific knowledge with anyone who may work in this part of the
codebase in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=K6G3gMcFyek),
and the slides on [Google Slides](https://docs.google.com/presentation/d/1bGutFH2AT3bxOPZuLMGl1ANWHqFnrxwQwjiwAZkF-TU/edit)
and in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/b5ad2f336e0afcfe0f99db0af0ccc71a/).
Everything covered in this deep dive was accurate as of GitLab 11.7, and while specific details may
have changed since then, it should still serve as a good introduction.
| `markdownlint-cli` | Specfic | `yarn global add markdownlint-cli@0.23.2` | The `@` indicates a specific version, and this example updates the tool to version `0.23.2`. |
| Vale | Latest | `brew update && brew upgrade vale` | This command is for macOS only. |
| Vale | Specific | n/a | Not possible using `brew`, but can be [directly downloaded](https://github.com/errata-ai/vale/releases). |
@@ -13,7 +13,7 @@ the [Elasticsearch integration documentation](../integration/elasticsearch.md#en
## Deep Dive
In June 2019, Mario de la Ossa hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/issues/1`) on GitLab's [Elasticsearch integration](../integration/elasticsearch.md) to share his domain specific knowledge with anyone who may work in this part of the codebase in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=vrvl-tN2EaA), and the slides on [Google Slides](https://docs.google.com/presentation/d/1H-pCzI_LNrgrL5pJAIQgvLX8Ji0-jIKOg1QeJQzChug/edit) and in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/c5aa32b6b07476fa8b597004899ec538/Elasticsearch_Deep_Dive.pdf). Everything covered in this deep dive was accurate as of GitLab 12.0, and while specific details may have changed since then, it should still serve as a good introduction.
In June 2019, Mario de la Ossa hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/issues/1`) on GitLab's [Elasticsearch integration](../integration/elasticsearch.md) to share his domain specific knowledge with anyone who may work in this part of the codebase in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=vrvl-tN2EaA), and the slides on [Google Slides](https://docs.google.com/presentation/d/1H-pCzI_LNrgrL5pJAIQgvLX8Ji0-jIKOg1QeJQzChug/edit) and in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/c5aa32b6b07476fa8b597004899ec538/Elasticsearch_Deep_Dive.pdf). Everything covered in this deep dive was accurate as of GitLab 12.0, and while specific details may have changed since then, it should still serve as a good introduction.
In August 2020, a second Deep Dive was hosted, focusing on [GitLab's specific architecture for multi-indices support](#zero-downtime-reindexing-with-multiple-indices). The [recording on YouTube](https://www.youtube.com/watch?v=0WdPR9oB2fg) and the [slides](https://lulalala.gitlab.io/gitlab-elasticsearch-deepdive/) are available. Everything covered in this deep dive was accurate as of GitLab 13.3.
...
...
@@ -192,7 +192,7 @@ NOTE:
This only supported for indices created with GitLab 13.0 or greater.
Migrations are stored in the [`ee/elastic/migrate/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/elastic/migrate) folder with `YYYYMMDDHHMMSS_migration_name.rb`
filename format, which is similar to Rails database migrations:
filename format, which is similar to Rails database migrations:
@@ -68,7 +68,7 @@ The editor follows the same public API as [provided by Monaco editor](https://mi
Editor Lite comes with the loading state built-in, making spinners and loaders rarely needed in HTML. To benefit the built-in loading state, set the `data-editor-loading` property on the HTML element that is supposed to contain the editor. Editor Lite will show the loader automatically while it's bootstrapping.
@@ -61,7 +61,9 @@ In case the column is not unique (no unique index definition), calling the `dist
When dealing with data migrations the preferred way to iterate over large volume of data is using `EachBatch`.
A special case of data migration is a background migration where the actual data modification is executed in a background job. The migration code that determines the data ranges (slices) and schedules the background jobs uses `each_batch`. More info: [background migration scheduling](background_migrations.md#scheduling)
A special case of data migration is a [background migration](background_migrations.md#scheduling)
where the actual data modification is executed in a background job. The migration code that determines
the data ranges (slices) and schedules the background jobs uses `each_batch`.
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Sometimes the business asks to change the name of a feature. Broadly speaking, there are 2 approaches to that task. They basically trade between immediate effort and future complexity/bug risk:
- Complete, rename everything in the repo.
- Complete, rename everything in the repository.
- Pros: does not increase code complexity.
- Cons: more work to execute, and higher risk of immediate bugs.
- Façade, rename as little as possible; only the user-facing content like interfaces,
@@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
In December 2018, Tiago Botelho hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/issues/1`)
on GitLab's [Pull Repository Mirroring functionality](../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository)
to share his domain specific knowledge with anyone who may work in this part of the
codebase in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=sSZq0fpdY-Y),
codebase in the future. You can find the [recording on YouTube](https://www.youtube.com/watch?v=sSZq0fpdY-Y),
and the slides in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/8693404888a941fd851f8a8ecdec9675/Gitlab_Create_-_Pull_Mirroring_Deep_Dive.pdf).
Everything covered in this deep dive was accurate as of GitLab 11.6, and while specific
details may have changed since then, it should still serve as a good introduction.
is an attack in which the attacker knows or controls the regular expression used.
The attacker may be able to enter user input that triggers this backtracking behavior in a
...
...
@@ -331,7 +331,7 @@ Once you've [determined when and where](#setting-expectations) the user submitte
- Content placed inside [HTML URL GET parameters](https://youtu.be/2VFavqfDS6w?t=3494) need to be [URL-encoded](https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html#rule-5---url-escape-before-inserting-untrusted-data-into-html-url-parameter-values)
-[Additional contexts may require context-specific encoding](https://youtu.be/2VFavqfDS6w?t=2341).
### Additional info
### Additional information
#### XSS mitigation and prevention in Rails
...
...
@@ -590,4 +590,3 @@ In order to prevent this from happening, it is recommended to use the method `us
@@ -63,7 +63,7 @@ Here's a list of the AWS services we will use, with links to pricing information
## Create an IAM EC2 instance role and profile
As we'll be using [Amazon S3 object storage](#amazon-s3-object-storage), our EC2 instances need to have read, write, and list permissions for our S3 buckets. To avoid embedding AWS keys in our GitLab config, we'll make use of an [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to allow our GitLab instance with this access. We'll need to create an IAM policy to attach to our IAM role:
As we'll be using [Amazon S3 object storage](#amazon-s3-object-storage), our EC2 instances need to have read, write, and list permissions for our S3 buckets. To avoid embedding AWS keys in our GitLab configuration, we'll make use of an [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to allow our GitLab instance with this access. We'll need to create an IAM policy to attach to our IAM role:
Make sure to edit the config file to match your setup. Also, ensure that you match your paths to GitLab, especially if installing for a user other than the `git` user:
Make sure to edit the configuration file to match your setup. Also, ensure that you match your paths to GitLab, especially if installing for a user other than the `git` user:
```shell
# Change YOUR_SERVER_FQDN to the fully-qualified
...
...
@@ -795,21 +795,21 @@ Make sure to edit the config file to match your setup. Also, ensure that you mat
sudo editor /etc/nginx/sites-available/gitlab
```
If you intend to enable GitLab Pages, there is a separate NGINX config you need
If you intend to enable GitLab Pages, there is a separate NGINX configuration you need
to use. Read all about the needed configuration at the
If you want to use HTTPS, replace the `gitlab` NGINX config with `gitlab-ssl`. See [Using HTTPS](#using-https) for HTTPS configuration details.
If you want to use HTTPS, replace the `gitlab` NGINX configuration with `gitlab-ssl`. See [Using HTTPS](#using-https) for HTTPS configuration details.
### Test Configuration
Validate your `gitlab` or `gitlab-ssl` NGINX config file with the following command:
Validate your `gitlab` or `gitlab-ssl` NGINX configuration file with the following command:
```shell
sudo nginx -t
```
You should receive `syntax is okay` and `test is successful` messages. If you receive errors check your `gitlab` or `gitlab-ssl` NGINX config file for typos, etc. as indicated in the error message given.
You should receive `syntax is okay` and `test is successful` messages. If you receive errors check your `gitlab` or `gitlab-ssl` NGINX configuration file for typos, etc. as indicated in the error message given.
Verify that the installed version is greater than 1.12.1:
...
...
@@ -878,7 +878,7 @@ To use GitLab with HTTPS:
1. In the `config.yml` of GitLab Shell:
1. Set `gitlab_url` option to the HTTPS endpoint of GitLab (e.g. `https://git.example.com`).
1. Set the certificates using either the `ca_file` or `ca_path` option.
1. Use the `gitlab-ssl` NGINX example config instead of the `gitlab` config.
1. Use the `gitlab-ssl` NGINX example configuration instead of the `gitlab` configuration.
1. Update `YOUR_SERVER_FQDN`.
1. Update `ssl_certificate` and `ssl_certificate_key`.
1. Review the configuration file and consider applying other security and performance enhancing features.
...
...
@@ -951,7 +951,7 @@ production:
### Custom SSH Connection
If you are running SSH on a non-standard port, you must change the GitLab user's SSH config.
If you are running SSH on a non-standard port, you must change the GitLab user's SSH configuration.
```plaintext
# Add to /home/git/.ssh/config
...
...
@@ -973,7 +973,7 @@ As of GitLab 12.9, [Puma](https://github.com/puma/puma) has replaced Unicorn as
If you want to switch back to Unicorn, follow these steps:
1. Finish the GitLab setup so you have it up and running.
1. Copy the supplied example Unicorn config file into place:
1. Copy the supplied example Unicorn configuration file into place:
@@ -111,7 +111,7 @@ Since file system performance may affect GitLab's overall performance, [we don't
### CPU
CPU requirements are dependent on the number of users and expected workload. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size.
CPU requirements are dependent on the number of users and expected workload. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repository/change size.
The following is the recommended minimum CPU hardware guidance for a handful of example GitLab user base sizes.
...
...
@@ -121,7 +121,7 @@ The following is the recommended minimum CPU hardware guidance for a handful of
### Memory
Memory requirements are dependent on the number of users and expected workload. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size.
Memory requirements are dependent on the number of users and expected workload. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repository/change size.
The following is the recommended minimum Memory hardware guidance for a handful of example GitLab user base sizes.
@@ -203,7 +203,7 @@ If those are present, the request is exceeding the
which is set to 10 seconds by default.
To fix this the `gitlab_rails['webhook_timeout']` value must be increased
in the `gitlab.rb` config file, followed by the [`gitlab-ctl reconfigure` command](../administration/restart_gitlab.md).
in the `gitlab.rb` configuration file, followed by the [`gitlab-ctl reconfigure` command](../administration/restart_gitlab.md).
If you don't find the errors above, but do find *duplicate* entries like below (in `/var/log/gitlab/gitlab-rail`), this
could also indicate that [webhook requests are timing out](../user/project/integrations/webhooks.md#webhook-fails-or-multiple-webhook-requests-are-triggered):
@@ -16,7 +16,7 @@ To enable the Shibboleth OmniAuth provider you must configure Apache Shibboleth
The installation and configuration of the module itself is out of the scope of this document.
Check <https://wiki.shibboleth.net/confluence/display/SP3/Apache> for more information.
You can find Apache config in [GitLab Recipes](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache).
You can find Apache configuration in [GitLab Recipes](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache).
The following changes are needed to enable Shibboleth:
...
...
@@ -40,7 +40,7 @@ The following changes are needed to enable Shibboleth:
</Location>
```
1. Exclude Shibboleth URLs from rewriting. Add `RewriteCond %{REQUEST_URI} !/Shibboleth.sso` and `RewriteCond %{REQUEST_URI} !/shibboleth-sp`. Config should look like this:
1. Exclude Shibboleth URLs from rewriting. Add `RewriteCond %{REQUEST_URI} !/Shibboleth.sso` and `RewriteCond %{REQUEST_URI} !/shibboleth-sp`. Configuration should look like this:
