Skip to content

G
gitlab-ce
Commit 529bc7e2 authored by GitLab Bot's avatar GitLab Bot
Browse files
Options

Add latest changes from gitlab-org/gitlab@master

parent e8baeb7f
Hide whitespace changes
Inline Side-by-side
Showing with 2213 additions and 2123 deletions
doc/administration/geo/replication/object_storage.md
View file @ 529bc7e2
......@@ -30,7 +30,7 @@ To enable GitLab replication, you must:
checkbox.
For LFS, follow the documentation to
[set up LFS object storage](../../../workflow/lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
[set up LFS object storage](../../lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
For CI job artifacts, there is similar documentation to configure
[jobs artifact object storage](../../job_artifacts.md#using-object-storage)
......
doc/administration/git_annex.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/git_annex.html'
---
# Git annex
> **Warning:** GitLab has [completely
removed][deprecate-annex-issue] in GitLab 9.0 (2017/03/22).
Read through the [migration guide from git-annex to Git LFS][guide].
The biggest limitation of Git, compared to some older centralized version
control systems, has been the maximum size of the repositories.
The general recommendation is to not have Git repositories larger than 1GB to
preserve performance. Although GitLab has no limit (some repositories in GitLab
are over 50GB!), we subscribe to the advice to keep repositories as small as
you can.
Not being able to version control large binaries is a big problem for many
larger organizations.
Videos, photos, audio, compiled binaries and many other types of files are too
large. As a workaround, people keep artwork-in-progress in a Dropbox folder and
only check in the final result. This results in using outdated files, not
having a complete history and increases the risk of losing work.
This problem is solved in GitLab Enterprise Edition by integrating the
[git-annex] application.
`git-annex` allows managing large binaries with Git without checking the
contents into Git.
You check-in only a symlink that contains the SHA-1 of the large binary. If you
need the large binary, you can sync it from the GitLab server over `rsync`, a
very fast file copying tool.
## GitLab git-annex Configuration
`git-annex` is disabled by default in GitLab. Below you will find the
configuration options required to enable it.
### Requirements
`git-annex` needs to be installed both on the server and the client side.
For Debian-like systems (e.g., Debian, Ubuntu) this can be achieved by running:
```
sudo apt-get update && sudo apt-get install git-annex
```
For RedHat-like systems (e.g., CentOS, RHEL) this can be achieved by running:
```
sudo yum install epel-release && sudo yum install git-annex
```
### Configuration for Omnibus packages
For Omnibus GitLab packages, only one configuration setting is needed.
The Omnibus package will internally set the correct options in all locations.
1. In `/etc/gitlab/gitlab.rb` add the following line:
```ruby
gitlab_shell['git_annex_enabled'] = true
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
### Configuration for installations from source
There are 2 settings to enable git-annex on your GitLab server.
One is located in `config/gitlab.yml` of the GitLab repository and the other
one is located in `config.yml` of GitLab Shell.
1. In `config/gitlab.yml` add or edit the following lines:
```yaml
gitlab_shell:
git_annex_enabled: true
```
1. In `config.yml` of GitLab Shell add or edit the following lines:
```yaml
git_annex_enabled: true
```
1. Save the files and [restart GitLab][] for the changes to take effect.
## Using GitLab git-annex
> **Note:**
> Your Git remotes must be using the SSH protocol, not HTTP(S).
Here is an example workflow of uploading a very large file and then checking it
into your Git repository:
```bash
git clone git@example.com:group/project.git
git annex init 'My Laptop' # initialize the annex project and give an optional description
cp ~/tmp/debian.iso ./ # copy a large file into the current directory
git annex add debian.iso # add the large file to git annex
git commit -am "Add Debian iso" # commit the file metadata
git annex sync --content # sync the Git repo and large file to the GitLab server
```
The output should look like this:
```
commit
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)
nothing to commit, working tree clean
ok
pull origin
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From example.com:group/project
497842b..5162f80 git-annex -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
copy debian.iso (checking origin...) (to origin...)
SHA256E-s26214400--8092b3d482fb1b7a5cf28c43bc1425c8f2d380e86869c0686c49aa7b0f086ab2.iso
26,214,400 100% 638.88kB/s 0:00:40 (xfr#1, to-chk=0/1)
ok
pull origin
ok
(recording state in git...)
push origin
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (15/15), 1.64 KiB | 0 bytes/s, done.
Total 15 (delta 1), reused 0 (delta 0)
To example.com:group/project.git
* [new branch] git-annex -> synced/git-annex
* [new branch] master -> synced/master
ok
```
Your files can be found in the `master` branch, but you'll notice that there
are more branches created by the `annex sync` command.
Git Annex will also create a new directory at `.git/annex/` and will record the
tracked files in the `.git/config` file. The files you assign to be tracked
with `git-annex` will not affect the existing `.git/config` records. The files
are turned into symbolic links that point to data in `.git/annex/objects/`.
The `debian.iso` file in the example will contain the symbolic link:
```
.git/annex/objects/ZW/1k/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.png/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.iso
```
Use `git annex info` to retrieve the information about the local copy of your
repository.
---
Downloading a single large file is also very simple:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync # sync Git branches but not the large file
git annex get debian.iso # download the large file
```
To download all files:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync --content # sync Git branches and download all the large files
```
By using `git-annex` without GitLab, anyone that can access the server can also
access the files of all projects, but GitLab Annex ensures that you can only
access files of projects you have access to (developer, maintainer, or owner role).
## How it works
Internally GitLab uses [GitLab Shell] to handle SSH access and this was a great
integration point for `git-annex`.
There is a setting in GitLab Shell so you can disable GitLab Annex support
if you want to.
## Troubleshooting tips
Differences in version of `git-annex` on the GitLab server and on local machines
can cause `git-annex` to raise unpredicted warnings and errors.
Consult the [Annex upgrade page][annex-upgrade] for more information about
the differences between versions. You can find out which version is installed
on your server by navigating to <https://pkgs.org/download/git-annex> and
searching for your distribution.
Although there is no general guide for `git-annex` errors, there are a few tips
on how to go around the warnings.
### `git-annex-shell: Not a git-annex or gcrypt repository`
This warning can appear on the initial `git annex sync --content` and is caused
by differences in `git-annex-shell`. You can read more about it
[in this git-annex issue][issue].
One important thing to note is that despite the warning, the `sync` succeeds
and the files are pushed to the GitLab repository.
If you get hit by this, you can run the following command inside the repository
that the warning was raised:
```
git config remote.origin.annex-ignore false
```
Consecutive runs of `git annex sync --content` **should not** produce this
warning and the output should look like this:
```
commit ok
pull origin
ok
pull origin
ok
push origin
```
[annex-upgrade]: https://git-annex.branchable.com/upgrades/
[deprecate-annex-issue]: https://gitlab.com/gitlab-org/gitlab/issues/1648
[git-annex]: https://git-annex.branchable.com/ "git-annex website"
[gitlab shell]: https://gitlab.com/gitlab-org/gitlab-shell "GitLab Shell repository"
[guide]: lfs/migrate_from_git_annex_to_git_lfs.html
[issue]: https://git-annex.branchable.com/forum/Error_from_git-annex-shell_on_creation_of_gcrypt_special_remote/ "git-annex issue"
[reconfigure GitLab]: restart_gitlab.md#omnibus-gitlab-reconfigure
[restart GitLab]: restart_gitlab.md#installations-from-source
doc/administration/gitaly/index.md
View file @ 529bc7e2
......@@ -551,7 +551,7 @@ a few things that you need to do:
to eliminate the need for a shared authorized_keys file.
1. Configure [object storage for job artifacts](../job_artifacts.md#using-object-storage)
including [incremental logging](../job_logs.md#new-incremental-logging-architecture).
1. Configure [object storage for LFS objects](../../workflow/lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
1. Configure [object storage for LFS objects](../lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
1. Configure [object storage for uploads](../uploads.md#using-object-storage-core-only).
NOTE: **Note:**
......
doc/administration/index.md
View file @ 529bc7e2
......@@ -161,9 +161,10 @@ Learn how to install, configure, update, and maintain your GitLab instance.
## Git configuration options
- [Custom Git hooks](custom_hooks.md): Custom Git hooks (on the filesystem) for when webhooks aren't enough.
- [Git LFS configuration](../workflow/lfs/lfs_administration.md): Learn how to configure LFS for GitLab.
- [Git LFS configuration](lfs/lfs_administration.md): Learn how to configure LFS for GitLab.
- [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast.
- [Configuring Git Protocol v2](git_protocol.md): Git protocol version 2 support.
- [Manage large files with `git-annex` (Deprecated)](git_annex.md)
## Monitoring GitLab
......
doc/administration/lfs/lfs_administration.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/lfs_administration.html'
---
# GitLab Git LFS Administration
Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md).
## Requirements
- Git LFS is supported in GitLab starting with version 8.2.
- Support for object storage, such as AWS S3, was introduced in 10.0.
- Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up.
## Configuration
Git LFS objects can be large in size. By default, they are stored on the server
GitLab is installed on.
There are various configuration options to help GitLab server administrators:
- Enabling/disabling Git LFS support
- Changing the location of LFS object storage
- Setting up object storage supported by [Fog](http://fog.io/about/provider_documentation.html)
### Configuration for Omnibus installations
In `/etc/gitlab/gitlab.rb`:
```ruby
# Change to true to enable lfs - enabled by default if not defined
gitlab_rails['lfs_enabled'] = false
# Optionally, change the storage path location. Defaults to
# `#{gitlab_rails['shared_path']}/lfs-objects`. Which evaluates to
# `/var/opt/gitlab/gitlab-rails/shared/lfs-objects` by default.
gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects"
```
### Configuration for installations from source
In `config/gitlab.yml`:
```yaml
# Change to true to enable lfs
lfs:
enabled: false
storage_path: /mnt/storage/lfs-objects
```
## Storing LFS objects in remote object storage
> [Introduced][ee-2760] in [GitLab Premium][eep] 10.0. Brought to GitLab Core in 10.7.
It is possible to store LFS objects in remote object storage which allows you
to offload local hard disk R/W operations, and free up disk space significantly.
GitLab is tightly integrated with `Fog`, so you can refer to its [documentation](http://fog.io/about/provider_documentation.html)
to check which storage services can be integrated with GitLab.
You can also use external object storage in a private local network. For example,
[MinIO](https://min.io/) is a standalone object storage service, is easy to set up, and works well with GitLab instances.
GitLab provides two different options for the uploading mechanism: "Direct upload" and "Background upload".
**Option 1. Direct upload**
1. User pushes an lfs file to the GitLab instance
1. GitLab-workhorse uploads the file directly to the external object storage
1. GitLab-workhorse notifies GitLab-rails that the upload process is complete
**Option 2. Background upload**
1. User pushes an lfs file to the GitLab instance
1. GitLab-rails stores the file in the local file storage
1. GitLab-rails then uploads the file to the external object storage asynchronously
The following general settings are supported.
| Setting | Description | Default |
|---------|-------------|---------|
| `enabled` | Enable/disable object storage | `false` |
| `remote_directory` | The bucket name where LFS objects will be stored| |
| `direct_upload` | Set to true to enable direct upload of LFS without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` |
| `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` |
| `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` |
| `connection` | Various connection options described below | |
The `connection` settings match those provided by [Fog](https://github.com/fog).
Here is a configuration example with S3.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | AWS |
| `aws_access_key_id` | AWS credentials, or compatible | `ABC123DEF456` |
| `aws_secret_access_key` | AWS credentials, or compatible | `ABC123DEF456ABC123DEF456ABC123DEF456` |
| `aws_signature_version` | AWS signature version to use. 2 or 4 are valid options. Digital Ocean Spaces and other providers may need 2. | 4 |
| `enable_signature_v4_streaming` | Set to true to enable HTTP chunked transfers with [AWS v4 signatures](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html). Oracle Cloud S3 needs this to be false | true |
| `region` | AWS region | us-east-1 |
| `host` | S3 compatible host for when not using AWS, e.g. `localhost` or `storage.example.com` | s3.amazonaws.com |
| `endpoint` | Can be used when configuring an S3 compatible service such as [MinIO](https://min.io), by entering a URL such as `http://127.0.0.1:9000` | (optional) |
| `path_style` | Set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Leave as false for AWS S3 | false |
| `use_iam_profile` | Set to true to use IAM profile instead of access keys | false
Here is a configuration example with GCS.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | `Google` |
| `google_project` | GCP project name | `gcp-project-12345` |
| `google_client_email` | The email address of the service account | `foo@gcp-project-12345.iam.gserviceaccount.com` |
| `google_json_key_location` | The json key path | `/path/to/gcp-project-12345-abcde.json` |
NOTE: **Note:**
The service account must have permission to access the bucket.
[See more](https://cloud.google.com/storage/docs/authentication)
Here is a configuration example with Rackspace Cloud Files.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | `Rackspace` |
| `rackspace_username` | The username of the Rackspace account with access to the container | `joe.smith` |
| `rackspace_api_key` | The API key of the Rackspace account with access to the container | `ABC123DEF456ABC123DEF456ABC123DE` |
| `rackspace_region` | The Rackspace storage region to use, a three letter code from the [list of service access endpoints](https://developer.rackspace.com/docs/cloud-files/v1/general-api-info/service-access/) | `iad` |
| `rackspace_temp_url_key` | The private key you have set in the Rackspace API for temporary URLs. Read more [here](https://developer.rackspace.com/docs/cloud-files/v1/use-cases/public-access-to-your-cloud-files-account/#tempurl) | `ABC123DEF456ABC123DEF456ABC123DE` |
NOTE: **Note:**
Regardless of whether the container has public access enabled or disabled, Fog will
use the TempURL method to grant access to LFS objects. If you see errors in logs referencing
instantiating storage with a temp-url-key, ensure that you have set they key properly
on the Rackspace API and in `gitlab.rb`. You can verify the value of the key Rackspace
has set by sending a GET request with token header to the service access endpoint URL
and comparing the output of the returned headers.
### Manual uploading to an object storage
There are two ways to manually do the same thing as automatic uploading (described above).
**Option 1: rake task**
```sh
rake gitlab:lfs:migrate
```
**Option 2: rails console**
```sh
$ sudo gitlab-rails console # Login to rails console
> # Upload LFS files manually
> LfsObject.where(file_store: [nil, 1]).find_each do |lfs_object|
> lfs_object.file.migrate!(ObjectStorage::Store::REMOTE) if lfs_object.file.file.exists?
> end
```
### S3 for Omnibus installations
On Omnibus installations, the settings are prefixed by `lfs_object_store_`:
1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with
the values you want:
```ruby
gitlab_rails['lfs_object_store_enabled'] = true
gitlab_rails['lfs_object_store_remote_directory'] = "lfs-objects"
gitlab_rails['lfs_object_store_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'aws_access_key_id' => '1ABCD2EFGHI34JKLM567N',
'aws_secret_access_key' => 'abcdefhijklmnopQRSTUVwxyz0123456789ABCDE',
# The below options configure an S3 compatible host instead of AWS
'host' => 'localhost',
'endpoint' => 'http://127.0.0.1:9000',
'path_style' => true
}
```
1. Save the file and [reconfigure GitLab]s for the changes to take effect.
1. Migrate any existing local LFS objects to the object storage:
```bash
gitlab-rake gitlab:lfs:migrate
```
This will migrate existing LFS objects to object storage. New LFS objects
will be forwarded to object storage unless
`gitlab_rails['lfs_object_store_background_upload']` is set to false.
### S3 for installations from source
For source installations the settings are nested under `lfs:` and then
`object_store:`:
1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following
lines:
```yaml
lfs:
enabled: true
object_store:
enabled: false
remote_directory: lfs-objects # Bucket name
connection:
provider: AWS
aws_access_key_id: 1ABCD2EFGHI34JKLM567N
aws_secret_access_key: abcdefhijklmnopQRSTUVwxyz0123456789ABCDE
region: eu-central-1
# Use the following options to configure an AWS compatible host such as Minio
host: 'localhost'
endpoint: 'http://127.0.0.1:9000'
path_style: true
```
1. Save the file and [restart GitLab][] for the changes to take effect.
1. Migrate any existing local LFS objects to the object storage:
```bash
sudo -u git -H bundle exec rake gitlab:lfs:migrate RAILS_ENV=production
```
This will migrate existing LFS objects to object storage. New LFS objects
will be forwarded to object storage unless `background_upload` is set to
false.
### Migrating back to local storage
In order to migrate back to local storage:
1. Set both `direct_upload` and `background_upload` to false under the LFS object storage settings. Don't forget to restart GitLab.
1. Run `rake gitlab:lfs:migrate_to_local` on your console.
1. Disable `object_storage` for LFS objects in `gitlab.rb`. Remember to restart GitLab afterwards.
## Storage statistics
You can see the total storage used for LFS objects on groups and projects
in the administration area, as well as through the [groups](../../api/groups.md)
and [projects APIs](../../api/projects.md).
## Troubleshooting: `Google::Apis::TransmissionError: execution expired`
If LFS integration is configred with Google Cloud Storage and background uploads (`background_upload: true` and `direct_upload: false`),
Sidekiq workers may encouter this error. This is because the uploading timed out with very large files.
LFS files up to 6Gb can be uploaded without any extra steps, otherwise you need to use the following workaround.
```shell
$ sudo gitlab-rails console # Login to rails console
> # Set up timeouts. 20 minutes is enough to upload 30GB LFS files.
> # These settings are only in effect for the same session, i.e. they are not effective for sidekiq workers.
> ::Google::Apis::ClientOptions.default.open_timeout_sec = 1200
> ::Google::Apis::ClientOptions.default.read_timeout_sec = 1200
> ::Google::Apis::ClientOptions.default.send_timeout_sec = 1200
> # Upload LFS files manually. This process does not use sidekiq at all.
> LfsObject.where(file_store: [nil, 1]).find_each do |lfs_object|
> lfs_object.file.migrate!(ObjectStorage::Store::REMOTE) if lfs_object.file.file.exists?
> end
```
See more information in [!19581](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/19581)
## Known limitations
- Support for removing unreferenced LFS objects was added in 8.14 onwards.
- LFS authentications via SSH was added with GitLab 8.12.
- Only compatible with the Git LFS client versions 1.1.0 and up, or 1.0.2.
- The storage statistics currently count each LFS object multiple times for
every project linking to it.
[reconfigure gitlab]: ../restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab"
[restart gitlab]: ../restart_gitlab.md#installations-from-source "How to restart GitLab"
[eep]: https://about.gitlab.com/pricing/ "GitLab Premium"
[ee-2760]: https://gitlab.com/gitlab-org/gitlab/merge_requests/2760
doc/administration/lfs/manage_large_binaries_with_git_lfs.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html'
---
# Git LFS
Managing large files such as audio, video and graphics files has always been one
of the shortcomings of Git. The general recommendation is to not have Git repositories
larger than 1GB to preserve performance.
![Git LFS tracking status](img/lfs-icon.png)
An LFS icon is shown on files tracked by Git LFS to denote if a file is stored
as a blob or as an LFS pointer.
## How it works
Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication
to authorize client requests. Once the request is authorized, Git LFS client receives
instructions from where to fetch or where to push the large file.
## GitLab server configuration
Documentation for GitLab instance administrators is under [LFS administration doc](lfs_administration.md).
## Requirements
- Git LFS is supported in GitLab starting with version 8.2
- Git LFS must be enabled under project settings
- [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up
## Known limitations
- Git LFS v1 original API is not supported since it was deprecated early in LFS
development
- When SSH is set as a remote, Git LFS objects still go through HTTPS
- Any Git LFS request will ask for HTTPS credentials to be provided so a good Git
credentials store is recommended
- Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have
to add the URL to Git config manually (see [troubleshooting](#troubleshooting))
NOTE: **Note:**
With 8.12 GitLab added LFS support to SSH. The Git LFS communication
still goes over HTTP, but now the SSH client passes the correct credentials
to the Git LFS client, so no action is required by the user.
## Using Git LFS
Lets take a look at the workflow when you need to check large files into your Git
repository with Git LFS. For example, if you want to upload a very large file and
check it into your Git repository:
```bash
git clone git@gitlab.example.com:group/project.git
git lfs install # initialize the Git LFS project
git lfs track "*.iso" # select the file extensions that you want to treat as large files
```
Once a certain file extension is marked for tracking as a LFS object you can use
Git as usual without having to redo the command to track a file with the same extension:
```bash
cp ~/tmp/debian.iso ./ # copy a large file into the current directory
git add . # add the large file to the project
git commit -am "Added Debian iso" # commit the file meta data
git push origin master # sync the git repo and large file to the GitLab server
```
**Make sure** that `.gitattributes` is tracked by Git. Otherwise Git
LFS will not be working properly for people cloning the project:
```bash
git add .gitattributes
```
Cloning the repository works the same as before. Git automatically detects the
LFS-tracked files and clones them via HTTP. If you performed the `git clone`
command with a SSH URL, you have to enter your GitLab credentials for HTTP
authentication.
```bash
git clone git@gitlab.example.com:group/project.git
```
If you already cloned the repository and you want to get the latest LFS object
that are on the remote repository, eg. for a branch from origin:
```bash
git lfs fetch origin master
```
### Migrate an existing repo to Git LFS
Read the documentation on how to [migrate an existing Git repo with Git LFS](../../topics/git/migrate_to_git_lfs/index.md).
## File Locking
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/35856) in GitLab 10.5.
The first thing to do before using File Locking is to tell Git LFS which
kind of files are lockable. The following command will store PNG files
in LFS and flag them as lockable:
```bash
git lfs track "*.png" --lockable
```
After executing the above command a file named `.gitattributes` will be
created or updated with the following content:
```bash
*.png filter=lfs diff=lfs merge=lfs -text lockable
```
You can also register a file type as lockable without using LFS
(In order to be able to lock/unlock a file you need a remote server that implements the LFS File Locking API),
in order to do that you can edit the `.gitattributes` file manually:
```bash
*.pdf lockable
```
After a file type has been registered as lockable, Git LFS will make
them readonly on the file system automatically. This means you will
need to lock the file before editing it.
### Managing Locked Files
Once you're ready to edit your file you need to lock it first:
```bash
git lfs lock images/banner.png
Locked images/banner.png
```
This will register the file as locked in your name on the server:
```bash
git lfs locks
images/banner.png joe ID:123
```
Once you have pushed your changes, you can unlock the file so others can
also edit it:
```bash
git lfs unlock images/banner.png
```
You can also unlock by id:
```bash
git lfs unlock --id=123
```
If for some reason you need to unlock a file that was not locked by you,
you can use the `--force` flag as long as you have a `maintainer` access on
the project:
```bash
git lfs unlock --id=123 --force
```
## Troubleshooting
### error: Repository or object not found
There are a couple of reasons why this error can occur:
- You don't have permissions to access certain LFS object
Check if you have permissions to push to the project or fetch from the project.
- Project is not allowed to access the LFS object
LFS object you are trying to push to the project or fetch from the project is not
available to the project anymore. Probably the object was removed from the server.
- Local Git repository is using deprecated LFS API
### Invalid status for `<url>` : 501
Git LFS will log the failures into a log file.
To view this log file, while in project directory:
```bash
git lfs logs last
```
If the status `error 501` is shown, it is because:
- Git LFS is not enabled in project settings. Check your project settings and
enable Git LFS.
- Git LFS support is not enabled on the GitLab server. Check with your GitLab
administrator why Git LFS is not enabled on the server. See
[LFS administration documentation](lfs_administration.md) for instructions
on how to enable LFS support.
- Git LFS client version is not supported by GitLab server. Check your Git LFS
version with `git lfs version`. Check the Git config of the project for traces
of deprecated API with `git lfs -l`. If `batch = false` is set in the config,
remove the line and try to update your Git LFS client. Only version 1.0.1 and
newer are supported.
### getsockopt: connection refused
If you push a LFS object to a project and you receive an error similar to:
`Post <URL>/info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`,
the LFS client is trying to reach GitLab through HTTPS. However, your GitLab
instance is being served on HTTP.
This behaviour is caused by Git LFS using HTTPS connections by default when a
`lfsurl` is not set in the Git config.
To prevent this from happening, set the lfs url in project Git config:
```bash
git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs"
```
### Credentials are always required when pushing an object
NOTE: **Note:**
With 8.12 GitLab added LFS support to SSH. The Git LFS communication
still goes over HTTP, but now the SSH client passes the correct credentials
to the Git LFS client, so no action is required by the user.
Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing
the LFS object on every push for every object, user HTTPS credentials are required.
By default, Git has support for remembering the credentials for each repository
you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials).
For example, you can tell Git to remember the password for a period of time in
which you expect to push the objects:
```bash
git config --global credential.helper 'cache --timeout=3600'
```
This will remember the credentials for an hour after which Git operations will
require re-authentication.
If you are using OS X you can use `osxkeychain` to store and encrypt your credentials.
For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases).
More details about various methods of storing the user credentials can be found
on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage).
### LFS objects are missing on push
GitLab checks files to detect LFS pointers on push. If LFS pointers are detected, GitLab tries to verify that those files already exist in LFS on GitLab.
Verify that LFS in installed locally and consider a manual push with `git lfs push --all`.
If you are storing LFS files outside of GitLab you can disable LFS on the project by setting `lfs_enabled: false` with the [projects API](../../api/projects.md#edit-project).
### Hosting LFS objects externally
It is possible to host LFS objects externally by setting a custom LFS url with `git config -f .lfsconfig lfs.url https://example.com/<project>.git/info/lfs`.
You might choose to do this if you are using an appliance like a Sonatype Nexus to store LFS data. If you choose to use an external LFS store,
GitLab will not be able to verify LFS objects which means that pushes will fail if you have GitLab LFS support enabled.
To stop push failure, LFS support can be disabled in the [Project settings](../../user/project/settings/index.md). This means you will lose GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS).
doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md 0 → 100644
View file @ 529bc7e2
# Migration guide from Git Annex to Git LFS
>**Note:**
Git Annex support [has been removed][issue-remove-annex] in GitLab Enterprise
Edition 9.0 (2017/03/22).
Both [Git Annex][] and [Git LFS][] are tools to manage large files in Git.
## History
Git Annex [was introduced in GitLab Enterprise Edition 7.8][post-3], at a time
where Git LFS didn't yet exist. A few months later, GitLab brought support for
Git LFS in [GitLab 8.2][post-2] and is available for both Community and
Enterprise editions.
## Differences between Git Annex and Git LFS
Some items below are general differences between the two protocols and some are
ones that GitLab developed.
- Git Annex works only through SSH, whereas Git LFS works both with SSH and HTTPS
(SSH support was added in GitLab 8.12).
- Annex files are stored in a sub-directory of the normal repositories, whereas
LFS files are stored outside of the repositories in a place you can define.
- Git Annex requires a more complex setup, but has much more options than Git
LFS. You can compare the commands each one offers by running `man git-annex`
and `man git-lfs`.
- Annex files cannot be browsed directly in GitLab's interface, whereas LFS
files can.
## Migration steps
>**Note:**
Since Git Annex files are stored in a sub-directory of the normal repositories
(`.git/annex/objects`) and LFS files are stored outside of the repositories,
they are not compatible as they are using a different scheme. Therefore, the
migration has to be done manually per repository.
There are basically two steps you need to take in order to migrate from Git
Annex to Git LFS.
### TL; DR
If you know what you are doing and want to skip the reading, this is what you
need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab-git-annex) in your
repository and that you have made backups in case something goes wrong).
Fire up a terminal, navigate to your Git repository and:
1. Disable `git-annex`:
```bash
git annex sync --content
git annex direct
git annex uninit
git annex indirect
```
1. Enable `git-lfs`:
```
git lfs install
git lfs track <files>
git add .
git commit -m "commit message"
git push
```
### Disabling Git Annex in your repo
Before changing anything, make sure you have a backup of your repository first.
There are a couple of ways to do that, but you can simply clone it to another
local path and maybe push it to GitLab if you want a remote backup as well.
Here you'll find a guide on
[how to back up a **git-annex** repository to an external hard drive][bkp-ext-drive].
Since Annex files are stored as objects with symlinks and cannot be directly
modified, we need to first remove those symlinks.
NOTE: **Note:**
Make sure the you read about the [`direct` mode][annex-direct] as it contains
useful information that may fit in your use case. Note that `annex direct` is
deprecated in Git Annex version 6, so you may need to upgrade your repository
if the server also has Git Annex 6 installed. Read more in the
[Git Annex troubleshooting tips](../git_annex.md#troubleshooting-tips) section.
1. Backup your repository
```bash
cd repository
git annex sync --content
cd ..
git clone repository repository-backup
cd repository-backup
git annex get
cd ..
```
1. Use `annex direct`:
```bash
cd repository
git annex direct
```
The output should be similar to this:
```bash
commit
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
ok
direct debian.iso ok
direct ok
```
1. Disable Git Annex with [`annex uninit`][uninit]:
```bash
git annex uninit
```
The output should be similar to this:
```bash
unannex debian.iso ok
Deleted branch git-annex (was 2534d2c).
```
This will `unannex` every file in the repository, leaving the original files.
1. Switch back to `indirect` mode:
```bash
git annex indirect
```
The output should be similar to this:
```bash
(merging origin/git-annex into git-annex...)
(recording state in git...)
commit (recording state in git...)
ok
(recording state in git...)
[master fac3194] commit before switching to indirect mode
1 file changed, 1 deletion(-)
delete mode 120000 alpine-virt-3.4.4-x86_64.iso
ok
indirect ok
ok
```
---
At this point, you have two options. Either add, commit and push the files
directly back to GitLab or switch to Git LFS. We will tackle the LFS switch in
the next section.
### Enabling Git LFS in your repo
Git LFS is enabled by default on all GitLab products (GitLab CE, GitLab EE,
GitLab.com), therefore, you don't need to do anything server-side.
1. First, make sure you have `git-lfs` installed locally:
```bash
git lfs help
```
If the terminal doesn't prompt you with a full response on `git-lfs` commands,
[install the Git LFS client][install-lfs] first.
1. Inside the repo, run the following command to initiate LFS:
```bash
git lfs install
```
1. Enable `git-lfs` for the group of files you want to track. You
can track specific files, all files containing the same extension, or an
entire directory:
```bash
git lfs track images/01.png # per file
git lfs track **/*.png # per extension
git lfs track images/ # per directory
```
Once you do that, run `git status` and you'll see `.gitattributes` added
to your repo. It collects all file patterns that you chose to track via
`git-lfs`.
1. Add the files, commit and push them to GitLab:
```bash
git add .
git commit -m "commit message"
git push
```
If your remote is set up with HTTP, you will be asked to enter your login
credentials. If you have [2FA enabled](../../user/profile/account/two_factor_authentication.md), make sure to use a
[personal access token](../../user/profile/account/two_factor_authentication.md#personal-access-tokens)
instead of your password.
## Removing the Git Annex branches
After the migration finishes successfully, you can remove all `git-annex`
related branches from your repository.
On GitLab, navigate to your project's **Repository ➔ Branches** and delete all
branches created by Git Annex: `git-annex`, and all under `synced/`.
![repository branches](img/git-annex-branches.png)
You can also do this on the command line with:
```bash
git branch -d synced/master
git branch -d synced/git-annex
git push origin :synced/master
git push origin :synced/git-annex
git push origin :git-annex
git remote prune origin
```
If there are still some Annex objects inside your repository (`.git/annex/`)
or references inside `.git/config`, run `annex uninit` again:
```bash
git annex uninit
```
## Further Reading
- (Blog Post) [Getting Started with Git FLS][post-1]
- (Blog Post) [Announcing LFS Support in GitLab][post-2]
- (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3]
- (GitLab Docs) [Git Annex](../git_annex.md)
- (GitLab Docs) [Git LFS](manage_large_binaries_with_git_lfs.md)
[annex-direct]: https://git-annex.branchable.com/direct_mode/
[bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive
[Git Annex]: http://git-annex.branchable.com/
[Git LFS]: https://git-lfs.github.com/
[install-lfs]: https://git-lfs.github.com/
[issue-remove-annex]: https://gitlab.com/gitlab-org/gitlab/issues/1648
[lfs-track]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/#tracking-files-with-lfs
[post-1]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/
[post-2]: https://about.gitlab.com/blog/2015/11/23/announcing-git-lfs-support-in-gitlab/
[post-3]: https://about.gitlab.com/blog/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/
[uninit]: https://git-annex.branchable.com/git-annex-uninit/
doc/administration/repository_storage_types.md
View file @ 529bc7e2
......@@ -183,7 +183,8 @@ CI Artifacts are S3 compatible since **9.4** (GitLab Premium), and available in
##### LFS Objects
LFS Objects implements a similar storage pattern using 2 chars, 2 level folders, following Git own implementation:
[LFS Objects in GitLab](lfs/manage_large_binaries_with_git_lfs.md) implement a similar
storage pattern using 2 chars, 2 level folders, following Git's own implementation:
```ruby
"shared/lfs-objects/#{oid[0..1}/#{oid[2..3]}/#{oid[4..-1]}"
......@@ -192,7 +193,7 @@ LFS Objects implements a similar storage pattern using 2 chars, 2 level folders,
"shared/lfs-objects/89/09/029eb962194cfb326259411b22ae3f4a814b5be4f80651735aeef9f3229c"
```
They are also S3 compatible since **10.0** (GitLab Premium), and available in GitLab Core since **10.7**.
LFS objects are also [S3 compatible](lfs/lfs_administration.md#storing-lfs-objects-in-remote-object-storage).
[ce-2821]: https://gitlab.com/gitlab-com/infrastructure/issues/2821
[ce-28283]: https://gitlab.com/gitlab-org/gitlab-foss/issues/28283
......
doc/ci/ci_cd_for_external_repos/bitbucket_integration.md
View file @ 529bc7e2
......@@ -151,7 +151,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines
configured in `.gitlab-ci.yml` and push the status to Bitbucket.
[pull-mirroring]: ../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter
[pull-mirroring]: ../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter
<!-- ## Troubleshooting
......
doc/ci/ci_cd_for_external_repos/github_integration.md
View file @ 529bc7e2
......@@ -46,7 +46,7 @@ repositories:
GitLab will:
1. Import the project.
1. Enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter)
1. Enable [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter)
1. Enable [GitHub project integration](../../user/project/integrations/github.md)
1. Create a web hook on GitHub to notify GitLab of new commits.
......
doc/ci/ci_cd_for_external_repos/index.md
View file @ 529bc7e2
......@@ -101,5 +101,5 @@ requests and not on branches you can add `except: [branches]` to the job specs.
[ee-4642]: https://gitlab.com/gitlab-org/gitlab/merge_requests/4642
[eep]: https://about.gitlab.com/pricing/
[mirroring]: ../../workflow/repository_mirroring.md
[mirroring]: ../../user/project/repository/repository_mirroring.md
[settings]: ../../user/project/settings/index.md#sharing-and-permissions
doc/ci/environments.md
View file @ 529bc7e2
......@@ -292,7 +292,7 @@ For the value of:
the web server to serve these requests is based on your setup.
We have used `$CI_ENVIRONMENT_SLUG` here because it is guaranteed to be unique. If
you're using a workflow like [GitLab Flow](../workflow/gitlab_flow.md), collisions
you're using a workflow like [GitLab Flow](../topics/gitlab_flow.md), collisions
are unlikely and you may prefer environment names to be more closely based on the
branch name. In that case, you could use `$CI_COMMIT_REF_SLUG` in `environment:url` in
the example above: `https://$CI_COMMIT_REF_SLUG.example.com`, which would give a URL
......
doc/ci/examples/laravel_with_gitlab_and_envoy/index.md
View file @ 529bc7e2
......@@ -379,7 +379,7 @@ These are persistent data and will be shared to every new release.
Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../environments.md), which will be described [later](#setting-up-gitlab-cicd) in this tutorial.
Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch.
To keep things simple, we commit directly to `master`, without using [feature-branches](../../../workflow/gitlab_flow.md#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial.
To keep things simple, we commit directly to `master`, without using [feature-branches](../../../topics/gitlab_flow.md#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial.
In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [Merge Requests](../../../user/project/merge_requests/index.md) to move their code across branches:
```bash
......
doc/ci/pipelines.md
View file @ 529bc7e2
......@@ -28,7 +28,7 @@ If all the jobs in a stage:
- Fail, the next stage is not (usually) executed and the pipeline ends early.
NOTE: **Note:**
If you have a [mirrored repository that GitLab pulls from](../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter),
If you have a [mirrored repository that GitLab pulls from](../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
......
doc/ci/quick_start/README.md
View file @ 529bc7e2
......@@ -143,7 +143,7 @@ Now if you go to the **Pipelines** page you will see that the pipeline is
pending.
NOTE: **Note:**
If you have a [mirrored repository where GitLab pulls from](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter),
If you have a [mirrored repository where GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
......
doc/ci/yaml/README.md
View file @ 529bc7e2
......@@ -23,7 +23,7 @@ We have complete examples of configuring pipelines:
- To see a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml).
NOTE: **Note:**
If you have a [mirrored repository where GitLab pulls from](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter),
If you have a [mirrored repository where GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
......
doc/development/contributing/merge_request_workflow.md
View file @ 529bc7e2
......@@ -36,7 +36,7 @@ include a regression test are merged quickly, while new features without proper
tests might be slower to receive feedback. The workflow to make a merge
request is as follows:
1. [Fork](../../workflow/forking_workflow.md#creating-a-fork) the project into
1. [Fork](../../user/project/repository/forking_workflow.md) the project into
your personal namespace (or group) on GitLab.com.
1. Create a feature branch in your fork (don't work off `master`).
1. Write [tests](../rake_tasks.md#run-tests) and code.
......
doc/development/git_object_deduplication.md
View file @ 529bc7e2
# How Git object deduplication works in GitLab
When a GitLab user [forks a project](../workflow/forking_workflow.md),
When a GitLab user [forks a project](../user/project/repository/forking_workflow.md),
GitLab creates a new Project with an associated Git repository that is a
copy of the original project at the time of the fork. If a large project
gets forked often, this can lead to a quick increase in Git repository
......
doc/development/repository_mirroring.md
View file @ 529bc7e2
......@@ -5,6 +5,6 @@
In December 2018, Tiago Botelho hosted a [Deep Dive] on GitLab's [Pull Repository Mirroring functionality] to share his domain specific knowledge with anyone who may work in this part of the code base in the future. You can find the [recording on YouTube], and the slides in [PDF]. Everything covered in this deep dive was accurate as of GitLab 11.6, and while specific details may have changed since then, it should still serve as a good introduction.
[Deep Dive]: https://gitlab.com/gitlab-org/create-stage/issues/1
[Pull Repository Mirroring functionality]: ../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter
[Pull Repository Mirroring functionality]: ../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter
[recording on YouTube]: https://www.youtube.com/watch?v=sSZq0fpdY-Y
[PDF]: https://gitlab.com/gitlab-org/create-stage/uploads/8693404888a941fd851f8a8ecdec9675/Gitlab_Create_-_Pull_Mirroring_Deep_Dive.pdf
doc/gitlab-basics/README.md
View file @ 529bc7e2
......@@ -30,7 +30,7 @@ The following are guides to basic GitLab functionality:
- [Create a merge request](add-merge-request.md), to request changes made in a branch
be merged into a project's repository.
- See how these features come together in the [GitLab Flow introduction video](https://youtu.be/InKNIvky2KE)
and [GitLab Flow page](../workflow/gitlab_flow.md).
and [GitLab Flow page](../topics/gitlab_flow.md).
## Working with Git from the command line
......
doc/gitlab-basics/fork-project.md
View file @ 529bc7e2
......@@ -8,4 +8,4 @@ A fork is a copy of an original repository that you put in another namespace
where you can experiment and apply changes that you can later decide whether or
not to share, without affecting the original project.
It takes just a few steps to [fork a project in GitLab](../workflow/forking_workflow.md#creating-a-fork).
It takes just a few steps to [fork a project in GitLab](../user/project/repository/forking_workflow.md#creating-a-fork).
doc/install/aws/index.md
View file @ 529bc7e2
......@@ -539,7 +539,7 @@ which would otherwise take much space.
In particular, you can store in S3:
- [The Git LFS objects](../../workflow/lfs/lfs_administration.md#s3-for-omnibus-installations) ((Omnibus GitLab installations))
- [The Git LFS objects](../../administration/lfs/lfs_administration.md#s3-for-omnibus-installations) ((Omnibus GitLab installations))
- [The Container Registry images](../../administration/packages/container_registry.md#container-registry-storage-driver) (Omnibus GitLab installations)
- [The GitLab CI/CD job artifacts](../../administration/job_artifacts.md#using-object-storage) (Omnibus GitLab installations)
......
doc/intro/README.md
View file @ 529bc7e2
......@@ -24,7 +24,7 @@ Create issues, labels, milestones, cast your vote, and review issues.
Create merge requests and review code.
- [Fork a project and contribute to it](../workflow/forking_workflow.md)
- [Fork a project and contribute to it](../user/project/repository/forking_workflow.md)
- [Create a new merge request](../gitlab-basics/add-merge-request.md)
- [Automatically close issues from merge requests](../user/project/issues/managing_issues.md#closing-issues-automatically)
- [Automatically merge when pipeline succeeds](../user/project/merge_requests/merge_when_pipeline_succeeds.md)
......
doc/raketasks/backup_restore.md
View file @ 529bc7e2
......@@ -877,7 +877,7 @@ including (but not restricted to):
- [Custom Pages domains](../user/project/pages/custom_domains_ssl_tls_certification/index.md)
- [Project error tracking](../user/project/operations/error_tracking.md)
- [Runner authentication](../ci/runners/README.md)
- [Project mirroring](../workflow/repository_mirroring.md)
- [Project mirroring](../user/project/repository/repository_mirroring.md)
- [Web hooks](../user/project/integrations/webhooks.md)
In cases like CI/CD variables and Runner authentication, you might
......
doc/topics/autodevops/quick_start_guide.md
View file @ 529bc7e2
......@@ -212,7 +212,7 @@ under **Settings > CI/CD > Environment variables**.
### Working with branches
Following the [GitLab flow](../../workflow/gitlab_flow.md#working-with-feature-branches),
Following the [GitLab flow](../gitlab_flow.md#working-with-feature-branches),
let's create a feature branch that will add some content to the application.
Under your repository, navigate to the following file: `app/views/welcome/index.html.erb`.
......
doc/topics/git/index.md
View file @ 529bc7e2
......@@ -32,7 +32,7 @@ The following resources will help you get started with Git:
- Commits:
- [Revert a commit](../../user/project/merge_requests/revert_changes.md#reverting-a-commit)
- [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit)
- [Squashing commits](../../workflow/gitlab_flow.md#squashing-commits-with-rebase)
- [Squashing commits](../gitlab_flow.md#squashing-commits-with-rebase)
### Concepts
......
doc/topics/git/migrate_to_git_lfs/index.md
View file @ 529bc7e2
......@@ -163,9 +163,9 @@ but commented out to help encourage others to add to it in the future. -->
## References
- [Getting Started with Git LFS](https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/)
- [Migrate from Git Annex to Git LFS](../../../workflow/lfs/migrate_from_git_annex_to_git_lfs.md)
- [GitLab's Git LFS user documentation](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md)
- [GitLab's Git LFS administrator documentation](../../../workflow/lfs/lfs_administration.md)
- [Migrate from Git Annex to Git LFS](../../../administration/lfs/migrate_from_git_annex_to_git_lfs.md)
- [GitLab's Git LFS user documentation](../../../administration/lfs/manage_large_binaries_with_git_lfs.md)
- [GitLab's Git LFS administrator documentation](../../../administration/lfs/lfs_administration.md)
- Alternative method to [migrate an existing repo to Git LFS](https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existing-repository-data-to-lfs)
<!--
......
doc/topics/gitlab_flow.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/gitlab_flow.html'
---
# Introduction to GitLab Flow
![GitLab Flow](img/gitlab_flow.png)
Git allows a wide variety of branching strategies and workflows.
Because of this, many organizations end up with workflows that are too complicated, not clearly defined, or not integrated with issue tracking systems.
Therefore, we propose GitLab flow as a clearly defined set of best practices.
It combines [feature-driven development](https://en.wikipedia.org/wiki/Feature-driven_development) and [feature branches](https://martinfowler.com/bliki/FeatureBranch.html) with issue tracking.
Organizations coming to Git from other version control systems frequently find it hard to develop a productive workflow.
This article describes GitLab flow, which integrates the Git workflow with an issue tracking system.
It offers a simple, transparent, and effective way to work with Git.
![Four stages (working copy, index, local repo, remote repo) and three steps between them](img/gitlab_flow_four_stages.png)
When converting to Git, you have to get used to the fact that it takes three steps to share a commit with colleagues.
Most version control systems have only one step: committing from the working copy to a shared server.
In Git, you add files from the working copy to the staging area. After that, you commit them to your local repo.
The third step is pushing to a shared remote repository.
After getting used to these three steps, the next challenge is the branching model.
![Multiple long-running branches and merging in all directions](img/gitlab_flow_messy_flow.png)
Since many organizations new to Git have no conventions for how to work with it, their repositories can quickly become messy.
The biggest problem is that many long-running branches emerge that all contain part of the changes.
People have a hard time figuring out which branch has the latest code, or which branch to deploy to production.
Frequently, the reaction to this problem is to adopt a standardized pattern such as [Git flow](https://nvie.com/posts/a-successful-git-branching-model/) and [GitHub flow](http://scottchacon.com/2011/08/31/github-flow.html).
We think there is still room for improvement. In this document, we describe a set of practices we call GitLab flow.
For a video introduction of how this works in GitLab, see [GitLab Flow](https://youtu.be/InKNIvky2KE).
## Git flow and its problems
![Git Flow timeline by Vincent Driessen, used with permission](img/gitlab_flow_gitdashflow.png)
Git flow was one of the first proposals to use Git branches, and it has received a lot of attention.
It suggests a `master` branch and a separate `develop` branch, as well as supporting branches for features, releases, and hotfixes.
The development happens on the `develop` branch, moves to a release branch, and is finally merged into the `master` branch.
Git flow is a well-defined standard, but its complexity introduces two problems.
The first problem is that developers must use the `develop` branch and not `master`. `master` is reserved for code that is released to production.
It is a convention to call your default branch `master` and to mostly branch from and merge to this.
Since most tools automatically use the `master` branch as the default, it is annoying to have to switch to another branch.
The second problem of Git flow is the complexity introduced by the hotfix and release branches.
These branches can be a good idea for some organizations but are overkill for the vast majority of them.
Nowadays, most organizations practice continuous delivery, which means that your default branch can be deployed.
Continuous delivery removes the need for hotfix and release branches, including all the ceremony they introduce.
An example of this ceremony is the merging back of release branches.
Though specialized tools do exist to solve this, they require documentation and add complexity.
Frequently, developers make mistakes such as merging changes only into `master` and not into the `develop` branch.
The reason for these errors is that Git flow is too complicated for most use cases.
For example, many projects do releases but don't need to do hotfixes.
## GitHub flow as a simpler alternative
![Master branch with feature branches merged in](img/gitlab_flow_github_flow.png)
In reaction to Git flow, GitHub created a simpler alternative.
[GitHub flow](https://guides.github.com/introduction/flow/index.html) has only feature branches and a `master` branch.
This flow is clean and straightforward, and many organizations have adopted it with great success.
Atlassian recommends [a similar strategy](https://www.atlassian.com/blog/git/simple-git-workflow-is-simple), although they rebase feature branches.
Merging everything into the `master` branch and frequently deploying means you minimize the amount of unreleased code, which is in line with lean and continuous delivery best practices.
However, this flow still leaves a lot of questions unanswered regarding deployments, environments, releases, and integrations with issues.
With GitLab flow, we offer additional guidance for these questions.
## Production branch with GitLab flow
![Master branch and production branch with an arrow that indicates a deployment](img/gitlab_flow_production_branch.png)
GitHub flow assumes you can deploy to production every time you merge a feature branch.
While this is possible in some cases, such as SaaS applications, there are many cases where this is not possible.
One case is where you don't control the timing of a release, for example, an iOS application that is released when it passes App Store validation.
Another case is when you have deployment windows &mdash; for example, workdays from 10&nbsp;AM to 4&nbsp;PM when the operations team is at full capacity &mdash; but you also merge code at other times.
In these cases, you can make a production branch that reflects the deployed code.
You can deploy a new version by merging `master` into the production branch.
If you need to know what code is in production, you can just checkout the production branch to see.
The approximate time of deployment is easily visible as the merge commit in the version control system.
This time is pretty accurate if you automatically deploy your production branch.
If you need a more exact time, you can have your deployment script create a tag on each deployment.
This flow prevents the overhead of releasing, tagging, and merging that happens with Git flow.
## Environment branches with GitLab flow
![Multiple branches with the code cascading from one to another](img/gitlab_flow_environment_branches.png)
It might be a good idea to have an environment that is automatically updated to the `master` branch.
Only, in this case, the name of this environment might differ from the branch name.
Suppose you have a staging environment, a pre-production environment, and a production environment.
In this case, deploy the `master` branch to staging.
To deploy to pre-production, create a merge request from the `master` branch to the pre-production branch.
Go live by merging the pre-production branch into the production branch.
This workflow, where commits only flow downstream, ensures that everything is tested in all environments.
If you need to cherry-pick a commit with a hotfix, it is common to develop it on a feature branch and merge it into `master` with a merge request.
In this case, do not delete the feature branch yet.
If `master` passes automatic testing, you then merge the feature branch into the other branches.
If this is not possible because more manual testing is required, you can send merge requests from the feature branch to the downstream branches.
## Release branches with GitLab flow
![Master and multiple release branches that vary in length with cherry-picks from master](img/gitlab_flow_release_branches.png)
You only need to work with release branches if you need to release software to the outside world.
In this case, each branch contains a minor version, for example, 2-3-stable, 2-4-stable, etc.
Create stable branches using `master` as a starting point, and branch as late as possible.
By doing this, you minimize the length of time during which you have to apply bug fixes to multiple branches.
After announcing a release branch, only add serious bug fixes to the branch.
If possible, first merge these bug fixes into `master`, and then cherry-pick them into the release branch.
If you start by merging into the release branch, you might forget to cherry-pick them into `master`, and then you'd encounter the same bug in subsequent releases.
Merging into `master` and then cherry-picking into release is called an "upstream first" policy, which is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](https://www.redhat.com/en/blog/a-community-for-using-openstack-with-red-hat-rdo).
Every time you include a bug fix in a release branch, increase the patch version (to comply with [Semantic Versioning](https://semver.org/)) by setting a new tag.
Some projects also have a stable branch that points to the same commit as the latest released branch.
In this flow, it is not common to have a production branch (or Git flow `master` branch).
## Merge/pull requests with GitLab flow
![Merge request with inline comments](img/gitlab_flow_mr_inline_comments.png)
Merge or pull requests are created in a Git management application. They ask an assigned person to merge two branches.
Tools such as GitHub and Bitbucket choose the name "pull request" since the first manual action is to pull the feature branch.
Tools such as GitLab and others choose the name "merge request" since the final action is to merge the feature branch.
In this article, we'll refer to them as merge requests.
If you work on a feature branch for more than a few hours, it is good to share the intermediate result with the rest of the team.
To do this, create a merge request without assigning it to anyone.
Instead, mention people in the description or a comment, for example, "/cc @mark @susan."
This indicates that the merge request is not ready to be merged yet, but feedback is welcome.
Your team members can comment on the merge request in general or on specific lines with line comments.
The merge request serves as a code review tool, and no separate code review tools should be needed.
If the review reveals shortcomings, anyone can commit and push a fix.
Usually, the person to do this is the creator of the merge request.
The diff in the merge request automatically updates when new commits are pushed to the branch.
When you are ready for your feature branch to be merged, assign the merge request to the person who knows most about the codebase you are changing.
Also, mention any other people from whom you would like feedback.
After the assigned person feels comfortable with the result, they can merge the branch.
If the assigned person does not feel comfortable, they can request more changes or close the merge request without merging.
In GitLab, it is common to protect the long-lived branches, e.g., the `master` branch, so that [most developers can't modify them](../user/permissions.md).
So, if you want to merge into a protected branch, assign your merge request to someone with maintainer permissions.
After you merge a feature branch, you should remove it from the source control software.
In GitLab, you can do this when merging.
Removing finished branches ensures that the list of branches shows only work in progress.
It also ensures that if someone reopens the issue, they can use the same branch name without causing problems.
NOTE: **Note:**
When you reopen an issue you need to create a new merge request.
![Remove checkbox for branch in merge requests](img/gitlab_flow_remove_checkbox.png)
## Issue tracking with GitLab flow
![Merge request with the branch name "15-require-a-password-to-change-it" and assignee field shown](img/gitlab_flow_merge_request.png)
GitLab flow is a way to make the relation between the code and the issue tracker more transparent.
Any significant change to the code should start with an issue that describes the goal.
Having a reason for every code change helps to inform the rest of the team and to keep the scope of a feature branch small.
In GitLab, each change to the codebase starts with an issue in the issue tracking system.
If there is no issue yet, create the issue, as long as the change will take a significant amount of work, i.e., more than 1 hour.
In many organizations, raising an issue is part of the development process because they are used in sprint planning.
The issue title should describe the desired state of the system.
For example, the issue title "As an administrator, I want to remove users without receiving an error" is better than "Admin can't remove users."
When you are ready to code, create a branch for the issue from the `master` branch.
This branch is the place for any work related to this change.
NOTE: **Note:**
The name of a branch might be dictated by organizational standards.
When you are done or want to discuss the code, open a merge request.
A merge request is an online place to discuss the change and review the code.
If you open the merge request but do not assign it to anyone, it is a "Work In Progress" merge request.
These are used to discuss the proposed implementation but are not ready for inclusion in the `master` branch yet.
Start the title of the merge request with `[WIP]` or `WIP:` to prevent it from being merged before it's ready.
When you think the code is ready, assign the merge request to a reviewer.
The reviewer can merge the changes when they think the code is ready for inclusion in the `master` branch.
When they press the merge button, GitLab merges the code and creates a merge commit that makes this event easily visible later on.
Merge requests always create a merge commit, even when the branch could be merged without one.
This merge strategy is called "no fast-forward" in Git.
After the merge, delete the feature branch since it is no longer needed.
In GitLab, this deletion is an option when merging.
Suppose that a branch is merged but a problem occurs and the issue is reopened.
In this case, it is no problem to reuse the same branch name since the first branch was deleted when it was merged.
At any time, there is at most one branch for every issue.
It is possible that one feature branch solves more than one issue.
## Linking and closing issues from merge requests
![Merge request showing the linked issues that will be closed](img/gitlab_flow_close_issue_mr.png)
Link to issues by mentioning them in commit messages or the description of a merge request, for example, "Fixes #16" or "Duck typing is preferred. See #12."
GitLab then creates links to the mentioned issues and creates comments in the issues linking back to the merge request.
To automatically close linked issues, mention them with the words "fixes" or "closes," for example, "fixes #14" or "closes #67." GitLab closes these issues when the code is merged into the default branch.
If you have an issue that spans across multiple repositories, create an issue for each repository and link all issues to a parent issue.
## Squashing commits with rebase
![Vim screen showing the rebase view](img/gitlab_flow_rebase.png)
With Git, you can use an interactive rebase (`rebase -i`) to squash multiple commits into one or reorder them.
This functionality is useful if you want to replace a couple of small commits with a single commit, or if you want to make the order more logical.
However, you should never rebase commits you have pushed to a remote server.
Rebasing creates new commits for all your changes, which can cause confusion because the same change would have multiple identifiers.
It also causes merge errors for anyone working on the same branch because their history would not match with yours.
Also, if someone has already reviewed your code, rebasing makes it hard to tell what changed since the last review.
You should also never rebase commits authored by other people.
Not only does this rewrite history, but it also loses authorship information.
Rebasing prevents the other authors from being attributed and sharing part of the [`git blame`](https://git-scm.com/docs/git-blame).
If a merge involves many commits, it may seem more difficult to undo.
You might think to solve this by squashing all the changes into one commit before merging, but as discussed earlier, it is a bad idea to rebase commits that you have already pushed.
Fortunately, there is an easy way to undo a merge with all its commits.
The way to do this is by reverting the merge commit.
Preserving this ability to revert a merge is a good reason to always use the "no fast-forward" (`--no-ff`) strategy when you merge manually.
NOTE: **Note:**
If you revert a merge commit and then change your mind, revert the revert commit to redo the merge.
Git does not allow you to merge the code again otherwise.
## Reducing merge commits in feature branches
![List of sequential merge commits](img/gitlab_flow_merge_commits.png)
Having lots of merge commits can make your repository history messy.
Therefore, you should try to avoid merge commits in feature branches.
Often, people avoid merge commits by just using rebase to reorder their commits after the commits on the `master` branch.
Using rebase prevents a merge commit when merging `master` into your feature branch, and it creates a neat linear history.
However, as discussed in [the section about rebasing](#squashing-commits-with-rebase), you should never rebase commits you have pushed to a remote server.
This restriction makes it impossible to rebase work in progress that you already shared with your team, which is something we recommend.
Rebasing also creates more work, since every time you rebase, you have to resolve similar conflicts.
Sometimes you can reuse recorded resolutions (`rerere`), but merging is better since you only have to resolve conflicts once.
Atlassian has a more thorough explanation of the tradeoffs between merging and rebasing [on their blog](https://www.atlassian.com/blog/git/git-team-workflows-merge-or-rebase).
A good way to prevent creating many merge commits is to not frequently merge `master` into the feature branch.
There are three reasons to merge in `master`: utilizing new code, resolving merge conflicts, and updating long-running branches.
If you need to utilize some code that was introduced in `master` after you created the feature branch, you can often solve this by just cherry-picking a commit.
If your feature branch has a merge conflict, creating a merge commit is a standard way of solving this.
NOTE: **Note:**
Sometimes you can use .gitattributes to reduce merge conflicts.
For example, you can set your changelog file to use the [union merge driver](https://git-scm.com/docs/gitattributes#gitattributes-union) so that multiple new entries don't conflict with each other.
The last reason for creating merge commits is to keep long-running feature branches up-to-date with the latest state of the project.
The solution here is to keep your feature branches short-lived.
Most feature branches should take less than one day of work.
If your feature branches often take more than a day of work, try to split your features into smaller units of work.
If you need to keep a feature branch open for more than a day, there are a few strategies to keep it up-to-date.
One option is to use continuous integration (CI) to merge in `master` at the start of the day.
Another option is to only merge in from well-defined points in time, for example, a tagged release.
You could also use [feature toggles](https://martinfowler.com/bliki/FeatureToggle.html) to hide incomplete features so you can still merge back into `master` every day.
> **Note:** Don't confuse automatic branch testing with continuous integration.
> Martin Fowler makes this distinction in [his article about feature branches](https://martinfowler.com/bliki/FeatureBranch.html):
>
> "I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit.
> That's continuous building, and a Good Thing, but there's no *integration*, so it's not CI."
In conclusion, you should try to prevent merge commits, but not eliminate them.
Your codebase should be clean, but your history should represent what actually happened.
Developing software happens in small, messy steps, and it is OK to have your history reflect this.
You can use tools to view the network graphs of commits and understand the messy history that created your code.
If you rebase code, the history is incorrect, and there is no way for tools to remedy this because they can't deal with changing commit identifiers.
## Commit often and push frequently
Another way to make your development work easier is to commit often.
Every time you have a working set of tests and code, you should make a commit.
Splitting up work into individual commits provides context for developers looking at your code later.
Smaller commits make it clear how a feature was developed, and they make it easy to roll back to a specific good point in time or to revert one code change without reverting several unrelated changes.
Committing often also makes it easy to share your work, which is important so that everyone is aware of what you are working on.
You should push your feature branch frequently, even when it is not yet ready for review.
By sharing your work in a feature branch or [a merge request](#mergepull-requests-with-gitlab-flow), you prevent your team members from duplicating work.
Sharing your work before it's complete also allows for discussion and feedback about the changes, which can help improve the code before it gets to review.
## How to write a good commit message
![Good and bad commit message](img/gitlab_flow_good_commit.png)
A commit message should reflect your intention, not just the contents of the commit.
It is easy to see the changes in a commit, so the commit message should explain why you made those changes.
An example of a good commit message is: "Combine templates to reduce duplicate code in the user views."
The words "change," "improve," "fix," and "refactor" don't add much information to a commit message.
For example, "Improve XML generation" could be better written as "Properly escape special characters in XML generation."
For more information about formatting commit messages, please see this excellent [blog post by Tim Pope](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
## Testing before merging
![Merge requests showing the test states: red, yellow, and green](img/gitlab_flow_ci_mr.png)
In old workflows, the continuous integration (CI) server commonly ran tests on the `master` branch only.
Developers had to ensure their code did not break the `master` branch.
When using GitLab flow, developers create their branches from this `master` branch, so it is essential that it never breaks.
Therefore, each merge request must be tested before it is accepted.
CI software like Travis CI and GitLab CI show the build results right in the merge request itself to make this easy.
There is one drawback to testing merge requests: the CI server only tests the feature branch itself, not the merged result.
Ideally, the server could also test the `master` branch after each change.
However, retesting on every commit to `master` is computationally expensive and means you are more frequently waiting for test results.
Since feature branches should be short-lived, testing just the branch is an acceptable risk.
If new commits in `master` cause merge conflicts with the feature branch, merge `master` back into the branch to make the CI server re-run the tests.
As said before, if you often have feature branches that last for more than a few days, you should make your issues smaller.
## Working with feature branches
![Shell output showing git pull output](img/gitlab_flow_git_pull.png)
When creating a feature branch, always branch from an up-to-date `master`.
If you know before you start that your work depends on another branch, you can also branch from there.
If you need to merge in another branch after starting, explain the reason in the merge commit.
If you have not pushed your commits to a shared location yet, you can also incorporate changes by rebasing on `master` or another feature branch.
Do not merge from upstream again if your code can work and merge cleanly without doing so.
Merging only when needed prevents creating merge commits in your feature branch that later end up littering the `master` history.
doc/topics/index.md
View file @ 529bc7e2
......@@ -11,6 +11,7 @@ tutorials, technical overviews, blog posts) and videos.
- [Authentication](authentication/index.md)
- [Continuous Integration (GitLab CI)](../ci/README.md)
- [Git](git/index.md)
- [GitLab Flow](gitlab_flow.md)
- [GitLab Installation](../install/README.md)
- [GitLab Pages](../user/project/pages/index.md)
......
doc/university/README.md
View file @ 529bc7e2
......@@ -129,7 +129,7 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres
1. [GitLab Flow vs Forking in GitLab - Video](https://www.youtube.com/watch?v=UGotqAUACZA)
1. [GitLab Flow Overview](https://about.gitlab.com/blog/2014/09/29/gitlab-flow/)
1. [Always Start with an Issue](https://about.gitlab.com/blog/2016/03/03/start-with-an-issue/)
1. [GitLab Flow Documentation](../workflow/gitlab_flow.md)
1. [GitLab Flow Documentation](../topics/gitlab_flow.md)
### 2.5. GitLab Comparisons
......
doc/university/training/gitlab_flow.md
View file @ 529bc7e2
......@@ -38,7 +38,7 @@ type: reference
## More details
For more information, read through the [GitLab Flow](../../workflow/gitlab_flow.md)
For more information, read through the [GitLab Flow](../../topics/gitlab_flow.md)
documentation.
<!-- ## Troubleshooting
......
doc/user/admin_area/settings/account_and_limit_settings.md
View file @ 529bc7e2
......@@ -32,7 +32,7 @@ For instance, consider the following workflow:
1. Your team develops apps which require large files to be stored in
the application repository.
1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md#git-lfs)
1. Although you have enabled [Git LFS](../../../administration/lfs/manage_large_binaries_with_git_lfs.md#git-lfs)
to your project, your storage has grown significantly.
1. Before you exceed available storage, you set up a limit of 10 GB
per repository.
......
doc/user/admin_area/settings/usage_statistics.md
View file @ 529bc7e2
......@@ -31,7 +31,7 @@ patches will need to be backported, making sure active GitLab instances remain
secure.
If you disable version check, this information will not be collected. Enable or
disable the version check at **Admin area > Settings > Usage statistics**.
disable the version check at **Admin area > Settings > Metrics and profiling > Usage statistics**.
## Usage ping **(CORE ONLY)**
......@@ -85,7 +85,7 @@ will be able to show [usage statistics](../../instance_statistics/index.md)
of your instance to your users.
This can be restricted to admins by selecting "Only admins" in the Instance
Statistics visibility section under **Admin area > Settings > Usage statistics**.
Statistics visibility section under **Admin area > Settings > Metrics and profiling > Usage statistics**.
<!-- ## Troubleshooting
......
doc/user/admin_area/settings/visibility_and_access_controls.md
View file @ 529bc7e2
......@@ -177,7 +177,7 @@ For more details, see [SSH key restrictions](../../../security/ssh_keys_restrict
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3586) in GitLab 10.3.
This option is enabled by default. By disabling it, both [pull and push mirroring](../../../workflow/repository_mirroring.md) will no longer
This option is enabled by default. By disabling it, both [pull and push mirroring](../../project/repository/repository_mirroring.md) will no longer
work in every repository and can only be re-enabled by an admin on a per-project basis.
![Mirror settings](img/mirror_settings.png)
......
doc/user/analytics/cycle_analytics.md
View file @ 529bc7e2
......@@ -176,7 +176,7 @@ Learn more about Cycle Analytics in the following resources:
[ce-5986]: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/5986
[ce-20975]: https://gitlab.com/gitlab-org/gitlab-foss/issues/20975
[environment]: ../../ci/yaml/README.md#environment
[GitLab flow]: ../../workflow/gitlab_flow.md
[GitLab flow]: ../../topics/gitlab_flow.md
[idea to production]: https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab
[permissions]: ../permissions.md
[yml]: ../../ci/yaml/README.md
doc/user/group/epics/index.md
View file @ 529bc7e2
......@@ -287,7 +287,7 @@ Once you wrote your comment, you can either:
## Notifications
- [Receive notifications](../../../workflow/notifications.md) for epic events.
- [Receive notifications](../../profile/notifications.md) for epic events.
<!-- ## Troubleshooting
......
doc/user/index.md
View file @ 529bc7e2
......@@ -58,7 +58,7 @@ With GitLab Enterprise Edition, you can also:
- Use [Burndown Charts](project/milestones/burndown_charts.md) to track progress during a sprint or while working on a new version of their software.
- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance.
- [Authenticate users with Kerberos](../integration/kerberos.md).
- [Mirror a repository](../workflow/repository_mirroring.md) from elsewhere on your local server.
- [Mirror a repository](project/repository/repository_mirroring.md) from elsewhere on your local server.
- [Export issues as CSV](project/issues/csv_export.md).
- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipeline_graphs.md).
- [Lock files](project/file_lock.md) to prevent conflicts.
......
doc/user/profile/index.md
View file @ 529bc7e2
......@@ -53,7 +53,7 @@ From there, you can:
[use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth)
- Manage [personal access tokens](personal_access_tokens.md) to access your account via API and authorized applications
- Add and delete emails linked to your account
- Choose which email to use for notifications, web-based commits, and display on your public profile
- Choose which email to use for [notifications](notifications.md), web-based commits, and display on your public profile
- Manage [SSH keys](../../ssh/README.md) to access your account via SSH
- Manage your [preferences](preferences.md#syntax-highlighting-theme)
to customize your own GitLab experience
......
doc/user/profile/notifications.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/notifications.html'
---
# GitLab Notification Emails
GitLab has a notification system in place to notify a user of events that are important for the workflow.
## Notification settings
You can find notification settings under the user profile.
![notification settings](img/notification_global_settings.png)
Notification settings are divided into three groups:
- Global settings
- Group settings
- Project settings
Each of these settings have levels of notification:
- Global: For groups and projects, notifications as per global settings.
- Watch: Receive notifications for any activity.
- Participate: Receive notifications for threads you have participated in.
- On Mention: Receive notifications when `@mentioned` in comments.
- Disabled: Turns off notifications.
- Custom: Receive notifications for custom selected events.
> Introduced in GitLab 12.0
You can also select an email address to receive notifications for each group you belong to.
### Global Settings
Global settings are at the bottom of the hierarchy.
Any setting set here will be overridden by a setting at the group or a project level.
Group or Project settings can use `global` notification setting which will then use
anything that is set at Global Settings.
### Group Settings
![notification settings](img/notification_group_settings.png)
Group settings are taking precedence over Global Settings but are on a level below Project or Subgroup settings:
```
Group < Subgroup < Project
```
This means that you can set a different level of notifications per group while still being able
to have a finer level setting per project or subgroup.
Organization like this is suitable for users that belong to different groups but don't have the
same need for being notified for every group they are member of.
These settings can be configured on group page under the name of the group. It will be the dropdown with the bell icon. They can also be configured on the user profile notifications dropdown.
The group owner can disable email notifications for a group, which includes
its subgroups and projects. If this is the case, you will not receive any corresponding notifications,
and the notification button will be disabled with an explanatory tooltip.
### Project Settings
![notification settings](img/notification_project_settings.png)
Project settings are at the top level and any setting placed at this level will take precedence of any
other setting.
This is suitable for users that have different needs for notifications per project basis.
These settings can be configured on project page under the name of the project. It will be the dropdown with the bell icon. They can also be configured on the user profile notifications dropdown.
The project owner (or its group owner) can disable email notifications for the project.
If this is the case, you will not receive any corresponding notifications, and the notification
button will be disabled with an explanatory tooltip.
## Notification events
Below is the table of events users can be notified of:
| Event | Sent to | Settings level |
|------------------------------|---------------------|------------------------------|
| New SSH key added | User | Security email, always sent. |
| New email added | User | Security email, always sent. |
| Email changed | User | Security email, always sent. |
| Password changed | User | Security email, always sent. |
| New user created | User | Sent on user creation, except for OmniAuth (LDAP)|
| User added to project | User | Sent when user is added to project |
| Project access level changed | User | Sent when user project access level is changed |
| User added to group | User | Sent when user is added to group |
| Group access level changed | User | Sent when user group access level is changed |
| Project moved | Project members (1) | (1) not disabled |
| New release | Project members | Custom notification |
### Issue / Epics / Merge request events
In most of the below cases, the notification will be sent to:
- Participants:
- the author and assignee of the issue/merge request
- authors of comments on the issue/merge request
- anyone mentioned by `@username` in the title or description of the issue, merge request or epic **(ULTIMATE)**
- anyone with notification level "Participating" or higher that is mentioned by `@username`
in any of the comments on the issue, merge request, or epic **(ULTIMATE)**
- Watchers: users with notification level "Watch"
- Subscribers: anyone who manually subscribed to the issue, merge request, or epic **(ULTIMATE)**
- Custom: Users with notification level "custom" who turned on notifications for any of the events present in the table below
| Event | Sent to |
|------------------------|---------|
| New issue | |
| Close issue | |
| Reassign issue | The above, plus the old assignee |
| Reopen issue | |
| Due issue | Participants and Custom notification level with this event selected |
| Change milestone issue | Subscribers, participants mentioned, and Custom notification level with this event selected |
| Remove milestone issue | Subscribers, participants mentioned, and Custom notification level with this event selected |
| New merge request | |
| Push to merge request | Participants and Custom notification level with this event selected |
| Reassign merge request | The above, plus the old assignee |
| Close merge request | |
| Reopen merge request | |
| Merge merge request | |
| Change milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected |
| Remove milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected |
| New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher |
| Failed pipeline | The author of the pipeline |
| Successful pipeline | The author of the pipeline, if they have the custom notification setting for successful pipelines set |
| New epic **(ULTIMATE)** | |
| Close epic **(ULTIMATE)** | |
| Reopen epic **(ULTIMATE)** | |
In addition, if the title or description of an Issue or Merge Request is
changed, notifications will be sent to any **new** mentions by `@username` as
if they had been mentioned in the original text.
You won't receive notifications for Issues, Merge Requests or Milestones created
by yourself (except when an issue is due). You will only receive automatic
notifications when somebody else comments or adds changes to the ones that
you've created or mentions you.
If an open merge request becomes unmergeable due to conflict, its author will be notified about the cause.
If a user has also set the merge request to automatically merge once pipeline succeeds,
then that user will also be notified.
### Email Headers
Notification emails include headers that provide extra content about the notification received:
| Header | Description |
|-----------------------------|-------------------------------------------------------------------------|
| X-GitLab-Project | The name of the project the notification belongs to |
| X-GitLab-Project-Id | The ID of the project |
| X-GitLab-Project-Path | The path of the project |
| X-GitLab-(Resource)-ID | The ID of the resource the notification is for, where resource is `Issue`, `MergeRequest`, `Commit`, etc|
| X-GitLab-Discussion-ID | Only in comment emails, the ID of the thread the comment is from |
| X-GitLab-Pipeline-Id | Only in pipeline emails, the ID of the pipeline the notification is for |
| X-GitLab-Reply-Key | A unique token to support reply by email |
| X-GitLab-NotificationReason | The reason for being notified. "mentioned", "assigned", etc |
| List-Id | The path of the project in a RFC 2919 mailing list identifier useful for email organization, for example, with Gmail filters |
#### X-GitLab-NotificationReason
This header holds the reason for the notification to have been sent out,
where reason can be `mentioned`, `assigned`, `own_activity`, etc.
Only one reason is sent out according to its priority:
- `own_activity`
- `assigned`
- `mentioned`
The reason in this header will also be shown in the footer of the notification email. For example an email with the
reason `assigned` will have this sentence in the footer:
`"You are receiving this email because you have been assigned an item on {configured GitLab hostname}"`
NOTE: **Note:**
Only reasons listed above have been implemented so far.
Further implementation is [being discussed](https://gitlab.com/gitlab-org/gitlab/issues/20689).
doc/user/project/import/github.md
View file @ 529bc7e2
......@@ -125,7 +125,7 @@ your GitHub repositories are listed.
## Mirroring and pipeline status sharing
Depending your GitLab tier, [project mirroring](../../../workflow/repository_mirroring.md) can be set up to keep
Depending your GitLab tier, [project mirroring](../repository/repository_mirroring.md) can be set up to keep
your imported project in sync with its GitHub copy.
Additionally, you can configure GitLab to send pipeline status updates back GitHub with the
......
doc/user/project/index.md
View file @ 529bc7e2
......@@ -26,6 +26,7 @@ When you create a project in GitLab, you'll have access to a large number of
from messing with history or pushing code without review
- [Protected tags](protected_tags.md): Control over who has
permission to create tags, and prevent accidental update or deletion
- [Repository mirroring](repository/repository_mirroring.md)
- [Signing commits](gpg_signed_commits/index.md): use GPG to sign your commits
- [Deploy tokens](deploy_tokens/index.md): Manage project-based deploy tokens that allow permanent access to the repository and Container Registry.
- [Web IDE](web_ide/index.md)
......
doc/user/project/integrations/prometheus.md
View file @ 529bc7e2
......@@ -355,6 +355,8 @@ Once enabled, an issue will be opened automatically when an alert is triggered w
- Optional list of attached annotations extracted from `annotations/*`
- Alert [GFM](../../markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown`
When GitLab recieves a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicated that it was closed automatically by the GitLab Alert bot.
To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field.
Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well.
......
doc/user/project/issues/issue_data_and_actions.md
View file @ 529bc7e2
......@@ -109,7 +109,7 @@ from which you can select **Create new label**.
#### 9. Weight **(STARTER)**
[Assign a weight](../../../workflow/issue_weight.md) to an issue.
[Assign a weight](issue_weight.md) to an issue.
Larger values are used to indicate more effort is required to complete the issue. Only
positive values or zero are allowed.
......@@ -131,7 +131,7 @@ or were mentioned in the description or threads.
#### 13. Notifications
Click on the icon to enable/disable [notifications](../../../workflow/notifications.md#issue--epics--merge-request-events)
Click on the icon to enable/disable [notifications](../../profile/notifications.md#issue--epics--merge-request-events)
for the issue. This will automatically enable if you participate in the issue in any way.
- **Enable**: If you are not a participant in the discussion on that issue, but
......@@ -162,7 +162,7 @@ allowing many formatting options.
You can mention a user or a group present in your GitLab instance with `@username` or
`@groupname` and they will be notified via todos and email, unless they have disabled
all notifications in their profile settings. This is controlled in the
[notification settings](../../../workflow/notifications.md).
[notification settings](../../profile/notifications.md).
Mentions for yourself (the current logged in user), will be highlighted in a different
color, allowing you to easily see which comments involve you, helping you focus on
......
doc/user/project/issues/issue_weight.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/issue_weight.html'
---
# Issue weight **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/76) in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3.
When you have a lot of issues, it can be hard to get an overview.
By adding a weight to each issue, you can get a better idea of how much time,
value or complexity a given issue has or will cost.
You can set the weight of an issue during its creation, by simply changing the
value in the dropdown menu. You can set it to a non-negative integer
value from 0, 1, 2, and so on. (The database stores a 4-byte value, so the
upper bound is essentially limitless).
You can remove weight from an issue
as well.
This value will appear on the right sidebar of an individual issue, as well as
in the issues page next to a distinctive balance scale icon.
As an added bonus, you can see the total sum of all issues on the milestone page.
![issue page](img/issue_weight.png)
doc/user/project/labels.md
View file @ 529bc7e2
......@@ -210,8 +210,8 @@ The following can be filtered by labels:
## Subscribing to labels
From the project label list page and the group label list page, you can subscribe
to [notifications](../../workflow/notifications.md) of a given label, to alert you
that the label has been assigned to an epic, issue, and merge request.
to [notifications](../profile/notifications.md) of a given label, to alert you
that the label has been assigned to an epic, issue, or merge request.
![Labels subscriptions](img/labels_subscriptions_v12_1.png)
......
doc/user/project/protected_branches.md
View file @ 529bc7e2
......@@ -55,7 +55,7 @@ the actions that different roles can perform with the protected branch.
For example, you could set "Allowed to push" to "No one", and "Allowed to merge"
to "Developers + Maintainers", to require _everyone_ to submit a merge request for
changes going into the protected branch. This is compatible with workflows like
the [GitLab workflow](../../workflow/gitlab_flow.md).
the [GitLab workflow](../../topics/gitlab_flow.md).
However, there are workflows where that is not needed, and only protecting from
force pushes and branch removal is useful. For those workflows, you can allow
......
doc/user/project/releases/index.md
View file @ 529bc7e2
......@@ -77,6 +77,26 @@ following modal window will be then displayed, from which you can select **New r
![Custom notification - New release](img/custom_notifications_new_release_v12_4.png)
## Add release notes to Git tags
You can add release notes to any Git tag using the notes feature. Release notes
behave like any other markdown form in GitLab so you can write text and
drag and drop files to it. Release notes are stored in GitLab's database.
There are several ways to add release notes:
- In the interface, when you create a new Git tag
- In the interface, by adding a note to an existing Git tag
- Using the GitLab API
### New tag page with release notes text area
![new_tag](img/new_tag.png)
### Tags page with button to add or edit release notes for existing Git tag
![tags](img/tags.png)
<!-- ## Troubleshooting
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
......
doc/user/project/repository/file_finder.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/file_finder.html'
---
# File finder
> [Introduced][gh-9889] in GitLab 8.4.
The file finder feature allows you to search for a file in a repository using the
GitLab UI.
You can find the **Find File** button when in the **Files** section of a
project.
![Find file button](img/file_finder_find_button.png)
For those who prefer to keep their fingers on the keyboard, there is a
[shortcut button](../../../workflow/shortcuts.md) as well, which you can invoke from _anywhere_
in a project.
Press `t` to launch the File search function when in **Issues**,
**Merge requests**, **Milestones**, even the project's settings.
Start typing what you are searching for and watch the magic happen. With the
up/down arrows, you go up and down the results, with `Esc` you close the search
and go back to **Files**.
## How it works
The File finder feature is powered by the [Fuzzy filter](https://github.com/jeancroy/fuzz-aldrin-plus) library.
It implements a fuzzy search with highlight, and tries to provide intuitive
results by recognizing patterns that people use while searching.
For example, consider the [GitLab CE repository][ce] and that we want to open
the `app/controllers/admin/deploy_keys_controller.rb` file.
Using fuzzy search, we start by typing letters that get us closer to the file.
**Protip:** To narrow down your search, include `/` in your search terms.
![Find file button](img/file_finder_find_file.png)
[gh-9889]: https://github.com/gitlabhq/gitlabhq/pull/9889 "File finder pull request"
[ce]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master "GitLab CE repository"
doc/user/project/repository/forking_workflow.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/forking_workflow.html'
---
# Project forking workflow
Forking a project to your own namespace is useful if you have no write
access to the project you want to contribute to. If you do have write
access or can request it, we recommend working together in the same
repository since it is simpler. See our [GitLab Flow](../../../workflow/gitlab_flow.md)
document more information about using branches to work together.
## Creating a fork
Forking a project is in most cases a two-step process.
1. Click on the fork button located located in between the star and clone buttons on the project's home page.
![Fork button](img/forking_workflow_fork_button.png)
1. Once you do that, you'll be presented with a screen where you can choose
the namespace to fork to. Only namespaces (groups and your own
namespace) where you have write access to, will be shown. Click on the
namespace to create your fork there.
![Choose namespace](img/forking_workflow_choose_namespace.png)
**Note:**
If the namespace you chose to fork the project to has another project with
the same path name, you will be presented with a warning that the forking
could not be completed. Try to resolve the error before repeating the forking
process.
![Path taken error](img/forking_workflow_path_taken_error.png)
After the forking is done, you can start working on the newly created
repository. There, you will have full [Owner](../../permissions.md)
access, so you can set it up as you please.
## Merging upstream
Once you are ready to send your code back to the main project, you need
to create a merge request. Choose your forked project's main branch as
the source and the original project's main branch as the destination and
create the [merge request](../merge_requests/index.md).
![Selecting branches](img/forking_workflow_branch_select.png)
You can then assign the merge request to someone to have them review
your changes. Upon pressing the 'Submit Merge Request' button, your
changes will be added to the repository and branch you're merging into.
![New merge request](img/forking_workflow_merge_request.png)
[gitlab flow]: https://about.gitlab.com/blog/2014/09/29/gitlab-flow/ "GitLab Flow blog post"
doc/user/project/repository/index.md
View file @ 529bc7e2
......@@ -11,7 +11,8 @@ A repository is part of a [project](../index.md), which has a lot of other featu
## Create a repository
To create a new repository, all you need to do is
[create a new project](../../../gitlab-basics/create-project.md).
[create a new project](../../../gitlab-basics/create-project.md) or
[fork an existing project](forking_workflow.md).
Once you create a new project, you can add new files via UI
(read the section below) or via command line.
......@@ -55,7 +56,7 @@ To get started with the command line, please read through the
### Find files
Use GitLab's [file finder](../../../workflow/file_finder.md) to search for files in a repository.
Use GitLab's [file finder](file_finder.md) to search for files in a repository.
### Supported markup languages and extensions
......
doc/user/project/repository/repository_mirroring.md 0 → 100644
View file @ 529bc7e2
---
disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html'
---
# Repository mirroring
Repository mirroring allows for mirroring of repositories to and from external sources. It can be
used to mirror branches, tags, and commits between repositories.
A repository mirror at GitLab will be updated automatically. You can also manually trigger an update
at most once every 5 minutes.
## Overview
Repository mirroring is useful when you want to use a repository outside of GitLab.
There are two kinds of repository mirroring supported by GitLab:
- Push: for mirroring a GitLab repository to another location.
- Pull: for mirroring a repository from another location to GitLab. **(STARTER)**
When the mirror repository is updated, all new branches, tags, and commits will be visible in the
project's activity feed.
Users with at least [developer access](../../permissions.md) to the project can also force an
immediate update, unless:
- The mirror is already being updated.
- 5 minutes haven't elapsed since its last update.
## Use cases
The following are some possible use cases for repository mirroring:
- You migrated to GitLab but still need to keep your project in another source. In that case, you
can simply set it up to mirror to GitLab (pull) and all the essential history of commits, tags,
and branches will be available in your GitLab instance. **(STARTER)**
- You have old projects in another source that you don't use actively anymore, but don't want to
remove for archiving purposes. In that case, you can create a push mirror so that your active
GitLab repository can push its changes to the old location.
## Pushing to a remote repository **(CORE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/249) in GitLab Enterprise Edition 8.7.
> - [Moved to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/18715) in 10.8.
For an existing project, you can set up push mirroring as follows:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories** section.
1. Enter a repository URL.
1. Select **Push** from the **Mirror direction** dropdown.
1. Select an authentication method from the **Authentication method** dropdown, if necessary.
1. Check the **Only mirror protected branches** box, if necessary.
1. Click the **Mirror repository** button to save the configuration.
![Repository mirroring push settings screen](img/repository_mirroring_push_settings.png)
When push mirroring is enabled, only push commits directly to the mirrored repository to prevent the
mirror diverging. All changes will end up in the mirrored repository whenever:
- Commits are pushed to GitLab.
- A [forced update](#forcing-an-update-core) is initiated.
Changes pushed to files in the repository are automatically pushed to the remote mirror at least:
- Within five minutes of being received.
- Within one minute if **Only mirror protected branches** is enabled.
In the case of a diverged branch, you will see an error indicated at the **Mirroring repositories**
section.
### Push only protected branches **(CORE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3350) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
> - [Moved to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/18715) in 10.8.
You can choose to only push your protected branches from GitLab to your remote repository.
To use this option, check the **Only mirror protected branches** box when creating a repository
mirror.
## Setting up a push mirror from GitLab to GitHub **(CORE)**
To set up a mirror from GitLab to GitHub, you need to follow these steps:
1. Create a [GitHub personal access token](https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line) with the `public_repo` box checked.
1. Fill in the **Git repository URL** field using this format: `https://<your_github_username>@github.com/<your_github_group>/<your_github_project>.git`.
1. Fill in **Password** field with your GitHub personal access token.
1. Click the **Mirror repository** button.
The mirrored repository will be listed. For example, `https://*****:*****@github.com/<your_github_group>/<your_github_project>.git`.
The repository will push soon. To force a push, click the appropriate button.
## Setting up a push mirror to another GitLab instance with 2FA activated
1. On the destination GitLab instance, create a [personal access token](../../profile/personal_access_tokens.md) with `API` scope.
1. On the source GitLab instance:
1. Fill in the **Git repository URL** field using this format: `https://oauth2@<destination host>/<your_gitlab_group_or_name>/<your_gitlab_project>.git`.
1. Fill in **Password** field with the GitLab personal access token created on the destination GitLab instance.
1. Click the **Mirror repository** button.
## Pulling from a remote repository **(STARTER)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/51) in GitLab Enterprise Edition 8.2.
> - [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab/issues/10871) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.11.
NOTE: **Note:** This feature [is available for free](https://gitlab.com/gitlab-org/gitlab/issues/10361) to
GitLab.com users until March 22nd, 2020.
You can set up a repository to automatically have its branches, tags, and commits updated from an
upstream repository.
This is useful when a repository you're interested in is located on a different server, and you want
to be able to browse its content and its activity using the familiar GitLab interface.
To configure mirror pulling for an existing project:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories**
section.
1. Enter a repository URL.
1. Select **Pull** from the **Mirror direction** dropdown.
1. Select an authentication method from the **Authentication method** dropdown, if necessary.
1. If necessary, check the following boxes:
- **Overwrite diverged branches**.
- **Trigger pipelines for mirror updates**.
- **Only mirror protected branches**.
1. Click the **Mirror repository** button to save the configuration.
![Repository mirroring pull settings screen - upper part](img/repository_mirroring_pull_settings_upper.png)
---
![Repository mirroring pull settings screen - lower part](img/repository_mirroring_pull_settings_lower.png)
Because GitLab is now set to pull changes from the upstream repository, you should not push commits
directly to the repository on GitLab. Instead, any commits should be pushed to the upstream repository.
Changes pushed to the upstream repository will be pulled into the GitLab repository, either:
- Automatically within a certain period of time.
- When a [forced update](#forcing-an-update-core) is initiated.
CAUTION: **Caution:**
If you do manually update a branch in the GitLab repository, the branch will become diverged from
upstream and GitLab will no longer automatically update this branch to prevent any changes from being lost.
### How it works
Once the pull mirroring feature has been enabled for a repository, the repository is added to a queue.
Once per minute, a Sidekiq cron job schedules repository mirrors to update, based on:
- The capacity available. This is determined by Sidekiq settings. For GitLab.com, see [GitLab.com Sidekiq settings](../../gitlab_com/index.md#sidekiq).
- The number of repository mirrors already in the queue that are due to be updated. Being due depends on when the repository mirror was last updated and how many times it's been retried.
Repository mirrors are updated as Sidekiq becomes available to process them. If the process of updating the repository mirror:
- Succeeds, an update will be enqueued again with at least a 30 minute wait.
- Fails (for example, a branch diverged from upstream), it will be attempted again later. Mirrors can fail
up to 14 times before they will not be enqueued for update again.
### SSH authentication
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/2551) for Pull mirroring in [GitLab Starter](https://about.gitlab.com/pricing/) 9.5.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/22982) for Push mirroring in [GitLab Core](https://about.gitlab.com/pricing/) 11.6
SSH authentication is mutual:
- You have to prove to the server that you're allowed to access the repository.
- The server also has to prove to *you* that it's who it claims to be.
You provide your credentials as a password or public key. The server that the
other repository resides on provides its credentials as a "host key", the
fingerprint of which needs to be verified manually.
If you're mirroring over SSH (that is, using an `ssh://` URL), you can authenticate using:
- Password-based authentication, just as over HTTPS.
- Public key authentication. This is often more secure than password authentication,
especially when the other repository supports [Deploy Keys](../../../ssh/README.md#deploy-keys).
To get started:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories** section.
1. Enter an `ssh://` URL for mirroring.
NOTE: **Note:**
SCP-style URLs (that is, `git@example.com:group/project.git`) are not supported at this time.
Entering the URL adds two buttons to the page:
- **Detect host keys**.
- **Input host keys manually**.
If you click the:
- **Detect host keys** button, GitLab will fetch the host keys from the server and display the fingerprints.
- **Input host keys manually** button, a field is displayed where you can paste in host keys.
Assuming you used the former, you now need to verify that the fingerprints are
those you expect. GitLab.com and other code hosting sites publish their
fingerprints in the open for you to check:
- [AWS CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints)
- [Bitbucket](https://confluence.atlassian.com/bitbucket/ssh-keys-935365775.html)
- [GitHub](https://help.github.com/en/articles/githubs-ssh-key-fingerprints)
- [GitLab.com](../../gitlab_com/index.md#ssh-host-keys-fingerprints)
- [Launchpad](https://help.launchpad.net/SSHFingerprints)
- [Savannah](http://savannah.gnu.org/maintenance/SshAccess/)
- [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/)
Other providers will vary. If you're running self-managed GitLab, or otherwise
have access to the server for the other repository, you can securely gather the
key fingerprints:
```sh
$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f -
256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA)
256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519)
2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA)
```
NOTE: **Note:**
You may need to exclude `-E md5` for some older versions of SSH.
When mirroring the repository, GitLab will now check that at least one of the
stored host keys matches before connecting. This can prevent malicious code from
being injected into your mirror, or your password being stolen.
### SSH public key authentication
To use SSH public key authentication, you'll also need to choose that option
from the **Authentication method** dropdown. When the mirror is created,
GitLab generates a 4096-bit RSA key that can be copied by clicking the **Copy SSH public key** button.
![Repository mirroring copy SSH public key to clipboard button](img/repository_mirroring_copy_ssh_public_key_button.png)
You then need to add the public SSH key to the other repository's configuration:
- If the other repository is hosted on GitLab, you should add the public SSH key
as a [Deploy Key](../../../ssh/README.md#deploy-keys).
- If the other repository is hosted elsewhere, you may need to add the key to
your user's `authorized_keys` file. Paste the entire public SSH key into the
file on its own line and save it.
If you need to change the key at any time, you can remove and re-add the mirror
to generate a new key. You'll have to update the other repository with the new
key to keep the mirror running.
### Overwrite diverged branches **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/4559) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.6.
You can choose to always update your local branches with remote versions, even if they have
diverged from the remote.
CAUTION: **Caution:**
For mirrored branches, enabling this option results in the loss of local changes.
To use this option, check the **Overwrite diverged branches** box when creating a repository mirror.
### Only mirror protected branches **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3326) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
You can choose to pull mirror only the protected branches from your remote repository to GitLab.
Non-protected branches are not mirrored and can diverge.
To use this option, check the **Only mirror protected branches** box when creating a repository mirror.
### Hard failure **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3117) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.2.
Once the mirroring process is unsuccessfully retried 14 times in a row, it will get marked as hard
failed. This will become visible in either the:
- Project's main dashboard.
- Pull mirror settings page.
When a project is hard failed, it will no longer get picked up for mirroring. A user can resume the
project mirroring again by [Forcing an update](#forcing-an-update-core).
### Trigger update using API **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3453) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
Pull mirroring uses polling to detect new branches and commits added upstream, often minutes
afterwards. If you notify GitLab by [API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter),
updates will be pulled immediately.
For more information, see [Start the pull mirroring process for a Project](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter).
## Forcing an update **(CORE)**
While mirrors are scheduled to update automatically, you can always force an update by using the
update button which is available on the **Mirroring repositories** section of the **Repository Settings** page.
![Repository mirroring force update user interface](img/repository_mirroring_force_update.png)
## Bidirectional mirroring **(STARTER)**
CAUTION: **Caution:**
Bidirectional mirroring may cause conflicts.
If you configure a GitLab repository to both pull from, and push to, the same remote source, there
is no guarantee that either repository will update correctly. If you set up a repository for
bidirectional mirroring, you should prepare for the likely conflicts by deciding who will resolve
them and how they will be resolved.
Rewriting any mirrored commit on either remote will cause conflicts and mirroring to fail. This can
be prevented by:
- [Pulling only protected branches](#only-mirror-protected-branches-starter).
- [Pushing only protected branches](#push-only-protected-branches-core).
You should [protect the branches](../protected_branches.md) you wish to mirror on both
remotes to prevent conflicts caused by rewriting history.
Bidirectional mirroring also creates a race condition where commits made close together to the same
branch causes conflicts. The race condition can be mitigated by reducing the mirroring delay by using
a [Push event webhook](../integrations/webhooks.md#push-events) to trigger an immediate
pull to GitLab. Push mirroring from GitLab is rate limited to once per minute when only push mirroring
protected branches.
### Preventing conflicts using a `pre-receive` hook
CAUTION: **Warning:**
The solution proposed will negatively impact the performance of
Git push operations because they will be proxied to the upstream Git
repository.
A server-side `pre-receive` hook can be used to prevent the race condition
described above by only accepting the push after first pushing the commit to
the upstream Git repository. In this configuration one Git repository acts as
the authoritative upstream, and the other as downstream. The `pre-receive` hook
will be installed on the downstream repository.
Read about [configuring custom Git hooks](../../../administration/custom_hooks.md) on the GitLab server.
A sample `pre-receive` hook is provided below.
```bash
#!/usr/bin/env bash
# --- Assume only one push mirror target
# Push mirroring remotes are named `remote_mirror_<id>`, this finds the first remote and uses that.
TARGET_REPO=$(git remote | grep -m 1 remote_mirror)
proxy_push()
{
# --- Arguments
OLDREV=$(git rev-parse $1)
NEWREV=$(git rev-parse $2)
REFNAME="$3"
# --- Pattern of branches to proxy pushes
whitelisted=$(expr "$branch" : "\(master\)")
case "$refname" in
refs/heads/*)
branch=$(expr "$refname" : "refs/heads/\(.*\)")
if [ "$whitelisted" = "$branch" ]; then
error="$(git push --quiet $TARGET_REPO $NEWREV:$REFNAME 2>&1)"
fail=$?
if [ "$fail" != "0" ]; then
echo >&2 ""
echo >&2 " Error: updates were rejected by upstream server"
echo >&2 " This is usually caused by another repository pushing changes"
echo >&2 " to the same ref. You may want to first integrate remote changes"
echo >&2 ""
return
fi
fi
;;
esac
}
# Allow dual mode: run from the command line just like the update hook, or
# if no arguments are given then run as a hook script
if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
# Output to the terminal in command line mode - if someone wanted to
# resend an email; they could redirect the output to sendmail
# themselves
PAGER= proxy_push $2 $3 $1
else
# Push is proxied upstream one ref at a time. Because of this it is possible
# for some refs to succeed, and others to fail. This will result in a failed
# push.
while read oldrev newrev refname
do
proxy_push $oldrev $newrev $refname
done
fi
```
### Mirroring with Perforce Helix via Git Fusion **(STARTER)**
CAUTION: **Warning:**
Bidirectional mirroring should not be used as a permanent configuration. Refer to
[Migrating from Perforce Helix](../import/perforce.md) for alternative migration approaches.
[Git Fusion](https://www.perforce.com/manuals/git-fusion/#Git-Fusion/section_avy_hyc_gl.html) provides a Git interface
to [Perforce Helix](https://www.perforce.com/products) which can be used by GitLab to bidirectionally
mirror projects with GitLab. This may be useful in some situations when migrating from Perforce Helix
to GitLab where overlapping Perforce Helix workspaces cannot be migrated simultaneously to GitLab.
If using mirroring with Perforce Helix, you should only mirror protected branches. Perforce Helix
will reject any pushes that rewrite history. Only the fewest number of branches should be mirrored
due to the performance limitations of Git Fusion.
When configuring mirroring with Perforce Helix via Git Fusion, the following Git Fusion
settings are recommended:
- `change-pusher` should be disabled. Otherwise, every commit will be rewritten as being committed
by the mirroring account, rather than being mapped to existing Perforce Helix users or the `unknown_git` user.
- `unknown_git` user will be used as the commit author if the GitLab user does not exist in
Perforce Helix.
Read about [Git Fusion settings on Perforce.com](https://www.perforce.com/manuals/git-fusion/Content/Git-Fusion/section_vss_bdw_w3.html#section_zdp_zz1_3l).
## Troubleshooting
Should an error occur during a push, GitLab will display an "Error" highlight for that repository. Details on the error can then be seen by hovering over the highlight text.
### 13:Received RST_STREAM with error code 2 with GitHub
If you receive an "13:Received RST_STREAM with error code 2" while mirroring to a GitHub repository, your GitHub settings might be set to block pushes that expose your email address used in commits. Either set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting.
doc/workflow/README.md
View file @ 529bc7e2
......@@ -9,7 +9,6 @@ comments: false
- [Cycle Analytics](../user/project/cycle_analytics.md)
- [Description templates](../user/project/description_templates.md)
- [Feature branch workflow](workflow.md)
- [GitLab Flow](gitlab_flow.md)
- [Groups](../user/group/index.md)
- Issues - The GitLab Issue Tracker is an advanced and complete tool for
tracking the evolution of a new idea or the process of solving a problem.
......@@ -18,13 +17,11 @@ comments: false
- [Due date for issues](../user/project/issues/due_dates.md)
- [Issue Board](../user/project/issue_board.md)
- [Keyboard shortcuts](shortcuts.md)
- [File finder](file_finder.md)
- [File finder](../user/project/repository/file_finder.md)
- [File lock](../user/project/file_lock.md) **(PREMIUM)**
- [Labels](../user/project/labels.md)
- [Issue weight](issue_weight.md) **(STARTER)**
- [Notification emails](notifications.md)
- [Projects](../user/project/index.md)
- [Project forking workflow](forking_workflow.md)
- [Project forking workflow](../user/project/repository/forking_workflow.md)
- [Project users](../user/project/members/index.md)
- [Protected branches](../user/project/protected_branches.md)
- [Protected tags](../user/project/protected_tags.md)
......@@ -32,7 +29,6 @@ comments: false
- [Sharing projects with groups](../user/project/members/share_project_with_groups.md)
- [Time tracking](time_tracking.md)
- [Web Editor](../user/project/repository/web_editor.md)
- [Releases](releases.md)
- [Milestones](../user/project/milestones/index.md)
- [Merge Requests](../user/project/merge_requests/index.md)
- [Authorization for merge requests](../user/project/merge_requests/authorization_for_merge_requests.md)
......@@ -45,9 +41,7 @@ comments: false
- ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md)
- [Fast-forward merge requests](../user/project/merge_requests/fast_forward_merge.md)
- [Merge request approvals](../user/project/merge_requests/merge_request_approvals.md) **(STARTER)**
- [Repository mirroring](repository_mirroring.md) **(STARTER)**
- [Service Desk](../user/project/service_desk.md) **(PREMIUM)**
- [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md)
- [Importing from SVN, GitHub, Bitbucket, etc](importing/README.md)
- [Todos](todos.md)
- [Snippets](../user/snippets.md)
......
doc/workflow/file_finder.md
View file @ 529bc7e2
# File finder
---
redirect_to: '../user/project/repository/file_finder.md'
---
> [Introduced][gh-9889] in GitLab 8.4.
The file finder feature allows you to quickly shortcut your way when you are
searching for a file in a repository using the GitLab UI.
You can find the **Find File** button when in the **Files** section of a
project.
![Find file button](img/file_finder_find_button.png)
For those who prefer to keep their fingers on the keyboard, there is a
[shortcut button](shortcuts.md) as well, which you can invoke from _anywhere_
in a project.
Press `t` to launch the File search function when in **Issues**,
**Merge requests**, **Milestones**, even the project's settings.
Start typing what you are searching for and watch the magic happen. With the
up/down arrows, you go up and down the results, with `Esc` you close the search
and go back to **Files**.
## How it works
The File finder feature is powered by the [Fuzzy filter](https://github.com/jeancroy/fuzz-aldrin-plus) library.
It implements a fuzzy search with highlight, and tries to provide intuitive
results by recognizing patterns that people use while searching.
For example, consider the [GitLab CE repository][ce] and that we want to open
the `app/controllers/admin/deploy_keys_controller.rb` file.
Using fuzzy search, we start by typing letters that get us closer to the file.
**Protip:** To narrow down your search, include `/` in your search terms.
![Find file button](img/file_finder_find_file.png)
[gh-9889]: https://github.com/gitlabhq/gitlabhq/pull/9889 "File finder pull request"
[ce]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master "GitLab CE repository"
This document was moved to [another location](../user/project/repository/file_finder.md).
doc/workflow/forking_workflow.md
View file @ 529bc7e2
# Project forking workflow
---
redirect_to: '../user/project/repository/forking_workflow.md'
---
Forking a project to your own namespace is useful if you have no write
access to the project you want to contribute to. If you do have write
access or can request it, we recommend working together in the same
repository since it is simpler. See our [GitLab Flow](gitlab_flow.md)
document more information about using branches to work together.
## Creating a fork
Forking a project is in most cases a two-step process.
1. Click on the fork button located located in between the star and clone buttons on the project's home page.
![Fork button](img/forking_workflow_fork_button.png)
1. Once you do that, you'll be presented with a screen where you can choose
the namespace to fork to. Only namespaces (groups and your own
namespace) where you have write access to, will be shown. Click on the
namespace to create your fork there.
![Choose namespace](img/forking_workflow_choose_namespace.png)
**Note:**
If the namespace you chose to fork the project to has another project with
the same path name, you will be presented with a warning that the forking
could not be completed. Try to resolve the error before repeating the forking
process.
![Path taken error](img/forking_workflow_path_taken_error.png)
After the forking is done, you can start working on the newly created
repository. There, you will have full [Owner](../user/permissions.md)
access, so you can set it up as you please.
## Merging upstream
Once you are ready to send your code back to the main project, you need
to create a merge request. Choose your forked project's main branch as
the source and the original project's main branch as the destination and
create the [merge request](merge_requests.md).
![Selecting branches](forking/branch_select.png)
You can then assign the merge request to someone to have them review
your changes. Upon pressing the 'Submit Merge Request' button, your
changes will be added to the repository and branch you're merging into.
![New merge request](forking/merge_request.png)
[gitlab flow]: https://about.gitlab.com/blog/2014/09/29/gitlab-flow/ "GitLab Flow blog post"
This document was moved to [another location](../user/project/repository/forking_workflow.md).
doc/workflow/git_annex.md
View file @ 529bc7e2
# Git annex
> **Warning:** GitLab has [completely
removed][deprecate-annex-issue] in GitLab 9.0 (2017/03/22).
Read through the [migration guide from git-annex to Git LFS][guide].
The biggest limitation of Git, compared to some older centralized version
control systems, has been the maximum size of the repositories.
The general recommendation is to not have Git repositories larger than 1GB to
preserve performance. Although GitLab has no limit (some repositories in GitLab
are over 50GB!), we subscribe to the advice to keep repositories as small as
you can.
Not being able to version control large binaries is a big problem for many
larger organizations.
Videos, photos, audio, compiled binaries and many other types of files are too
large. As a workaround, people keep artwork-in-progress in a Dropbox folder and
only check in the final result. This results in using outdated files, not
having a complete history and increases the risk of losing work.
This problem is solved in GitLab Enterprise Edition by integrating the
[git-annex] application.
`git-annex` allows managing large binaries with Git without checking the
contents into Git.
You check-in only a symlink that contains the SHA-1 of the large binary. If you
need the large binary, you can sync it from the GitLab server over `rsync`, a
very fast file copying tool.
## GitLab git-annex Configuration
`git-annex` is disabled by default in GitLab. Below you will find the
configuration options required to enable it.
### Requirements
`git-annex` needs to be installed both on the server and the client side.
For Debian-like systems (e.g., Debian, Ubuntu) this can be achieved by running:
```
sudo apt-get update && sudo apt-get install git-annex
```
For RedHat-like systems (e.g., CentOS, RHEL) this can be achieved by running:
```
sudo yum install epel-release && sudo yum install git-annex
```
### Configuration for Omnibus packages
For Omnibus GitLab packages, only one configuration setting is needed.
The Omnibus package will internally set the correct options in all locations.
1. In `/etc/gitlab/gitlab.rb` add the following line:
```ruby
gitlab_shell['git_annex_enabled'] = true
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
### Configuration for installations from source
There are 2 settings to enable git-annex on your GitLab server.
One is located in `config/gitlab.yml` of the GitLab repository and the other
one is located in `config.yml` of GitLab Shell.
1. In `config/gitlab.yml` add or edit the following lines:
```yaml
gitlab_shell:
git_annex_enabled: true
```
1. In `config.yml` of GitLab Shell add or edit the following lines:
```yaml
git_annex_enabled: true
```
1. Save the files and [restart GitLab][] for the changes to take effect.
## Using GitLab git-annex
> **Note:**
> Your Git remotes must be using the SSH protocol, not HTTP(S).
Here is an example workflow of uploading a very large file and then checking it
into your Git repository:
```bash
git clone git@example.com:group/project.git
git annex init 'My Laptop' # initialize the annex project and give an optional description
cp ~/tmp/debian.iso ./ # copy a large file into the current directory
git annex add debian.iso # add the large file to git annex
git commit -am "Add Debian iso" # commit the file metadata
git annex sync --content # sync the Git repo and large file to the GitLab server
```
The output should look like this:
```
commit
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)
nothing to commit, working tree clean
ok
pull origin
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From example.com:group/project
497842b..5162f80 git-annex -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
copy debian.iso (checking origin...) (to origin...)
SHA256E-s26214400--8092b3d482fb1b7a5cf28c43bc1425c8f2d380e86869c0686c49aa7b0f086ab2.iso
26,214,400 100% 638.88kB/s 0:00:40 (xfr#1, to-chk=0/1)
ok
pull origin
ok
(recording state in git...)
push origin
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (15/15), 1.64 KiB | 0 bytes/s, done.
Total 15 (delta 1), reused 0 (delta 0)
To example.com:group/project.git
* [new branch] git-annex -> synced/git-annex
* [new branch] master -> synced/master
ok
```
Your files can be found in the `master` branch, but you'll notice that there
are more branches created by the `annex sync` command.
Git Annex will also create a new directory at `.git/annex/` and will record the
tracked files in the `.git/config` file. The files you assign to be tracked
with `git-annex` will not affect the existing `.git/config` records. The files
are turned into symbolic links that point to data in `.git/annex/objects/`.
The `debian.iso` file in the example will contain the symbolic link:
```
.git/annex/objects/ZW/1k/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.png/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.iso
```
Use `git annex info` to retrieve the information about the local copy of your
repository.
---
redirect_to: '../administration/git_annex.md'
---
Downloading a single large file is also very simple:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync # sync Git branches but not the large file
git annex get debian.iso # download the large file
```
To download all files:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync --content # sync Git branches and download all the large files
```
By using `git-annex` without GitLab, anyone that can access the server can also
access the files of all projects, but GitLab Annex ensures that you can only
access files of projects you have access to (developer, maintainer, or owner role).
## How it works
Internally GitLab uses [GitLab Shell] to handle SSH access and this was a great
integration point for `git-annex`.
There is a setting in GitLab Shell so you can disable GitLab Annex support
if you want to.
## Troubleshooting tips
Differences in version of `git-annex` on the GitLab server and on local machines
can cause `git-annex` to raise unpredicted warnings and errors.
Consult the [Annex upgrade page][annex-upgrade] for more information about
the differences between versions. You can find out which version is installed
on your server by navigating to <https://pkgs.org/download/git-annex> and
searching for your distribution.
Although there is no general guide for `git-annex` errors, there are a few tips
on how to go around the warnings.
### `git-annex-shell: Not a git-annex or gcrypt repository`
This warning can appear on the initial `git annex sync --content` and is caused
by differences in `git-annex-shell`. You can read more about it
[in this git-annex issue][issue].
One important thing to note is that despite the warning, the `sync` succeeds
and the files are pushed to the GitLab repository.
If you get hit by this, you can run the following command inside the repository
that the warning was raised:
```
git config remote.origin.annex-ignore false
```
Consecutive runs of `git annex sync --content` **should not** produce this
warning and the output should look like this:
```
commit ok
pull origin
ok
pull origin
ok
push origin
```
[annex-upgrade]: https://git-annex.branchable.com/upgrades/
[deprecate-annex-issue]: https://gitlab.com/gitlab-org/gitlab/issues/1648
[git-annex]: https://git-annex.branchable.com/ "git-annex website"
[gitlab shell]: https://gitlab.com/gitlab-org/gitlab-shell "GitLab Shell repository"
[guide]: lfs/migrate_from_git_annex_to_git_lfs.html
[issue]: https://git-annex.branchable.com/forum/Error_from_git-annex-shell_on_creation_of_gcrypt_special_remote/ "git-annex issue"
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source
This document was moved to [another location](../administration/git_annex.md).
doc/workflow/git_lfs.md
View file @ 529bc7e2
---
redirect_to: 'lfs/manage_large_binaries_with_git_lfs.md'
redirect_to: '../administration/lfs/manage_large_binaries_with_git_lfs.md'
---
This document was moved to [another location](lfs/manage_large_binaries_with_git_lfs.md).
This document was moved to [another location](../administration/lfs/manage_large_binaries_with_git_lfs.md).
doc/workflow/gitlab_flow.md
View file @ 529bc7e2
# Introduction to GitLab Flow
---
redirect_to: '../topics/gitlab_flow.md'
---
![GitLab Flow](img/gitlab_flow.png)
Git allows a wide variety of branching strategies and workflows.
Because of this, many organizations end up with workflows that are too complicated, not clearly defined, or not integrated with issue tracking systems.
Therefore, we propose GitLab flow as a clearly defined set of best practices.
It combines [feature-driven development](https://en.wikipedia.org/wiki/Feature-driven_development) and [feature branches](https://martinfowler.com/bliki/FeatureBranch.html) with issue tracking.
Organizations coming to Git from other version control systems frequently find it hard to develop a productive workflow.
This article describes GitLab flow, which integrates the Git workflow with an issue tracking system.
It offers a simple, transparent, and effective way to work with Git.
![Four stages (working copy, index, local repo, remote repo) and three steps between them](img/four_stages.png)
When converting to Git, you have to get used to the fact that it takes three steps to share a commit with colleagues.
Most version control systems have only one step: committing from the working copy to a shared server.
In Git, you add files from the working copy to the staging area. After that, you commit them to your local repo.
The third step is pushing to a shared remote repository.
After getting used to these three steps, the next challenge is the branching model.
![Multiple long-running branches and merging in all directions](img/messy_flow.png)
Since many organizations new to Git have no conventions for how to work with it, their repositories can quickly become messy.
The biggest problem is that many long-running branches emerge that all contain part of the changes.
People have a hard time figuring out which branch has the latest code, or which branch to deploy to production.
Frequently, the reaction to this problem is to adopt a standardized pattern such as [Git flow](https://nvie.com/posts/a-successful-git-branching-model/) and [GitHub flow](http://scottchacon.com/2011/08/31/github-flow.html).
We think there is still room for improvement. In this document, we describe a set of practices we call GitLab flow.
For a video introduction of how this works in GitLab, see [GitLab Flow](https://youtu.be/InKNIvky2KE).
## Git flow and its problems
![Git Flow timeline by Vincent Driessen, used with permission](img/gitdashflow.png)
Git flow was one of the first proposals to use Git branches, and it has received a lot of attention.
It suggests a `master` branch and a separate `develop` branch, as well as supporting branches for features, releases, and hotfixes.
The development happens on the `develop` branch, moves to a release branch, and is finally merged into the `master` branch.
Git flow is a well-defined standard, but its complexity introduces two problems.
The first problem is that developers must use the `develop` branch and not `master`. `master` is reserved for code that is released to production.
It is a convention to call your default branch `master` and to mostly branch from and merge to this.
Since most tools automatically use the `master` branch as the default, it is annoying to have to switch to another branch.
The second problem of Git flow is the complexity introduced by the hotfix and release branches.
These branches can be a good idea for some organizations but are overkill for the vast majority of them.
Nowadays, most organizations practice continuous delivery, which means that your default branch can be deployed.
Continuous delivery removes the need for hotfix and release branches, including all the ceremony they introduce.
An example of this ceremony is the merging back of release branches.
Though specialized tools do exist to solve this, they require documentation and add complexity.
Frequently, developers make mistakes such as merging changes only into `master` and not into the `develop` branch.
The reason for these errors is that Git flow is too complicated for most use cases.
For example, many projects do releases but don't need to do hotfixes.
## GitHub flow as a simpler alternative
![Master branch with feature branches merged in](img/github_flow.png)
In reaction to Git flow, GitHub created a simpler alternative.
[GitHub flow](https://guides.github.com/introduction/flow/index.html) has only feature branches and a `master` branch.
This flow is clean and straightforward, and many organizations have adopted it with great success.
Atlassian recommends [a similar strategy](https://www.atlassian.com/blog/git/simple-git-workflow-is-simple), although they rebase feature branches.
Merging everything into the `master` branch and frequently deploying means you minimize the amount of unreleased code, which is in line with lean and continuous delivery best practices.
However, this flow still leaves a lot of questions unanswered regarding deployments, environments, releases, and integrations with issues.
With GitLab flow, we offer additional guidance for these questions.
## Production branch with GitLab flow
![Master branch and production branch with an arrow that indicates a deployment](img/production_branch.png)
GitHub flow assumes you can deploy to production every time you merge a feature branch.
While this is possible in some cases, such as SaaS applications, there are many cases where this is not possible.
One case is where you don't control the timing of a release, for example, an iOS application that is released when it passes App Store validation.
Another case is when you have deployment windows &mdash; for example, workdays from 10&nbsp;AM to 4&nbsp;PM when the operations team is at full capacity &mdash; but you also merge code at other times.
In these cases, you can make a production branch that reflects the deployed code.
You can deploy a new version by merging `master` into the production branch.
If you need to know what code is in production, you can just checkout the production branch to see.
The approximate time of deployment is easily visible as the merge commit in the version control system.
This time is pretty accurate if you automatically deploy your production branch.
If you need a more exact time, you can have your deployment script create a tag on each deployment.
This flow prevents the overhead of releasing, tagging, and merging that happens with Git flow.
## Environment branches with GitLab flow
![Multiple branches with the code cascading from one to another](img/environment_branches.png)
It might be a good idea to have an environment that is automatically updated to the `master` branch.
Only, in this case, the name of this environment might differ from the branch name.
Suppose you have a staging environment, a pre-production environment, and a production environment.
In this case, deploy the `master` branch to staging.
To deploy to pre-production, create a merge request from the `master` branch to the pre-production branch.
Go live by merging the pre-production branch into the production branch.
This workflow, where commits only flow downstream, ensures that everything is tested in all environments.
If you need to cherry-pick a commit with a hotfix, it is common to develop it on a feature branch and merge it into `master` with a merge request.
In this case, do not delete the feature branch yet.
If `master` passes automatic testing, you then merge the feature branch into the other branches.
If this is not possible because more manual testing is required, you can send merge requests from the feature branch to the downstream branches.
## Release branches with GitLab flow
![Master and multiple release branches that vary in length with cherry-picks from master](img/release_branches.png)
You only need to work with release branches if you need to release software to the outside world.
In this case, each branch contains a minor version, for example, 2-3-stable, 2-4-stable, etc.
Create stable branches using `master` as a starting point, and branch as late as possible.
By doing this, you minimize the length of time during which you have to apply bug fixes to multiple branches.
After announcing a release branch, only add serious bug fixes to the branch.
If possible, first merge these bug fixes into `master`, and then cherry-pick them into the release branch.
If you start by merging into the release branch, you might forget to cherry-pick them into `master`, and then you'd encounter the same bug in subsequent releases.
Merging into `master` and then cherry-picking into release is called an "upstream first" policy, which is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](https://www.redhat.com/en/blog/a-community-for-using-openstack-with-red-hat-rdo).
Every time you include a bug fix in a release branch, increase the patch version (to comply with [Semantic Versioning](https://semver.org/)) by setting a new tag.
Some projects also have a stable branch that points to the same commit as the latest released branch.
In this flow, it is not common to have a production branch (or Git flow `master` branch).
## Merge/pull requests with GitLab flow
![Merge request with inline comments](img/mr_inline_comments.png)
Merge or pull requests are created in a Git management application. They ask an assigned person to merge two branches.
Tools such as GitHub and Bitbucket choose the name "pull request" since the first manual action is to pull the feature branch.
Tools such as GitLab and others choose the name "merge request" since the final action is to merge the feature branch.
In this article, we'll refer to them as merge requests.
If you work on a feature branch for more than a few hours, it is good to share the intermediate result with the rest of the team.
To do this, create a merge request without assigning it to anyone.
Instead, mention people in the description or a comment, for example, "/cc @mark @susan."
This indicates that the merge request is not ready to be merged yet, but feedback is welcome.
Your team members can comment on the merge request in general or on specific lines with line comments.
The merge request serves as a code review tool, and no separate code review tools should be needed.
If the review reveals shortcomings, anyone can commit and push a fix.
Usually, the person to do this is the creator of the merge request.
The diff in the merge request automatically updates when new commits are pushed to the branch.
When you are ready for your feature branch to be merged, assign the merge request to the person who knows most about the codebase you are changing.
Also, mention any other people from whom you would like feedback.
After the assigned person feels comfortable with the result, they can merge the branch.
If the assigned person does not feel comfortable, they can request more changes or close the merge request without merging.
In GitLab, it is common to protect the long-lived branches, e.g., the `master` branch, so that [most developers can't modify them](../permissions/permissions.md).
So, if you want to merge into a protected branch, assign your merge request to someone with maintainer permissions.
After you merge a feature branch, you should remove it from the source control software.
In GitLab, you can do this when merging.
Removing finished branches ensures that the list of branches shows only work in progress.
It also ensures that if someone reopens the issue, they can use the same branch name without causing problems.
NOTE: **Note:**
When you reopen an issue you need to create a new merge request.
![Remove checkbox for branch in merge requests](img/remove_checkbox.png)
## Issue tracking with GitLab flow
![Merge request with the branch name "15-require-a-password-to-change-it" and assignee field shown](img/merge_request.png)
GitLab flow is a way to make the relation between the code and the issue tracker more transparent.
Any significant change to the code should start with an issue that describes the goal.
Having a reason for every code change helps to inform the rest of the team and to keep the scope of a feature branch small.
In GitLab, each change to the codebase starts with an issue in the issue tracking system.
If there is no issue yet, create the issue, as long as the change will take a significant amount of work, i.e., more than 1 hour.
In many organizations, raising an issue is part of the development process because they are used in sprint planning.
The issue title should describe the desired state of the system.
For example, the issue title "As an administrator, I want to remove users without receiving an error" is better than "Admin can't remove users."
When you are ready to code, create a branch for the issue from the `master` branch.
This branch is the place for any work related to this change.
NOTE: **Note:**
The name of a branch might be dictated by organizational standards.
When you are done or want to discuss the code, open a merge request.
A merge request is an online place to discuss the change and review the code.
If you open the merge request but do not assign it to anyone, it is a "Work In Progress" merge request.
These are used to discuss the proposed implementation but are not ready for inclusion in the `master` branch yet.
Start the title of the merge request with `[WIP]` or `WIP:` to prevent it from being merged before it's ready.
When you think the code is ready, assign the merge request to a reviewer.
The reviewer can merge the changes when they think the code is ready for inclusion in the `master` branch.
When they press the merge button, GitLab merges the code and creates a merge commit that makes this event easily visible later on.
Merge requests always create a merge commit, even when the branch could be merged without one.
This merge strategy is called "no fast-forward" in Git.
After the merge, delete the feature branch since it is no longer needed.
In GitLab, this deletion is an option when merging.
Suppose that a branch is merged but a problem occurs and the issue is reopened.
In this case, it is no problem to reuse the same branch name since the first branch was deleted when it was merged.
At any time, there is at most one branch for every issue.
It is possible that one feature branch solves more than one issue.
## Linking and closing issues from merge requests
![Merge request showing the linked issues that will be closed](img/close_issue_mr.png)
Link to issues by mentioning them in commit messages or the description of a merge request, for example, "Fixes #16" or "Duck typing is preferred. See #12."
GitLab then creates links to the mentioned issues and creates comments in the issues linking back to the merge request.
To automatically close linked issues, mention them with the words "fixes" or "closes," for example, "fixes #14" or "closes #67." GitLab closes these issues when the code is merged into the default branch.
If you have an issue that spans across multiple repositories, create an issue for each repository and link all issues to a parent issue.
## Squashing commits with rebase
![Vim screen showing the rebase view](img/rebase.png)
With Git, you can use an interactive rebase (`rebase -i`) to squash multiple commits into one or reorder them.
This functionality is useful if you want to replace a couple of small commits with a single commit, or if you want to make the order more logical.
However, you should never rebase commits you have pushed to a remote server.
Rebasing creates new commits for all your changes, which can cause confusion because the same change would have multiple identifiers.
It also causes merge errors for anyone working on the same branch because their history would not match with yours.
Also, if someone has already reviewed your code, rebasing makes it hard to tell what changed since the last review.
You should also never rebase commits authored by other people.
Not only does this rewrite history, but it also loses authorship information.
Rebasing prevents the other authors from being attributed and sharing part of the [`git blame`](https://git-scm.com/docs/git-blame).
If a merge involves many commits, it may seem more difficult to undo.
You might think to solve this by squashing all the changes into one commit before merging, but as discussed earlier, it is a bad idea to rebase commits that you have already pushed.
Fortunately, there is an easy way to undo a merge with all its commits.
The way to do this is by reverting the merge commit.
Preserving this ability to revert a merge is a good reason to always use the "no fast-forward" (`--no-ff`) strategy when you merge manually.
NOTE: **Note:**
If you revert a merge commit and then change your mind, revert the revert commit to redo the merge.
Git does not allow you to merge the code again otherwise.
## Reducing merge commits in feature branches
![List of sequential merge commits](img/merge_commits.png)
Having lots of merge commits can make your repository history messy.
Therefore, you should try to avoid merge commits in feature branches.
Often, people avoid merge commits by just using rebase to reorder their commits after the commits on the `master` branch.
Using rebase prevents a merge commit when merging `master` into your feature branch, and it creates a neat linear history.
However, as discussed in [the section about rebasing](#squashing-commits-with-rebase), you should never rebase commits you have pushed to a remote server.
This restriction makes it impossible to rebase work in progress that you already shared with your team, which is something we recommend.
Rebasing also creates more work, since every time you rebase, you have to resolve similar conflicts.
Sometimes you can reuse recorded resolutions (`rerere`), but merging is better since you only have to resolve conflicts once.
Atlassian has a more thorough explanation of the tradeoffs between merging and rebasing [on their blog](https://www.atlassian.com/blog/git/git-team-workflows-merge-or-rebase).
A good way to prevent creating many merge commits is to not frequently merge `master` into the feature branch.
There are three reasons to merge in `master`: utilizing new code, resolving merge conflicts, and updating long-running branches.
If you need to utilize some code that was introduced in `master` after you created the feature branch, you can often solve this by just cherry-picking a commit.
If your feature branch has a merge conflict, creating a merge commit is a standard way of solving this.
NOTE: **Note:**
Sometimes you can use .gitattributes to reduce merge conflicts.
For example, you can set your changelog file to use the [union merge driver](https://git-scm.com/docs/gitattributes#gitattributes-union) so that multiple new entries don't conflict with each other.
The last reason for creating merge commits is to keep long-running feature branches up-to-date with the latest state of the project.
The solution here is to keep your feature branches short-lived.
Most feature branches should take less than one day of work.
If your feature branches often take more than a day of work, try to split your features into smaller units of work.
If you need to keep a feature branch open for more than a day, there are a few strategies to keep it up-to-date.
One option is to use continuous integration (CI) to merge in `master` at the start of the day.
Another option is to only merge in from well-defined points in time, for example, a tagged release.
You could also use [feature toggles](https://martinfowler.com/bliki/FeatureToggle.html) to hide incomplete features so you can still merge back into `master` every day.
> **Note:** Don't confuse automatic branch testing with continuous integration.
> Martin Fowler makes this distinction in [his article about feature branches](https://martinfowler.com/bliki/FeatureBranch.html):
>
> "I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit.
> That's continuous building, and a Good Thing, but there's no *integration*, so it's not CI."
In conclusion, you should try to prevent merge commits, but not eliminate them.
Your codebase should be clean, but your history should represent what actually happened.
Developing software happens in small, messy steps, and it is OK to have your history reflect this.
You can use tools to view the network graphs of commits and understand the messy history that created your code.
If you rebase code, the history is incorrect, and there is no way for tools to remedy this because they can't deal with changing commit identifiers.
## Commit often and push frequently
Another way to make your development work easier is to commit often.
Every time you have a working set of tests and code, you should make a commit.
Splitting up work into individual commits provides context for developers looking at your code later.
Smaller commits make it clear how a feature was developed, and they make it easy to roll back to a specific good point in time or to revert one code change without reverting several unrelated changes.
Committing often also makes it easy to share your work, which is important so that everyone is aware of what you are working on.
You should push your feature branch frequently, even when it is not yet ready for review.
By sharing your work in a feature branch or [a merge request](#mergepull-requests-with-gitlab-flow), you prevent your team members from duplicating work.
Sharing your work before it's complete also allows for discussion and feedback about the changes, which can help improve the code before it gets to review.
## How to write a good commit message
![Good and bad commit message](img/good_commit.png)
A commit message should reflect your intention, not just the contents of the commit.
It is easy to see the changes in a commit, so the commit message should explain why you made those changes.
An example of a good commit message is: "Combine templates to reduce duplicate code in the user views."
The words "change," "improve," "fix," and "refactor" don't add much information to a commit message.
For example, "Improve XML generation" could be better written as "Properly escape special characters in XML generation."
For more information about formatting commit messages, please see this excellent [blog post by Tim Pope](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
## Testing before merging
![Merge requests showing the test states: red, yellow, and green](img/ci_mr.png)
In old workflows, the continuous integration (CI) server commonly ran tests on the `master` branch only.
Developers had to ensure their code did not break the `master` branch.
When using GitLab flow, developers create their branches from this `master` branch, so it is essential that it never breaks.
Therefore, each merge request must be tested before it is accepted.
CI software like Travis CI and GitLab CI show the build results right in the merge request itself to make this easy.
There is one drawback to testing merge requests: the CI server only tests the feature branch itself, not the merged result.
Ideally, the server could also test the `master` branch after each change.
However, retesting on every commit to `master` is computationally expensive and means you are more frequently waiting for test results.
Since feature branches should be short-lived, testing just the branch is an acceptable risk.
If new commits in `master` cause merge conflicts with the feature branch, merge `master` back into the branch to make the CI server re-run the tests.
As said before, if you often have feature branches that last for more than a few days, you should make your issues smaller.
## Working with feature branches
![Shell output showing git pull output](img/git_pull.png)
When creating a feature branch, always branch from an up-to-date `master`.
If you know before you start that your work depends on another branch, you can also branch from there.
If you need to merge in another branch after starting, explain the reason in the merge commit.
If you have not pushed your commits to a shared location yet, you can also incorporate changes by rebasing on `master` or another feature branch.
Do not merge from upstream again if your code can work and merge cleanly without doing so.
Merging only when needed prevents creating merge commits in your feature branch that later end up littering the `master` history.
This document was moved to [another location](../topics/gitlab_flow.md).
doc/workflow/issue_weight.md
View file @ 529bc7e2
# Issue weight **(STARTER)**
---
redirect_to: '../user/project/issues/issue_weight.md'
---
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/76) in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3.
When you have a lot of issues, it can be hard to get an overview.
By adding a weight to each issue, you can get a better idea of how much time,
value or complexity a given issue has or will cost.
You can set the weight of an issue during its creation, by simply changing the
value in the dropdown menu. You can set it to a non-negative integer
value from 0, 1, 2, and so on. (The database stores a 4-byte value, so the
upper bound is essentially limitless).
You can remove weight from an issue
as well.
This value will appear on the right sidebar of an individual issue, as well as
in the issues page next to a distinctive balance scale icon.
As an added bonus, you can see the total sum of all issues on the milestone page.
![issue page](issue_weight/issue.png)
This document was moved to [another location](../user/project/issues/issue_weight.md).
doc/workflow/lfs/lfs_administration.md
View file @ 529bc7e2
# GitLab Git LFS Administration
---
redirect_to: '../../administration/lfs/lfs_administration.md'
---
Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md).
## Requirements
- Git LFS is supported in GitLab starting with version 8.2.
- Support for object storage, such as AWS S3, was introduced in 10.0.
- Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up.
## Configuration
Git LFS objects can be large in size. By default, they are stored on the server
GitLab is installed on.
There are various configuration options to help GitLab server administrators:
- Enabling/disabling Git LFS support
- Changing the location of LFS object storage
- Setting up object storage supported by [Fog](http://fog.io/about/provider_documentation.html)
### Configuration for Omnibus installations
In `/etc/gitlab/gitlab.rb`:
```ruby
# Change to true to enable lfs - enabled by default if not defined
gitlab_rails['lfs_enabled'] = false
# Optionally, change the storage path location. Defaults to
# `#{gitlab_rails['shared_path']}/lfs-objects`. Which evaluates to
# `/var/opt/gitlab/gitlab-rails/shared/lfs-objects` by default.
gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects"
```
### Configuration for installations from source
In `config/gitlab.yml`:
```yaml
# Change to true to enable lfs
lfs:
enabled: false
storage_path: /mnt/storage/lfs-objects
```
## Storing LFS objects in remote object storage
> [Introduced][ee-2760] in [GitLab Premium][eep] 10.0. Brought to GitLab Core in 10.7.
It is possible to store LFS objects in remote object storage which allows you
to offload local hard disk R/W operations, and free up disk space significantly.
GitLab is tightly integrated with `Fog`, so you can refer to its [documentation](http://fog.io/about/provider_documentation.html)
to check which storage services can be integrated with GitLab.
You can also use external object storage in a private local network. For example,
[MinIO](https://min.io/) is a standalone object storage service, is easy to set up, and works well with GitLab instances.
GitLab provides two different options for the uploading mechanism: "Direct upload" and "Background upload".
**Option 1. Direct upload**
1. User pushes an lfs file to the GitLab instance
1. GitLab-workhorse uploads the file directly to the external object storage
1. GitLab-workhorse notifies GitLab-rails that the upload process is complete
**Option 2. Background upload**
1. User pushes an lfs file to the GitLab instance
1. GitLab-rails stores the file in the local file storage
1. GitLab-rails then uploads the file to the external object storage asynchronously
The following general settings are supported.
| Setting | Description | Default |
|---------|-------------|---------|
| `enabled` | Enable/disable object storage | `false` |
| `remote_directory` | The bucket name where LFS objects will be stored| |
| `direct_upload` | Set to true to enable direct upload of LFS without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` |
| `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` |
| `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` |
| `connection` | Various connection options described below | |
The `connection` settings match those provided by [Fog](https://github.com/fog).
Here is a configuration example with S3.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | AWS |
| `aws_access_key_id` | AWS credentials, or compatible | `ABC123DEF456` |
| `aws_secret_access_key` | AWS credentials, or compatible | `ABC123DEF456ABC123DEF456ABC123DEF456` |
| `aws_signature_version` | AWS signature version to use. 2 or 4 are valid options. Digital Ocean Spaces and other providers may need 2. | 4 |
| `enable_signature_v4_streaming` | Set to true to enable HTTP chunked transfers with [AWS v4 signatures](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html). Oracle Cloud S3 needs this to be false | true |
| `region` | AWS region | us-east-1 |
| `host` | S3 compatible host for when not using AWS, e.g. `localhost` or `storage.example.com` | s3.amazonaws.com |
| `endpoint` | Can be used when configuring an S3 compatible service such as [MinIO](https://min.io), by entering a URL such as `http://127.0.0.1:9000` | (optional) |
| `path_style` | Set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Leave as false for AWS S3 | false |
| `use_iam_profile` | Set to true to use IAM profile instead of access keys | false
Here is a configuration example with GCS.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | `Google` |
| `google_project` | GCP project name | `gcp-project-12345` |
| `google_client_email` | The email address of the service account | `foo@gcp-project-12345.iam.gserviceaccount.com` |
| `google_json_key_location` | The json key path | `/path/to/gcp-project-12345-abcde.json` |
NOTE: **Note:**
The service account must have permission to access the bucket.
[See more](https://cloud.google.com/storage/docs/authentication)
Here is a configuration example with Rackspace Cloud Files.
| Setting | Description | example |
|---------|-------------|---------|
| `provider` | The provider name | `Rackspace` |
| `rackspace_username` | The username of the Rackspace account with access to the container | `joe.smith` |
| `rackspace_api_key` | The API key of the Rackspace account with access to the container | `ABC123DEF456ABC123DEF456ABC123DE` |
| `rackspace_region` | The Rackspace storage region to use, a three letter code from the [list of service access endpoints](https://developer.rackspace.com/docs/cloud-files/v1/general-api-info/service-access/) | `iad` |
| `rackspace_temp_url_key` | The private key you have set in the Rackspace API for temporary URLs. Read more [here](https://developer.rackspace.com/docs/cloud-files/v1/use-cases/public-access-to-your-cloud-files-account/#tempurl) | `ABC123DEF456ABC123DEF456ABC123DE` |
NOTE: **Note:**
Regardless of whether the container has public access enabled or disabled, Fog will
use the TempURL method to grant access to LFS objects. If you see errors in logs referencing
instantiating storage with a temp-url-key, ensure that you have set they key properly
on the Rackspace API and in `gitlab.rb`. You can verify the value of the key Rackspace
has set by sending a GET request with token header to the service access endpoint URL
and comparing the output of the returned headers.
### Manual uploading to an object storage
There are two ways to manually do the same thing as automatic uploading (described above).
**Option 1: rake task**
```sh
rake gitlab:lfs:migrate
```
**Option 2: rails console**
```sh
$ sudo gitlab-rails console # Login to rails console
> # Upload LFS files manually
> LfsObject.where(file_store: [nil, 1]).find_each do |lfs_object|
> lfs_object.file.migrate!(ObjectStorage::Store::REMOTE) if lfs_object.file.file.exists?
> end
```
### S3 for Omnibus installations
On Omnibus installations, the settings are prefixed by `lfs_object_store_`:
1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with
the values you want:
```ruby
gitlab_rails['lfs_object_store_enabled'] = true
gitlab_rails['lfs_object_store_remote_directory'] = "lfs-objects"
gitlab_rails['lfs_object_store_connection'] = {
'provider' => 'AWS',
'region' => 'eu-central-1',
'aws_access_key_id' => '1ABCD2EFGHI34JKLM567N',
'aws_secret_access_key' => 'abcdefhijklmnopQRSTUVwxyz0123456789ABCDE',
# The below options configure an S3 compatible host instead of AWS
'host' => 'localhost',
'endpoint' => 'http://127.0.0.1:9000',
'path_style' => true
}
```
1. Save the file and [reconfigure GitLab]s for the changes to take effect.
1. Migrate any existing local LFS objects to the object storage:
```bash
gitlab-rake gitlab:lfs:migrate
```
This will migrate existing LFS objects to object storage. New LFS objects
will be forwarded to object storage unless
`gitlab_rails['lfs_object_store_background_upload']` is set to false.
### S3 for installations from source
For source installations the settings are nested under `lfs:` and then
`object_store:`:
1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following
lines:
```yaml
lfs:
enabled: true
object_store:
enabled: false
remote_directory: lfs-objects # Bucket name
connection:
provider: AWS
aws_access_key_id: 1ABCD2EFGHI34JKLM567N
aws_secret_access_key: abcdefhijklmnopQRSTUVwxyz0123456789ABCDE
region: eu-central-1
# Use the following options to configure an AWS compatible host such as Minio
host: 'localhost'
endpoint: 'http://127.0.0.1:9000'
path_style: true
```
1. Save the file and [restart GitLab][] for the changes to take effect.
1. Migrate any existing local LFS objects to the object storage:
```bash
sudo -u git -H bundle exec rake gitlab:lfs:migrate RAILS_ENV=production
```
This will migrate existing LFS objects to object storage. New LFS objects
will be forwarded to object storage unless `background_upload` is set to
false.
### Migrating back to local storage
In order to migrate back to local storage:
1. Set both `direct_upload` and `background_upload` to false under the LFS object storage settings. Don't forget to restart GitLab.
1. Run `rake gitlab:lfs:migrate_to_local` on your console.
1. Disable `object_storage` for LFS objects in `gitlab.rb`. Remember to restart GitLab afterwards.
## Storage statistics
You can see the total storage used for LFS objects on groups and projects
in the administration area, as well as through the [groups](../../api/groups.md)
and [projects APIs](../../api/projects.md).
## Troubleshooting: `Google::Apis::TransmissionError: execution expired`
If LFS integration is configred with Google Cloud Storage and background uploads (`background_upload: true` and `direct_upload: false`),
Sidekiq workers may encouter this error. This is because the uploading timed out with very large files.
LFS files up to 6Gb can be uploaded without any extra steps, otherwise you need to use the following workaround.
```shell
$ sudo gitlab-rails console # Login to rails console
> # Set up timeouts. 20 minutes is enough to upload 30GB LFS files.
> # These settings are only in effect for the same session, i.e. they are not effective for sidekiq workers.
> ::Google::Apis::ClientOptions.default.open_timeout_sec = 1200
> ::Google::Apis::ClientOptions.default.read_timeout_sec = 1200
> ::Google::Apis::ClientOptions.default.send_timeout_sec = 1200
> # Upload LFS files manually. This process does not use sidekiq at all.
> LfsObject.where(file_store: [nil, 1]).find_each do |lfs_object|
> lfs_object.file.migrate!(ObjectStorage::Store::REMOTE) if lfs_object.file.file.exists?
> end
```
See more information in [!19581](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/19581)
## Known limitations
- Support for removing unreferenced LFS objects was added in 8.14 onwards.
- LFS authentications via SSH was added with GitLab 8.12.
- Only compatible with the Git LFS client versions 1.1.0 and up, or 1.0.2.
- The storage statistics currently count each LFS object multiple times for
every project linking to it.
[reconfigure gitlab]: ../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab"
[restart gitlab]: ../../administration/restart_gitlab.md#installations-from-source "How to restart GitLab"
[eep]: https://about.gitlab.com/pricing/ "GitLab Premium"
[ee-2760]: https://gitlab.com/gitlab-org/gitlab/merge_requests/2760
This document was moved to [another location](../../administration/lfs/lfs_administration.md).
doc/workflow/lfs/manage_large_binaries_with_git_lfs.md
View file @ 529bc7e2
# Git LFS
---
redirect_to: '../../administration/lfs/manage_large_binaries_with_git_lfs.md'
---
Managing large files such as audio, video and graphics files has always been one
of the shortcomings of Git. The general recommendation is to not have Git repositories
larger than 1GB to preserve performance.
![Git LFS tracking status](img/lfs-icon.png)
An LFS icon is shown on files tracked by Git LFS to denote if a file is stored
as a blob or as an LFS pointer.
## How it works
Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication
to authorize client requests. Once the request is authorized, Git LFS client receives
instructions from where to fetch or where to push the large file.
## GitLab server configuration
Documentation for GitLab instance administrators is under [LFS administration doc](lfs_administration.md).
## Requirements
- Git LFS is supported in GitLab starting with version 8.2
- Git LFS must be enabled under project settings
- [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up
## Known limitations
- Git LFS v1 original API is not supported since it was deprecated early in LFS
development
- When SSH is set as a remote, Git LFS objects still go through HTTPS
- Any Git LFS request will ask for HTTPS credentials to be provided so a good Git
credentials store is recommended
- Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have
to add the URL to Git config manually (see [troubleshooting](#troubleshooting))
NOTE: **Note:**
With 8.12 GitLab added LFS support to SSH. The Git LFS communication
still goes over HTTP, but now the SSH client passes the correct credentials
to the Git LFS client, so no action is required by the user.
## Using Git LFS
Lets take a look at the workflow when you need to check large files into your Git
repository with Git LFS. For example, if you want to upload a very large file and
check it into your Git repository:
```bash
git clone git@gitlab.example.com:group/project.git
git lfs install # initialize the Git LFS project
git lfs track "*.iso" # select the file extensions that you want to treat as large files
```
Once a certain file extension is marked for tracking as a LFS object you can use
Git as usual without having to redo the command to track a file with the same extension:
```bash
cp ~/tmp/debian.iso ./ # copy a large file into the current directory
git add . # add the large file to the project
git commit -am "Added Debian iso" # commit the file meta data
git push origin master # sync the git repo and large file to the GitLab server
```
**Make sure** that `.gitattributes` is tracked by Git. Otherwise Git
LFS will not be working properly for people cloning the project:
```bash
git add .gitattributes
```
Cloning the repository works the same as before. Git automatically detects the
LFS-tracked files and clones them via HTTP. If you performed the `git clone`
command with a SSH URL, you have to enter your GitLab credentials for HTTP
authentication.
```bash
git clone git@gitlab.example.com:group/project.git
```
If you already cloned the repository and you want to get the latest LFS object
that are on the remote repository, eg. for a branch from origin:
```bash
git lfs fetch origin master
```
### Migrate an existing repo to Git LFS
Read the documentation on how to [migrate an existing Git repo with Git LFS](../../topics/git/migrate_to_git_lfs/index.md).
## File Locking
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/35856) in GitLab 10.5.
The first thing to do before using File Locking is to tell Git LFS which
kind of files are lockable. The following command will store PNG files
in LFS and flag them as lockable:
```bash
git lfs track "*.png" --lockable
```
After executing the above command a file named `.gitattributes` will be
created or updated with the following content:
```bash
*.png filter=lfs diff=lfs merge=lfs -text lockable
```
You can also register a file type as lockable without using LFS
(In order to be able to lock/unlock a file you need a remote server that implements the LFS File Locking API),
in order to do that you can edit the `.gitattributes` file manually:
```bash
*.pdf lockable
```
After a file type has been registered as lockable, Git LFS will make
them readonly on the file system automatically. This means you will
need to lock the file before editing it.
### Managing Locked Files
Once you're ready to edit your file you need to lock it first:
```bash
git lfs lock images/banner.png
Locked images/banner.png
```
This will register the file as locked in your name on the server:
```bash
git lfs locks
images/banner.png joe ID:123
```
Once you have pushed your changes, you can unlock the file so others can
also edit it:
```bash
git lfs unlock images/banner.png
```
You can also unlock by id:
```bash
git lfs unlock --id=123
```
If for some reason you need to unlock a file that was not locked by you,
you can use the `--force` flag as long as you have a `maintainer` access on
the project:
```bash
git lfs unlock --id=123 --force
```
## Troubleshooting
### error: Repository or object not found
There are a couple of reasons why this error can occur:
- You don't have permissions to access certain LFS object
Check if you have permissions to push to the project or fetch from the project.
- Project is not allowed to access the LFS object
LFS object you are trying to push to the project or fetch from the project is not
available to the project anymore. Probably the object was removed from the server.
- Local Git repository is using deprecated LFS API
### Invalid status for `<url>` : 501
Git LFS will log the failures into a log file.
To view this log file, while in project directory:
```bash
git lfs logs last
```
If the status `error 501` is shown, it is because:
- Git LFS is not enabled in project settings. Check your project settings and
enable Git LFS.
- Git LFS support is not enabled on the GitLab server. Check with your GitLab
administrator why Git LFS is not enabled on the server. See
[LFS administration documentation](lfs_administration.md) for instructions
on how to enable LFS support.
- Git LFS client version is not supported by GitLab server. Check your Git LFS
version with `git lfs version`. Check the Git config of the project for traces
of deprecated API with `git lfs -l`. If `batch = false` is set in the config,
remove the line and try to update your Git LFS client. Only version 1.0.1 and
newer are supported.
### getsockopt: connection refused
If you push a LFS object to a project and you receive an error similar to:
`Post <URL>/info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`,
the LFS client is trying to reach GitLab through HTTPS. However, your GitLab
instance is being served on HTTP.
This behaviour is caused by Git LFS using HTTPS connections by default when a
`lfsurl` is not set in the Git config.
To prevent this from happening, set the lfs url in project Git config:
```bash
git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs"
```
### Credentials are always required when pushing an object
NOTE: **Note:**
With 8.12 GitLab added LFS support to SSH. The Git LFS communication
still goes over HTTP, but now the SSH client passes the correct credentials
to the Git LFS client, so no action is required by the user.
Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing
the LFS object on every push for every object, user HTTPS credentials are required.
By default, Git has support for remembering the credentials for each repository
you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials).
For example, you can tell Git to remember the password for a period of time in
which you expect to push the objects:
```bash
git config --global credential.helper 'cache --timeout=3600'
```
This will remember the credentials for an hour after which Git operations will
require re-authentication.
If you are using OS X you can use `osxkeychain` to store and encrypt your credentials.
For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases).
More details about various methods of storing the user credentials can be found
on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage).
### LFS objects are missing on push
GitLab checks files to detect LFS pointers on push. If LFS pointers are detected, GitLab tries to verify that those files already exist in LFS on GitLab.
Verify that LFS in installed locally and consider a manual push with `git lfs push --all`.
If you are storing LFS files outside of GitLab you can disable LFS on the project by setting `lfs_enabled: false` with the [projects API](../../api/projects.md#edit-project).
### Hosting LFS objects externally
It is possible to host LFS objects externally by setting a custom LFS url with `git config -f .lfsconfig lfs.url https://example.com/<project>.git/info/lfs`.
You might choose to do this if you are using an appliance like a Sonatype Nexus to store LFS data. If you choose to use an external LFS store,
GitLab will not be able to verify LFS objects which means that pushes will fail if you have GitLab LFS support enabled.
To stop push failure, LFS support can be disabled in the [Project settings](../../user/project/settings/index.md). This means you will lose GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS).
This document was moved to [another location](../../administration/lfs/manage_large_binaries_with_git_lfs.md).
doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md
View file @ 529bc7e2
# Migration guide from Git Annex to Git LFS
>**Note:**
Git Annex support [has been removed][issue-remove-annex] in GitLab Enterprise
Edition 9.0 (2017/03/22).
Both [Git Annex][] and [Git LFS][] are tools to manage large files in Git.
## History
Git Annex [was introduced in GitLab Enterprise Edition 7.8][post-3], at a time
where Git LFS didn't yet exist. A few months later, GitLab brought support for
Git LFS in [GitLab 8.2][post-2] and is available for both Community and
Enterprise editions.
## Differences between Git Annex and Git LFS
Some items below are general differences between the two protocols and some are
ones that GitLab developed.
- Git Annex works only through SSH, whereas Git LFS works both with SSH and HTTPS
(SSH support was added in GitLab 8.12).
- Annex files are stored in a sub-directory of the normal repositories, whereas
LFS files are stored outside of the repositories in a place you can define.
- Git Annex requires a more complex setup, but has much more options than Git
LFS. You can compare the commands each one offers by running `man git-annex`
and `man git-lfs`.
- Annex files cannot be browsed directly in GitLab's interface, whereas LFS
files can.
## Migration steps
>**Note:**
Since Git Annex files are stored in a sub-directory of the normal repositories
(`.git/annex/objects`) and LFS files are stored outside of the repositories,
they are not compatible as they are using a different scheme. Therefore, the
migration has to be done manually per repository.
There are basically two steps you need to take in order to migrate from Git
Annex to Git LFS.
### TL; DR
If you know what you are doing and want to skip the reading, this is what you
need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab-git-annex) in your
repository and that you have made backups in case something goes wrong).
Fire up a terminal, navigate to your Git repository and:
1. Disable `git-annex`:
```bash
git annex sync --content
git annex direct
git annex uninit
git annex indirect
```
1. Enable `git-lfs`:
```
git lfs install
git lfs track <files>
git add .
git commit -m "commit message"
git push
```
### Disabling Git Annex in your repo
Before changing anything, make sure you have a backup of your repository first.
There are a couple of ways to do that, but you can simply clone it to another
local path and maybe push it to GitLab if you want a remote backup as well.
Here you'll find a guide on
[how to back up a **git-annex** repository to an external hard drive][bkp-ext-drive].
Since Annex files are stored as objects with symlinks and cannot be directly
modified, we need to first remove those symlinks.
NOTE: **Note:**
Make sure the you read about the [`direct` mode][annex-direct] as it contains
useful information that may fit in your use case. Note that `annex direct` is
deprecated in Git Annex version 6, so you may need to upgrade your repository
if the server also has Git Annex 6 installed. Read more in the
[Git Annex troubleshooting tips](../git_annex.md#troubleshooting-tips) section.
1. Backup your repository
```bash
cd repository
git annex sync --content
cd ..
git clone repository repository-backup
cd repository-backup
git annex get
cd ..
```
1. Use `annex direct`:
```bash
cd repository
git annex direct
```
The output should be similar to this:
```bash
commit
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
ok
direct debian.iso ok
direct ok
```
1. Disable Git Annex with [`annex uninit`][uninit]:
```bash
git annex uninit
```
The output should be similar to this:
```bash
unannex debian.iso ok
Deleted branch git-annex (was 2534d2c).
```
This will `unannex` every file in the repository, leaving the original files.
1. Switch back to `indirect` mode:
```bash
git annex indirect
```
The output should be similar to this:
```bash
(merging origin/git-annex into git-annex...)
(recording state in git...)
commit (recording state in git...)
ok
(recording state in git...)
[master fac3194] commit before switching to indirect mode
1 file changed, 1 deletion(-)
delete mode 120000 alpine-virt-3.4.4-x86_64.iso
ok
indirect ok
ok
```
---
redirect_to: '../../administration/lfs/migrate_from_git_annex_to_git_lfs.md'
---
At this point, you have two options. Either add, commit and push the files
directly back to GitLab or switch to Git LFS. We will tackle the LFS switch in
the next section.
### Enabling Git LFS in your repo
Git LFS is enabled by default on all GitLab products (GitLab CE, GitLab EE,
GitLab.com), therefore, you don't need to do anything server-side.
1. First, make sure you have `git-lfs` installed locally:
```bash
git lfs help
```
If the terminal doesn't prompt you with a full response on `git-lfs` commands,
[install the Git LFS client][install-lfs] first.
1. Inside the repo, run the following command to initiate LFS:
```bash
git lfs install
```
1. Enable `git-lfs` for the group of files you want to track. You
can track specific files, all files containing the same extension, or an
entire directory:
```bash
git lfs track images/01.png # per file
git lfs track **/*.png # per extension
git lfs track images/ # per directory
```
Once you do that, run `git status` and you'll see `.gitattributes` added
to your repo. It collects all file patterns that you chose to track via
`git-lfs`.
1. Add the files, commit and push them to GitLab:
```bash
git add .
git commit -m "commit message"
git push
```
If your remote is set up with HTTP, you will be asked to enter your login
credentials. If you have [2FA enabled](../../user/profile/account/two_factor_authentication.md), make sure to use a
[personal access token](../../user/profile/account/two_factor_authentication.md#personal-access-tokens)
instead of your password.
## Removing the Git Annex branches
After the migration finishes successfully, you can remove all `git-annex`
related branches from your repository.
On GitLab, navigate to your project's **Repository ➔ Branches** and delete all
branches created by Git Annex: `git-annex`, and all under `synced/`.
![repository branches](images/git-annex-branches.png)
You can also do this on the command line with:
```bash
git branch -d synced/master
git branch -d synced/git-annex
git push origin :synced/master
git push origin :synced/git-annex
git push origin :git-annex
git remote prune origin
```
If there are still some Annex objects inside your repository (`.git/annex/`)
or references inside `.git/config`, run `annex uninit` again:
```bash
git annex uninit
```
## Further Reading
- (Blog Post) [Getting Started with Git FLS][post-1]
- (Blog Post) [Announcing LFS Support in GitLab][post-2]
- (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3]
- (GitLab Docs) [Git Annex](../git_annex.md)
- (GitLab Docs) [Git LFS](manage_large_binaries_with_git_lfs.md)
[annex-direct]: https://git-annex.branchable.com/direct_mode/
[bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive
[Git Annex]: http://git-annex.branchable.com/
[Git LFS]: https://git-lfs.github.com/
[install-lfs]: https://git-lfs.github.com/
[issue-remove-annex]: https://gitlab.com/gitlab-org/gitlab/issues/1648
[lfs-track]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/#tracking-files-with-lfs
[post-1]: https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/
[post-2]: https://about.gitlab.com/blog/2015/11/23/announcing-git-lfs-support-in-gitlab/
[post-3]: https://about.gitlab.com/blog/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/
[uninit]: https://git-annex.branchable.com/git-annex-uninit/
This document was moved to [another location](../../administration/lfs/migrate_from_git_annex_to_git_lfs.md).
doc/workflow/notifications.md
View file @ 529bc7e2
# GitLab Notification Emails
---
redirect_to: '../user/profile/notifications.md'
---
GitLab has a notification system in place to notify a user of events that are important for the workflow.
## Notification settings
You can find notification settings under the user profile.
![notification settings](img/notification_global_settings.png)
Notification settings are divided into three groups:
- Global settings
- Group settings
- Project settings
Each of these settings have levels of notification:
- Global: For groups and projects, notifications as per global settings.
- Watch: Receive notifications for any activity.
- Participate: Receive notifications for threads you have participated in.
- On Mention: Receive notifications when `@mentioned` in comments.
- Disabled: Turns off notifications.
- Custom: Receive notifications for custom selected events.
> Introduced in GitLab 12.0
You can also select an email address to receive notifications for each group you belong to.
### Global Settings
Global settings are at the bottom of the hierarchy.
Any setting set here will be overridden by a setting at the group or a project level.
Group or Project settings can use `global` notification setting which will then use
anything that is set at Global Settings.
### Group Settings
![notification settings](img/notification_group_settings.png)
Group settings are taking precedence over Global Settings but are on a level below Project or Subgroup settings:
```
Group < Subgroup < Project
```
This means that you can set a different level of notifications per group while still being able
to have a finer level setting per project or subgroup.
Organization like this is suitable for users that belong to different groups but don't have the
same need for being notified for every group they are member of.
These settings can be configured on group page under the name of the group. It will be the dropdown with the bell icon. They can also be configured on the user profile notifications dropdown.
The group owner can disable email notifications for a group, which includes
its subgroups and projects. If this is the case, you will not receive any corresponding notifications,
and the notification button will be disabled with an explanatory tooltip.
### Project Settings
![notification settings](img/notification_project_settings.png)
Project settings are at the top level and any setting placed at this level will take precedence of any
other setting.
This is suitable for users that have different needs for notifications per project basis.
These settings can be configured on project page under the name of the project. It will be the dropdown with the bell icon. They can also be configured on the user profile notifications dropdown.
The project owner (or its group owner) can disable email notifications for the project.
If this is the case, you will not receive any corresponding notifications, and the notification
button will be disabled with an explanatory tooltip.
## Notification events
Below is the table of events users can be notified of:
| Event | Sent to | Settings level |
|------------------------------|---------------------|------------------------------|
| New SSH key added | User | Security email, always sent. |
| New email added | User | Security email, always sent. |
| Email changed | User | Security email, always sent. |
| Password changed | User | Security email, always sent. |
| New user created | User | Sent on user creation, except for OmniAuth (LDAP)|
| User added to project | User | Sent when user is added to project |
| Project access level changed | User | Sent when user project access level is changed |
| User added to group | User | Sent when user is added to group |
| Group access level changed | User | Sent when user group access level is changed |
| Project moved | Project members (1) | (1) not disabled |
| New release | Project members | Custom notification |
### Issue / Epics / Merge request events
In most of the below cases, the notification will be sent to:
- Participants:
- the author and assignee of the issue/merge request
- authors of comments on the issue/merge request
- anyone mentioned by `@username` in the title or description of the issue, merge request or epic **(ULTIMATE)**
- anyone with notification level "Participating" or higher that is mentioned by `@username`
in any of the comments on the issue, merge request, or epic **(ULTIMATE)**
- Watchers: users with notification level "Watch"
- Subscribers: anyone who manually subscribed to the issue, merge request, or epic **(ULTIMATE)**
- Custom: Users with notification level "custom" who turned on notifications for any of the events present in the table below
| Event | Sent to |
|------------------------|---------|
| New issue | |
| Close issue | |
| Reassign issue | The above, plus the old assignee |
| Reopen issue | |
| Due issue | Participants and Custom notification level with this event selected |
| Change milestone issue | Subscribers, participants mentioned, and Custom notification level with this event selected |
| Remove milestone issue | Subscribers, participants mentioned, and Custom notification level with this event selected |
| New merge request | |
| Push to merge request | Participants and Custom notification level with this event selected |
| Reassign merge request | The above, plus the old assignee |
| Close merge request | |
| Reopen merge request | |
| Merge merge request | |
| Change milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected |
| Remove milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected |
| New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher |
| Failed pipeline | The author of the pipeline |
| Successful pipeline | The author of the pipeline, if they have the custom notification setting for successful pipelines set |
| New epic **(ULTIMATE)** | |
| Close epic **(ULTIMATE)** | |
| Reopen epic **(ULTIMATE)** | |
In addition, if the title or description of an Issue or Merge Request is
changed, notifications will be sent to any **new** mentions by `@username` as
if they had been mentioned in the original text.
You won't receive notifications for Issues, Merge Requests or Milestones created
by yourself (except when an issue is due). You will only receive automatic
notifications when somebody else comments or adds changes to the ones that
you've created or mentions you.
If an open merge request becomes unmergeable due to conflict, its author will be notified about the cause.
If a user has also set the merge request to automatically merge once pipeline succeeds,
then that user will also be notified.
### Email Headers
Notification emails include headers that provide extra content about the notification received:
| Header | Description |
|-----------------------------|-------------------------------------------------------------------------|
| X-GitLab-Project | The name of the project the notification belongs to |
| X-GitLab-Project-Id | The ID of the project |
| X-GitLab-Project-Path | The path of the project |
| X-GitLab-(Resource)-ID | The ID of the resource the notification is for, where resource is `Issue`, `MergeRequest`, `Commit`, etc|
| X-GitLab-Discussion-ID | Only in comment emails, the ID of the thread the comment is from |
| X-GitLab-Pipeline-Id | Only in pipeline emails, the ID of the pipeline the notification is for |
| X-GitLab-Reply-Key | A unique token to support reply by email |
| X-GitLab-NotificationReason | The reason for being notified. "mentioned", "assigned", etc |
| List-Id | The path of the project in a RFC 2919 mailing list identifier useful for email organization, for example, with Gmail filters |
#### X-GitLab-NotificationReason
This header holds the reason for the notification to have been sent out,
where reason can be `mentioned`, `assigned`, `own_activity`, etc.
Only one reason is sent out according to its priority:
- `own_activity`
- `assigned`
- `mentioned`
The reason in this header will also be shown in the footer of the notification email. For example an email with the
reason `assigned` will have this sentence in the footer:
`"You are receiving this email because you have been assigned an item on {configured GitLab hostname}"`
NOTE: **Note:**
Only reasons listed above have been implemented so far.
Further implementation is [being discussed](https://gitlab.com/gitlab-org/gitlab-foss/issues/42062).
This document was moved to [another location](../user/profile/notifications.md).
doc/workflow/releases.md
View file @ 529bc7e2
# Releases
---
redirect_to: '../user/project/releases/index.md#add-release-notes-to-git-tags'
---
NOTE: In GitLab 11.7, we introduced the full fledged [Releases](../user/project/releases/index.md)
feature. You can still create release notes on this page, but the new method is preferred.
You can add release notes to any Git tag using the notes feature. Release notes
behave like any other markdown form in GitLab so you can write text and
drag-n-drop files to it. Release notes are stored in GitLab's database.
There are several ways to add release notes:
- In the interface, when you create a new Git tag
- In the interface, by adding a note to an existing Git tag
- Using the GitLab API
## New tag page with release notes text area
![new_tag](releases/new_tag.png)
## Tags page with button to add or edit release notes for existing Git tag
![tags](releases/tags.png)
This document was moved to [another location](../user/project/releases/index.md#add-release-notes-to-git-tags).
doc/workflow/repository_mirroring.md
View file @ 529bc7e2
# Repository mirroring
Repository mirroring allows for mirroring of repositories to and from external sources. It can be
used to mirror branches, tags, and commits between repositories.
A repository mirror at GitLab will be updated automatically. You can also manually trigger an update
at most once every 5 minutes.
## Overview
Repository mirroring is useful when you want to use a repository outside of GitLab.
There are two kinds of repository mirroring supported by GitLab:
- Push: for mirroring a GitLab repository to another location.
- Pull: for mirroring a repository from another location to GitLab. **(STARTER)**
When the mirror repository is updated, all new branches, tags, and commits will be visible in the
project's activity feed.
Users with at least [developer access](../user/permissions.md) to the project can also force an
immediate update, unless:
- The mirror is already being updated.
- 5 minutes haven't elapsed since its last update.
## Use cases
The following are some possible use cases for repository mirroring:
- You migrated to GitLab but still need to keep your project in another source. In that case, you
can simply set it up to mirror to GitLab (pull) and all the essential history of commits, tags,
and branches will be available in your GitLab instance. **(STARTER)**
- You have old projects in another source that you don't use actively anymore, but don't want to
remove for archiving purposes. In that case, you can create a push mirror so that your active
GitLab repository can push its changes to the old location.
## Pushing to a remote repository **(CORE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/249) in GitLab Enterprise Edition 8.7.
> - [Moved to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/18715) in 10.8.
For an existing project, you can set up push mirroring as follows:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories** section.
1. Enter a repository URL.
1. Select **Push** from the **Mirror direction** dropdown.
1. Select an authentication method from the **Authentication method** dropdown, if necessary.
1. Check the **Only mirror protected branches** box, if necessary.
1. Click the **Mirror repository** button to save the configuration.
![Repository mirroring push settings screen](img/repository_mirroring_push_settings.png)
When push mirroring is enabled, only push commits directly to the mirrored repository to prevent the
mirror diverging. All changes will end up in the mirrored repository whenever:
- Commits are pushed to GitLab.
- A [forced update](#forcing-an-update-core) is initiated.
Changes pushed to files in the repository are automatically pushed to the remote mirror at least:
- Within five minutes of being received.
- Within one minute if **Only mirror protected branches** is enabled.
In the case of a diverged branch, you will see an error indicated at the **Mirroring repositories**
section.
### Push only protected branches **(CORE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3350) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
> - [Moved to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/18715) in 10.8.
You can choose to only push your protected branches from GitLab to your remote repository.
To use this option, check the **Only mirror protected branches** box when creating a repository
mirror.
## Setting up a push mirror from GitLab to GitHub **(CORE)**
To set up a mirror from GitLab to GitHub, you need to follow these steps:
1. Create a [GitHub personal access token](https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line) with the `public_repo` box checked.
1. Fill in the **Git repository URL** field using this format: `https://<your_github_username>@github.com/<your_github_group>/<your_github_project>.git`.
1. Fill in **Password** field with your GitHub personal access token.
1. Click the **Mirror repository** button.
The mirrored repository will be listed. For example, `https://*****:*****@github.com/<your_github_group>/<your_github_project>.git`.
The repository will push soon. To force a push, click the appropriate button.
## Setting up a push mirror to another GitLab instance with 2FA activated
1. On the destination GitLab instance, create a [personal access token](../user/profile/personal_access_tokens.md) with `API` scope.
1. On the source GitLab instance:
1. Fill in the **Git repository URL** field using this format: `https://oauth2@<destination host>/<your_gitlab_group_or_name>/<your_gitlab_project>.git`.
1. Fill in **Password** field with the GitLab personal access token created on the destination GitLab instance.
1. Click the **Mirror repository** button.
## Pulling from a remote repository **(STARTER)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/51) in GitLab Enterprise Edition 8.2.
> - [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab/issues/10871) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.11.
NOTE: **Note:** This feature [is available for free](https://gitlab.com/gitlab-org/gitlab/issues/10361) to
GitLab.com users until March 22nd, 2020.
You can set up a repository to automatically have its branches, tags, and commits updated from an
upstream repository.
This is useful when a repository you're interested in is located on a different server, and you want
to be able to browse its content and its activity using the familiar GitLab interface.
To configure mirror pulling for an existing project:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories**
section.
1. Enter a repository URL.
1. Select **Pull** from the **Mirror direction** dropdown.
1. Select an authentication method from the **Authentication method** dropdown, if necessary.
1. If necessary, check the following boxes:
- **Overwrite diverged branches**.
- **Trigger pipelines for mirror updates**.
- **Only mirror protected branches**.
1. Click the **Mirror repository** button to save the configuration.
![Repository mirroring pull settings screen - upper part](img/repository_mirroring_pull_settings_upper.png)
---
redirect_to: '../user/project/repository/repository_mirroring.md'
---
![Repository mirroring pull settings screen - lower part](img/repository_mirroring_pull_settings_lower.png)
Because GitLab is now set to pull changes from the upstream repository, you should not push commits
directly to the repository on GitLab. Instead, any commits should be pushed to the upstream repository.
Changes pushed to the upstream repository will be pulled into the GitLab repository, either:
- Automatically within a certain period of time.
- When a [forced update](#forcing-an-update-core) is initiated.
CAUTION: **Caution:**
If you do manually update a branch in the GitLab repository, the branch will become diverged from
upstream and GitLab will no longer automatically update this branch to prevent any changes from being lost.
### How it works
Once the pull mirroring feature has been enabled for a repository, the repository is added to a queue.
Once per minute, a Sidekiq cron job schedules repository mirrors to update, based on:
- The capacity available. This is determined by Sidekiq settings. For GitLab.com, see [GitLab.com Sidekiq settings](../user/gitlab_com/index.md#sidekiq).
- The number of repository mirrors already in the queue that are due to be updated. Being due depends on when the repository mirror was last updated and how many times it's been retried.
Repository mirrors are updated as Sidekiq becomes available to process them. If the process of updating the repository mirror:
- Succeeds, an update will be enqueued again with at least a 30 minute wait.
- Fails (for example, a branch diverged from upstream), it will be attempted again later. Mirrors can fail
up to 14 times before they will not be enqueued for update again.
### SSH authentication
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/2551) for Pull mirroring in [GitLab Starter](https://about.gitlab.com/pricing/) 9.5.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/22982) for Push mirroring in [GitLab Core](https://about.gitlab.com/pricing/) 11.6
SSH authentication is mutual:
- You have to prove to the server that you're allowed to access the repository.
- The server also has to prove to *you* that it's who it claims to be.
You provide your credentials as a password or public key. The server that the
other repository resides on provides its credentials as a "host key", the
fingerprint of which needs to be verified manually.
If you're mirroring over SSH (that is, using an `ssh://` URL), you can authenticate using:
- Password-based authentication, just as over HTTPS.
- Public key authentication. This is often more secure than password authentication,
especially when the other repository supports [Deploy Keys](../ssh/README.md#deploy-keys).
To get started:
1. Navigate to your project's **Settings > Repository** and expand the **Mirroring repositories** section.
1. Enter an `ssh://` URL for mirroring.
NOTE: **Note:**
SCP-style URLs (that is, `git@example.com:group/project.git`) are not supported at this time.
Entering the URL adds two buttons to the page:
- **Detect host keys**.
- **Input host keys manually**.
If you click the:
- **Detect host keys** button, GitLab will fetch the host keys from the server and display the fingerprints.
- **Input host keys manually** button, a field is displayed where you can paste in host keys.
Assuming you used the former, you now need to verify that the fingerprints are
those you expect. GitLab.com and other code hosting sites publish their
fingerprints in the open for you to check:
- [AWS CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints)
- [Bitbucket](https://confluence.atlassian.com/bitbucket/ssh-keys-935365775.html)
- [GitHub](https://help.github.com/en/articles/githubs-ssh-key-fingerprints)
- [GitLab.com](../user/gitlab_com/index.md#ssh-host-keys-fingerprints)
- [Launchpad](https://help.launchpad.net/SSHFingerprints)
- [Savannah](http://savannah.gnu.org/maintenance/SshAccess/)
- [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/)
Other providers will vary. If you're running self-managed GitLab, or otherwise
have access to the server for the other repository, you can securely gather the
key fingerprints:
```sh
$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f -
256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA)
256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519)
2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA)
```
NOTE: **Note:**
You may need to exclude `-E md5` for some older versions of SSH.
When mirroring the repository, GitLab will now check that at least one of the
stored host keys matches before connecting. This can prevent malicious code from
being injected into your mirror, or your password being stolen.
### SSH public key authentication
To use SSH public key authentication, you'll also need to choose that option
from the **Authentication method** dropdown. When the mirror is created,
GitLab generates a 4096-bit RSA key that can be copied by clicking the **Copy SSH public key** button.
![Repository mirroring copy SSH public key to clipboard button](img/copy_ssh_public_key_button.png)
You then need to add the public SSH key to the other repository's configuration:
- If the other repository is hosted on GitLab, you should add the public SSH key
as a [Deploy Key](../ssh/README.md#deploy-keys).
- If the other repository is hosted elsewhere, you may need to add the key to
your user's `authorized_keys` file. Paste the entire public SSH key into the
file on its own line and save it.
If you need to change the key at any time, you can remove and re-add the mirror
to generate a new key. You'll have to update the other repository with the new
key to keep the mirror running.
### Overwrite diverged branches **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/4559) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.6.
You can choose to always update your local branches with remote versions, even if they have
diverged from the remote.
CAUTION: **Caution:**
For mirrored branches, enabling this option results in the loss of local changes.
To use this option, check the **Overwrite diverged branches** box when creating a repository mirror.
### Only mirror protected branches **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3326) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
You can choose to pull mirror only the protected branches from your remote repository to GitLab.
Non-protected branches are not mirrored and can diverge.
To use this option, check the **Only mirror protected branches** box when creating a repository mirror.
### Hard failure **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3117) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.2.
Once the mirroring process is unsuccessfully retried 14 times in a row, it will get marked as hard
failed. This will become visible in either the:
- Project's main dashboard.
- Pull mirror settings page.
When a project is hard failed, it will no longer get picked up for mirroring. A user can resume the
project mirroring again by [Forcing an update](#forcing-an-update-core).
### Trigger update using API **(STARTER)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/3453) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3.
Pull mirroring uses polling to detect new branches and commits added upstream, often minutes
afterwards. If you notify GitLab by [API](../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter),
updates will be pulled immediately.
For more information, see [Start the pull mirroring process for a Project](../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter).
## Forcing an update **(CORE)**
While mirrors are scheduled to update automatically, you can always force an update by using the
update button which is available on the **Mirroring repositories** section of the **Repository Settings** page.
![Repository mirroring force update user interface](img/repository_mirroring_force_update.png)
## Bidirectional mirroring **(STARTER)**
CAUTION: **Caution:**
Bidirectional mirroring may cause conflicts.
If you configure a GitLab repository to both pull from, and push to, the same remote source, there
is no guarantee that either repository will update correctly. If you set up a repository for
bidirectional mirroring, you should prepare for the likely conflicts by deciding who will resolve
them and how they will be resolved.
Rewriting any mirrored commit on either remote will cause conflicts and mirroring to fail. This can
be prevented by:
- [Pulling only protected branches](#only-mirror-protected-branches-starter).
- [Pushing only protected branches](#push-only-protected-branches-core).
You should [protect the branches](../user/project/protected_branches.md) you wish to mirror on both
remotes to prevent conflicts caused by rewriting history.
Bidirectional mirroring also creates a race condition where commits made close together to the same
branch causes conflicts. The race condition can be mitigated by reducing the mirroring delay by using
a [Push event webhook](../user/project/integrations/webhooks.md#push-events) to trigger an immediate
pull to GitLab. Push mirroring from GitLab is rate limited to once per minute when only push mirroring
protected branches.
### Preventing conflicts using a `pre-receive` hook
CAUTION: **Warning:**
The solution proposed will negatively impact the performance of
Git push operations because they will be proxied to the upstream Git
repository.
A server-side `pre-receive` hook can be used to prevent the race condition
described above by only accepting the push after first pushing the commit to
the upstream Git repository. In this configuration one Git repository acts as
the authoritative upstream, and the other as downstream. The `pre-receive` hook
will be installed on the downstream repository.
Read about [configuring custom Git hooks](../administration/custom_hooks.md) on the GitLab server.
A sample `pre-receive` hook is provided below.
```bash
#!/usr/bin/env bash
# --- Assume only one push mirror target
# Push mirroring remotes are named `remote_mirror_<id>`, this finds the first remote and uses that.
TARGET_REPO=$(git remote | grep -m 1 remote_mirror)
proxy_push()
{
# --- Arguments
OLDREV=$(git rev-parse $1)
NEWREV=$(git rev-parse $2)
REFNAME="$3"
# --- Pattern of branches to proxy pushes
whitelisted=$(expr "$branch" : "\(master\)")
case "$refname" in
refs/heads/*)
branch=$(expr "$refname" : "refs/heads/\(.*\)")
if [ "$whitelisted" = "$branch" ]; then
error="$(git push --quiet $TARGET_REPO $NEWREV:$REFNAME 2>&1)"
fail=$?
if [ "$fail" != "0" ]; then
echo >&2 ""
echo >&2 " Error: updates were rejected by upstream server"
echo >&2 " This is usually caused by another repository pushing changes"
echo >&2 " to the same ref. You may want to first integrate remote changes"
echo >&2 ""
return
fi
fi
;;
esac
}
# Allow dual mode: run from the command line just like the update hook, or
# if no arguments are given then run as a hook script
if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
# Output to the terminal in command line mode - if someone wanted to
# resend an email; they could redirect the output to sendmail
# themselves
PAGER= proxy_push $2 $3 $1
else
# Push is proxied upstream one ref at a time. Because of this it is possible
# for some refs to succeed, and others to fail. This will result in a failed
# push.
while read oldrev newrev refname
do
proxy_push $oldrev $newrev $refname
done
fi
```
### Mirroring with Perforce Helix via Git Fusion **(STARTER)**
CAUTION: **Warning:**
Bidirectional mirroring should not be used as a permanent configuration. Refer to
[Migrating from Perforce Helix](../user/project/import/perforce.md) for alternative migration approaches.
[Git Fusion](https://www.perforce.com/manuals/git-fusion/#Git-Fusion/section_avy_hyc_gl.html) provides a Git interface
to [Perforce Helix](https://www.perforce.com/products) which can be used by GitLab to bidirectionally
mirror projects with GitLab. This may be useful in some situations when migrating from Perforce Helix
to GitLab where overlapping Perforce Helix workspaces cannot be migrated simultaneously to GitLab.
If using mirroring with Perforce Helix, you should only mirror protected branches. Perforce Helix
will reject any pushes that rewrite history. Only the fewest number of branches should be mirrored
due to the performance limitations of Git Fusion.
When configuring mirroring with Perforce Helix via Git Fusion, the following Git Fusion
settings are recommended:
- `change-pusher` should be disabled. Otherwise, every commit will be rewritten as being committed
by the mirroring account, rather than being mapped to existing Perforce Helix users or the `unknown_git` user.
- `unknown_git` user will be used as the commit author if the GitLab user does not exist in
Perforce Helix.
Read about [Git Fusion settings on Perforce.com](https://www.perforce.com/manuals/git-fusion/Content/Git-Fusion/section_vss_bdw_w3.html#section_zdp_zz1_3l).
## Troubleshooting
Should an error occur during a push, GitLab will display an "Error" highlight for that repository. Details on the error can then be seen by hovering over the highlight text.
### 13:Received RST_STREAM with error code 2 with GitHub
If you receive an "13:Received RST_STREAM with error code 2" while mirroring to a GitHub repository, your GitHub settings might be set to block pushes that expose your email address used in commits. Either set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting.
This document was moved to [another location](../user/project/repository/repository_mirroring.md).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7