gitlab_flow.md 25 KB
Newer Older
1 2
![GitLab Flow](gitlab_flow.png)

Ciro Santilli's avatar
Ciro Santilli committed
3
## Introduction
4 5 6 7 8 9

Version management with git makes branching and merging much easier than older versioning systems such as SVN.
This allows a wide variety of branching strategies and workflows.
Almost all of these are an improvement over the methods used before git.
But many organizations end up with a workflow that is not clearly defined, overly complex or not integrated with issue tracking systems.
Therefore we propose the GitLab flow as clearly defined set of best practices.
10
It combines [feature driven development](https://en.wikipedia.org/wiki/Feature-driven_development) and [feature branches](http://martinfowler.com/bliki/FeatureBranch.html) with issue tracking.
11 12 13 14 15

Organizations coming to git from other version control systems frequently find it hard to develop an effective workflow.
This article describes the GitLab flow that integrates the git workflow with an issue tracking system.
It offers a simple, transparent and effective way to work with git.

16 17 18
![Four stages (working copy, index, local repo, remote repo) and three steps between them](four_stages.png)

When converting to git you have to get used to the fact that there are three steps before a commit is shared with colleagues.
19 20 21 22 23
Most version control systems have only step, committing from the working copy to a shared server.
In git you add files from the working copy to the staging area. After that you commit them to the local repo.
The third step is pushing to a shared remote repository.
After getting used to these three steps the branching model becomes the challenge.

24 25 26
![Multiple long running branches and merging in all directions](messy_flow.png)

Since many organizations new to git have no conventions how to work with it, it can quickly become a mess.
27 28
The biggest problem they run into is that many long running branches that each contain part of the changes are around.
People have a hard time figuring out which branch they should develop on or deploy to production.
Derek Robati's avatar
Derek Robati committed
29
Frequently the reaction to this problem is to adopt a standardized pattern such as [git flow](http://nvie.com/posts/a-successful-git-branching-model/) and [GitHub flow](http://scottchacon.com/2011/08/31/github-flow.html).
30 31
We think there is still room for improvement and will detail a set of practices we call GitLab flow.

Ciro Santilli's avatar
Ciro Santilli committed
32
## Git flow and its problems
33

Arno Fleming's avatar
Arno Fleming committed
34
![Git Flow timeline by Vincent Driessen, used with permission](gitdashflow.png)
35 36

Git flow was one of the first proposals to use git branches and it has gotten a lot of attention.
37 38 39
It advocates a master branch and a separate develop branch as well as supporting branches for features, releases and hotfixes.
The development happens on the develop branch, moves to a release branch and is finally merged into the master branch.
Git flow is a well defined standard but its complexity introduces two problems.
40
The first problem is that developers must use the develop branch and not master, master is reserved for code that is released to production.
41 42 43
It is a convention to call your default branch master and to mostly branch from and merge to this.
Since most tools automatically make the master branch the default one and display that one by default it is annoying to have to switch to another one.
The second problem of git flow is the complexity introduced by the hotfix and release branches.
44
These branches can be a good idea for some organizations but are overkill for the vast majority of them.
45
Nowadays most organizations practice continuous delivery which means that your default branch can be deployed.
46
This means that hotfix and release branches can be prevented including all the ceremony they introduce.
47
An example of this ceremony is the merging back of release branches.
48
Though specialized tools do exist to solve this, they require documentation and add complexity.
49 50 51 52
Frequently developers make a mistake and for example changes are only merged into master and not into the develop branch.
The root cause of these errors is that git flow is too complex for most of the use cases.
And doing releases doesn't automatically mean also doing hotfixes.

Ciro Santilli's avatar
Ciro Santilli committed
53
## GitHub flow as a simpler alternative
54

55 56
![Master branch with feature branches merged in](github_flow.png)

Arno Fleming's avatar
Arno Fleming committed
57
In reaction to git flow a simpler alternative was detailed, [GitHub flow](https://guides.github.com/introduction/flow/index.html).
58 59 60
This flow has only feature branches and a master branch.
This is very simple and clean, many organizations have adopted it with great success.
Atlassian recommends [a similar strategy](http://blogs.atlassian.com/2014/01/simple-git-workflow-simple/) although they rebase feature branches.
61
Merging everything into the master branch and deploying often means you minimize the amount of code in 'inventory' which is in line with the lean and continuous delivery best practices.
62 63 64
But this flow still leaves a lot of questions unanswered regarding deployments, environments, releases and integrations with issues.
With GitLab flow we offer additional guidance for these questions.

Ciro Santilli's avatar
Ciro Santilli committed
65
## Production branch with GitLab flow
66

67 68 69
![Master branch and production branch with arrow that indicate deployments](production_branch.png)

GitHub flow does assume you are able to deploy to production every time you merge a feature branch.
70
This is possible for SaaS applications but are many cases where this is not possible.
Ciro Santilli's avatar
Ciro Santilli committed
71
One would be a situation where you are not in control of the exact release moment, for example an iOS application that needs to pass App Store validation.
72 73 74 75 76 77 78 79 80
Another example is when you have deployment windows (workdays from 10am to 4pm when the operations team is at full capacity) but you also merge code at other times.
In these cases you can make a production branch that reflects the deployed code.
You can deploy a new version by merging in master to the production branch.
If you need to know what code is in production you can just checkout the production branch to see.
The approximate time of deployment is easily visible as the merge commit in the version control system.
This time is pretty accurate if you automatically deploy your production branch.
If you need a more exact time you can have your deployment script create a tag on each deployment.
This flow prevents the overhead of releasing, tagging and merging that is common to git flow.

Ciro Santilli's avatar
Ciro Santilli committed
81
## Environment branches with GitLab flow
82

83 84 85
![Multiple branches with the code cascading from one to another](environment_branches.png)

It might be a good idea to have an environment that is automatically updated to the master branch.
86
Only in this case, the name of this environment might differ from the branch name.
87 88 89 90 91 92 93
Suppose you have a staging environment, a pre-production environment and a production environment.
In this case the master branch is deployed on staging. When someone wants to deploy to pre-production they create a merge request from the master branch to the pre-production branch.
And going live with code happens by merging the pre-production branch into the production branch.
This workflow where commits only flow downstream ensures that everything has been tested on all environments.
If you need to cherry-pick a commit with a hotfix it is common to develop it on a feature branch and merge it into master with a merge request, do not delete the feature branch.
If master is good to go (it should be if you a practicing [continuous delivery](http://martinfowler.com/bliki/ContinuousDelivery.html)) you then merge it to the other branches.
If this is not possible because more manual testing is required you can send merge requests from the feature branch to the downstream branches.
94
An 'extreme' version of environment branches are setting up an environment for each feature branch as done by [Teatro](https://teatro.io/).
95

Ciro Santilli's avatar
Ciro Santilli committed
96
## Release branches with GitLab flow
97

98
![Master and multiple release branches that vary in length with cherry-picks from master](release_branches.png)
99 100

Only in case you need to release software to the outside world you need to work with release branches.
101
In this case, each branch contains a minor version (2-3-stable, 2-4-stable, etc.).
102
The stable branch uses master as a starting point and is created as late as possible.
103
By branching as late as possible you minimize the time you have to apply bug fixes to multiple branches.
104
After a release branch is announced, only serious bug fixes are included in the release branch.
105
If possible these bug fixes are first merged into master and then cherry-picked into the release branch.
106
This way you can't forget to cherry-pick them into master and encounter the same bug on subsequent releases.
107
This is called an 'upstream first' policy that is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](https://www.redhat.com/about/news/archive/2013/5/a-community-for-using-openstack-with-red-hat-rdo).
108
Every time a bug-fix is included in a release branch the patch version is raised (to comply with [Semantic Versioning](http://semver.org/)) by setting a new tag.
109 110 111
Some projects also have a stable branch that points to the same commit as the latest released branch.
In this flow it is not common to have a production branch (or git flow master branch).

Ciro Santilli's avatar
Ciro Santilli committed
112
## Merge/pull requests with GitLab flow
113

114 115 116
![Merge request with line comments](mr_inline_comments.png)

Merge or pull requests are created in a git management application and ask an assigned person to merge two branches.
117 118 119 120 121
Tools such as GitHub and Bitbucket choose the name pull request since the first manual action would be to pull the feature branch.
Tools such as GitLab and Gitorious choose the name merge request since that is the final action that is requested of the assignee.
In this article we'll refer to them as merge requests.

If you work on a feature branch for more than a few hours it is good to share the intermediate result with the rest of the team.
122
This can be done by creating a merge request without assigning it to anyone, instead you mention people in the description or a comment (/cc @mark @susan).
123 124 125 126 127 128 129 130 131 132 133
This means it is not ready to be merged but feedback is welcome.
Your team members can comment on the merge request in general or on specific lines with line comments.
The merge requests serves as a code review tool and no separate tools such as Gerrit and reviewboard should be needed.
If the review reveals shortcomings anyone can commit and push a fix.
Commonly the person to do this is the creator of the merge/pull request.
The diff in the merge/pull requests automatically updates when new commits are pushed on the branch.

When you feel comfortable with it to be merged you assign it to the person that knows most about the codebase you are changing and mention any other people you would like feedback from.
There is room for more feedback and after the assigned person feels comfortable with the result the branch is merged.
If the assigned person does not feel comfortable they can close the merge request without merging.

Arno Fleming's avatar
Arno Fleming committed
134
In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://doc.gitlab.com/ce/permissions/permissions.html).
135 136
So if you want to merge it into a protected branch you assign it to someone with master authorizations.

Ciro Santilli's avatar
Ciro Santilli committed
137
## Issues with GitLab flow
138

139 140 141
![Merge request with the branch name 15-require-a-password-to-change-it and assignee field shown](merge_request.png)

GitLab flow is a way to make the relation between the code and the issue tracker more transparent.
142 143 144

Any significant change to the code should start with an issue where the goal is described.
Having a reason for every code change is important to inform everyone on the team and to help people keep the scope of a feature branch small.
145 146
In GitLab each change to the codebase starts with an issue in the issue tracking system.
If there is no issue yet it should be created first provided there is significant work involved (more than 1 hour).
147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163
For many organizations this will be natural since the issue will have to be estimated for the sprint.
Issue titles should describe the desired state of the system, e.g. "As an administrator I want to remove users without receiving an error" instead of "Admin can't remove users.".

When you are ready to code you start a branch for the issue from the master branch.
The name of this branch should start with the issue number, for example '15-require-a-password-to-change-it'.

When you are done or want to discuss the code you open a merge request.
This is an online place to discuss the change and review the code.
Creating a branch is a manual action since you do not always want to merge a new branch you push, it could be a long-running environment or release branch.
If you create the merge request but do not assign it to anyone it is a 'work-in-process' merge request.
These are used to discuss the proposed implementation but are not ready for inclusion in the master branch yet.

When the author thinks the code is ready the merge request is assigned to reviewer.
The reviewer presses the merge button when they think the code is ready for inclusion in the master branch.
In this case the code is merged and a merge commit is generated that makes this event easily visible later on.
Merge requests always create a merge commit even when the commit could be added without one.
This merge strategy is called 'no fast-forward' in git.
164
After the merge the feature branch is deleted since it is no longer needed, in GitLab this deletion is an option when merging.
165 166 167 168

Suppose that a branch is merged but a problem occurs and the issue is reopened.
In this case it is no problem to reuse the same branch name since it was deleted when the branch was merged.
At any time there is at most one branch for every issue.
169
It is possible that one feature branch solves more than one issue.
170

Ciro Santilli's avatar
Ciro Santilli committed
171
## Linking and closing issues from merge requests
172

173 174 175
![Merge request showing the linked issues that will be closed](close_issue_mr.png)

Linking to the issue can happen by mentioning them from commit messages (fixes #14, closes #67, etc.) or from the merge request description.
176
In GitLab this creates a comment in the issue that the merge requests mentions the issue.
177
And the merge request shows the linked issues.
178
These issues are closed once code is merged into the default branch.
179

180
If you only want to make the reference without closing the issue you can also just mention it: "Duck typing is preferred. #12".
181

182
If you have an issue that spans across multiple repositories, the best thing is to create an issue for each repository and link all issues to a parent issue.
183

Ciro Santilli's avatar
Ciro Santilli committed
184
## Squashing commits with rebase
185

186 187 188
![Vim screen showing the rebase view](rebase.png)

With git you can use an interactive rebase (rebase -i) to squash multiple commits into one and reorder them.
189
This functionality is useful if you made a couple of commits for small changes during development and want to replace them with a single commit or if you want to make the order more logical.
190 191
However you should never rebase commits you have pushed to a remote server.
Somebody can have referred to the commits or cherry-picked them.
Ciro Santilli's avatar
Ciro Santilli committed
192
When you rebase you change the identifier (SHA-1) of the commit and this is confusing.
193
If you do that the same change will be known under multiple identifiers and this can cause much confusion.
194 195
If people already reviewed your code it will be hard for them to review only the improvements you made since then if you have rebased everything into one commit.

196
People are encouraged to commit often and to frequently push to the remote repository so other people are aware what everyone is working on.
197 198
This will lead to many commits per change which makes the history harder to understand.
But the advantages of having stable identifiers outweigh this drawback.
199
And to understand a change in context one can always look at the merge commit that groups all the commits together when the code is merged into the master branch.
200 201 202

After you merge multiple commits from a feature branch into the master branch this is harder to undo.
If you would have squashed all the commits into one you could have just reverted this commit but as we indicated you should not rebase commits after they are pushed.
203
Fortunately [reverting a merge made some time ago](https://git-scm.com/blog/2010/03/02/undoing-merges.html) can be done with git.
204
This however, requires having specific merge commits for the commits your want to revert.
205 206 207
If you revert a merge and you change your mind, revert the revert instead of merging again since git will not allow you to merge the code again otherwise.

Being able to revert a merge is a good reason always to create a merge commit when you merge manually with the `--no-ff` option.
208 209
Git management software will always create a merge commit when you accept a merge request.

Ciro Santilli's avatar
Ciro Santilli committed
210
## Do not order commits with rebase
211

212 213 214
![List of sequential merge commits](merge_commits.png)

With git you can also rebase your feature branch commits to order them after the commits on the master branch.
215 216 217
This prevents creating a merge commit when merging master into your feature branch and creates a nice linear history.
However, just like with squashing you should never rebase commits you have pushed to a remote server.
This makes it impossible to rebase work in progress that you already shared with your team which is something we recommend.
218
When using rebase to keep your feature branch updated you [need to resolve similar conflicts again and again](https://blogs.atlassian.com/2013/10/git-team-workflows-merge-or-rebase/).
Arno Fleming's avatar
Arno Fleming committed
219
You can reuse recorded resolutions (rerere) sometimes, but without rebasing you only have to solve the conflicts one time and you’re set.
220 221 222 223 224
There has to be a better way to avoid many merge commits.

The way to prevent creating many merge commits is to not frequently merge master into the feature branch.
We'll discuss the three reasons to merge in master: leveraging code, solving merge conflicts and long running branches.
If you need to leverage some code that was introduced in master after you created the feature branch you can sometimes solve this by just cherry-picking a commit.
225
If your feature branch has a merge conflict, creating a merge commit is a normal way of solving this.
226 227 228 229 230
You should aim to prevent merge conflicts where they are likely to occur.
One example is the CHANGELOG file where each significant change in the codebase is documented under a version header.
Instead of everyone adding their change at the bottom of the list for the current version it is better to randomly insert it in the current list for that version.
This it is likely that multiple feature branches that add to the CHANGELOG can be merged before a conflict occurs.
The last reason for creating merge commits is having long lived branches that you want to keep up to date with the latest state of the project.
231 232 233
Martin Fowler, in [his article about feature branches](http://martinfowler.com/bliki/FeatureBranch.html) talks about this Continuous Integration (CI).
At GitLab we are guilty of confusing CI with branch testing. Quoting Martin Fowler: "I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit.
That's continuous building, and a Good Thing, but there's no integration, so it's not CI.".
Ciro Santilli's avatar
Ciro Santilli committed
234 235
The solution to prevent many merge commits is to keep your feature branches short-lived, the vast majority should take less than one day of work.
If your feature branches commonly take more than a day of work, look into ways to create smaller units of work and/or use [feature toggles](http://martinfowler.com/bliki/FeatureToggle.html).
236
As for the long running branches that take more than one day there are two strategies.
237 238
In a CI strategy you can merge in master at the start of the day to prevent painful merges at a later time.
In a synchronization point strategy you only merge in from well defined points in time, for example a tagged release.
239
This strategy is [advocated by Linus Torvalds](https://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html) because the state of the code at these points is better known.
240

241
In conclusion, we can say that you should try to prevent merge commits, but not eliminate them.
242 243 244
Your codebase should be clean but your history should represent what actually happened.
Developing software happen in small messy steps and it is OK to have your history reflect this.
You can use tools to view the network graphs of commits and understand the messy history that created your code.
245
If you rebase code the history is incorrect, and there is no way for tools to remedy this because they can't deal with changing commit identifiers.
246

Ciro Santilli's avatar
Ciro Santilli committed
247
## Voting on merge requests
248

249 250 251
![Voting slider in GitLab](voting_slider.png)

It is common to voice approval or disapproval by using +1 or -1 emoticons.
252
In GitLab the +1 and -1 are aggregated and shown at the top of the merge request.
253
As a rule of thumb anything that doesn't have two times more +1's than -1's is suspect and should not be merged yet.
254

Ciro Santilli's avatar
Ciro Santilli committed
255
## Pushing and removing branches
256

257 258 259
![Remove checkbox for branch in merge requests](remove_checkbox.png)

We recommend that people push their feature branches frequently, even when they are not ready for review yet.
260 261
By doing this you prevent team members from accidentally starting to work on the same issue.
Of course this situation should already be prevented by assigning someone to the issue in the issue tracking software.
262
However sometimes one of the two parties forgets to assign someone in the issue tracking software.
263 264 265 266 267 268
After a branch is merged it should be removed from the source control software.
In GitLab and similar systems this is an option when merging.
This ensures that the branch overview in the repository management software shows only work in progress.
This also ensures that when someone reopens the issue a new branch with the same name can be used without problem.
When you reopen an issue you need to create a new merge request.

Ciro Santilli's avatar
Ciro Santilli committed
269
## Committing often and with the right message
270

271 272 273
![Good and bad commit message](good_commit.png)

We recommend to commit early and often.
274 275 276 277 278 279
Each time you have a functioning set of tests and code a commit can be made.
The advantage is that when an extension or refactor goes wrong it is easy to revert to a working version.
This is quite a change for programmers that used SVN before, they used to commit when their work was ready to share.
The trick is to use the merge/pull request with multiple commits when your work is ready to share.
The commit message should reflect your intention, not the contents of the commit.
The contents of the commit can be easily seen anyway, the question is why you did it.
280 281
An example of a good commit message is: "Combine templates to dry up the user views.".
Some words that are bad commit messages because they don't contain munch information are: change, improve and refactor.
282
The word fix or fixes is also a red flag, unless it comes after the commit sentence and references an issue number.
283
To see more information about the formatting of commit messages please see this great [blog post by Tim Pope](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
284

Ciro Santilli's avatar
Ciro Santilli committed
285
## Testing before merging
286

287 288 289
![Merge requests showing the test states, red, yellow and green](ci_mr.png)

In old workflows the Continuous Integration (CI) server commonly ran tests on the master branch only.
290
Developers had to ensure their code did not break the master branch.
291
When using GitLab flow developers create their branches from this master branch so it is essential it is green.
292 293 294 295 296 297 298
Therefore each merge request must be tested before it is accepted.
CI software like Travis and GitLab CI show the build results right in the merge request itself to make this easy.
One drawback is that they are testing the feature branch itself and not the merged result.
What one can do to improve this is to test the merged result itself.
The problem is that the merge result changes every time something is merged into master.
Retesting on every commit to master is computationally expensive and means you are more frequently waiting for test results.
If there are no merge conflicts and the feature branches are short lived the risk is acceptable.
299 300
If there are merge conflicts you merge the master branch into the feature branch and the CI server will rerun the tests.
If you have long lived feature branches that last for more than a few days you should make your issues smaller.
301

Ciro Santilli's avatar
Ciro Santilli committed
302
## Merging in other code
303

304 305 306
![Shell output showing git pull output](git_pull.png)

When initiating a feature branch, always start with an up to date master to branch off from.
307
If you know beforehand that your work absolutely depends on another branch you can also branch from there.
308
If you need to merge in another branch after starting explain the reason in the merge commit.
309
If you have not pushed your commits to a shared location yet you can also rebase on master or another feature branch.
310
Do not merge in upstream if your code will work and merge cleanly without doing so, Linus even says that [you should never merge in upstream at random points, only at major releases](https://lwn.net/Articles/328438/).
311
Merging only when needed prevents creating merge commits in your feature branch that later end up littering the master history.
312 313 314 315 316

### References

- [Sketch file](https://www.dropbox.com/s/58dvsj5votbwrzv/git_flows.sketch?dl=0) with vectors of images in this article
- [Git Flow by Vincent Driessen](http://nvie.com/posts/a-successful-git-branching-model/)