Skip to content

G
gitlab-ce
Commit c5ab4136 authored by Hordur Freyr Yngvason's avatar Hordur Freyr Yngvason Committed by Matija Čupić
Browse files
Options

Add Helm-2to3.gitlab-ci.yml to Auto DevOps

parent 15ee1a41
Hide whitespace changes
Inline Side-by-side
Showing with 209 additions and 43 deletions
doc/topics/autodevops/upgrading_auto_deploy_dependencies.md
View file @ c5ab4136
......@@ -77,7 +77,7 @@ The v2 auto-deploy-image drops support for Kubernetes 1.15 and lower. If you nee
Kubernetes cluster, follow your cloud provider's instructions. Here's
[an example on GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/upgrading-a-cluster).
#### Helm 3
#### Helm v3
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228609) in GitLab 13.4.
......@@ -86,47 +86,38 @@ Previously, `auto-deploy-image` used Helm v2, which used Tiller in a cluster.
In the v2 `auto-deploy-image`, it uses Helm v3 that doesn't require Tiller anymore.
If your Auto DevOps project has an active environment that was deployed with the v1
`auto-deploy-image`, use the following steps to upgrade to v2, which uses Helm 3:
1. Modify your `.gitlab-ci.yml` with:
```yaml
include:
- template: Auto-DevOps.gitlab-ci.yml
- remote: https://gitlab.com/hfyngvason/ci-templates/-/raw/master/Helm-2to3.gitlab-ci.yml
variables:
# If this variable is not present, the migration jobs will not show up
MIGRATE_HELM_2TO3: "true"
.auto-deploy:
# Optional: If you are on GitLab 13.12 or older, pin the auto-deploy-image
# image: registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.6.0
variables:
AUTO_DEVOPS_FORCE_DEPLOY_V2: 1
# If you have non-public pipelines, you can back up the entire namespace in a job artifact
# prior to the migration by setting the CI variable BACKUP_NAMESPACE to a non-empty value.
# WARNING: If you have public pipelines, this artifact will be public and can
# expose your secrets.
# BACKUP_HELM2_RELEASES: 1
```
1. Run the `<environment-name>:helm-2to3:migrate` job.
1. Deploy your environment as usual. This deployment uses Helm 3.
1. If the deployment succeeds, you can safely run `environment:helm-2to3:cleanup`.
This deletes all Helm 2 release data from the namespace.
If you set `BACKUP_HELM2_RELEASES` to a non-empty value, the `<environment-name>:helm2to3:migrate`
job saves a backup for 1 week in a job artifact called `helm-2-release-backups`.
If you accidentally delete the Helm 2 releases before you are ready, then
this backup is in a Kubernetes manifest file that can be restored using
`kubectl apply -f $backup`.
**WARNING:**
This artifact can contain secrets and is visible to any
user who can see your job.
1. Remove the `MIGRATE_HELM_2TO3` CI/CD variable.
`auto-deploy-image`, use the following steps to upgrade to v2, which uses Helm v3:
1. Include the [Helm 2to3 migration CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Jobs/Helm-2to3.gitlab-ci.yml):
- If you are on GitLab.com, or GitLab 14.0.1 or later, this template is already included in Auto DevOps.
- On other versions of GitLab, you can modify your `.gitlab-ci.yml` to include the templates:
```yaml
include:
- template: Auto-DevOps.gitlab-ci.yml
- remote: https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Jobs/Helm-2to3.gitlab-ci.yml
```
1. Set the following CI/CD variables:
- `MIGRATE_HELM_2TO3` to `true`. If this variable is not present, migration jobs do not run.
- `AUTO_DEVOPS_FORCE_DEPLOY_V2` to `1`.
- **Optional:** `BACKUP_HELM2_RELEASES` to `1`. If you set this variable, the migration
job saves a backup for 1 week in a job artifact called `helm-2-release-backups`.
If you accidentally delete the Helm v2 releases before you are ready, you can restore
this backup from a Kubernetes manifest file by using `kubectl apply -f $backup`.
**WARNING:**
*Do not use this if you have public pipelines*.
This artifact can contain secrets and is visible to any
user who can see your job.
1. Run a pipeline and trigger the `<environment-name>:helm-2to3:migrate` job.
1. Deploy your environment as usual. This deployment uses Helm v3.
1. If the deployment succeeds, you can safely run `<environment-name>:helm-2to3:cleanup`.
This deletes all Helm v2 release data from the namespace.
1. Remove the `MIGRATE_HELM_2TO3` CI/CD variable or set it to `false`. You can do this one environment at a time using [environment scopes](../../ci/environments/index.md#scoping-environments-with-specs).
#### In-Cluster PostgreSQL Channel 2
......
lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml
View file @ c5ab4136
......@@ -166,6 +166,7 @@ include:
- template: Jobs/Deploy/EC2.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/EC2.gitlab-ci.yml
- template: Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml
- template: Jobs/Browser-Performance-Testing.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml
- template: Jobs/Helm-2to3.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Helm-2to3.gitlab-ci.yml
- template: Security/DAST.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml
- template: Security/Container-Scanning.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml
......
lib/gitlab/ci/templates/Jobs/Helm-2to3.gitlab-ci.yml 0 → 100644
View file @ c5ab4136
# This is a pre-release of a Helm 2to3 migration template to facilitate
# the migration of Auto DevOps releases to Helm 3.
#
# To use, set the CI variable MIGRATE_HELM_2TO3 to "true".
# For more details, go to https://docs.gitlab.com/ee/topics/autodevops/upgrading_auto_deploy_dependencies.html#helm-v3
.helm-2to3-migrate:
image: registry.gitlab.com/gitlab-org/cluster-integration/helm-install-image/releases/helm-2to3-2.17.0-3.5.3-kube-1.16.15-alpine-3.12
# NOTE: We use the deploy stage because:
# - It exists in all versions of Auto DevOps.
# - It is _empty_.
# - It precedes any Kubernetes deployments.
# Users on older versions of GitLab can include this template without adjusting their stages.
stage: deploy
needs: []
artifacts:
when: always
expire_in: 1 week
paths:
- helm-2-release-backups/
before_script:
- mkdir helm-2-release-backups
- apk add jq
- export TILLER_NAMESPACE=$KUBE_NAMESPACE
- export HELM_HOST="localhost:44134"
- nohup tiller -listen "${HELM_HOST}" >tiller.log 2>&1 &
- helm2 init --client-only
script:
# check for releases
- releases=$(helm2 ls --output json | jq -r '.Releases[].Name')
# back up entire namespace if the user opts into it
- |
if [[ -n "$releases" && -n "$BACKUP_HELM2_RELEASES" ]]; then
echo "Backing up releases"
kubectl get configmap -n "$KUBE_NAMESPACE" -l "OWNER=TILLER" -o yaml > "helm-2-release-backups/$KUBE_NAMESPACE.yaml"
fi
# adopt manifests from each release
- |
for release in $releases; do
chart=$(helm2 ls "^$release\$" --output json | jq -r '.Releases[0].Chart')
echo "Adopting Helm v2 manifests from $release"
# some resource kinds must be listed explicitly https://github.com/kubernetes/kubernetes/issues/42885
for name in $(kubectl -n "$KUBE_NAMESPACE" get all,ingress,daemonset -o name -l chart="$chart"); do
kubectl annotate --overwrite "$name" meta.helm.sh/release-name="$release"
kubectl annotate --overwrite "$name" meta.helm.sh/release-namespace="$KUBE_NAMESPACE"
kubectl label --overwrite "$name" app.kubernetes.io/managed-by=Helm
done
done
# migrate each release
- |
for release in $releases; do
echo "Migrating release: $release"
helm3 2to3 convert --ignore-already-migrated --release-storage configmaps --tiller-out-cluster --tiller-ns "$TILLER_NAMESPACE" "$release"
done
.helm-2to3-cleanup:
image: registry.gitlab.com/gitlab-org/cluster-integration/helm-install-image/releases/helm-2to3-2.17.0-3.5.3-kube-1.16.15-alpine-3.12
stage: cleanup
environment:
action: prepare
before_script:
- export TILLER_NAMESPACE=$KUBE_NAMESPACE
- export HELM_HOST="localhost:44134"
- nohup tiller -listen "${HELM_HOST}" >tiller.log 2>&1 &
- helm2 init --client-only
script:
- helm3 2to3 cleanup --skip-confirmation --release-storage configmaps --tiller-out-cluster --tiller-ns "$TILLER_NAMESPACE"
.review:
environment:
name: review/$CI_COMMIT_REF_NAME
rules:
- if: '$MIGRATE_HELM_2TO3 != "true"'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: never
- if: '$REVIEW_DISABLED'
when: never
- if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH'
when: manual
review:helm-2to3:migrate:
extends: [.review, .helm-2to3-migrate]
review:helm-2to3:cleanup:
extends: [.review, .helm-2to3-cleanup]
rules:
- if: '$MIGRATE_HELM_2TO3 != "true" && $CLEANUP_HELM_2TO3 == null'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: never
- if: '$REVIEW_DISABLED'
when: never
- if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH'
when: manual
.staging:
environment:
name: staging
rules:
- if: '$MIGRATE_HELM_2TO3 != "true"'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
when: never
- if: '$STAGING_ENABLED'
when: manual
staging:helm-2to3:migrate:
extends: [.staging, .helm-2to3-migrate]
staging:helm-2to3:cleanup:
extends: [.staging, .helm-2to3-cleanup]
rules:
- if: '$MIGRATE_HELM_2TO3 != "true" && $CLEANUP_HELM_2TO3 == null'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
when: never
- if: '$STAGING_ENABLED'
when: manual
.production:
environment:
name: production
rules:
- if: '$MIGRATE_HELM_2TO3 != "true"'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: manual
production:helm-2to3:migrate:
extends: [.production, .helm-2to3-migrate]
production:helm-2to3:cleanup:
extends: [.production, .helm-2to3-cleanup]
rules:
- if: '$MIGRATE_HELM_2TO3 != "true" && $CLEANUP_HELM_2TO3 == null'
when: never
- if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
when: never
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: manual
spec/lib/gitlab/ci/templates/auto_devops_gitlab_ci_yaml_spec.rb
View file @ c5ab4136
......@@ -34,6 +34,7 @@ RSpec.describe 'Auto-DevOps.gitlab-ci.yml' do
expect(build_names).not_to include('canary')
expect(build_names).not_to include('review')
expect(build_names).not_to include(a_string_matching(/rollout \d+%/))
expect(build_names).not_to include(a_string_matching(/helm-2to3\d+%/))
end
end
......@@ -190,6 +191,17 @@ RSpec.describe 'Auto-DevOps.gitlab-ci.yml' do
expect(build_names).not_to include(a_string_matching(/rollout \d+%/))
end
end
context 'when MIGRATE_HELM_2TO3=true' do
before do
create(:ci_variable, project: project, key: 'MIGRATE_HELM_2TO3', value: 'true')
end
it 'includes a helm-2to3:migrate and a helm-2to3:cleanup job' do
expect(build_names).to include('production:helm-2to3:migrate')
expect(build_names).to include('production:helm-2to3:cleanup')
end
end
end
context 'outside of default branch' do
......@@ -207,12 +219,23 @@ RSpec.describe 'Auto-DevOps.gitlab-ci.yml' do
expect(build_names).to include('review')
expect(build_names).not_to include(a_string_matching(/rollout \d+%/))
end
context 'when MIGRATE_HELM_2TO3=true' do
before do
create(:ci_variable, project: project, key: 'MIGRATE_HELM_2TO3', value: 'true')
end
it 'includes a helm-2to3:migrate and a helm-2to3:cleanup job' do
expect(build_names).to include('review:helm-2to3:migrate')
expect(build_names).to include('review:helm-2to3:cleanup')
end
end
end
end
end
end
describe 'build-pack detection' do
describe 'buildpack detection' do
using RSpec::Parameterized::TableSyntax
where(:case_name, :files, :variables, :include_build_names, :not_include_build_names) do
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7