Skip to content

G
gitlab-ce
Commit ffffd1c4 authored by Craig Norris's avatar Craig Norris
Browse files
Options

Merge branch 'docs-move-spring-example' into 'master' 

Move CI example to source project

See merge request gitlab-org/gitlab!51861
parents b1bb448c c783d90b
Hide whitespace changes
Inline Side-by-side
Showing with 32 additions and 1044 deletions
doc/ci/README.md
View file @ ffffd1c4
...@@ -160,8 +160,6 @@ Its feature set is listed on the table below according to DevOps stages. ...@@ -160,8 +160,6 @@ Its feature set is listed on the table below according to DevOps stages.
Find example project code and tutorials for using GitLab CI/CD with a variety of app frameworks, languages, and platforms Find example project code and tutorials for using GitLab CI/CD with a variety of app frameworks, languages, and platforms
on the [CI Examples](examples/README.md) page. on the [CI Examples](examples/README.md) page.
GitLab also provides [example projects](https://gitlab.com/gitlab-examples) pre-configured to use GitLab CI/CD.
## Administration **(CORE ONLY)** ## Administration **(CORE ONLY)**
As a GitLab administrator, you can change the default behavior As a GitLab administrator, you can change the default behavior
......
doc/ci/examples/README.md
View file @ ffffd1c4
...@@ -23,31 +23,35 @@ Examples are available in several forms. As a collection of: ...@@ -23,31 +23,35 @@ Examples are available in several forms. As a collection of:
The following table lists examples with step-by-step tutorials that are contained in this section: The following table lists examples with step-by-step tutorials that are contained in this section:
| Use case | Resource | | Use case | Resource |
|:------------------------------|:---------| |-------------------------------|----------|
| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). |
| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | | Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). |
| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). |
| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. |
| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | | End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). |
| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). |
| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). |
| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). |
| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | | Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). |
| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | | Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). |
| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | | NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). |
| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | | PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). |
| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | | PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). |
| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | | PHP with PHPunit, atoum | [Testing PHP projects](php.md). |
| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). |
| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | | Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). |
| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). |
| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). |
| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | | Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). |
### How to contributing examples ### Contributed examples
You can help people that use your favorite programming language by submitting a link
to a guide for that language. These contributed guides are hosted externally or in
separate example projects:
Contributions are welcome! You can help your favorite programming | Use case | Resource |
language users and GitLab by sending a merge request with a guide for that language. |-------------------------------|----------|
| Game development | [DevOps and Game Dev with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). |
| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). |
| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). |
| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). |
| Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). |
## CI/CD templates ## CI/CD templates
......
doc/ci/examples/artifactory_and_gitlab/index.md
View file @ ffffd1c4
--- ---
stage: Verify redirect_to: '../README.md#contributed-examples'
group: Continuous Integration
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html'
author: Fabio Busatto
author_gitlab: bikebilly
type: tutorial
date: 2017-08-15
--- ---
<!-- vale off --> This document was moved to [another location](../README.md#contributed-examples).
# How to deploy Maven projects to Artifactory with GitLab CI/CD <!-- This redirect file can be deleted after 2021-04-18. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
## Introduction
In this article, we show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/)
to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://jfrog.com/artifactory/), and then use it from another Maven application as a dependency.
You'll create two different projects:
- `simple-maven-dep`: the app built and deployed to Artifactory (see the [simple-maven-dep](https://gitlab.com/gitlab-examples/maven/simple-maven-dep) example project)
- `simple-maven-app`: the app using the previous one as a dependency (see the [simple-maven-app](https://gitlab.com/gitlab-examples/maven/simple-maven-app) example project)
We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/).
We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it.
## Create the simple Maven dependency
First, you need an application to work with: in this specific case we'll use a
simple one, but it could be any Maven application. This will be the dependency
you want to package and deploy to Artifactory, to be available to other
projects.
### Prepare the dependency application
For this article you'll use a Maven app that can be cloned from our example
project:
1. Log in to your GitLab account
1. Create a new project by selecting **Import project from > Repo by URL**
1. Add the following URL:
```plaintext
https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git
```
1. Click **Create project**
This application is nothing more than a basic class with a stub for a JUnit based test suite.
It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen.
The project structure is really simple, and you should consider these two resources:
- `pom.xml`: project object model (POM) configuration file
- `src/main/java/com/example/dep/Dep.java`: source of our application
### Configure the Artifactory deployment
The application is ready to use, but you need some additional steps to deploy it to Artifactory:
1. Log in to Artifactory with your user's credentials.
1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel.
1. Copy to clipboard the configuration snippet under the **Deploy** paragraph.
1. Change the `url` value to have it configurable by using variables.
1. Copy the snippet in the `pom.xml` file for your project, just after the
`dependencies` section. The snippet should look like this:
```xml
<distributionManagement>
<repository>
<id>central</id>
<name>83d43b5afeb5-releases</name>
<url>${env.MAVEN_REPO_URL}/libs-release-local</url>
</repository>
</distributionManagement>
```
Another step you need to do before you can deploy the dependency to Artifactory
is to configure the authentication data. It is a simple task, but Maven requires
it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory
in the user's homedir.
Since you want to use a runner to automatically deploy the application, you
should create the file in the project's home directory and set a command line
parameter in `.gitlab-ci.yml` to use the custom location instead of the default one:
1. Create a folder called `.m2` in the root of your repository
1. Create a file called `settings.xml` in the `.m2` folder
1. Copy the following content into a `settings.xml` file:
```xml
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd"
xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<servers>
<server>
<id>central</id>
<username>${env.MAVEN_REPO_USER}</username>
<password>${env.MAVEN_REPO_PASS}</password>
</server>
</servers>
</settings>
```
Username and password will be replaced by the correct values using variables.
### Configure GitLab CI/CD for `simple-maven-dep`
Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and deploy the dependency!
GitLab CI/CD uses a file in the root of the repository, named `.gitlab-ci.yml`, to read the definitions for jobs
that will be executed by the configured runners. You can read more about this file in the [GitLab Documentation](../../yaml/README.md).
First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Environment variables** page
and add the following ones (replace them with your current values, of course):
- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL)
- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username)
- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password)
Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repository:
```yaml
image: maven:latest
variables:
MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
paths:
- .m2/repository/
- target/
build:
stage: build
script:
- mvn $MAVEN_CLI_OPTS compile
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS test
deploy:
stage: deploy
script:
- mvn $MAVEN_CLI_OPTS deploy
only:
- master
```
The runner uses the latest [Maven Docker image](https://hub.docker.com/_/maven/),
which contains all of the tools and dependencies needed to manage the project
and to run the jobs.
Environment variables are set to instruct Maven to use the `homedir` of the repository instead of the user's home when searching for configuration and dependencies.
Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process
by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary.
Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application.
Deploy to Artifactory is done as defined by the variables we have just set up.
The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published.
Done! Now you have all the changes in the GitLab repository, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening.
If the deployment has been successful, the deploy job log will output:
```plaintext
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.983 s
```
>**Note**:
the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it.
Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repository.
## Create the main Maven application
Now that you have the dependency available on Artifactory, it's time to use it!
Let's see how we can have it as a dependency to our main application.
### Prepare the main application
We'll use again a Maven app that can be cloned from our example project:
1. Create a new project by selecting **Import project from ➔ Repo by URL**
1. Add the following URL:
```plaintext
https://gitlab.com/gitlab-examples/maven/simple-maven-app.git
```
1. Click **Create project**
This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java`
file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter.
Since Maven doesn't know how to resolve the dependency, you need to modify the configuration:
1. Go back to Artifactory
1. Browse the `libs-release-local` repository
1. Select the `simple-maven-dep-1.0.jar` file
1. Find the configuration snippet from the **Dependency Declaration** section of the main panel
1. Copy the snippet in the `dependencies` section of the `pom.xml` file.
The snippet should look like this:
```xml
<dependency>
<groupId>com.example.dep</groupId>
<artifactId>simple-maven-dep</artifactId>
<version>1.0</version>
</dependency>
```
### Configure the Artifactory repository location
At this point you defined the dependency for the application, but you still miss where you can find the required files.
You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables.
Here is how you can get the content of the file directly from Artifactory:
1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel
1. Click on **Generate Maven Settings**
1. Click on **Generate Settings**
1. Copy to clipboard the configuration file
1. Save the file as `.m2/settings.xml` in your repository
Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application!
### Configure GitLab CI/CD for `simple-maven-app`
You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`.
You want to leverage [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and run your awesome application,
and see if you can get the greeting as expected!
All you need to do is to add the following `.gitlab-ci.yml` to the repository:
```yaml
image: maven:latest
stages:
- build
- test
- run
variables:
MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
paths:
- .m2/repository/
- target/
build:
stage: build
script:
- mvn $MAVEN_CLI_OPTS compile
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS test
run:
stage: run
script:
- mvn $MAVEN_CLI_OPTS package
- mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App"
```
It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job.
Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically.
And that's it! In the `run` job output log you will find a friendly hello to GitLab!
## Conclusion
In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts.
A similar approach could be used to interact with any other Maven compatible Binary Repository Manager.
Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow.
doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md
View file @ ffffd1c4
--- ---
stage: Release redirect_to: '../README.md#contributed-examples'
group: Release
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
author: Dylan Griffith
author_gitlab: DylanGriffith
type: tutorial
date: 2018-06-07
description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD"
--- ---
<!-- vale off --> This document was moved to [another location](../README.md#contributed-examples).
# Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD <!-- This redirect file can be deleted after 2021-04-18. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
## Introduction
This article demonstrates how to use the [Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment)
method to deploy a [Spring Boot](https://projects.spring.io/spring-boot/) application to
[Cloud Foundry (CF)](https://www.cloudfoundry.org/)
with GitLab CI/CD.
All the code for this project can be found in this [GitLab
repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo).
In case you're interested in deploying Spring Boot applications to Kubernetes
using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/blog/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/).
## Requirements
This tutorial assumes you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD.
To follow along, you need:
- An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any
other Cloud Foundry (CF) instance.
- An account on GitLab.
NOTE:
If you're not deploying to PWS, you must replace the `api.run.pivotal.io` URL in all the below
commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html)
of your CF instance.
## Create your project
To create your Spring Boot application you can use the Spring template in
GitLab when creating a new project:
![New Project From Template](img/create_from_template.png)
## Configure the deployment to Cloud Foundry
To deploy to Cloud Foundry you must add a `manifest.yml` file. This
is the configuration for the CF CLI you must use to deploy the application.
Create this in the root directory of your project with the following
content:
```yaml
---
applications:
- name: gitlab-hello-world
random-route: true
memory: 1G
path: target/demo-0.0.1-SNAPSHOT.jar
```
## Configure GitLab CI/CD to deploy your application
Now you must add the GitLab CI/CD configuration file
([`.gitlab-ci.yml`](../../yaml/README.md))
to your project's root. This is how GitLab figures out what commands must run whenever
code is pushed to your repository. Add the following `.gitlab-ci.yml`
file to the root directory of the repository. GitLab detects it
automatically and runs the defined steps once you push your code:
```yaml
image: java:8
stages:
- build
- deploy
before_script:
- chmod +x mvnw
build:
stage: build
script: ./mvnw package
artifacts:
paths:
- target/demo-0.0.1-SNAPSHOT.jar
production:
stage: deploy
script:
- curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx
- ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io
- ./cf push
only:
- master
```
This uses the `java:8` [Docker image](../../docker/using_docker_images.md)
to build your application, as it provides the up-to-date Java 8 JDK on [Docker Hub](https://hub.docker.com/).
You also added the [`only` clause](../../yaml/README.md#onlyexcept-basic)
to ensure your deployments only happen when you push to the master branch.
Because the steps defined in `.gitlab-ci.yml` require credentials to sign in to
CF, you must add your CF credentials as
[environment variables](../../variables/README.md#predefined-environment-variables)
in GitLab CI/CD. To set the environment variables, navigate to your project's
**Settings > CI/CD**, and then expand **Variables**. Name the variables
`CF_USERNAME` and `CF_PASSWORD` and set them to the correct values.
![Variable Settings in GitLab](img/cloud_foundry_variables.png)
After set up, GitLab CI/CD deploys your app to CF at every push to your
repository's default branch. To review the build logs or watch your builds
running live, navigate to **CI/CD > Pipelines**.
WARNING:
It's considered best practice for security to create a separate deploy user for
your application and add its credentials to GitLab instead of using a
developer's credentials.
To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click
**Run Pipeline**. After the app is finished deploying, it displays the
URL of your application in the logs for the `production` job:
```shell
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: gitlab-hello-world-undissembling-hotchpot.cfapps.io
last uploaded: Mon Nov 6 10:02:25 UTC 2017
stack: cflinuxfs2
buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1...
state since cpu memory disk details
#0 running 2017-11-06 09:03:22 PM 120.4% 291.9M of 1G 137.6M of 1G
```
You can then visit your deployed application (for this example,
`https://gitlab-hello-world-undissembling-hotchpot.cfapps.io/`) and you should
see the "Spring is here!" message.
doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md
View file @ ffffd1c4
--- ---
stage: Verify redirect_to: '../README.md#ci-cd-examples'
group: Continuous Integration
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
author: Ryan Hall
author_gitlab: blitzgren
type: tutorial
date: 2018-03-07
--- ---
<!-- vale off --> This document was moved to [another location](../README.md#contributed-examples).
# DevOps and Game Dev with GitLab CI/CD <!-- This redirect file can be deleted after 2021-04-19. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
With advances in WebGL and WebSockets, browsers are extremely viable as game development
platforms without the use of plugins like Adobe Flash. Furthermore, by using GitLab and [AWS](https://aws.amazon.com/),
single game developers, as well as game dev teams, can easily host browser-based games online.
In this tutorial, we'll focus on DevOps, as well as testing and hosting games with Continuous
Integration/Deployment methods using [GitLab CI/CD](../../README.md). We assume you are familiar with GitLab, JavaScript,
and the basics of game development.
## The game
Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction.
Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](https://www.darknova.io),
was essential for the fast pace the team worked at. This tutorial will build upon my
[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps:
1. Using code from the previous article to start with a bare-bones [Phaser](https://phaser.io) game built by a gulp file
1. Adding and running unit tests
1. Creating a `Weapon` class that can be triggered to spawn a `Bullet` in a given direction
1. Adding a `Player` class that uses this weapon and moves around the screen
1. Adding the sprites we will use for the `Player` and `Weapon`
1. Testing and deploying with Continuous Integration and Continuous Deployment methods
By the end, we'll have the core of a [playable game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/)
that's tested and deployed on every push to the `master` branch of the [codebase](https://gitlab.com/blitzgren/gitlab-game-demo).
This will also provide
boilerplate code for starting a browser-based game with the following components:
- Written in [TypeScript](https://www.typescriptlang.org/) and [PhaserJs](https://phaser.io)
- Building, running, and testing with [Gulp](https://gulpjs.com)
- Unit tests with [Chai](https://www.chaijs.com) and [Mocha](https://mochajs.org/)
- CI/CD with GitLab
- Hosting the codebase on GitLab.com
- Hosting the game on AWS
- Deploying to AWS
## Requirements and setup
Please refer to my previous article [DevOps and Game Dev](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) to learn the foundational
development tools, running a Hello World-like game, and building this game using GitLab
CI/CD from every new push to master. The `master` branch for this game's [repository](https://gitlab.com/blitzgren/gitlab-game-demo)
contains a completed version with all configurations. If you would like to follow along
with this article, you can clone and work from the `devops-article` branch:
```shell
git clone git@gitlab.com:blitzgren/gitlab-game-demo.git
git checkout devops-article
```
Next, we'll create a small subset of tests that exemplify most of the states I expect
this `Weapon` class to go through. To get started, create a folder called `lib/tests`
and add the following code to a new file `weaponTests.ts`:
```typescript
import { expect } from 'chai';
import { Weapon, BulletFactory } from '../lib/weapon';
describe('Weapon', () => {
var subject: Weapon;
var shotsFired: number = 0;
// Mocked bullet factory
var bulletFactory: BulletFactory = <BulletFactory>{
generate: function(px, py, vx, vy, rot) {
shotsFired++;
}
};
var parent: any = { x: 0, y: 0 };
beforeEach(() => {
shotsFired = 0;
subject = new Weapon(bulletFactory, parent, 0.25, 1);
});
it('should shoot if not in cooldown', () => {
subject.trigger(true);
subject.update(0.1);
expect(shotsFired).to.equal(1);
});
it('should not shoot during cooldown', () => {
subject.trigger(true);
subject.update(0.1);
subject.update(0.1);
expect(shotsFired).to.equal(1);
});
it('should shoot after cooldown ends', () => {
subject.trigger(true);
subject.update(0.1);
subject.update(0.3); // longer than timeout
expect(shotsFired).to.equal(2);
});
it('should not shoot if not triggered', () => {
subject.update(0.1);
subject.update(0.1);
expect(shotsFired).to.equal(0);
});
});
```
To build and run these tests using gulp, let's also add the following gulp functions
to the existing `gulpfile.js` file:
```typescript
gulp.task('build-test', function () {
return gulp.src('src/tests/**/*.ts', { read: false })
.pipe(tap(function (file) {
// replace file contents with browserify's bundle stream
file.contents = browserify(file.path, { debug: true })
.plugin(tsify, { project: "./tsconfig.test.json" })
.bundle();
}))
.pipe(buffer())
.pipe(sourcemaps.init({loadMaps: true}) )
.pipe(gulp.dest('built/tests'));
});
gulp.task('run-test', function() {
gulp.src(['./built/tests/**/*.ts']).pipe(mocha());
});
```
We will start implementing the first part of our game and get these `Weapon` tests to pass.
The `Weapon` class will expose a method to trigger the generation of a bullet at a given
direction and speed. Later we will implement a `Player` class that ties together the user input
to trigger the weapon. In the `src/lib` folder create a `weapon.ts` file. We'll add two classes
to it: `Weapon` and `BulletFactory` which will encapsulate Phaser's **sprite** and
**group** objects, and the logic specific to our game.
```typescript
export class Weapon {
private isTriggered: boolean = false;
private currentTimer: number = 0;
constructor(private bulletFactory: BulletFactory, private parent: Phaser.Sprite, private cooldown: number, private bulletSpeed: number) {
}
public trigger(on: boolean): void {
this.isTriggered = on;
}
public update(delta: number): void {
this.currentTimer -= delta;
if (this.isTriggered && this.currentTimer <= 0) {
this.shoot();
}
}
private shoot(): void {
// Reset timer
this.currentTimer = this.cooldown;
// Get velocity direction from player rotation
var parentRotation = this.parent.rotation + Math.PI / 2;
var velx = Math.cos(parentRotation);
var vely = Math.sin(parentRotation);
// Apply a small forward offset so bullet shoots from head of ship instead of the middle
var posx = this.parent.x - velx * 10
var posy = this.parent.y - vely * 10;
this.bulletFactory.generate(posx, posy, -velx * this.bulletSpeed, -vely * this.bulletSpeed, this.parent.rotation);
}
}
export class BulletFactory {
constructor(private bullets: Phaser.Group, private poolSize: number) {
// Set all the defaults for this BulletFactory's bullet object
this.bullets.enableBody = true;
this.bullets.physicsBodyType = Phaser.Physics.ARCADE;
this.bullets.createMultiple(30, 'bullet');
this.bullets.setAll('anchor.x', 0.5);
this.bullets.setAll('anchor.y', 0.5);
this.bullets.setAll('outOfBoundsKill', true);
this.bullets.setAll('checkWorldBounds', true);
}
public generate(posx: number, posy: number, velx: number, vely: number, rot: number): Phaser.Sprite {
// Pull a bullet from Phaser's Group pool
var bullet = this.bullets.getFirstExists(false);
// Set the few unique properties about this bullet: rotation, position, and velocity
if (bullet) {
bullet.reset(posx, posy);
bullet.rotation = rot;
bullet.body.velocity.x = velx;
bullet.body.velocity.y = vely;
}
return bullet;
}
}
```
Lastly, we'll redo our entry point, `game.ts`, to tie together both `Player` and `Weapon` objects
as well as add them to the update loop. Here is what the updated `game.ts` file looks like:
```typescript
import { Player } from "./player";
import { Weapon, BulletFactory } from "./weapon";
window.onload = function() {
var game = new Phaser.Game(800, 600, Phaser.AUTO, 'gameCanvas', { preload: preload, create: create, update: update });
var player: Player;
var weapon: Weapon;
// Import all assets prior to loading the game
function preload () {
game.load.image('player', 'assets/player.png');
game.load.image('bullet', 'assets/bullet.png');
}
// Create all entities in the game, after Phaser loads
function create () {
// Create and position the player
var playerSprite = game.add.sprite(400, 550, 'player');
playerSprite.anchor.setTo(0.5);
player = new Player(game.input, playerSprite, 150);
var bulletFactory = new BulletFactory(game.add.group(), 30);
weapon = new Weapon(bulletFactory, player.sprite, 0.25, 1000);
player.loadWeapon(weapon);
}
// This function is called once every tick, default is 60fps
function update() {
var deltaSeconds = game.time.elapsedMS / 1000; // convert to seconds
player.update(deltaSeconds);
weapon.update(deltaSeconds);
}
}
```
Run `gulp serve` and you can run around and shoot. Wonderful! Let's update our CI
pipeline to include running the tests along with the existing build job.
## Continuous Integration
To ensure our changes don't break the build and all tests still pass, we use
Continuous Integration (CI) to run these checks automatically for every push.
Read through this article to understand [Continuous Integration, Continuous Delivery, and Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/),
and how these methods are leveraged by GitLab.
From the [last tutorial](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) we already have a `.gitlab-ci.yml` file set up for building our app from
every push. We need to set up a new CI job for testing, which GitLab CI/CD will run after the build job using our generated artifacts from gulp.
Please read through the [documentation on CI/CD configuration](../../../ci/yaml/README.md) file to explore its contents and adjust it to your needs.
### Build your game with GitLab CI/CD
We need to update our build job to ensure tests get run as well. Add `gulp build-test`
to the end of the `script` array for the existing `build` job. After these commands run,
we know we will need to access everything in the `built` folder, given by GitLab CI/CD's `artifacts`.
We'll also cache `node_modules` to avoid having to do a full re-pull of those dependencies:
just pack them up in the cache. Here is the full `build` job:
```yaml
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
```
### Test your game with GitLab CI/CD
For testing locally, we simply run `gulp run-tests`, which requires gulp to be installed
globally like in the `build` job. We pull `node_modules` from the cache, so the `npm i`
command won't have to do much. In preparation for deployment, we know we will still need
the `built` folder in the artifacts, which will be brought over as default behavior from
the previous job. Lastly, by convention, we let GitLab CI/CD know this needs to be run after
the `build` job by giving it a `test` [stage](../../../ci/yaml/README.md#stages).
Following the YAML structure, the `test` job should look like this:
```yaml
test:
stage: test
script:
- npm i gulp -g
- npm i
- gulp run-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
```
We have added unit tests for a `Weapon` class that shoots on a specified interval.
The `Player` class implements `Weapon` along with the ability to move around and shoot. Also,
we've added test artifacts and a test stage to our GitLab CI/CD pipeline using `.gitlab-ci.yml`,
allowing us to run our tests by every push.
Our entire `.gitlab-ci.yml` file should now look like this:
```yaml
image: node:10
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
test:
stage: test
script:
- npm i gulp -g
- npm i
- gulp run-test
cache:
policy: pull
paths:
- node_modules/
artifacts:
paths:
- built/
```
### Run your CI/CD pipeline
That's it! Add all your new files, commit, and push. For a reference of what our repository should
look like at this point, please refer to the [final commit related to this article on my sample repository](https://gitlab.com/blitzgren/gitlab-game-demo/commit/8b36ef0ecebcf569aeb251be4ee13743337fcfe2).
By applying both build and test stages, GitLab will run them sequentially at every push to
our repository. If all goes well you'll end up with a green check mark on each job for the pipeline:
![Passing Pipeline](img/test_pipeline_pass.png)
You can confirm that the tests passed by clicking on the `test` job to enter the full build logs.
Scroll to the bottom and observe, in all its passing glory:
```shell
$ gulp run-test
[18:37:24] Using gulpfile /builds/blitzgren/gitlab-game-demo/gulpfile.js
[18:37:24] Starting 'run-test'...
[18:37:24] Finished 'run-test' after 21 ms
Weapon
✓ should shoot if not in cooldown
✓ should not shoot during cooldown
✓ should shoot after cooldown ends
✓ should not shoot if not triggered
4 passing (18ms)
Uploading artifacts...
built/: found 17 matching files
Uploading artifacts to coordinator... ok id=17095874 responseStatus=201 Created token=aaaaaaaa Job succeeded
```
## Continuous Deployment
We have our codebase built and tested on every push. To complete the full pipeline with Continuous Deployment,
let's set up [free web hosting with AWS S3](https://aws.amazon.com/free/) and a job through which our build artifacts get
deployed. GitLab also has a free static site hosting service we can use, [GitLab Pages](https://about.gitlab.com/stages-devops-lifecycle/pages/),
however Dark Nova specifically uses other AWS tools that necessitates using `AWS S3`.
Read through this article that describes [deploying to both S3 and GitLab Pages](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/)
and further delves into the principles of GitLab CI/CD than discussed in this article.
### Set up S3 Bucket
1. Log into your AWS account and go to [S3](https://console.aws.amazon.com/s3/home)
1. Click the **Create Bucket** link at the top
1. Enter a name of your choosing and click next
1. Keep the default **Properties** and click next
1. Click the **Manage group permissions** and allow **Read** for the **Everyone** group, click next
1. Create the bucket, and select it in your S3 bucket list
1. On the right side, click **Properties** and enable the **Static website hosting** category
1. Update the radio button to the **Use this bucket to host a website** selection. Fill in `index.html` and `error.html` respectively
### Set up AWS Secrets
We need to be able to deploy to AWS with our AWS account credentials, but we certainly
don't want to put secrets into source code. Luckily GitLab provides a solution for this
with [Variables](../../../ci/variables/README.md). This can get complicated
due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't
use root security credentials. Proper IAM credential management is beyond the scope of this
article, but AWS will remind you that using root credentials is unadvised and against their
best practices, as they should. Feel free to follow best practices and use a custom IAM user's
credentials, which will be the same two credentials (Key ID and Secret). It's a good idea to
fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html). We need to add these credentials to GitLab:
1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential)
1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the ID and secret around, you'll need them later
![AWS Access Key Configuration](img/aws_config_window.png)
1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar
1. Expand the **Variables** section
![GitLab Secret Configuration](img/gitlab_config.png)
1. Add a key named `AWS_KEY_ID` and copy the key ID from Step 2 into the **Value** field
1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** field
### Deploy your game with GitLab CI/CD
To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on
the shared runner. The shared runner also needs to be able to authenticate with your AWS
account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID`
and `AWS_SECRET_ACCESS_KEY`. GitLab CI/CD gives us a way to pass the variables we
set up in the prior section using the `variables` portion of the `deploy` job. At the end,
we add directives to ensure deployment `only` happens on pushes to `master`. This way, every
single branch still runs through CI, and only merging (or committing directly) to master will
trigger the `deploy` job of our pipeline. Put these together to get the following:
```yaml
deploy:
stage: deploy
variables:
AWS_ACCESS_KEY_ID: "$AWS_KEY_ID"
AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET"
script:
- apt-get update
- apt-get install -y python3-dev python3-pip
- easy_install3 -U pip
- pip3 install --upgrade awscli
- aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete
only:
- master
```
Be sure to update the region and S3 URL in that last script command to fit your setup.
Our final configuration file `.gitlab-ci.yml` looks like:
```yaml
image: node:10
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
test:
stage: test
script:
- npm i gulp -g
- gulp run-test
cache:
policy: pull
paths:
- node_modules/
artifacts:
paths:
- built/
deploy:
stage: deploy
variables:
AWS_ACCESS_KEY_ID: "$AWS_KEY_ID"
AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET"
script:
- apt-get update
- apt-get install -y python3-dev python3-pip
- easy_install3 -U pip
- pip3 install --upgrade awscli
- aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete
only:
- master
```
## Conclusion
Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get
[TypeScript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](https://gulpjs.com/) and [Phaser](https://phaser.io) all playing
together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](https://www.darknova.io).
Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation,
and unit tests, all running and deployed at every push to master - with shockingly little code.
Errors can be easily debugged through GitLab build logs, and within minutes of a successful commit,
you can see the changes live on your game.
Setting up Continuous Integration and Continuous Deployment from the start with Dark Nova enables
rapid but stable development. We can easily test changes in a separate [environment](../../environments/index.md),
or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing
and tedious, but having faith in a stable deployment with GitLab CI/CD allows
a lot of breathing room in quickly getting changes to players.
## Further settings
Here are some ideas to further investigate that can speed up or improve your pipeline:
- [Yarn](https://yarnpkg.com) instead of npm
- Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) image that can pre-load dependencies and tools (like AWS CLI)
- Forward a [custom domain](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website
- Combine jobs if you find it unnecessary for a small project
- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/blog/2016/03/01/gitlab-runner-with-docker/)
doc/ci/examples/test-phoenix-application.md
View file @ ffffd1c4
...@@ -3,3 +3,6 @@ redirect_to: '../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md' ...@@ -3,3 +3,6 @@ redirect_to: '../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md'
--- ---
The content of this page was incorporated in [this document](../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md). The content of this page was incorporated in [this document](../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md).
<!-- This redirect file can be deleted after February 1, 2021. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
doc/ci/examples/test-scala-application.md
View file @ ffffd1c4
--- ---
stage: Verify redirect_to: '../README.md#contributed-examples'
group: Continuous Integration
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
type: tutorial
--- ---
# Test and deploy a Scala application to Heroku This document was moved to [another location](../README.md#contributed-examples).
This example demonstrates the integration of GitLab CI/CD with Scala <!-- This redirect file can be deleted after 2021-04-18. -->
applications using SBT. You can view or fork the [example project](https://gitlab.com/gitlab-examples/scala-sbt) <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
and view the logs of its past [CI jobs](https://gitlab.com/gitlab-examples/scala-sbt/-/jobs?scope=finished).
## Add `.gitlab-ci.yml` file to project
The following `.gitlab-ci.yml` should be added in the root of your
repository to trigger CI:
``` yaml
image: openjdk:8
stages:
- test
- deploy
before_script:
- apt-get update -y
- apt-get install apt-transport-https -y
## Install SBT
- echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
- apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823
- apt-get update -y
- apt-get install sbt -y
- sbt sbtVersion
test:
stage: test
script:
- sbt clean coverage test coverageReport
deploy:
stage: deploy
script:
- apt-get update -yq
- apt-get install rubygems ruby-dev -y
- gem install dpl
- dpl --provider=heroku --app=gitlab-play-sample-app --api-key=$HEROKU_API_KEY
```
In the above configuration:
- The `before_script` installs [SBT](https://www.scala-sbt.org/) and
displays the version that is being used.
- The `test` stage executes SBT to compile and test the project.
- [sbt-scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT
plugin to measure test coverage.
- The `deploy` stage automatically deploys the project to Heroku using dpl.
You can use other versions of Scala and SBT by defining them in
`build.sbt`.
## Display test coverage in job
Add the `Coverage was \[\d+.\d+\%\]` regular expression in the
**Settings > Pipelines > Coverage report** project setting to
retrieve the [test coverage](../pipelines/settings.md#test-coverage-report-badge)
rate from the build trace and have it displayed with your jobs.
**Pipelines** must be enabled for this option to appear.
## Heroku application
A Heroku application is required. You can create one through the
[Dashboard](https://dashboard.heroku.com/). Substitute `gitlab-play-sample-app`
in the `.gitlab-ci.yml` file with your application's name.
## Heroku API key
You can look up your Heroku API key in your
[account](https://dashboard.heroku.com/account). Add a [protected variable](../variables/README.md#protect-a-custom-variable) with
this value in **Project ➔ Variables** with key `HEROKU_API_KEY`.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7