api_styleguide.md 12.5 KB
Newer Older
Amy Qualls's avatar
Amy Qualls committed
1 2 3 4 5 6
---
stage: none
group: unassigned
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
---

7
# API style guide
Rémy Coutable's avatar
Rémy Coutable committed
8

9
This style guide recommends best practices for API development.
Rémy Coutable's avatar
Rémy Coutable committed
10

11 12 13 14 15 16 17
## Instance variables

Please do not use instance variables, there is no need for them (we don't need
to access them as we do in Rails views), local variables are fine.

## Entities

18
Always use an [Entity](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/api/entities) to present the endpoint's payload.
19

20 21
## Documentation

22
Each new or updated API endpoint must come with documentation, unless it is internal or behind a feature flag.
23
The docs should be in the same merge request, or, if strictly necessary,
24
in a follow-up with the same milestone as the original merge request.
25

26
See the [Documentation Style Guide RESTful API page](documentation/restful_api_styleguide.md) for details on documenting API resources in Markdown as well as in OpenAPI definition files.
27

28 29 30
## Methods and parameters description

Every method must be described using the [Grape DSL](https://github.com/ruby-grape/grape#describing-methods)
31
(see <https://gitlab.com/gitlab-org/gitlab/blob/master/lib/api/environments.rb>
32 33 34 35
for a good example):

- `desc` for the method summary. You should pass it a block for additional
  details such as:
36
  - The GitLab version when the endpoint was added. If it is behind a feature flag, mention that instead: _This feature is gated by the :feature\_flag\_symbol feature flag._
Rémy Coutable's avatar
Rémy Coutable committed
37
  - If the endpoint is deprecated, and if so, when will it be removed
38

39
- `params` for the method parameters. This acts as description,
40
  [validation, and coercion of the parameters](https://github.com/ruby-grape/grape#parameter-validation-and-coercion)
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

A good example is as follows:

```ruby
desc 'Get all broadcast messages' do
  detail 'This feature was introduced in GitLab 8.12.'
  success Entities::BroadcastMessage
end
params do
  optional :page,     type: Integer, desc: 'Current page number'
  optional :per_page, type: Integer, desc: 'Number of messages per page'
end
get do
  messages = BroadcastMessage.all

  present paginate(messages), with: Entities::BroadcastMessage
end
```

60
## Declared parameters
Rémy Coutable's avatar
Rémy Coutable committed
61 62

> Grape allows you to access only the parameters that have been declared by your
63
`params` block. It filters out the parameters that have been passed, but are not
Rémy Coutable's avatar
Rémy Coutable committed
64 65
allowed.

66
<https://github.com/ruby-grape/grape#declared>
Rémy Coutable's avatar
Rémy Coutable committed
67

68
### Exclude parameters from parent namespaces
Rémy Coutable's avatar
Rémy Coutable committed
69

70
> By default `declared(params)`includes parameters that were defined in all
Rémy Coutable's avatar
Rémy Coutable committed
71 72
parent namespaces.

73
<https://github.com/ruby-grape/grape#include-parent-namespaces>
Rémy Coutable's avatar
Rémy Coutable committed
74

75
In most cases you will want to exclude parameters from the parent namespaces:
Rémy Coutable's avatar
Rémy Coutable committed
76 77 78 79 80

```ruby
declared(params, include_parent_namespaces: false)
```

81
### When to use `declared(params)`
Rémy Coutable's avatar
Rémy Coutable committed
82

83
You should always use `declared(params)` when you pass the parameters hash as
Rémy Coutable's avatar
Rémy Coutable committed
84 85 86 87 88 89 90 91 92 93 94 95
arguments to a method call.

For instance:

```ruby
# bad
User.create(params) # imagine the user submitted `admin=1`... :)

# good
User.create(declared(params, include_parent_namespaces: false).to_h)
```

96
NOTE: **Note:**
Rémy Coutable's avatar
Rémy Coutable committed
97 98 99
`declared(params)` return a `Hashie::Mash` object, on which you will have to
call `.to_h`.

Rémy Coutable's avatar
Rémy Coutable committed
100
But we can use `params[key]` directly when we access single elements.
Rémy Coutable's avatar
Rémy Coutable committed
101 102 103 104 105 106 107 108

For instance:

```ruby
# good
Model.create(foo: params[:foo])
```

Stan Hu's avatar
Stan Hu committed
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148
## Array types

With Grape v1.3+, Array types must be defined with a `coerce_with`
block, or parameters will fail to validate when passed a string from an
API request. See the [Grape upgrading
documentation](https://github.com/ruby-grape/grape/blob/master/UPGRADING.md#ensure-that-array-types-have-explicit-coercions)
for more details.

### Automatic coercion of nil inputs

Prior to Grape v1.3.3, Array parameters with `nil` values would
automatically be coerced to an empty Array. However, due to [this pull
request in v1.3.3](https://github.com/ruby-grape/grape/pull/2040), this
is no longer the case. For example, suppose you define a PUT `/test`
request that has an optional parameter:

```ruby
optional :user_ids, type: Array[Integer], coerce_with: ::API::Validations::Types::CommaSeparatedToIntegerArray.coerce, desc: 'The user ids for this rule'
```

Normally, a request to PUT `/test?user_ids` would cause Grape to pass
`params` of `{ user_ids: nil }`.

This may introduce errors with endpoints that expect a blank array and
do not handle `nil` inputs properly. To preserve the previous behavior,
there is a helper method `coerce_nil_params_to_array!` that is used
in the `before` block of all API calls:

```ruby
before do
  coerce_nil_params_to_array!
end
```

With this change, a request to PUT `/test?user_ids` will cause Grape to
pass `params` to be `{ user_ids: [] }`.

There is [an open issue in the Grape tracker](https://github.com/ruby-grape/grape/issues/2068)
to make this easier.

149 150
## Using HTTP status helpers

151
For non-200 HTTP responses, use the provided helpers in `lib/api/helpers.rb` to ensure correct behavior (`not_found!`, `no_content!` etc.). These will `throw` inside Grape and abort the execution of your endpoint.
152 153 154

For `DELETE` requests, you should also generally use the `destroy_conditionally!` helper which by default returns a `204 No Content` response on success, or a `412 Precondition Failed` response if the given `If-Unmodified-Since` header is out of range. This helper calls `#destroy` on the passed resource, but you can also implement a custom deletion method by passing a block.

155 156
## Using API path helpers in GitLab Rails codebase

157
Because we support [installing GitLab under a relative URL](../install/relative_url.md), one must take this
158 159 160 161 162 163 164 165 166
into account when using API path helpers generated by Grape. Any such API path
helper usage must be in wrapped into the `expose_path` helper call.

For instance:

```haml
- endpoint = expose_path(api_v4_projects_issues_related_merge_requests_path(id: @project.id, issue_iid: @issue.iid))
```

167 168 169 170
## Custom Validators

In order to validate some parameters in the API request, we validate them
before sending them further (say Gitaly). The following are the
Amy Qualls's avatar
Amy Qualls committed
171
[custom validators](https://GitLab.com/gitlab-org/gitlab/-/tree/master/lib/api/validations/validators),
172 173 174 175 176 177 178 179 180 181 182 183
which we have added so far and how to use them. We also wrote a
guide on how you can add a new custom validator.

### Using custom validators

- `FilePath`:

  GitLab supports various functionalities where we need to traverse a file path.
  The [`FilePath` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/file_path.rb)
  validates the parameter value for different cases. Mainly, it checks whether a
  path is relative and does it contain `../../` relative traversal using
  `File::Separator` or not, and whether the path is absolute, for example
184
  `/etc/passwd/`. By default, absolute paths are not allowed. However, you can optionally pass in an allowlist for allowed absolute paths in the following way:
185
  `requires :file_path, type: String, file_path: { allowlist: ['/foo/bar/', '/home/foo/', '/app/home'] }`
186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209

- `Git SHA`:

  The [`Git SHA` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/git_sha.rb)
  checks whether the Git SHA parameter is a valid SHA.
  It checks by using the regex mentioned in [`commit.rb`](https://gitlab.com/gitlab-org/gitlab/-/commit/b9857d8b662a2dbbf54f46ecdcecb44702affe55#d1c10892daedb4d4dd3d4b12b6d071091eea83df_30_30) file.

- `Absence`:

  The [`Absence` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/absence.rb)
  checks whether a particular parameter is absent in a given parameters hash.

- `IntegerNoneAny`:

  The [`IntegerNoneAny` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/integer_none_any.rb)
  checks if the value of the given parameter is either an `Integer`, `None`, or `Any`.
  It allows only either of these mentioned values to move forward in the request.

- `ArrayNoneAny`:

  The [`ArrayNoneAny` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/array_none_any.rb)
  checks if the value of the given parameter is either an `Array`, `None`, or `Any`.
  It allows only either of these mentioned values to move forward in the request.

210 211 212 213 214 215
- `EmailOrEmailList`:

  The [`EmailOrEmailList` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/email_or_email_list.rb)
  checks if the value of a string or a list of strings contains only valid
  email addresses. It allows only lists with all valid email addresses to move forward in the request.

216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
### Adding a new custom validator

Custom validators are a great way to validate parameters before sending
them to platform for further processing. It saves some back-and-forth
from the server to the platform if we identify invalid parameters at the beginning.

If you need to add a custom validator, it would be added to
it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators) directory.
Since we use [Grape](https://github.com/ruby-grape/grape) to add our API
we inherit from the `Grape::Validations::Base` class in our validator class.
Now, all you have to do is define the `validate_param!` method which takes
in two parameters: the `params` hash and the `param` name to validate.

The body of the method does the hard work of validating the parameter value
and returns appropriate error messages to the caller method.

Lastly, we register the validator using the line below:

234
```ruby
235 236 237 238 239 240
Grape::Validations.register_validator(<validator name as symbol>, ::API::Helpers::CustomValidators::<YourCustomValidatorClassName>)
```

Once you add the validator, make sure you add the `rspec`s for it into
it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/api/validations/validators) directory.

241 242 243 244 245
## Internal API

The [internal API](./internal_api.md) is documented for internal use. Please keep it up to date so we know what endpoints
different components are making use of.

246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263
## Avoiding N+1 problems

In order to avoid N+1 problems that are common when returning collections
of records in an API endpoint, we should use eager loading.

A standard way to do this within the API is for models to implement a
scope called `with_api_entity_associations` that will preload the
associations and data returned in the API. An example of this scope can
be seen in
[the `Issue` model](https://gitlab.com/gitlab-org/gitlab/blob/2fedc47b97837ea08c3016cf2fb773a0300a4a25/app%2Fmodels%2Fissue.rb#L62).

In situations where the same model has multiple entities in the API
(for instance, `UserBasic`, `User` and `UserPublic`) you should use your
discretion with applying this scope. It may be that you optimize for the
most basic entity, with successive entities building upon that scope.

The `with_api_entity_associations` scope will also [automatically preload
data](https://gitlab.com/gitlab-org/gitlab/blob/19f74903240e209736c7668132e6a5a735954e7c/app%2Fmodels%2Ftodo.rb#L34)
264
for `Todo` _targets_ when returned in the [to-dos API](../api/todos.md).
265 266

For more context and discussion about preloading see
267
[this merge request](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/25711)
268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297
which introduced the scope.

### Verifying with tests

When an API endpoint returns collections, always add a test to verify
that the API endpoint does not have an N+1 problem, now and in the future.
We can do this using [`ActiveRecord::QueryRecorder`](query_recorder.md).

Example:

```ruby
def make_api_request
  get api('/foo', personal_access_token: pat)
end

it 'avoids N+1 queries', :request_store do
  # Firstly, record how many PostgreSQL queries the endpoint will make
  # when it returns a single record
  create_record

  control = ActiveRecord::QueryRecorder.new { make_api_request }

  # Now create a second record and ensure that the API does not execute
  # any more queries than before
  create_record

  expect { make_api_request }.not_to exceed_query_limit(control)
end
```

298 299 300 301 302 303 304
## Testing

When writing tests for new API endpoints, consider using a schema [fixture](./testing_guide/best_practices.md#fixtures) located in `/spec/fixtures/api/schemas`. You can `expect` a response to match a given schema:

```ruby
expect(response).to match_response_schema('merge_requests')
```
305 306

Also see [verifying N+1 performance](#verifying-with-tests) in tests.
307 308 309 310 311

## Include a changelog entry

All client-facing changes **must** include a [changelog entry](changelog.md).
This does not include internal APIs.