Commit 8924a813 authored by Vladimir Shushlin's avatar Vladimir Shushlin

Add pages_file_entries to plan_limits

Adds file entries limit to pages deployments:
https://gitlab.com/gitlab-org/gitlab/-/issues/320793.

GitLab Pages daemon actually caches index of each archive in memory,
so archives with too many entries can increase the memory usage.

To prevent this, we adding the limit. It also will work for
self-managed,  but we provide a workaround to change it in docs.

Changelog: added
parent 9c1fe8d1
...@@ -157,6 +157,10 @@ module Projects ...@@ -157,6 +157,10 @@ module Projects
entries_count = build.artifacts_metadata_entry("", recursive: true).entries.count entries_count = build.artifacts_metadata_entry("", recursive: true).entries.count
sha256 = build.job_artifacts_archive.file_sha256 sha256 = build.job_artifacts_archive.file_sha256
if pages_file_entries_limit > 0 && entries_count > pages_file_entries_limit
raise InvalidStateError, "pages site contains #{entries_count} file entries, while limit is set to #{pages_file_entries_limit}"
end
deployment = nil deployment = nil
File.open(artifacts_path) do |file| File.open(artifacts_path) do |file|
deployment = project.pages_deployments.create!(file: file, deployment = project.pages_deployments.create!(file: file,
...@@ -198,6 +202,12 @@ module Projects ...@@ -198,6 +202,12 @@ module Projects
max_pages_size max_pages_size
end end
def pages_file_entries_limit
return 0 unless Feature.enabled?(:pages_limit_entries_count, project, default_enabled: :yaml)
project.actual_limits.pages_file_entries
end
def tmp_path def tmp_path
@tmp_path ||= File.join(::Settings.pages.path, TMP_EXTRACT_PATH) @tmp_path ||= File.join(::Settings.pages.path, TMP_EXTRACT_PATH)
end end
......
---
name: pages_limit_entries_count
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64925/diffs
rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/334765
milestone: '14.1'
type: development
group: group::release
default_enabled: false
# frozen_string_literal: true
class AddPagesFileEntriesToPlanLimits < ActiveRecord::Migration[6.1]
def change
add_column(:plan_limits, :pages_file_entries, :integer, default: 100_000, null: false)
end
end
28b31b6e8aba1b8feec2b9a29b5f91f7145431be5d8b9875bddb8183f89700f7
\ No newline at end of file
...@@ -16475,7 +16475,8 @@ CREATE TABLE plan_limits ( ...@@ -16475,7 +16475,8 @@ CREATE TABLE plan_limits (
ci_daily_pipeline_schedule_triggers integer DEFAULT 0 NOT NULL, ci_daily_pipeline_schedule_triggers integer DEFAULT 0 NOT NULL,
ci_max_artifact_size_running_container_scanning integer DEFAULT 0 NOT NULL, ci_max_artifact_size_running_container_scanning integer DEFAULT 0 NOT NULL,
ci_max_artifact_size_cluster_image_scanning integer DEFAULT 0 NOT NULL, ci_max_artifact_size_cluster_image_scanning integer DEFAULT 0 NOT NULL,
ci_jobs_trace_size_limit integer DEFAULT 100 NOT NULL ci_jobs_trace_size_limit integer DEFAULT 100 NOT NULL,
pages_file_entries integer DEFAULT 100000 NOT NULL
); );
CREATE SEQUENCE plan_limits_id_seq CREATE SEQUENCE plan_limits_id_seq
...@@ -457,6 +457,21 @@ installation, run the following in the [GitLab Rails console](operations/rails_c ...@@ -457,6 +457,21 @@ installation, run the following in the [GitLab Rails console](operations/rails_c
Plan.default.actual_limits.update!(ci_max_artifact_size_junit: 10) Plan.default.actual_limits.update!(ci_max_artifact_size_junit: 10)
``` ```
### Number of files per GitLab Pages web-site
The total number of file entries (including directories and symlinks) is limited to `100000` per
GitLab Pages website.
This is the default limit for all [GitLab self-managed and SaaS plans](https://about.gitlab.com/pricing/).
You can update the limit in your self-managed instance using the
[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session).
For example, to change the limit to `100`:
```ruby
Plan.default.actual_limits.update!(pages_file_entries: 100)
```
### Number of registered runners per scope ### Number of registered runners per scope
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321368) in GitLab 13.12. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321368) in GitLab 13.12.
......
...@@ -158,6 +158,21 @@ RSpec.describe Projects::UpdatePagesService do ...@@ -158,6 +158,21 @@ RSpec.describe Projects::UpdatePagesService do
expect(execute).not_to eq(:success) expect(execute).not_to eq(:success)
end end
it 'limits pages file count' do
create(:plan_limits, :default_plan, pages_file_entries: 2)
expect(execute).not_to eq(:success)
expect(GenericCommitStatus.last.description).to eq("pages site contains 3 file entries, while limit is set to 2")
end
it 'does not limit pages file count if feature is disabled' do
stub_feature_flags(pages_limit_entries_count: false)
create(:plan_limits, :default_plan, pages_file_entries: 2)
expect(execute).to eq(:success)
end
it 'removes pages after destroy' do it 'removes pages after destroy' do
expect(PagesWorker).to receive(:perform_in) expect(PagesWorker).to receive(:perform_in)
expect(project.pages_deployed?).to be_falsey expect(project.pages_deployed?).to be_falsey
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment