Merge branch '208717-remove-ff-for-bulk-indexer' into 'master'

Remove feature flag elastic_bulk_incremental_updates

See merge request gitlab-org/gitlab!27293

doc/development/elasticsearch.md

@@ -36,11 +36,7 @@ Additionally, if you need large repos or multiple forks for testing, please cons
 
 The Elasticsearch integration depends on an external indexer. We ship an [indexer written in Go](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer). The user must trigger the initial indexing via a rake task but, after this is done, GitLab itself will trigger reindexing when required via `after_` callbacks on create, update, and destroy that are inherited from [/ee/app/models/concerns/elastic/application_versioned_search.rb](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/models/concerns/elastic/application_versioned_search.rb).
 
-After initial indexing is complete, updates proceed in one of two ways, depending on the `:elastic_bulk_incremental_updates` feature flag.
-
-If disabled, every create, update, or delete operation on an Elasticsearch-tracked model enqueues a new `ElasticIndexerWorker` Sidekiq job which takes care of updating just that document. This is quite inefficient.
-
-If the feature flag is enabled, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/topics/data-types#sorted-sets) instead. A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html).
+After initial indexing is complete, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/topics/data-types#sorted-sets). A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html).
 
 Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them!
 

ee/app/models/concerns/elastic/application_versioned_search.rb

@@ -45,36 +45,15 @@ module Elastic
     end
 
     def maintain_elasticsearch_create
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(:index, self.class.to_s, self.id, self.es_id)
+      ::Elastic::ProcessBookkeepingService.track!(self)
     end
 
     def maintain_elasticsearch_update
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(
-        :update,
-        self.class.to_s,
-        self.id,
-        self.es_id
-      )
+      ::Elastic::ProcessBookkeepingService.track!(self)
     end
 
     def maintain_elasticsearch_destroy
-      return if maintain_elasticsearch_incremental_bulk
-
-      ElasticIndexerWorker.perform_async(
-        :delete, self.class.to_s, self.id, self.es_id, es_parent: self.es_parent
-      )
-    end
-
-    def maintain_elasticsearch_incremental_bulk
-      return false unless Feature.enabled?(:elastic_bulk_incremental_updates, self.project)
-
       ::Elastic::ProcessBookkeepingService.track!(self)
-
-      true
     end
 
     class_methods do

ee/app/models/concerns/elastic/projects_search.rb

@@ -19,12 +19,20 @@ module Elastic
         ::Gitlab::CurrentSettings.elasticsearch_indexes_project?(self)
       end
 
-      def maintain_elasticsearch_incremental_bulk
       # TODO: ElasticIndexerWorker does extra work for project hooks, so we
       # can't use the incremental-bulk indexer for projects yet.
       #
       # https://gitlab.com/gitlab-org/gitlab/issues/207494
-        false
+      def maintain_elasticsearch_create
+        ElasticIndexerWorker.perform_async(:index, self.class.to_s, self.id, self.es_id)
+      end
+
+      def maintain_elasticsearch_update
+        ElasticIndexerWorker.perform_async(:update, self.class.to_s, self.id, self.es_id)
+      end
+
+      def maintain_elasticsearch_destroy
+        ElasticIndexerWorker.perform_async(:delete, self.class.to_s, self.id, self.es_id, es_parent: self.es_parent)
       end
 
       def each_indexed_association

ee/changelogs/unreleased/208717-remove-ff-for-bulk-indexer.yml

+---
+title: Remove feature flag elastic_bulk_incremental_updates
+merge_request: 27293
+author:
+type: performance

ee/spec/models/concerns/elastic/note_spec.rb

@@ -106,21 +106,7 @@ describe Note, :elastic do
     expect(note.__elasticsearch__.as_indexed_json).to eq(expected_hash)
   end
 
-  it "does not create ElasticIndexerWorker job for system messages" do
-    stub_feature_flags(elastic_bulk_incremental_updates: false)
-
-    project = create :project, :repository
-    # We have to set one minute delay because of https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/15682
-    issue = create :issue, project: project, updated_at: 1.minute.ago
-
-    # Only issue should be updated
-    expect(ElasticIndexerWorker).to receive(:perform_async).with(:update, 'Issue', anything, anything)
-    create :note, :system, project: project, noteable: issue
-  end
-
-  it 'does not track system note updates via the bulk updater' do
-    stub_feature_flags(elastic_bulk_incremental_updates: true)
-
+  it 'does not track system note updates' do
     note = create(:note, :system)
 
     expect(Elastic::ProcessBookkeepingService).not_to receive(:track!)