Track usage data for Network Policies UI

Uses a redis HLL counter to track the number of clusters visiting the
network policies page with network policies present in the cluster.
Since any ultimate customer can visit the page, we require policies to
be present in the response as an extra signal for the feature having
provided value to the user.

See https://gitlab.com/gitlab-org/gitlab/-/issues/334090

Changelog: added
MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64731
EE: true

doc/development/usage_ping/dictionary.md

@@ -11712,7 +11712,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_assigned_monthly`
 
-Count of an alert's assignments per month
+Count of unique users assigning an alert per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180533_incident_management_alert_assigned_monthly.yml)
 
@@ -11724,7 +11724,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_assigned_weekly`
 
-Count of an alert's assignments per week
+Count of unique users assigning an alert per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180532_incident_management_alert_assigned_weekly.yml)
 
@@ -11736,7 +11736,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_status_changed_monthly`
 
-Count of an alert's status changes per month
+Count of unique users changing alert's status changes per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180530_incident_management_alert_status_changed_monthly.yml)
 
@@ -11748,7 +11748,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_status_changed_weekly`
 
-Count of an alert's status changes per week
+Count of unique users changing alert's status per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180528_incident_management_alert_status_changed_weekly.yml)
 
@@ -11760,7 +11760,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_todo_monthly`
 
-Count of alerts added to the TODO list per month
+Count of unique users adding alerts to the TODO list per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180537_incident_management_alert_todo_monthly.yml)
 
@@ -11772,7 +11772,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_alert_todo_weekly`
 
-Count of alerts added to the TODO list per week
+Count of unique users adding alerts to the TODO list per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180535_incident_management_alert_todo_weekly.yml)
 
@@ -11784,7 +11784,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_assigned_monthly`
 
-Count of an incidents's assignments per month
+Count of users assigning incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180552_incident_management_incident_assigned_monthly.yml)
 
@@ -11796,7 +11796,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_assigned_weekly`
 
-Count of an incidents's assignments per week
+Count of unique users assiging incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180550_incident_management_incident_assigned_weekly.yml)
 
@@ -11808,7 +11808,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_change_confidential_monthly`
 
-Count of incidents changed to confidential per month
+Count of users changing incidents to confidential per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180618_incident_management_incident_change_confidential_monthly.yml)
 
@@ -11820,7 +11820,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_change_confidential_weekly`
 
-Count of incidents changed to confidential per week
+Count of unique users changing incidents to confidential per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180616_incident_management_incident_change_confidential_weekly.yml)
 
@@ -11832,7 +11832,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_closed_monthly`
 
-Count of incidents closed per month
+Count of users closing incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180548_incident_management_incident_closed_monthly.yml)
 
@@ -11844,7 +11844,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_closed_weekly`
 
-Count of incidents closed per week
+Count of users closing incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180546_incident_management_incident_closed_weekly.yml)
 
@@ -11856,7 +11856,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_comment_monthly`
 
-Count of comments per month on incidents
+Count of unique users adding comments per month on incidents
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180559_incident_management_incident_comment_monthly.yml)
 
@@ -11868,7 +11868,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_comment_weekly`
 
-Count of comments per week on incidents
+Count of unique users adding comments on incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180558_incident_management_incident_comment_weekly.yml)
 
@@ -11880,7 +11880,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_created_monthly`
 
-Count of incidents created per month
+Count of unique users creating incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180541_incident_management_incident_created_monthly.yml)
 
@@ -11892,7 +11892,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_created_weekly`
 
-Count of incidents created per week
+Count of unique users creating incidents  per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180539_incident_management_incident_created_weekly.yml)
 
@@ -11904,7 +11904,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_published_monthly`
 
-Count of published incidents per month
+Count of unique users that published incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/counts_28d/20210216180607_incident_management_incident_published_monthly.yml)
 
@@ -11916,7 +11916,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_published_weekly`
 
-Count of published incidents per week
+Count of unique users that published incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/counts_7d/20210216180605_incident_management_incident_published_weekly.yml)
 
@@ -11928,7 +11928,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_relate_monthly`
 
-Count of issues added per month that are related to an incident
+Count of unique users adding issues per month that are related to an incident
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180611_incident_management_incident_relate_monthly.yml)
 
@@ -11940,7 +11940,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_relate_weekly`
 
-Count of issues added per week that are related to an incident
+Count of unique users adding issues per that are related to an incident week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180609_incident_management_incident_relate_weekly.yml)
 
@@ -11952,7 +11952,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_reopened_monthly`
 
-Count of incidents reopened per month
+Count of unique users reopening incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180545_incident_management_incident_reopened_monthly.yml)
 
@@ -11964,7 +11964,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_reopened_weekly`
 
-Count of incidents reopened per week
+Count of unique users reopening incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180543_incident_management_incident_reopened_weekly.yml)
 
@@ -11976,7 +11976,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_todo_monthly`
 
-Count of incidents added to the TODO list per month
+Count of unique users adding incidents to the TODO list per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180556_incident_management_incident_todo_monthly.yml)
 
@@ -11988,7 +11988,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_todo_weekly`
 
-Count of incidents added to the TODO list per week
+Count of unique users adding incidents to the TODO list per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180554_incident_management_incident_todo_weekly.yml)
 
@@ -12000,7 +12000,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_unrelate_monthly`
 
-Count of issues removed per month that are related to an incident
+Count of users removing issues that are related to an incident per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180614_incident_management_incident_unrelate_monthly.yml)
 
@@ -12012,7 +12012,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_unrelate_weekly`
 
-Count of issues removed per week that are related to an incident
+Count of unique users removing issue that are related to an incident per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180612_incident_management_incident_unrelate_weekly.yml)
 
@@ -12024,7 +12024,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_zoom_meeting_monthly`
 
-Count of Zoom meetings about incidents per month
+Count of users creating Zoom meetings about incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180603_incident_management_incident_zoom_meeting_monthly.yml)
 
@@ -12036,7 +12036,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_incident_zoom_meeting_weekly`
 
-Count of Zoom meetings about incidents per week
+Count of unique users creating Zoom meetings about incidents per week
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180601_incident_management_incident_zoom_meeting_weekly.yml)
 
@@ -12048,7 +12048,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_total_unique_counts_monthly`
 
-Count of total unique incidents per month
+Count of unique users performing events related with incidents per month
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216180622_incident_management_total_unique_counts_monthly.yml)
 
@@ -12060,7 +12060,7 @@ Tiers: `free`, `premium`, `ultimate`
 
 ### `redis_hll_counters.incident_management.incident_management_total_unique_counts_weekly`
 
-Count of total unique incidents per week
+Count of unique users performing events related to the incident management
 
 [YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216180620_incident_management_total_unique_counts_weekly.yml)
 
@@ -12934,6 +12934,30 @@ Status: `data_available`
 
 Tiers: `free`, `premium`, `ultimate`
 
+### `redis_hll_counters.network_policies.clusters_using_network_policies_ui_monthly`
+
+Monthly number of distinct clusters with network policies using the network policies UI
+
+[YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/counts_28d/20210623202402_clusters_using_network_policies_ui_monthly.yml)
+
+Group: `group::container security`
+
+Status: `implemented`
+
+Tiers: `ultimate`
+
+### `redis_hll_counters.network_policies.clusters_using_network_policies_ui_weekly`
+
+Weekly number of distinct clusters with network policies using the network policies UI
+
+[YAML definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/counts_7d/20210623202358_clusters_using_network_policies_ui_weekly.yml)
+
+Group: `group::container security`
+
+Status: `implemented`
+
+Tiers: `ultimate`
+
 ### `redis_hll_counters.pipeline_authoring.o_pipeline_authoring_unique_users_committing_ciconfigfile_monthly`
 
 Monthly unique user count doing commits which contains the CI config file

ee/app/services/network_policies/resources_service.rb

@@ -3,6 +3,7 @@
 module NetworkPolicies
   class ResourcesService
     include NetworkPolicies::Responses
+    include Gitlab::Utils::UsageData
 
     LIMIT = 100
 
@@ -26,6 +27,12 @@ module NetworkPolicies
 
     private
 
+    def track_usage_data_for_cluster(platform, policies)
+      return if policies.empty?
+
+      track_usage_event(:clusters_using_network_policies_ui, platform.cluster_id)
+    end
+
     def execute_per_environment(platform, namespace)
       policies = platform.kubeclient
         .get_network_policies(namespace: namespace)
@@ -33,6 +40,7 @@ module NetworkPolicies
       policies += platform.kubeclient
         .get_cilium_network_policies(namespace: namespace)
         .map { |resource| Gitlab::Kubernetes::CiliumNetworkPolicy.from_resource(resource) }
+      track_usage_data_for_cluster(platform, policies)
       [policies, nil]
     rescue Kubeclient::HttpError => e
       [policies, e]

ee/config/metrics/counts_28d/20210623202402_clusters_using_network_policies_ui_monthly.yml

+---
+key_path: redis_hll_counters.network_policies.clusters_using_network_policies_ui_monthly
+description: Monthly number of distinct clusters with network policies using the network policies UI
+product_section: sec
+product_stage: protect
+product_group: group::container security
+product_category: container_network_security
+value_type: number
+status: implemented
+milestone: "14.1"
+introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64731
+time_frame: 28d
+data_source: redis_hll
+data_category: Operational
+distribution:
+- ee
+tier:
+- ultimate

ee/config/metrics/counts_7d/20210623202358_clusters_using_network_policies_ui_weekly.yml

+---
+key_path: redis_hll_counters.network_policies.clusters_using_network_policies_ui_weekly
+description: Weekly number of distinct clusters with network policies using the network policies UI
+product_section: sec
+product_stage: protect
+product_group: group::container security
+product_category: container_network_security
+value_type: number
+status: implemented
+milestone: "14.1"
+introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64731
+time_frame: 7d
+data_source: redis_hll
+data_category: Operational
+distribution:
+- ee
+tier:
+- ultimate

ee/spec/services/network_policies/resources_service_spec.rb

@@ -9,7 +9,7 @@ RSpec.describe NetworkPolicies::ResourcesService do
   let(:project) { create(:project) }
   let(:cluster) { create(:cluster, :instance) }
   let!(:cluster_kubernetes_namespace) { create(:cluster_kubernetes_namespace, project: project, cluster: cluster, environment: environment, namespace: 'namespace') }
-  let(:platform) { double('Clusters::Platforms::Kubernetes', kubeclient: kubeclient) }
+  let(:platform) { double('Clusters::Platforms::Kubernetes', kubeclient: kubeclient, cluster_id: cluster.id) }
   let(:kubeclient) { double('Kubeclient::Client') }
   let(:policy) do
     Gitlab::Kubernetes::NetworkPolicy.new(
@@ -46,6 +46,23 @@ RSpec.describe NetworkPolicies::ResourcesService do
       expect(subject.payload.last.as_json).to eq(cilium_policy.as_json)
     end
 
+    it_behaves_like 'tracking unique hll events' do
+      subject(:request) { service.execute }
+
+      let(:target_id) { 'clusters_using_network_policies_ui' }
+      let(:expected_type) { instance_of(Integer) }
+
+      before do
+        allow(kubeclient).to receive(:get_network_policies)
+          .with(namespace: cluster_kubernetes_namespace.namespace)
+          .and_return [policy.generate]
+
+        allow(kubeclient).to receive(:get_cilium_network_policies)
+          .with(namespace: cluster_kubernetes_namespace.namespace)
+          .and_return [cilium_policy.generate]
+      end
+    end
+
     context 'without deployment_platform' do
       let(:platform) { nil }
 

lib/gitlab/usage_data_counters/known_events/common.yml

@@ -369,3 +369,8 @@
   category: testing
   aggregation: weekly
   feature_flag: users_expanding_widgets_usage_data
+# Container Security - Network Policies
+- name: clusters_using_network_policies_ui
+  redis_slot: network_policies
+  category: network_policies
+  aggregation: weekly

spec/lib/gitlab/usage_data_counters/hll_redis_counter_spec.rb

@@ -46,7 +46,8 @@ RSpec.describe Gitlab::UsageDataCounters::HLLRedisCounter, :clean_gitlab_redis_s
         'pipeline_authoring',
         'epics_usage',
         'epic_boards_usage',
-        'secure'
+        'secure',
+        'network_policies'
       )
     end
   end

spec/lib/gitlab/usage_data_spec.rb

@@ -1269,7 +1269,7 @@ RSpec.describe Gitlab::UsageData, :aggregate_failures do
 
     let(:categories) { ::Gitlab::UsageDataCounters::HLLRedisCounter.categories }
     let(:ineligible_total_categories) do
-      %w[source_code ci_secrets_management incident_management_alerts snippets terraform incident_management_oncall secure]
+      %w[source_code ci_secrets_management incident_management_alerts snippets terraform incident_management_oncall secure network_policies]
     end
 
     context 'with redis_hll_tracking feature enabled' do