Grouping

The PerfectScale grouping configuration stands as a powerful tool, facilitating the aggregation of data pertaining to redundant workloads exhibiting similar patterns. By preventing overload through batches of identical information, it enables you to focus on what matters, thereby streamlining analysis processes.

Custom grouping is an aggregation rule that aggregates multiple workloads into a single entity (for example, all GitLab runners are aggregated into a single entity—the GitLab workload), merging their data (resource usage, limits, requests, etc.).

This feature is especially convenient when using Spark, GitLab, or any other operators that produce short-living small workloads that often reflect just a single pod in the cluster.

🛠️ How to create Grouping Profile

There are two options for creating a Profile: from the Settings tab or directly from the Overview.

From the Settings tab

Go to the Settings tab on the left panel -> select Grouping -> click the +Add Profile button -> name the profile and put the needed values in the relevant fields -> click Save button.

From the Overview tab

Go to the Overview tab on the left panel -> find the cluster to which you want to apply the Grouping Profile and click gear button -> go to Customizations -> click on Add New Profile in the Custom Grouping Profile drop-down list -> name and configure your profile -> click the Save And Apply button -> click the Save Changes button.

Grouping Profile configuration

custom_grouping:
  - name: spark-driver
    filter:
      priority: 10
      owner_type: SparkApplication
      namespace: default
      label_exists: label_app
      pod_regex: 'spark-job-.*'
      namespace_regex: "namespace1-.*"
      containers_exist: "prometheus-server"
    target_workload:
      workload_name:
        static_value: 'grouped-spark-job'
      workload_type:
        static_value: GroupedSparkApplication
    container_grouping:
      - name: ps-container
        container_filter:
          container_regex: ps-.*
        target_container:
          container_name:
            static_value: grouped-ps-container

Here is the list of configurable grouping fields:

name - the group name of the aggregated workloads

filter - a set of conditions that define to which workloads the grouping will be applied. The workloads that meet the conditions will be assigned to a group:

• priority - (optional) indicates the filter's priority. This is particularly helpful when multiple filters match the workload.

• owner_type - (case sensitive) the equivalent of Type in Podfit.

• namespace - (case sensitive) checks if the workload belongs to the particular namespace

• label_exists - checks if the specific key is attached to the workloads (single key or comma-separated list of keys). 💡NOTE: label keys should be provided in the Prometheus format. Example: Kubernetes: app.kubernetes.io/instance Prometheus: label_app_kubernetes_io_instance

• pod_regex - a regular expression identifies the workload by its name (for example: loki-.* searches for the workloads with names start with loki- and then have at least one more symbol).

• namespace_regex - a regular expression identifies the namespace by its name. 💡NOTE: namespace_regex should be paired with some other filter condition.

• containers_exists - checks if workload holds the list of the containers described in this filter.

target_workload - a meta workload represents the group of workloads matched by the filter.

• workload_name - (mandatory) the name of the grouped workload to which workloads will be grouped.

• workload_type - (mandatory) the type of the grouped workload to which workloads will be grouped.

• workload_namespace - (optional) the namespace of the grouped workload to which workloads will be grouped.

container_grouping - an array of profiles for container grouping for the workload Custom Grouping entry. This is particularly helpful when the container names are auto-generated.

• container_filter - a set of conditions that define to which containers the grouping will be applied (supports only regexes).

  • container_regex - a regular expression identifies the container by its name.

• target_container - the container in which containers will be grouped by the entry.

Grouping of short-living workloads

You can seamlessly group short-lived workloads into a single entity using a grouping profile, which helps eliminate unnecessary clutter. This feature is particularly helpful when managing many short-lived workloads, allowing you to focus on the most important aspects. By grouping them, you can highlight the most relevant data, helping you prioritize actions and optimize your cluster more effectively.

To group short-living workloads, use the following configuration:

short_living_grouping:
  - name: default
    filter:
      namespace: ''
      label_exists: ''
      owner_type: ''
      pod_regex: ''
      namespace_regex: ''
      priority: 0
      containers_exist: ''
      type: ''
      workload_type: ''
    older_than: 240h0m0s
    running_time_less_than: 10m0s
    target_workload:
      workload_name:
        label_value: ''
        static_value: __grouped__
        labels_template: ''
      workload_type:
        label_value: ''
        static_value: __grouped__
        labels_template: ''
      workload_namespace:
        label_value: ''
        static_value: __grouped__
        labels_template: ''
custom_grouping: []

Custom Grouping Preview

The Custom Grouping Preview feature allows you to quickly evaluate how the cluster will look after re-aggregation with the applied profile.

Click the Custom Grouping Preview button to navigate directly to the preview screen.

1. Select the cluster that you want to group with the Grouping Profile. Get an estimated number of workloads, namespaces, and clusters after the grouping is applied.

2. Choose to which data, whether aggregated or raw, you want to apply the Grouping profile along with the desired timeframe.

3. Filter workloads by column values in the workloads table and view the number of workloads in each group.

Auto-Generated Profiles

PerfectScale can auto-generate the Grouping profile with a single click.

To auto-generate a Grouping Profile, go to the Overview tab on the left panel -> find the cluster to which you want to apply the Grouping Profile and click three dots button -> select Cluster Settings -> go to Customizations -> click Auto-Generate Profile-> name the profile and click Save button.

🛠️ How to apply Grouping Profile

Apply to a single cluster

To apply Grouping Profile to the cluster, go to the Overview tab on the left panel -> find the cluster to which you want to apply the Grouping Profile and click gear button -> go to Customizations -> select the needed profile in the Custom Grouping Profile drop-down list.

Apply to multiple clusters

To apply the profile to multiple clusters from a single view, use the Manage Assignments feature.

Go to the Settings tab on the left panel -> select the Grouping -> click the Manage Assignments button -> apply the profiles for the needed clusters -> click the Save Changes button.