Automation customization

Learn how to customize your Automation setup

All restrictions are compared to the original spec values of the workload

Customize your automation with the following settings:

timeConstraints:
          wasteMaxAutomationFrequency: "30m" # Applies recommendations to decrease resources every 30 minutes
          fixResiliencyMaxAutomationFrequency: "30m" # Applies recommendations to increase resources every 30 minutes
          maintenanceWindow:
            monday:
              - "00:00-23:59"
            tuesday:
              - "00:00-23:59"
            wednesday:
              - "00:00-23:59"
            thursday:
              - "00:00-23:59"
            friday:
              - "00:00-23:59"
            saturday:
              - "00:00-23:59"
            sunday:
              - "00:00-23:59"
          maintenanceWindowIgnoredForResiliency: false # deprecated, use "maintenanceWindowIgnoredMinResiliencyLevel" instead
          maintenanceWindowIgnoredMinResiliencyLevel: None # Indicates the minimum resiliency risk level required to bypass maintenance window constraints and resolve the issue 
restrictions:
    workloadMinWasteUSDPerMonth: 5 # Activates automation only if monthly waste exceeds a set threshold (for example, $5)
    cpuManagement:
      request:
        increaseEnabled: true/false   # default false
        decreaseEnabled: true/false   # default true
        minimumCores: 0.5             # default unset, to override to unset, use -1
        maximumCores: 8               # default unset, to override to unset, use -1
      limit:
        keepLimit: true/false         # default true
    memoryManagement:
      request:
        increaseEnabled: true/false  # default false
        decreaseEnabled: true/false   # default true
        minimumGib: 1                  # default unset, to override to unset, use -1
        maximumGib: 16                 # default unset, to override to unset, use -1
      limit:
        increaseEnabled: true/false  # default false
        decreaseEnabled: true/false  # default false
        minimumGib: 2                  # default unset, to override to unset, use -1
        maximumGib: 31                 # default unset, to override to unset, use -1
        memoryLeakDetection:
          maxMemoryIncreaseIterations:
            daily: 4                  # default 3
            weekly: 7                 # default 7

Time constraints

spec:
  automation:
    operational:
      timeConstraints:
        wasteMaxAutomationFrequency: "30m" # Applies recommendations to decrease resources every 30 minutes
        fixResiliencyMaxAutomationFrequency: "30m" # Applies recommendations to increase resources every 30 minutes
        maintenanceWindow:
          sunday: [00:00-23:59]
        maintenanceWindowIgnoredForResiliency: false # deprecated, use "maintenanceWindowIgnoredMinResiliencyLevel" instead
        maintenanceWindowIgnoredMinResiliencyLevel: None # Indicates the minimum resiliency risk level required to bypass maintenance window constraints and resolve the issue

🗑️ wasteMaxAutomationFrequency

This optional field defines the maximum frequency at which automated actions can be executed to apply recommendations when waste is detected, and resource reduction is needed.

If no wasteMaxAutomationFrequency is specified, PerfectScale defaults to values based on the type of workload:

Deployment: "30m" Rollout: "30m" DaemonSet: "4h" StatefulSet: "24h" CronJob: "30m" Job: "30m"

🚨 fixResiliencyMaxAutomationFrequency

This optional field defines the maximum frequency at which automated actions can be executed to apply recommendations when resiliency risks are detected, and resource increase is needed.

If no fixResiliencyMaxAutomationFrequency is specified, PerfectScale defaults to values based on the type of workload:

Deployment: "30m" Rollout: "30m" DaemonSet: "4h" StatefulSet: "24h" CronJob: "30m" Job: "30m"

🕙 Maintenance window

The maintenance window defines the timeframe for PerfectScale to implement the recommendations.

The maintenance window is particularly helpful in minimizing the impact of changes on business operations and provides a structured approach to system updates and changes.

Time is indicated in UTC.

Example 1

The automation actions are scheduled daily between 00:00 and 23:59 UTC:

timeConstraints:
          maintenanceWindow:
            monday:
              - "00:00-23:59"
            tuesday:
              - "00:00-23:59"
            wednesday:
              - "00:00-23:59"
            thursday:
              - "00:00-23:59"
            friday:
              - "00:00-23:59"
            saturday:
              - "00:00-23:59"
            sunday:
              - "00:00-23:59"

Example 2

Excluding a specific day from the automation maintenanceWindow.

timeConstraints:
          maintenanceWindow:
            monday:
              - "00:00-00:00"
            tuesday:
              - "00:00-00:00"
            wednesday:
              - "00:00-00:00"
            thursday:
              - "00:00-00:00"
            friday:
              - "00:00-00:00"

To exclude a specific day(s) from an automation maintenanceWindow, set the timeframe to 00:00-00:00 (e.g., Monday - Friday in the example above).

In the example above, the maintenanceWindow does not specify a timeframe for Saturday and Sunday. As a result, these days default to a 00:00-23:59 window.

Example 3

The automation actions are scheduled for the end of one day and the beginning of the next.

timeConstraints:
          maintenanceWindow:
            monday:
            - "00:00-02:30"
            - "23:30-23:59"
          tuesday:
            - "00:00-02:30"
            - "23:30-23:59"
          wednesday:
            - "00:00-02:30"
            - "23:30-23:59"
          thursday:
            - "00:00-02:30"
            - "23:30-23:59"
          friday:
            - "00:00-02:30"
            - "23:30-23:59"
          saturday:
            - "00:00-02:30"
            - "23:30-23:59"
          sunday:
            - "00:00-02:30"
            - "23:30-23:59"

🙅 maintenanceWindowIgnoredMinResiliencyLevel

In some cases, you may want Automation to override the maintenance window to ensure that critical resiliency issues, such as CPU throttling or OOM occurring outside the maintenance window, are properly addressed by an immediate resource increase.

spec:
  automation:
    operational:
      timeConstraints:
        maintenanceWindowIgnoredMinResiliencyLevel: None

This field indicates the minimum resiliency risk level required to bypass maintenance window constraints and resolve the issue. When set to None (default), the maintenance window constraints are enforced. However, when one of the resiliency levels is selected (Low, Medium, High), PerfectScale Automation will ignore maintenance window constraints for risks with the indicated resiliency level and above, applying changes to resolve the issue.

Example 1

The maintenance window will be ignored for risks at any level.

maintenanceWindowIgnoredMinResiliencyLevel: Low

Example 2

The maintenance window will be ignored for risks with Medium and High risk levels

maintenanceWindowIgnoredMinResiliencyLevel: Medium

Example 3

The maintenance window will be ignored for High risks only.

maintenanceWindowIgnoredMinResiliencyLevel: High

💡cpuManagement: keepLimit

In general, we recommend removing CPU limits when possible. Doing so can improve the performance and efficiency of your workloads by allowing them to use available CPU resources more freely.

However, be aware that removing misconfigured CPU limits can increase memory consumption in many cases. This requires adjusting both memory requests and limits to ensure optimal performance and efficient use of resources.

🔄 maxMemoryIncreaseIterations

maxMemoryIncreaseIterations sets the maximum allowed number of daily and weekly memory limit increase iterations.

max_memory_increase_iterations is a mandatory field. Make sure it exists and doesn't equal to 0.

Once the number of daily or weekly memory increase iterations reaches the maximum value, to prevent potential memory leaks, PerfectScale Automation will stop recursive memory increase.

In order to notify the customer, PerfectScale will raise a relevant indicator and create an Alert.

Last updated