PerfectScale Prometheus Exporter is a powerful feature that converts PerfectScale's optimization recommendations, cost insights, and resource utilization metrics into Prometheus format. Exposing these insights as Prometheus metrics enables you to seamlessly incorporate PerfectScale's detailed analysis into your standard monitoring and alerting workflows, enhancing visibility into cost-efficiency, resource usage across your infrastructure, and overall system performance.
To expose insights from PerfectScale as Prometheus metrics, a Kubernetes cluster with Helm v3 or higher and the PerfectScale agent installed and running is required.
Installing the Chart
You can seamlessly install the chart in two simple steps:
The Psc-Prom-Exporter must be installed in the same namespace as the PerfectScale Agent to reuse the existing credentials secret.
This will install Prometheus exporter with basic configuration, without manually configured scrapers, dashboards, and alerts. To configure it according to your needs, please follow the documentation. or and sections.
The general behavior of the Prom-Exporter is defined in the config section of the values file. Monitoring tools integration is configured in the scrapers, alerts, and dashboards sections of the values file.
Quick install:
Prom-Exporter with Prometheus Operator with Alerts example and Grafana dashboard
recommendations allows managing the exposure of resource recommendation metrics and, when enabled, exposes the following metrics:
ps_recommended_memory_request_bytes
ps_recommended_cpu_request_cores
ps_recommended_memory_limit_bytes
ps_recommended_cpu_limit_cores
ps_current_memory_request_bytes
ps_current_cpu_request_cores
ps_current_memory_limit_bytes
ps_current_cpu_limit_cores
costs
costs allows managing the exposure of cost analysis metrics and, when enabled, exposes the following metrics:
ps_cost_usd - hourly workload cos in USD
ps_waste_usd - estimated hourly waste in USD
indicators
indicators allows managing the exposure of resource utilization indicators and, when enabled, exposes the ps_workload_indicators metric with labels for different indicator types.
Filters configuration
filters:
workloads:
includeMuted: true # Include muted workloads
minRunningMinutes: 30 # Minimum runtime to include
types: # Workload types to include
- "*" # All types
# Or specify specific types:
# - "Deployment"
# - "StatefulSet"
# - "DaemonSet"
namespaces:
exclude: # Namespaces to exclude from metrics
- kube-system
- default
indicators:
types: # Types of indicators to expose
- waste # Resource waste indicators
- risk # Risk indicators
workloads
workloads allows managing which workload should be included in the metric based on its type, run-time, etc.
namespaces
namespaces allows managing which namespace should be excluded from the metric.
indicators
indicators allows managing indicators to expose
Labels configuration
The labels section allows managing Kubernetes labels to be included in the metrics.
The excludedLabels take precedence over includedLabels.
labels:
includedLabels:
- "*" # Include all labels
# Or specify specific labels:
# - "label_app"
# - "label_environment"
excludedLabels:
- "/.*perfectscale.*/" # Exclude labels matching regex
# - "label_internal_id" # Exclude specific label
excludeClusterUID: false # Include cluster UID in metrics
In the Prometheus format, all special label characters are replaced with an underscore _.
Example:
Kubernetes: app.kubernetes.io/instance
Prometheus: label_app_kubernetes_io_instance
Label pattern support
Pattern
Description
"label_full_name"
Matches the exact label name
"*"
Matches all labels
"/some-regexp/"
Matches labels with the specific regular expression
Configuration examples
Here are a few configuration examples designed to help you effortlessly integrate PerfectScale's insights into your monitoring workflow.
Recommended configuration for production environments with short-living workloads excluded
The PerfectScale Prometheus Exporter is compatible with a range of out-of-the-box monitoring systems to enhance your monitoring process and make it more efficient. This support allows seamless integration, enabling you to quickly leverage PerfectScale's insights without requiring extensive configuration.
Prometheus Operator integration
This integration will create a Service Monitor Custom Resource to scrape Prom-exporter with Prometheus Operator.
This integration allows you to pull metrics directly into Datadog. It supports both v1 and v2 autodiscovery annotations.
scrapers:
datadog:
enabled: true
# Choose AD version: "v1" or "v2"
adVersion: "v2"
containerName: "psc-prom-exporter"
# Common configuration for both versions
config:
endpoint: "/metrics"
port: 8080
namespace: "perfectscale"
# Set maxReturnedMetrics, DataDog default value is 2000. Increase if needed.
maxReturnedMetrics: 10000
# List of metrics to collect with optional renaming
metrics:
# Wildcard pattern for ps_ metrics
- name: "ps_.*"
# Renaming exampple:
#- name: "ps_waste_usd"
#rename: "perfectscale.waste.usd"
# Additional configuration options
options: {}
Dashboards
PerfectScale provides a pre-built Grafana dashboard that offers valuable insights. These dashboards allow you to visualize key metrics and enhance your monitoring experience with minimal configuration required.
Single Workload Dashboard
This dashboard offers a comprehensive view of your application's performance. It showcases detailed workload metrics and a clear overview of how resources are utilized, enabling you to monitor efficiency trends and identify optimization opportunities. It includes the following data:
You can effortlessly set up custom alert rules in the prometheusRule Helm values configuration section. This will help you stay ahead of key indicator changes, allowing you to address potential issues proactively and prevent them from impacting the system.
Configuring alerts
Cost optimization alert
There is an example of a cost optimization alert designed to help you proactively manage expenses and enhance resource efficiency for Prometheus and DataDog.
rules:
- alert: "PerfectScale Waste Cost Surge"
enabled: true
expr: |
(
ps_waste_usd > 0
and
(
ps_waste_usd
/
(ps_waste_usd offset 1h)
) > 1.5
)
for: 15m
labels:
severity: warning
team: cost-optimization
type: waste
annotations:
summary: "Waste cost increased by more than 50%"
description: "Workload {{ $labels.workload_name }} in namespace {{ $labels.namespace }} has increased its waste by more than 50% in the last hour. Current waste: {{ $value | humanizePercentage }} USD/hour"
- alert: "PerfectScale High Absolute Waste"
enabled: true
expr: |
ps_waste_usd > 100
for: 30m
labels:
severity: warning
team: cost-optimization
type: waste
annotations:
summary: "High waste cost detected"
description: "Workload {{ $labels.workload_name }} in namespace {{ $labels.namespace }} has waste cost exceeding 100 USD/hour. Current waste: {{ $value | humanize }} USD/hour"
Enabling alerts
To activate the pre-configured alerts for Alert Manager (Prometheus Rule):
alerts:
prometheusRule:
enabled: true
# Additional labels to the PrometheusRule
#labels:
#release: prometheus
