Understanding 'At Risk' indicators
Explore wide range of 'At Risk' indicators that PerfectScale provides
Last updated
Explore wide range of 'At Risk' indicators that PerfectScale provides
Last updated
Out-of-Memory events usually occur in the following situations:
The memory limit for a pod is set too low. An event will be triggered when the memory usage of the pod reaches a defined limit.
CPU Throttling occurs when the pod reaches its defined CPU limit and could create latency in application response.
Frequent restarts indicate the presence of a problem with a high potential of harming the desired SLA.
Eviction indicates forcefully terminating and removing a running pod from a node. Eviction events usually occur due to memory or CPU pressure on a node.
As demand for a service or application increases, HPA will scale the system to handle the additional load by dynamically adding more replicas. Once the maximum configured limit of replicas is reached, PerfectScale will raise the HPAAtMaxReplicasObserved indicator, which means the system cannot scale further based on the existing settings.
Depending on a workload's running time at maximum replicas, the severity of the indicator will vary. For example, the longer a workload runs at maximum replicas, the higher the severity indicator.
Setting proper CPU requests helps the Kubernetes scheduler to allocate the right amount of CPU for each container, making sure that the cluster's nodes capacity meets the demand.
Setting proper MEMORY requests helps the Kubernetes scheduler to allocate the right amount of memory for each container, making sure that the cluster's nodes capacity meets the demand.
Setting proper MEMORY limit helps to protect your worker node from OOM, preventing the risk of memory over-allocation.
Setting proper CPU requests helps the Kubernetes scheduler to allocate the right amount of CPU for each container, making sure that the cluster's nodes capacity meets the demand.
Setting proper MEMORY requests helps the Kubernetes scheduler to allocate the right amount of memory for each container, making sure that the cluster's nodes capacity meets the demand.
Setting proper MEMORY limit helps to protect your worker node from OOM, preventing the risk of memory over-allocation. However, under-provisioned MEMORY limit could cause unwanted OOM events on a pod level, potentially harming the desired SLA.
Setting proper CPU requests helps the Kubernetes scheduler allocate the right amount of CPU for each container, ensuring that the cluster's nodes capacity meets the demand. In cases of over-provisioned CPU requests, cloud resources are unnecessarily wasted due to allocation without utilization.
Setting proper MEMORY requests helps the Kubernetes scheduler allocate the right amount of memory for each container, ensuring that the cluster's nodes capacity meets the demand. However, when a memory request is over-provisioned, it wastes cloud resources, which are allocated but never used.
Node is experiencing memory pressure and tries to evict some pods.
Kubernetes use to implement the limit. The quota is based on the time period and not based on available CPU power. cfs_period_us is used to define the time period, it’s always 100000us (100ms).
For example, a container with 1 core limit will be throttled after 50ms when running on 2 cores node and after 25ms when running on 4 cores node regardless of the number of consumed CPU cores.
When eviction is observed, an will be triggered immediately to inform the users. Make sure that you have configured and assigned the to the cluster to receive timely notifications on or channel.
