What Is HPA? Auto-Scaling Pods With HPA and KEDA

What is Auto Scaling Pod?

This process happens when one or more events occur, for example: CPU exceeding 70%, the number of requests to the server exceeding 500 req/s, etc.

What is Auto Scaling for?

Auto Scaling helps an application get more resources, letting it handle more requests from clients, avoiding overload and downtime for the application.

Using Horizontal Pod Autoscale (HPA) in K8s

Horizontal pod autoscale comes built into a K8s cluster and can be used to scale up/down pods for ReplicaSet or Deployment objects.

Prerequisites

Even though HPA comes installed by default in K8s, to get info for monitoring (CPU, Memory) you need to install an additional metrics server at:

github.com/kubernetes-sigs/metrics-server

How does HPA work?

Each HPA has its own condition to check — if that condition (CPU > 70%) is met, it scales up, and when the condition is no longer met, it scales down. The pod count keeps getting scaled by doubling — for example, if there are 3 pods running, it scales to 6, then 12, until the condition is no longer met (CPU < 70%) and scaling stops, or until it hits the pod-count limit that can be created on the node.

The needed pod count gets calculated with the formula:

desiredReplicas = ceil[currentReplicas ( currentMetricValue / desiredMetricValue )]

Or: the needed pod count equals the current pod count multiplied by the ratio of the current condition value to the desired condition value, rounded up.

Example: the current pod count is 3, the scale-up condition threshold is CPU > 50%, and the current condition level is 170%

⇒ 3 x (170/50) = 10.2 rounded up to 11 => the needed pod count is 11

By default, every 15s HPA pulls info from the metric server to compare against the condition — this interval is set by the --horizontal-pod-autoscaler-sync-period parameter.
If the value of currentMetricValue / desiredMetricValue is less than 1 or close to 1 (an insignificant difference), it skips and keeps listening from the metric server — this insignificant value is set by the --horizontal-pod-autoscaler-tolerance parameter.
If the value is less than 1, by default it keeps listening for 300s — if the value is still less than 1, it gradually scales down. This continued-listening duration is set by the --horizontal-pod-autoscaler-downscale-stabilization parameter.

These default parameters can be changed via the behavior field in the yaml file used to create an HPA.

Learn more in the official documentation.

A detailed guide to the behavior field

behavior:
  scaleDown:
    policies:
    - type: Pods
      value: 4
      periodSeconds: 60
    - type: Percent
      value: 10
      periodSeconds: 60

Creating an HPA

Creating via the kubectl command line

kubectl autoscale deployment http-app --cpu-percent=70 --min=1 --max=10

The command above creates an HPA for the http-app Deployment object, with the condition CPU > 70% and a pod count that can scale from 1 to 10 pods.

Creating an HPA via a Yaml file

First we need to create a file auto-scale.yml and save the following content into it

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: http-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: appdeploy
  minReplicas: 1
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

Then run the file with kubectl to create the HPA:

kubectl apply -f auto-scale.yml

Use the following command to check whether the HPA was created successfully

kubectl get hpa

Pros:

Built-in, so it's most optimal for K8s components.
Scales quickly.

Cons:

Only supports a few basic metrics (cpu, ram, etc.).

Using KEDA to autoscale K8s pods

KEDA (keda.sh) stands for Kubernetes-based Event Driven Autoscaler — KEDA helps scale our application based on pre-configured events. KEDA is fairly lightweight and also very easy to install, with just 1-2 commands

How does KEDA work?

KEDA operates as a Metric Server, translating metrics received from an external server into a structure HPA can understand, in order to scale through HPA. So when using KEDA, we don't need to install an additional metrics server from K8s like above.

With KEDA we have a concept called ScaledObject — when you create a ScaleObject, it also automatically creates an HPA to handle scaling pods.

Where does KEDA pull metrics from?

KEDA supports a lot of sources for pulling data to create custom events (MySQL, Prometheus, etc.) — see all supported sources and docs for each source here.

Installing KEDA

Installing with Helm

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
kubectl create namespace keda
helm install keda kedacore/keda --namespace keda

Installing with a Yaml file

kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.4.0/keda-2.4.0.yaml

Creating a Scaled Object

In this section I'll create a ScaleObject that pulls a metric from Prometheus and creates an event. First, create a yaml file scaleObject.yml with the content

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: prometheus-scale
  namespace: default
spec:
  scaleTargetRef:
    name: appdeploy (1)
  minReplicaCount: 3 (2)
  maxReplicaCount: 10 (3)
  triggers:
  - type: prometheus (4)
    metadata:
      serverAddress: http://103.56.156.199:9090/ (5)
      metricName: total_http_request (6)
      threshold: '60' (7)
      query: sum(irate(by_path_counter_total{}[60s])) (8)

Explanation:

The name of the Deployment you want to scale
The minimum pod count
The maximum pod count
The type of external Server
The external Server's address
The metric's name
The value that triggers the event (here, if req/s > 60)
The query used to pull data from the external Server

Run the command to create the ScaledObject object:

kubectl apply -f scaledObject.yml

Once created, you'll get an object as shown: