Vertical Scaling on Kubernetes Without Restarting Pods? What's New in v1.33

This was especially inconvenient for stateful applications or workloads that take a long time to restart, and it could disrupt users of the system.

The good news is that with Kubernetes v1.33, the resize container resources in-place feature now lets you scale a container's CPU and memory within a pod without restarting it, depending on the configured policy. This is a big step forward for more efficient resource optimization and reduced downtime for applications — especially handy for firefighting when there's a "fire" in production 🤣.

⚙️ How it works

The Pod Resize feature lets you update resources.requests and resources.limits without deleting the pod. This is coordinated by the kubelet and must be enabled via the InPlacePodVerticalScaling feature gate.

Resizing a container's resources is simple:

1. You update the Pod or Deployment with new CPU/Memory configuration.
2. If the Pod and container support in-place resize (no restart needed), the kubelet will try to apply the change directly.
3. The resize status gets updated in the Pod's status.resize.status.

Resize status

While a container in a pod is being resized, it can be in one of these resize states:

• PodResizePending: The resize process is deferred or has failed. The message and reason fields below give the error details. Here are a few common reasons:
  • reason: Infeasible: The post-resize configuration may exceed the remaining resources available on the node.
  • reason: Deferred: The post-resize configuration currently isn't available; the kubelet will retry after some time.
• PodResizeInProgress: The resource change request has been accepted and is being applied.

Resize Policy

The resize policy lets you configure how a container behaves when a resource configuration change is requested.

resources:
  resizePolicy:
    - resourceName: cpu
      policy: NoRestart
    - resourceName: memory
      policy: RestartContainer

• NoRestart: Resize directly without restarting the container.
• RestartContainer: The container gets restarted on resize (the old behavior).

With a container configured as above:

• Changing only the CPU configuration => the container resizes without restarting
• Changing only the Memory configuration => the container resizes and restarts
• Changing both resource types => the container restarts, because of the memory policy

Note: If a pod's restartPolicy is Never, then the container's resizePolicy must be set to NotRequired for both CPU and Memory. You cannot use the RestartContainer policy in these pods.

Limitations

Even though this new feature fixes vertical scaling's fatal flaw on K8s, there are still some limitations:

• Currently only CPU and memory can be resized.
• Memory can't be scaled down if resizePolicy is NoRestart. Reducing memory still requires a container restart.
• QoS (Quality of Service) Class must remain the same after resizing.
• Containers like init-containers or ephemeral containers don't allow resizing.
• You can't remove CPU and memory configuration (BestEffort QoS) — you can only change it to a different number.
• Windows Pods currently don't support this resize feature.
• Pods using swap memory can only be updated by restarting the container.
• Some applications written in languages like Java may not be able to resize Heap memory in-place after the container's memory is resized, requiring a container restart.

Demo

Enough theory — let's try resizing a few cases and see how this feature actually works.

A successful resize case

First, you'll need a K8s cluster running version 1.33.

Create a pod with the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: demo-resize-success
spec:
  containers:
    - name: nginx
      image: nginx
      resources:
        requests:
          memory: "64Mi"
          cpu: "250m"
        limits:
          memory: "64Mi"
          cpu: "250m"
      resizePolicy:
      - resourceName: cpu
        restartPolicy: NotRequired
      - resourceName: memory
        restartPolicy: NotRequired

Update the manifest to increase both CPU and memory as follows, then apply it with the command:

apiVersion: v1
kind: Pod
metadata:
  name: demo-resize-success
spec:
  containers:
    - name: nginx
      image: nginx
      resources:
        requests:
          memory: "600Mi"
          cpu: "500m"
        limits:
          memory: "600Mi"
          cpu: "500m"
      resizePolicy:
      - resourceName: cpu
        restartPolicy: NotRequired
      - resourceName: memory
        restartPolicy: NotRequired

kubectl apply -f pod-resize-1.yaml --subresource resize --force-conflicts --server-side

Check the pod again and you'll see it's been resized without restarting.

A failed resize case

Create a new pod with the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: demo-resize-fail
spec:
  containers:
    - name: nginx
      image: nginx
      resources:
        requests:
          memory: "64Mi"
          cpu: "250m"
        limits:
          memory: "64Mi"
          cpu: "250m"
      resizePolicy:
      - resourceName: memory
        restartPolicy: NotRequired

We'll try scaling memory down, expecting this to fail since scaling memory down requires restarting the container. Scale the pod down to 50MB memory and see what happens:

You can also try other cases, like sizing a pod larger than the node's currently available resources, etc.

Wrapping up

The in-place pod vertical scaling feature opens up more flexible resource-optimization possibilities without affecting application availability. This is a feature worth paying attention to for DevOps/SRE teams wanting to reduce downtime in production environments.

In the near future, I believe we'll see new controllers that continuously monitor pod resource usage and perform this vertical scaling automatically, instead of requiring manual edits like today.

Thanks for reading all the way through — I hope this post brought you some value.

If you found it useful, please Upvote and Follow me for more posts to come 😄

If you're running into technical challenges, need help with systems, DevOps tools, or career direction, I'm confident I can help. Get in touch at hoangviet.io.vn

Further reading: kubernetes.io — Resize Container Resources