Interview Forge

CLOUD & INFRASTRUCTURE / SYSTEM CONCEPT BRIEF

Kubernetes scaling

Kubernetes scales at two levels.

IntermediatePhase 07 / Topic 13 of 17RequirementsTrade-offsFailure modes
01

Overview

Kubernetes scales at two levels. Pod scaling changes how many replicas run (Horizontal Pod Autoscaler, HPA) or how much CPU and memory each gets (Vertical Pod Autoscaler, VPA). Node scaling changes how many machines the cluster has (Cluster Autoscaler or Karpenter), adding nodes when pods cannot be scheduled and removing underused ones.

HPA scales on CPU, memory, or custom metrics like requests per second or queue depth; KEDA extends this to event sources (Kafka lag, SQS depth) and can scale to zero. Correct resource requests are essential, because the scheduler and autoscalers make decisions based on them.

A supermarket opening checkout lanes

When lines grow, the manager opens more lanes (pods). When all lanes are open and there is still a line, the store adds a temporary annex with more lanes (nodes). At night, lanes and annexes close to save money.

02

When to use it

  • Services with daily or seasonal traffic patterns.
  • Workers that should scale with queue backlog.
  • Reducing cost by removing idle nodes.
  • Handling spikes without manual intervention.
03

Where it shows up in interviews

Traffic-driven scaling

Recognize it when: requests vary through the day.

  • Design an e-commerce API for Black Friday
  • Handle viral traffic spikes
Backlog-driven scaling

Recognize it when: workers should match queue depth.

  • Design a video processing fleet
  • Design a notification worker pool
04

Where it is used in real software

Karpenter

AWS's open-source node provisioner launches right-sized nodes in seconds based on pending pods, including spot instances.

KEDA

Kubernetes Event-driven Autoscaling scales on 60+ event sources and to zero.

GKE Autopilot

Google manages nodes entirely; you pay per pod resources.

05

Key terms

HPA
Adjusts replicas based on metrics.
VPA
Adjusts pod CPU and memory requests.
Cluster Autoscaler / Karpenter
Adds and removes nodes.
Resource requests
Guaranteed CPU/memory used for scheduling.
PodDisruptionBudget
Limits voluntary evictions during scale-down.
06

How it works, step by step

  1. 1
    Set accurate requests

    Based on observed usage.

  2. 2
    Configure HPA

    Target 60-70% CPU or a custom metric.

  3. 3
    Pods pending?

    Node autoscaler adds nodes.

  4. 4
    Load drops

    HPA reduces replicas after a stabilization window.

  5. 5
    Nodes underused

    Autoscaler drains and removes them, respecting PDBs.

07

HPA calculation

desiredReplicas = ceil(currentReplicas x currentMetric / targetMetric)

Step 1 / 4
Current replicasAvg CPUTargetDesired replicas
490%60%ceil(4 x 90 / 60) = 6
665%60%ceil(6 x 65 / 60) = 7
730%60%ceil(7 x 30 / 60) = 4 (after stabilization)
40% (no traffic)60%minReplicas (e.g. 2)

NOWCurrent replicas: 4 | Avg CPU: 90% | Target: 60% | Desired replicas: ceil(4 x 90 / 60) = 6

HPA reacts proportionally; the scale-down stabilization window prevents flapping.

08

Implementation

apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: { name: api }spec:  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }  minReplicas: 3  maxReplicas: 60  metrics:    - type: Resource      resource: { name: cpu, target: { type: Utilization, averageUtilization: 65 } }  behavior:    scaleDown: { stabilizationWindowSeconds: 300 }---apiVersion: keda.sh/v1alpha1kind: ScaledObjectmetadata: { name: thumbnail-worker }spec:  scaleTargetRef: { name: thumbnail-worker }  minReplicaCount: 0          # scale to zero when the queue is empty  maxReplicaCount: 100  triggers:    - type: aws-sqs-queue      metadata: { queueURL: "https://sqs.us-east-1.amazonaws.com/123456789012/thumbnails", queueLength: "20", awsRegion: us-east-1 }
09

Complexity and performance

Pod scale-upSeconds

If nodes have room.

Node scale-up~30 s - few minutes

Instance launch + image pull.

10

Trade-offs

Responsiveness vs cost

Low targets and spare nodes react fast to spikes but waste capacity; overprovisioning with low-priority placeholder pods is a common middle ground.

HPA vs VPA

They conflict when both act on CPU; use VPA for recommendations or different metrics.

11

Variants and related techniques

Scheduled scaling

Pre-scale before known events (sales, games).

Scale to zero

KEDA or Knative for idle services.

12

Common mistakes

  • No resource requests.

    Fix: HPA utilization cannot be computed and scheduling is unreliable.

  • Scaling on CPU for I/O-bound services.

    Fix: Scale on RPS, latency, or queue depth.

  • Slow startup.

    Fix: Large images and long warm-up delay scaling; optimize startup and readiness.

13

Interview questions

How does Kubernetes handle a traffic spike?

HPA sees metrics above target and adds replicas; if pods cannot fit, they stay pending and the cluster autoscaler or Karpenter adds nodes; the Service starts routing to new pods once readiness probes pass.

How would you scale queue workers?

Use KEDA or an HPA on an external metric like queue depth or consumer lag, with scale to zero when idle and a max based on downstream capacity.

14

Practice problems

ProblemDifficultyWhat it trains
Configure HPA with a custom RPS metricMediumMetrics.
Plan scaling for a 10x flash saleHardPre-scaling and node headroom.