Interview Forge

CLOUD & INFRASTRUCTURE / SYSTEM CONCEPT BRIEF

Auto scaling

Auto scaling automatically adjusts the amount of compute to match demand: adding instances when load rises and removing them when it falls.

BeginnerPhase 07 / Topic 15 of 17RequirementsTrade-offsFailure modes
01

Overview

Auto scaling automatically adjusts the amount of compute to match demand: adding instances when load rises and removing them when it falls. This keeps latency stable during peaks and cuts cost during quiet periods. It applies to VMs (EC2 Auto Scaling groups), containers (HPA, ECS service scaling), serverless (automatic concurrency), and databases (Aurora Serverless, DynamoDB on-demand).

Policies include target tracking (keep CPU at 60%), step scaling (add 3 instances if CPU > 80%), scheduled scaling (scale up before 9 AM), and predictive scaling (forecast from history). Effective auto scaling requires stateless instances, fast startup, good health checks, and awareness of downstream limits.

Staffing a call center

More agents are scheduled for Monday mornings (scheduled), extra agents are called in when hold times exceed two minutes (target tracking), and staff go home when calls drop.

02

When to use it

  • Traffic that varies by time of day, week, or season.
  • Unpredictable spikes (viral content, launches).
  • Batch and worker fleets driven by queue depth.
  • Reducing cloud costs.
03

Where it shows up in interviews

Elastic web tier

Recognize it when: handle 10x peaks.

  • Design a ticket booking site for a concert sale
  • Design Black Friday readiness
Worker fleets

Recognize it when: scale with backlog.

  • Design a video processing pipeline
  • Design batch report generation
04

Where it is used in real software

EC2 Auto Scaling

Target tracking, step, scheduled, and predictive policies with health-check-based replacement.

Netflix Scryer

Netflix built predictive auto scaling that pre-scales ahead of daily viewing patterns.

Lambda concurrency

Scales automatically per request up to account and reserved concurrency limits.

05

Key terms

Target tracking
Keep a metric near a target value.
Cooldown / warm-up
Wait after scaling before acting again / time before a new instance counts.
Min / max / desired
Capacity bounds and current target.
Predictive scaling
Forecast demand and scale in advance.
Scale-in protection
Prevent specific instances from being terminated.
06

How it works, step by step

  1. 1
    Make instances stateless

    Sessions and files in shared services.

  2. 2
    Pick the metric

    CPU, RPS per target, queue depth, latency.

  3. 3
    Set min, max, and target

    Min for availability, max to protect downstream and budget.

  4. 4
    Tune warm-up and cooldowns

    Avoid flapping and overshoot.

  5. 5
    Add scheduled or predictive scaling

    For known peaks.

07

Scaling policies

When to use each

Step 1 / 5
PolicyHow it worksGood for
Target trackingKeep metric at targetMost services (default)
Step scalingAdd or remove by thresholdsCustom, aggressive responses
ScheduledChange capacity at set timesKnown peaks (business hours)
PredictiveForecast from historyRegular daily or weekly patterns
Queue-basedWorkers per backlogAsync workers

NOWPolicy: Target tracking | How it works: Keep metric at target | Good for: Most services (default)

Combine predictive or scheduled scaling for baselines with target tracking for surprises.

08

Implementation

resource "aws_autoscaling_group" "api" {  name                = "api"  min_size            = 3  max_size            = 60  vpc_zone_identifier = aws_subnet.private[*].id  target_group_arns   = [aws_lb_target_group.api.arn]  health_check_type   = "ELB"  default_instance_warmup = 120  launch_template {    id      = aws_launch_template.api.id    version = "$Latest"  }} resource "aws_autoscaling_policy" "requests_per_target" {  name                   = "rps-target"  autoscaling_group_name = aws_autoscaling_group.api.name  policy_type            = "TargetTrackingScaling"  target_tracking_configuration {    predefined_metric_specification {      predefined_metric_type = "ALBRequestCountPerTarget"      resource_label         = "${aws_lb.api.arn_suffix}/${aws_lb_target_group.api.arn_suffix}"    }    target_value = 800  }}
09

Complexity and performance

VM scale-out~1-5 minutes

Boot + app start.

Container scale-outSeconds

If nodes available.

10

Trade-offs

Reaction time vs stability

Aggressive policies respond quickly but can oscillate; conservative ones are stable but may lag spikes.

Cost vs headroom

Keeping a higher minimum wastes money but absorbs sudden spikes before new capacity arrives.

11

Variants and related techniques

Serverless scaling

Per-request scaling with concurrency limits.

Database auto scaling

Aurora replicas, DynamoDB capacity, storage growth.

12

Common mistakes

  • Scaling beyond downstream capacity.

    Fix: Set max limits and protect databases with pooling and rate limits.

  • Slow startup.

    Fix: Bake images, keep warm pools, or pre-scale for known events.

  • Stateful instances.

    Fix: Scale-in destroys local sessions or data.

13

Interview questions

How would you prepare a system for a flash sale?

Pre-scale (scheduled) compute, caches, and databases ahead of time, raise auto scaling limits, load test, add queues and rate limits for write paths, use a waiting room if needed, and keep target tracking for unexpected load.

What metric would you scale a queue worker on?

Backlog per worker or age of the oldest message, rather than CPU, because it directly reflects the work waiting.

14

Practice problems

ProblemDifficultyWhat it trains
Configure target tracking for an APIEasyMetrics.
Design auto scaling for a concert ticket saleHardPre-scaling and protection.