Interview Forge

CLOUD & INFRASTRUCTURE / SYSTEM CONCEPT BRIEF

Container orchestration

Container orchestration automates deploying, scheduling, networking, scaling, and healing containers across a cluster of machines.

IntermediatePhase 07 / Topic 17 of 17RequirementsTrade-offsFailure modes
01

Overview

Container orchestration automates deploying, scheduling, networking, scaling, and healing containers across a cluster of machines. Running a few containers by hand is easy; running thousands across hundreds of hosts, with rolling updates, service discovery, secrets, and failure recovery, requires an orchestrator.

Kubernetes is the dominant orchestrator; alternatives include Amazon ECS (simpler, AWS-native), HashiCorp Nomad (lightweight, also runs non-container workloads), and serverless container platforms (AWS Fargate, Google Cloud Run, Azure Container Apps) that hide the cluster entirely.

A port's logistics system

Thousands of shipping containers arrive daily. The port's system decides which crane unloads each, where to stack it, which truck takes it, and reroutes when a crane breaks. Orchestrators do this for software containers.

02

When to use it

  • Running many containerized services in production.
  • Needing self-healing, rolling updates, and autoscaling.
  • Efficient bin-packing of workloads onto shared machines.
  • Standardizing deployments across teams.
03

Where it shows up in interviews

Platform choice

Recognize it when: Kubernetes or something simpler?

  • Choose a runtime for a startup
  • Design an internal developer platform
04

Where it is used in real software

Kubernetes everywhere

EKS, GKE, and AKS run most large container fleets.

Amazon ECS

Used by many AWS-centric companies for simpler container operations, often with Fargate.

Google Cloud Run

Runs containers serverlessly with request-based scaling to zero.

05

Key terms

Scheduling
Placing containers on nodes based on resources and constraints.
Bin packing
Fitting workloads efficiently onto machines.
Desired state
Declared configuration the orchestrator maintains.
Self-healing
Restarting and rescheduling failed containers.
Affinity / anti-affinity
Rules to co-locate or spread workloads.
06

How it works, step by step

  1. 1
    Declare the workload

    Image, replicas, resources, ports, config.

  2. 2
    Scheduler places containers

    Spreading across zones and nodes.

  3. 3
    Networking connects them

    Service discovery and load balancing.

  4. 4
    Controllers maintain state

    Replace failures, roll out updates.

  5. 5
    Autoscalers adjust capacity

    Pods and nodes.

07

Orchestration options

Choose by team size and needs

Step 1 / 4
PlatformComplexityFlexibilityBest for
Kubernetes (EKS/GKE/AKS)HighHighest, huge ecosystemLarge platforms, multi-cloud
Amazon ECS + FargateLow-mediumAWS-focusedAWS teams wanting simplicity
NomadLowContainers + VMs + binariesMixed workloads, HashiCorp stacks
Cloud Run / Container AppsLowestRequest-driven servicesSmall teams, spiky HTTP services

NOWPlatform: Kubernetes (EKS/GKE/AKS) | Complexity: High | Flexibility: Highest, huge ecosystem | Best for: Large platforms, multi-cloud

Kubernetes is powerful but not always necessary; pick the simplest platform that meets requirements.

08

Implementation

{  "family": "api",  "requiresCompatibilities": ["FARGATE"],  "networkMode": "awsvpc",  "cpu": "512",  "memory": "1024",  "containerDefinitions": [    {      "name": "api",      "image": "containers.artifactory.tools.bestbuy.com/team/api:abc1234",      "portMappings": [{ "containerPort": 3000 }],      "healthCheck": { "command": ["CMD-SHELL", "wget -qO- http://localhost:3000/health || exit 1"], "interval": 15 },      "secrets": [{ "name": "DATABASE_URL", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:api-db" }],      "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/ecs/api", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "api" } }    }  ]}
09

Complexity and performance

Scheduling decisionMilliseconds

Per pod.

Cluster sizesTens to thousands of nodes

Per cluster.

10

Trade-offs

Control vs simplicity

Kubernetes offers the most control and ecosystem; managed serverless containers offer the least operational work.

Portability vs lock-in

Kubernetes is portable across clouds; ECS and Cloud Run are tied to their provider.

11

Variants and related techniques

Docker Swarm

Simple orchestrator built into Docker; less common today.

Edge orchestration

K3s and similar lightweight distributions for edge devices.

12

Common mistakes

  • Adopting Kubernetes too early.

    Fix: Small teams may move faster on managed container platforms.

  • All replicas in one zone.

    Fix: Use topology spread constraints or anti-affinity.

13

Interview questions

What problems does a container orchestrator solve?

Scheduling containers across machines, service discovery and load balancing, rolling updates and rollbacks, self-healing, autoscaling, configuration and secrets management, and efficient resource utilization.

When would you choose ECS or Cloud Run over Kubernetes?

When the team is small, workloads are standard web services or workers, and you want minimal operations on one cloud, without needing Kubernetes' ecosystem or portability.

14

Practice problems

ProblemDifficultyWhat it trains
Compare EKS and ECS for a 10-service appMediumTrade-offs.
Design zone-resilient schedulingMediumSpread constraints.