Overview
Compute is the resource that runs your code. In the cloud it comes in several forms: virtual machines (EC2, Compute Engine), containers on orchestrators (ECS, EKS, GKE, Cloud Run), serverless functions (Lambda, Cloud Functions), and specialized hardware (GPUs for ML, ARM processors like Graviton for price-performance).
Choosing compute is a trade-off between control, operational effort, startup time, cost model, and limits. VMs give full control, containers give portability and density, and functions give zero server management with per-request billing but with execution time limits and cold starts.
Buying (on-prem servers), leasing (reserved VMs), renting by the day (on-demand VMs), car-sharing by the minute (containers on shared clusters), or taking a taxi only when needed (serverless functions).
When to use it
- Deciding how to run an API, worker, or batch job.
- Optimizing cost for steady vs spiky workloads.
- Running GPU workloads for ML training or inference.
- Interview discussions of deployment targets.
Where it shows up in interviews
Recognize it when: where should this workload run?
- Design a video transcoding service
- Design a low-traffic internal API
Recognize it when: reduce compute cost for batch jobs.
- Design a nightly data pipeline
- Design ML training infrastructure
Where it is used in real software
Unused EC2 capacity at up to 90% discount; ideal for fault-tolerant batch and CI workloads.
ARM-based instances offering better price-performance; many companies migrated for 20-40% savings.
Services like iRobot and Coca-Cola run event-driven backends on Lambda without managing servers.
Key terms
- Instance type
- CPU, memory, network, and storage combination (m7g.large).
- On-demand / reserved / spot
- Pay per second / commit for discount / interruptible discount.
- Cold start
- Delay when a function or container starts from zero.
- Fargate
- Serverless containers: no nodes to manage.
- Right-sizing
- Matching instance size to actual usage.
How it works, step by step
- 1Characterize the workload
Request-driven or batch, steady or spiky, duration, CPU/memory/GPU needs.
- 2Pick the model
Functions for event-driven short tasks, containers for services, VMs for special needs.
- 3Pick the size and architecture
Right-size; consider ARM.
- 4Pick the pricing
Savings plans for baseline, on-demand for peaks, spot for interruptible work.
- 5Automate scaling
Auto scaling groups, HPA, or function concurrency.
Compute options compared
Choose by workload
| Option | Startup | Max duration | Operations | Best for |
|---|---|---|---|---|
| VM (EC2) | Minutes | Unlimited | Most (OS, patching) | Legacy apps, special tuning |
| Containers (ECS/EKS) | Seconds | Unlimited | Medium | Microservices, workers |
| Serverless containers (Fargate, Cloud Run) | Seconds | Unlimited / long | Low | Services without node management |
| Functions (Lambda) | ms-seconds (cold start) | 15 min (Lambda) | Lowest | Event handlers, glue, spiky APIs |
NOWOption: VM (EC2) | Startup: Minutes | Max duration: Unlimited | Operations: Most (OS, patching) | Best for: Legacy apps, special tuning
Containers are the default for services; functions for event-driven glue; VMs when you need full control.
Implementation
import type { S3Event } from "aws-lambda";import sharp from "sharp";import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({}); // created once per container, reused across invocations export const handler = async (event: S3Event) => { for (const record of event.Records) { const Bucket = record.s3.bucket.name; const Key = decodeURIComponent(record.s3.object.key.replace(/\+/g, " ")); const original = await s3.send(new GetObjectCommand({ Bucket, Key })); const thumb = await sharp(Buffer.from(await original.Body!.transformToByteArray())).resize(256).webp().toBuffer(); await s3.send(new PutObjectCommand({ Bucket: `${Bucket}-thumbs`, Key: Key.replace(/\.\w+$/, ".webp"), Body: thumb })); }};Complexity and performance
Use containers for longer jobs.
2-minute interruption notice.
Trade-offs
VMs give full control but need patching and capacity planning; serverless removes that but imposes limits.
Functions are cheapest for spiky low traffic; containers or reserved VMs are cheaper for steady high traffic.
Variants and related techniques
For ML training and inference.
Direct hardware access for specialized workloads.
Common mistakes
- Over-provisioned instances.
Fix: Right-size using utilization metrics.
- Heavy initialization inside the function handler.
Fix: Initialize clients outside the handler to reuse across invocations.
- Spot for stateful services without handling interruptions.
Fix: Use spot for stateless or checkpointed work.
Interview questions
When would you use Lambda instead of containers?
For event-driven, short-running, spiky workloads such as S3 triggers, queue consumers, and low-traffic APIs, where scaling to zero and no server management outweigh cold starts and time limits.
How do you reduce compute costs?
Right-size instances, use ARM where possible, cover baseline with savings plans, use spot for interruptible work, auto scale to demand, and turn off idle environments.
Practice problems
| Problem | Difficulty | What it trains |
|---|---|---|
| Choose compute for 5 workloads | Easy | Selection. |
| Design a cost-efficient transcoding fleet | Medium | Spot and queues. |