Large Language Models

Overview

A large language model (LLM) is a neural network trained to predict the next token in a sequence. Given "The capital of France is", it assigns a probability to every token in its vocabulary and picks one, then repeats with the new token appended. Trained on trillions of tokens, this simple objective produces a model that can answer questions, summarize, translate, and write code.

An LLM goes through three stages. Pre-training on large text corpora teaches language and world knowledge. Instruction tuning on curated examples teaches it to follow requests. Preference tuning (for example reinforcement learning from human feedback) makes answers more helpful and safer. At serving time the model only runs inference, and its knowledge is frozen at the training cutoff unless you supply context in the prompt.

When to use it

• Open-ended language tasks such as drafting, summarizing, rewriting, and answering questions.
• Turning unstructured text into structured data.
• Conversational interfaces and assistants.
• Code generation and explanation.

Where it shows up in interviews

Where it is used in real software

Key terms

Token

A chunk of text, often a word piece, that the model reads and writes. Roughly four English characters per token.

Context window

The maximum number of tokens the model can consider at once, covering both prompt and output.

Parameters

The learned weights of the network. More parameters generally mean more capacity and higher serving cost.

Temperature

A sampling setting. Low values make output focused and repeatable; high values make it more varied.

Hallucination

Fluent output that is not supported by facts or the provided context.

How it works, step by step

1. 1
   Tokenize the input

   The prompt is split into token IDs using the model's tokenizer.

2. 2
   Embed and add position

   Each token ID becomes a vector, combined with information about its position.

3. 3
   Run transformer layers

   Dozens of attention and feed-forward layers let each token gather context from earlier tokens.

4. 4
   Predict the next token

   The final layer outputs a probability for every vocabulary token; sampling picks one.

5. 5
   Repeat until done

   The new token is appended and the loop continues until a stop token or length limit. Cached keys and values (the KV cache) avoid recomputing earlier tokens.

Sizing an LLM feature

A support tool summarizes 50,000 tickets per day. Each ticket averages 1,200 input tokens and a 150-token summary.

Token volume drives cost, and output tokens drive latency. Batch the job overnight if nobody waits on it.

Implementation

1import OpenAI from "openai";2 3const client = new OpenAI();4 5// Stream tokens to the user as they are generated.6const stream = await client.chat.completions.create({7  model: "gpt-4o-mini",8  temperature: 0.2,                       // focused, repeatable answers9  max_tokens: 300,                        // cap cost and latency10  stream: true,11  messages: [12    { role: "system", content: "You summarize support tickets in three bullet points." },13    { role: "user", content: ticketText },14  ],15});16 17for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

Complexity and performance

Trade-offs

Variants and related techniques

Common mistakes

• Trusting the model's knowledge for recent or private facts.

  Fix: Provide the facts in the prompt through retrieval (RAG) or tool calls.

• Ignoring output token cost and latency.

  Fix: Set max tokens, ask for concise formats, and stream responses.

• Parsing free-form text in code.

  Fix: Request structured output (JSON schema) and validate it.

Interview questions

Practice problems