Embeddings are lists of numbers that represent the meaning of a piece of data, such as a word, sentence, image, or product, so that similar things end up close together in a mathematical space. With embeddings explained that simply, the practical payoff is clear: once meaning becomes geometry, you can search, cluster, deduplicate, and recommend by measuring distance instead of matching keywords. They are the foundation of semantic search and retrieval-augmented generation.
What are embeddings?
An embedding is a fixed-length vector, for example 384, 768, or 1,536 numbers, produced by a model trained so that inputs with related meaning get similar vectors. "How do I reset my password?" and "I forgot my login credentials" share almost no words, yet a good text embedding model places them near each other. "Reset the router" lands somewhere else.
No single number in the vector has a human-readable meaning. Meaning is spread across all dimensions, and only relative positions matter: what is near what, and in which direction.
A tiny illustration
Imagine a toy space with only three dimensions. The numbers below are made up purely for illustration:
"puppy" -> [0.81, 0.10, 0.62]
"dog" -> [0.78, 0.14, 0.58]
"invoice" -> [0.05, 0.92, 0.11]
"puppy" and "dog" point in nearly the same direction; "invoice" points elsewhere. Real embeddings work the same way, just with hundreds or thousands of dimensions capturing far subtler relationships.
How do embeddings capture meaning?
Embedding models learn from large datasets by adjusting their weights so that related inputs move closer and unrelated inputs move apart. Early word embeddings learned from the idea that words appearing in similar contexts have similar meanings. Modern text embedding models are usually transformer networks, the same family as a large language model, often trained with contrastive learning: the model sees pairs that should match, such as a question and its answer, alongside examples that should not, and learns to separate them.
Before any of this, text is split into tokens, so the same limits apply: each model has a maximum input length, and longer text must be chunked. See tokenization for how that step works.
How is similarity between embeddings measured?
Three measures show up constantly:
| Measure | What it compares | Range | Notes |
|---|---|---|---|
| Cosine similarity | Angle between vectors | -1 to 1 | Most common for text; ignores vector length |
| Dot product | Angle and length | Unbounded | Equals cosine similarity when vectors are normalized |
| Euclidean (L2) distance | Straight-line distance | 0 and up | Smaller means more similar |
Many embedding models output normalized vectors, in which case cosine similarity and dot product rank results identically and dot product is cheaper to compute. Always use the metric your model's documentation recommends.
Here is a small, runnable Python example of semantic search over a handful of sentences, using the open-source sentence-transformers library:
import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
docs = [
"How to reset your account password",
"Updating billing details and invoices",
"Troubleshooting Wi-Fi connection drops",
"Recovering access when you forget your login",
]
doc_vecs = model.encode(docs, normalize_embeddings=True)
query = "I can't sign in to my account"
q_vec = model.encode([query], normalize_embeddings=True)[0]
scores = doc_vecs @ q_vec # cosine similarity, since vectors are normalized
for i in np.argsort(-scores):
print(f"{scores[i]:.3f} {docs[i]}")
The password and login documents should rank at the top even though the query shares few exact words with them. That is the core behavior that keyword search cannot provide.
What are embeddings used for?
- Semantic search: find documents by meaning rather than exact words.
- Retrieval-augmented generation: retrieve relevant chunks to ground an LLM's answer. See RAG explained and what is RAG.
- Recommendations: suggest items whose embeddings are near what a user engaged with.
- Clustering and topic discovery: group support tickets or feedback without predefined labels.
- Deduplication: find near-duplicate records, questions, or listings.
- Classification: train a light classifier on top of embeddings, or label items by nearest labeled examples.
- Semantic caching: reuse an LLM answer when a new question is close enough to a previous one, a technique covered in how to reduce LLM API costs.
Embeddings vs keyword search
| Aspect | Keyword search | Embedding search |
|---|---|---|
| Matches | Exact terms and variants | Meaning and intent |
| Synonyms and paraphrases | Missed unless configured | Handled naturally |
| Exact identifiers and codes | Excellent | Often weak |
| Explainability | Easy to see why it matched | Harder to explain |
| Infrastructure | Inverted index | Vector index |
Because each covers the other's weaknesses, many production systems run both and merge results, known as hybrid search.
How to use embeddings well in production
Pick the model deliberately
Consider language coverage, domain (general text vs code vs multilingual), maximum input length, vector dimensions, cost, and whether you need to self-host. Evaluate a few candidates on your own queries rather than relying on general leaderboards alone.
Never mix models in one index
Vectors from different models, or even different versions of the same model, live in different spaces and are not comparable. Store the model name and version with every vector, and plan a full re-embed when you upgrade.
Chunk thoughtfully
Embedding an entire long document blurs many topics into one vector. Split content into coherent passages, include a title or heading for context, and keep chunks within the model's input limit.
Store and index efficiently
For small collections, a matrix in memory and a dot product is enough. As you grow, you need an approximate nearest neighbor index such as HNSW, typically provided by a vector database or a database extension. Our guide on how to choose a vector database compares the options. Storage adds up: vector count times dimensions times bytes per number, plus index overhead, so smaller dimensions or quantization can matter at scale.
Measure retrieval quality
Build a set of real queries with known relevant results and track recall at k when you change models, chunking, or index settings.
Key takeaways
- Embeddings turn data into vectors where distance reflects similarity in meaning.
- Cosine similarity is the standard measure for text; with normalized vectors it equals the dot product.
- Embeddings power semantic search, RAG, recommendations, clustering, and deduplication.
- Combine embeddings with keyword search for exact identifiers and names.
- Never compare vectors from different models, and evaluate models on your own data.
Frequently asked questions
What is an embedding in simple terms?
An embedding is a list of numbers that describes what something means, created by a model so that similar items get similar numbers. Comparing embeddings lets software judge how related two pieces of text, images, or products are.
Are embeddings the same as a large language model?
No. An embedding model outputs a vector that represents its input, while an LLM generates text. They are often built on similar transformer architectures and are commonly used together, with embeddings retrieving context that an LLM then uses to write an answer.
How many dimensions should an embedding have?
It depends on the model; you usually use whatever size it produces. Higher dimensions can capture more nuance but cost more memory and compute, so smaller models or reduced dimensions are often a good trade-off when quality on your evaluation set holds up.
Can I use embeddings for images and audio?
Yes. Embedding models exist for images, audio, and code, and some multimodal models place text and images in the same space, which enables searching images with a text query.