Embeddings and Vector Search

An embedding is a list of numbers — a vector — produced by a model trained to place text with similar meaning near each other in that numeric space. "A rich cup of coffee" and "a strong espresso" end up close together; "a rich cup of coffee" and "a tortoise moves slowly" end up far apart. Nothing about the vector is human-readable on its own; the only property that matters is its distance to other vectors.

That one property is the whole idea behind retrieval-augmented generation. Instead of searching for exact keyword matches, you turn "find the passages relevant to this question" into "find the vectors nearest to this question's vector" — a problem with fast, well-understood solutions at scale, in a way full-text semantic matching isn't. Embeddings also power search, clustering, recommendations, and classification more broadly, but retrieval for RAG is the use this module focuses on: embed a corpus once, embed each incoming query, and compare.

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-small",
    input="A rich cup of coffee.",
)
vector = response.data[0].embedding
print(len(vector))

OpenAI's third-generation embedding models are the most common starting point: text-embedding-3-small returns a 1536-dimension vector by default, text-embedding-3-large returns 3072, and both accept up to 8192 tokens of input per call. Both were also trained with a technique that lets you shrink the vector — the dimensions parameter — without destroying its concept-representing properties: OpenAI's own benchmark shows a text-embedding-3-large embedding shortened to 256 dimensions still outperforming an unshortened, older text-embedding-ada-002 embedding at 1536 dimensions. Smaller vectors mean less storage and faster comparisons, so this trade-off is worth using deliberately rather than defaulting to the largest size available.

If you shrink a vector yourself after the fact — slicing it down — rather than passing dimensions at request time, you have to re-normalize it to unit length afterward, or your distance comparisons will be off.

from openai import OpenAI
import math

client = OpenAI()

docs = ["A rich cup of coffee.", "A bright herbal tea.", "A tortoise moves slowly."]
query = "delicious espresso"

data = client.embeddings.create(model="text-embedding-3-small", input=docs + [query]).data
doc_vecs, query_vec = [d.embedding for d in data[:-1]], data[-1].embedding

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.hypot(*a) * math.hypot(*b))

ranked = sorted(zip(docs, doc_vecs), key=lambda pair: -cosine(pair[1], query_vec))
print([doc for doc, _ in ranked])

Cosine similarity — the angle between two vectors, ignoring their length — is the standard recommendation for comparing embeddings, and OpenAI's own guidance says the choice of distance function typically doesn't matter much beyond that. OpenAI's embeddings happen to already be normalized to unit length, which has two side effects worth knowing: cosine similarity can be computed as a slightly cheaper plain dot product, and cosine similarity and Euclidean distance produce identical rankings on these vectors. That equivalence is specific to normalized embeddings, though — it's not a universal property of vector search, so don't assume it holds for every embedding model you might use.