Late-Interaction Embeddings in plain English.

Standard embeddings turn a passage into a single vector, which is compact but lossy: fine details get averaged away. Late-interaction models, popularised by ColBERT, keep one small vector per token and compare query tokens to document tokens at search time, taking the best match for each query token and summing (the MaxSim operation). The "late" refers to when the query and document meet: after both are embedded, not before. The result is retrieval that is more precise on specifics such as names, numbers, and rare terms, at the cost of larger indexes and a vector store that supports multi-vector search. The approach has spread to visual documents: Perplexity's pplx-embed-v2-late models, released as open MIT weights on October 7, 2026, embed rendered PDF pages and slides as images so you can search them without OCR, and share one space between a 9B indexing model and a 0.6B serving model. Qdrant, Weaviate, and Vespa support multi-vector indexes; most managed stores are adding them.