Vector Indexes
The previous lesson left us with a problem: brute-force nearest-neighbor search is accurate but scans every vector on every query. A vector index is a data structure that lets you find the closest vectors without comparing against all of them — trading a sliver of accuracy for a large gain in speed. Choosing whether you need one, and which kind, is a core architecture decision.
Exact (brute-force) search
Exact search computes the distance to every stored vector and is guaranteed to return the true top-K nearest neighbors. There is nothing to build, nothing to tune, and nothing to go stale — and that simplicity is genuinely valuable. For corpora up to roughly the low hundreds of thousands of vectors on modern hardware, a brute-force scan returns in milliseconds and is the correct choice. It scales linearly: double the data, double the work.
LyraLearn sits comfortably here. A few hundred lesson chunks means an exact VECTOR_DISTANCE
scan over the whole table is effectively free, and every search is perfectly accurate. You do
not pay for an index you don't need — reaching for ANN at this scale is premature optimization.
Approximate nearest neighbor (ANN)
When the corpus grows into the millions, linear scans stop being free and you switch to approximate nearest neighbor (ANN) search. ANN indexes pre-organize the vectors so a query only explores a promising fraction of them. The two dominant families:
- HNSW (Hierarchical Navigable Small World) — a layered graph you "walk" toward the query, taking ever-shorter hops. Very fast queries, but the graph lives in RAM, so memory cost scales with the corpus.
- DiskANN — a graph designed to live on SSD instead of RAM, so a single node can index far more vectors economically. This is the family Microsoft built into SQL Server and Azure AI Search, which matters for large enterprise corpora where keeping everything in memory is impractical.
The word that defines ANN is approximate: it may occasionally miss a true neighbor.
The speed-versus-accuracy tradeoff
Every ANN index exposes a knob — how much of the graph to explore per query. Explore more and you approach exact accuracy but spend more time; explore less and you get answers faster but may drop a relevant result. This is measured as recall: the fraction of the true top-K that the approximate search actually returned. A well-tuned index often hits 95–99% recall while touching a tiny fraction of the data.
The practical rule: start exact, measure, and only add an ANN index when latency at your real data volume demands it. Index builds add memory or disk cost, tuning effort, and a small, permanent accuracy tax. For a public-sector knowledge base of a few thousand documents, that tax buys nothing. For a national records system with tens of millions of vectors, ANN is the only way queries stay sub-second — and DiskANN keeps that affordable.