Embeddings in Practice

Knowing what an embedding is matters less than running an embedding pipeline that stays correct in production. Most failures aren't exotic — they're chunking that's too coarse, a column that doesn't match the model, or stale vectors after a content edit. This lesson is the operational checklist for embeddings in a .NET / SQL Server system.

Chunk before you embed

You almost never embed a whole document. A model produces one vector per input, so a 30-page policy collapsed into a single vector becomes a blurry average that matches nothing well. Instead you chunk: split the document into passages and embed each one.

Guidelines that hold up in enterprise content:

• Size for one idea — a few hundred tokens (a paragraph or two) so each chunk has a focused meaning. LyraLearn chunks lessons roughly along the ## headings you see, which is why those subheadings matter: each becomes a retrievable unit.
• Overlap a little — carry ~10–15% of the previous chunk into the next so a sentence split across a boundary isn't lost to both.
• Keep metadata — store the source document, section, and order with each chunk so a retrieved vector can be traced back and cited.

Store vectors where you can query them

Store the vector next to the text it came from, in a column whose size exactly matches the model's dimensions. With SQL Server 2025:

CREATE TABLE LessonChunks (
    Id          INT IDENTITY PRIMARY KEY,
    LessonId    INT          NOT NULL,
    ChunkText   NVARCHAR(MAX) NOT NULL,
    Embedding   VECTOR(768)   NOT NULL   -- must equal the model's output size
);

If the model emits 768 numbers, the column must be VECTOR(768). A mismatch — VECTOR(1536) for a 768-dim model — fails at insert time, which is the good outcome; the bad outcome is silently storing vectors from two different models in one column, where distances become nonsense. Treat (model, dimension, column) as a single locked contract.

Know when to re-embed

Embeddings are a derived artifact: they go stale the moment their source changes. Re-embed when:

1. The text changes — edit a lesson or policy paragraph, and that chunk's old vector now describes text that no longer exists. Re-embed just the affected chunks.
2. The chunking strategy changes — new sizes or boundaries mean every vector must be rebuilt.
3. The embedding model changes — the cardinal rule from the previous lesson: a new model puts vectors in a new space, so you must re-embed the entire corpus, not mix old and new.

Build the pipeline so re-embedding is routine — ideally triggered automatically whenever content is saved — rather than a scary one-off migration.

The whole loop, end to end

Putting it together, here is the LyraLearn pipeline and the shape of any production system:

1. Ingest content and split it into chunks.
2. Embed each chunk with the local nomic-embed-text model (768-dim).
3. Store the vector in a VECTOR(768) column with its text and metadata.
4. Retrieve at query time by embedding the question and running a VECTOR_DISTANCE cosine search for the nearest chunks.
5. Re-embed whenever the text, chunking, or model changes.

Get those five steps right and embeddings stop being theory — they become the dependable retrieval layer under every RAG feature you build.