How Embeddings Are Created

You don't write the rules that turn text into a vector — a trained embedding model does it. Understanding roughly how that model works, and what knobs you actually control, keeps you from making expensive mistakes like mixing vectors from two different models or storing them in a column of the wrong size.

From tokens to a single vector

An embedding model is a neural network trained on enormous amounts of text. When you send it a string, three things happen:

1. Tokenization — the text is split into tokens (word fragments). "Embeddings" might become embed + dings.
2. Contextual encoding — the network processes all tokens together, so each token's representation is shaped by the words around it. "Bank" near "river" lands differently than "bank" near "loan."
3. Pooling — the per-token representations are combined (often averaged) into one fixed-length vector for the whole input.

The output is the embedding. The model was trained so that texts humans consider similar are pushed together in this space and dissimilar texts are pushed apart — that training objective is the entire reason nearby vectors mean similar meaning.

What "dimensions" actually are

Each number in the vector is a dimension — a learned feature of meaning. You can loosely imagine one dimension leaning toward "formal vs. casual," another toward "about money," another toward "past tense" — but in reality the model invents these axes during training and they don't map cleanly to human concepts. What matters in practice:

• The dimension count is fixed by the model: nomic-embed-text always emits 768.
• You cannot change it after the fact, and you cannot compare vectors of different lengths.
• More dimensions ≠ strictly better; it's a trade-off between expressiveness, storage, and speed.

Local vs. hosted embedding models

In an enterprise .NET stack you'll choose between two sourcing models:

• Hosted (e.g., Azure OpenAI text-embedding-3-small, 1536-dim) — a REST call, no infrastructure, but every chunk of text leaves your boundary and you pay per token.
• Local (e.g., Ollama running nomic-embed-text, 768-dim) — runs on your own hardware, no data egress, no per-call cost. This is what LyraLearn uses precisely because lesson content and learner questions never need to leave the box. For public-sector workloads with data- residency rules, local-first is often the deciding factor.

What you control — and what you must not change

You control the input (which text, how it's chunked, any cleanup) and the choice of model. You do not control the dimension count or the meaning of the numbers.

The cardinal rule: embed everything with the same model. Vectors from nomic-embed-text and from an Azure model live in completely different spaces — their distances are meaningless to each other. If you ever swap embedding models, you must re-embed your entire corpus, because old and new vectors can no longer be compared. The next lesson covers exactly how those comparisons work.