What an LLM Actually Does

The single most common wrong mental model about LLMs — held by many working engineers — is that the model "searches its knowledge base" for the closest match to your question. It feels right because vector search exists nearby (in RAG systems), but it describes the wrong component. If you carry this model into an interview, an AI-literate panelist will hear it immediately. This lesson replaces it.

The correct model: a next-token predictor

An LLM does exactly one thing. Given a sequence of tokens, it outputs a probability distribution over what token comes next. That's the whole machine.

Tokens are the model's alphabet: chunks of text, usually word-fragments. "Credential" might be one token; "credentialing" might be two. The model never sees words or letters — only token IDs.
Training is where the "knowledge" comes from. The model read an enormous corpus and billions of numeric weights were gradually adjusted so its next-token predictions got better across all of it. Facts, grammar, reasoning patterns — all of it ends up smeared across the weights as statistical tendencies. There is no facts table, no document store, no index inside the model. Nothing is "looked up," ever.
Inference (answering you) is generation, not retrieval: your prompt is tokenized, the model predicts a next token, that token is appended, and the process repeats — one token at a time — until it emits a stop. Every answer you've ever seen from an LLM was produced this way: sequential guesses, each conditioned on everything before it.

Two consequences worth saying out loud in interviews:

The context window is the model's entire working memory. The model can only condition on the tokens in front of it — the system prompt, the conversation, any documents you pasted. That's why "give the model good context" is the core engineering activity of the whole LLM era: what's in the window is the only thing the model knows for sure, as opposed to what it vaguely absorbed at training time.

Temperature is choice among plausible tokens. The model produces a probability distribution; sampling settings decide whether it always takes the top choice (deterministic, good for structured tasks) or samples more adventurously (varied, good for creative tasks).

Why hallucination is structural, not a bug

Now the hallucination answer writes itself, and it's much stronger than "vector search is approximate":

The model's objective is plausibility, not truth. It always produces a fluent next token — including where its training data was thin, contradictory, or silent. A correct answer and a confident fabrication are generated by the same mechanism; the model has no internal fact-checker and, by default, no mechanism for "I don't know." It doesn't know that it doesn't know.

This is why hallucination can't be "fixed" from inside — and why the entire responsible-AI toolkit exists on the outside: retrieval grounds answers in real documents, citations make them checkable, instructions permit "I don't know," thresholds route uncertainty to humans, and evaluation measures how often the whole assembly is right. One sentence to keep: truth is added to an LLM system from outside the model.

Where the "vector search" confusion comes from

Vector search is real — it's just a different component. Embedding models and vector indexes live in the RAG layer that engineers bolt on around an LLM (next lesson). The confusion is understandable: both involve "closeness of meaning." Keep them apart with this rule: embeddings retrieve, the LLM generates. If someone asks "how does the model find the answer?", the correct reply is: it doesn't find anything — it writes the answer, and if you want the writing grounded in real sources, you must find those sources and put them in the context window first.

Interview drill

Practice these three out loud until they're automatic:

"What is an LLM doing when it answers?" — Next-token prediction: knowledge lives in trained weights, answers are generated token by token, nothing is retrieved internally.
"Why do LLMs hallucinate?" — Objective is plausibility, not truth; same mechanism produces right and wrong answers; no built-in fact-checker; that's why grounding and verification are external.
"What's the context window?" — The model's only working memory; everything reliable the system does depends on what engineering puts into it.