What Is an LLM? How ChatGPT Actually Works

It all starts with prediction

An LLM is, at its core, a giant next-token predictor. Trained on enormous amounts of text, it learns what word (token) most plausibly comes next in any given context. That sounds underwhelming until you realise it is the whole game: everything a language model does is the same operation repeated, one token at a time.

The training works like a fill-in-the-blank exercise at a massive scale. Take a passage, hide a word, ask the model to guess it, and repeat across trillions of tokens. Getting better at that single task, at sufficient scale, produces models that write coherent essays, translate languages, and write working code.

Tokenization

Text is split into tokens, which are chunks of a few characters each. The model consumes tokens, not raw characters, which is why context length is measured in tokens rather than words or characters.

Tokens are not words. Common words are usually a single token, but a rare technical term like "quantization" might split into three. This has real consequences: token cost is billed per token, and a model with a 8,000-token window holds roughly 6,000 English words, fewer if your text is in a language that tokenizes less efficiently.

Attention

The attention mechanism lets the model weigh which earlier tokens matter most for the token it is about to produce. Every token looks at every other token in the context and decides how much each one matters to what comes next.

This is what lets a model keep track of a 5,000-word document while still attending to the topic from the first paragraph, and it is why the order and content of your prompt matter so much. If the information the model needs is not inside the context window, no amount of cleverness helps, because the model genuinely cannot see it.

Training and scale

The training data is enormous and mostly unlabelled, which is what made this possible — nobody had to hand-label trillions of tokens. But raw text alone is not enough. Modern models are trained in stages: broad pretraining on a huge general corpus, then further training on higher-quality data, then instruction tuning so the model responds to requests rather than just continuing text, and often a final stage using human or AI feedback to align behaviour with what people actually want.

Why it feels smart

With billions of parameters and trillions of examples, next-token prediction produces text that is coherent, relevant, and often impressive. The model is not thinking in the human sense — it is extremely good pattern completion, and it is fluent enough that the distinction is easy to miss.

The practical takeaway: the better your context, the better the output. That single insight is the entire field of prompt and context engineering. Everything from a clear system instruction to putting the most important information near the end of the prompt follows from the fact that the model is a conditional predictor working only with what you gave it.

Why hallucinations happen

Because the model always produces the most plausible next token, it has no built-in mechanism for saying "I don't know". When it lacks the fact, it produces something statistically smooth instead. This is not a bug that a better prompt can always fix, which is why techniques like retrieval (RAG) exist — to put the true answer inside the context so the model has something plausible to be right about.

Go deeper

• Learn to engineer better prompts now that you understand the model.
• See how teams fix hallucination with RAG.
• Go hands-on with production LLM skills in our LLM and Agentic AI training.