A large language model (LLM) - the technology behind ChatGPT, Claude, and similar tools - can feel like magic. The core mechanism is more understandable than it seems.
Next-word prediction, at massive scale
At its core, an LLM is trained to do one thing extremely well: predict the next word (technically, "token") given the text so far. Trained on a huge amount of text, it learns patterns of language, facts, reasoning steps, and style well enough that repeatedly predicting "what comes next" produces coherent, useful responses.
Tokens, not words
Models do not process whole words - they break text into tokens, which might be a whole word, part of a word, or punctuation. This is why AI pricing and context limits are usually measured in tokens, not words or characters.
# Roughly, as a mental model:
"unbelievable" -> ["un", "believ", "able"] # one word can be multiple tokens
Context window: the model's short-term memory
A model can only "see" a limited amount of text at once - its context window. Anything outside that window (earlier in a long conversation, for example) is simply not visible to the model when generating its next response. This is why very long conversations can cause a model to "forget" something mentioned much earlier.
Why this explains common LLM behavior
Understanding "predicts plausible next text" explains a lot: why models can sound confident while being wrong (they are generating plausible text, not looking up verified facts), why extremely specific or unusual requests can trip them up (less similar to their training data), and why giving clear, well-structured input produces clearly better output - the next lesson.