Inference

Actually using a trained model: you give it input, it gives you output. Training is building the engine; inference is driving it, and it’s the part that happens every time you hit send.

A 50s ad illustration: a cook in a tall white hat flipping a whole flight of pancakes at once over a stove, the dinner rush in full swing.

Inference is running a model that’s already been trained: feed it a prompt, an image, or a sound, and get its answer back. Nothing is being learned at this stage; the model’s parameters are frozen, and the computer is just pushing your input through them.

For an LLM , inference happens one token at a time: the model reads everything so far, picks the next token, adds it to the pile, and goes again. That loop is why answers stream onto the screen word by word, and why tricks like the KV cache matter so much for speed.

Training gets the headlines, but inference is where most of the money and electricity go, because it happens every single time anyone uses the thing. It’s also what “running a model locally” means: doing inference on your own machine, usually with a shrunken (quantized ) copy in a format like GGUF .