Inference is what happens every time you actually use an AI model. Training is the long, expensive phase where a model learns patterns from data; inference is the everyday phase where the finished model applies those patterns to your input — answering your question, summarizing your text, or generating an image. When you press "Generate" in any AI tool, you are running inference.
The distinction matters in practice. Inference speed determines how fast a tool feels, and inference cost is why many AI services meter usage by tokens or requests. A model's knowledge is also frozen at training time — inference does not teach it anything new, which is why models can be out of date and why techniques like RAG exist to feed them fresh information at answer time.
Related terms
Token · RAG (Retrieval-Augmented Generation) · Machine Learning (ML)