The Training Loop

Every time you practice a free throw, something happens in your brain: you shoot, you see where the ball went wrong, you adjust your form, you shoot again. Over hundreds of attempts, your accuracy climbs. Machine learning works on the exact same logic — just running millions of times per second instead of once per day. This cycle is called the training loop, and it is the engine behind every AI model you have ever used.

The Four Steps of the Loop

The training loop has four steps that repeat over and over until the model stops improving. Step 1 — Make a prediction. The model receives an input (say, a photo) and produces a guess (say, 'cat'). Early in training, this guess is essentially random. Step 2 — Compare to the correct answer. The model's prediction is compared to the true label (the actual right answer for that example). If the model predicted 'cat' and the true label was 'dog,' it got it wrong. Step 3 — Calculate the error. A mathematical score called the loss (or error) measures exactly how wrong the prediction was. A large loss means the model is far off; a loss near zero means it is almost right. Step 4 — Adjust and repeat. The model tweaks its internal numbers — its parameters — to make the loss smaller next time. Then the loop starts again with the next training example. After thousands or millions of loops, the parameters settle into values that produce consistently good predictions. That collection of tuned parameters is the trained model.

The adjustment step deserves more attention because it is where the real 'learning' happens. The model does not randomly change its parameters and hope for the best — it uses a mathematical technique called gradient descent to find the direction of change that will reduce the loss most efficiently. Think of it like a hiker trying to find the lowest point in a hilly valley by always taking a step downhill. The hiker does not need a map of the whole valley — just the direction that goes down from wherever they are standing right now. The amount each parameter changes per step is controlled by a setting called the learning rate. A learning rate that is too large causes the model to overshoot and bounce around; too small, and learning is painfully slow. Choosing a good learning rate is part of the art of training models.

When Does the Loop Stop?

The training loop does not run forever. It stops when one of several conditions is met: the loss drops below an acceptable threshold, the model stops improving after many iterations, or a preset number of epochs (full passes through the training data) is complete. A model that trains for too few iterations will underperform — it has not learned enough yet. A model that trains for too many iterations on the same data can actually start to memorize the training examples rather than learn general patterns — a problem called overfitting that you will encounter in Lesson 8 of this module. Finding the sweet spot is part of what makes training machine learning models a craft as much as a science.