The Forward Pass

You now understand neurons, weights, layers, and activation functions. Put them together and you have everything needed to understand the forward pass: the journey a piece of data takes through a neural network from raw input all the way to a final prediction. Every time you ask a trained network a question — Is this email spam? What digit is this? What is the next word? — the forward pass happens in milliseconds.

Step by Step Through the Network

The forward pass is called 'forward' because information moves in one direction only: from the input layer, through each hidden layer in order, to the output layer. Nothing goes backward during a forward pass. (Backward is for learning — that comes next lesson.) Here is the exact sequence of events: Step 1 — Load the inputs. Raw data is converted to numbers and placed into the input neurons. An image becomes a list of pixel values. A word becomes a vector of numbers representing its meaning. Step 2 — Compute the first hidden layer. Each neuron in hidden layer 1 takes all input values, multiplies each by its own weight, adds them up (the weighted sum), then applies its activation function. The result is the neuron's activation — the number it sends forward. Step 3 — Repeat for each subsequent layer. Hidden layer 2 receives the activations from hidden layer 1. Each of its neurons computes its own weighted sum of those activations, applies its activation function, and sends results forward. This continues layer by layer. Step 4 — Reach the output layer. The final layer applies its activation function (sigmoid for binary, softmax for multi-class) and produces the network's answer.

Let us trace a tiny forward pass completely. The network decides if a number is closer to 1 or to 0. Architecture: 2 inputs, 1 hidden layer with 2 neurons (using ReLU), 1 output neuron (using sigmoid). Inputs: x1 = 0.8, x2 = 0.3 Hidden neuron H1: Weights: w1=1.0, w2=-0.5 Weighted sum: (0.8×1.0) + (0.3×-0.5) = 0.8 - 0.15 = 0.65 ReLU(0.65) = 0.65 (positive, so unchanged) Activation: 0.65 Hidden neuron H2: Weights: w1=-1.2, w2=2.0 Weighted sum: (0.8×-1.2) + (0.3×2.0) = -0.96 + 0.6 = -0.36 ReLU(-0.36) = 0 (negative, so zeroed out) Activation: 0 Output neuron: Inputs: H1=0.65, H2=0 Weights: wH1=0.9, wH2=0.4 Weighted sum: (0.65×0.9) + (0×0.4) = 0.585 Sigmoid(0.585) ≈ 0.642 Output: 0.642 Interpretation: 0.642 — closer to 1 than to 0, so the network predicts class 1 with 64% confidence. Every neural network, no matter how large, runs this same operation: weighted sum, activation, next layer.

What the Network Is Actually Doing

The forward pass looks like arithmetic — and it is — but something remarkable happens across layers. The first hidden layer detects simple features. If the input is an image, early neurons might activate strongly for edges, colors, or corners. The second hidden layer combines those features into textures and shapes. Deeper layers combine those into parts of objects. The output layer combines those into a final decision. This hierarchical feature learning is what emerges from training with stacked layers and activation functions. Nobody programs 'look for edges first.' The network discovers useful intermediate representations on its own, guided only by the goal of getting the output right.