Classification vs Regression

Supervised learning splits into two major families depending on what kind of output you need. One family predicts a category — is this tumor malignant or benign? Which digit is handwritten here: 0 through 9? The other family predicts a number — how much will this house sell for? What will the temperature be tomorrow? Getting this distinction right is the first decision in any supervised learning project, because it determines which algorithms apply, which loss functions make sense, and how you evaluate success.

Classification: Predicting Discrete Categories

Classification is the supervised task where the target variable y belongs to a finite set of categories, called classes. Binary classification involves exactly two classes. The canonical encoding is y ∈ {0, 1}, where 0 is the negative class and 1 is the positive class. Examples: - Spam detection: spam (1) or not spam (0) - Medical diagnosis: disease present (1) or absent (0) - Fraud detection: fraudulent transaction (1) or legitimate (0) Multiclass classification involves three or more classes. y ∈ {0, 1, 2, ..., K-1} for K classes. Examples: - Handwritten digit recognition: y ∈ {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} - Plant species identification: y ∈ {species_1, species_2, ..., species_200} - Document topic classification: y ∈ {sports, politics, finance, science} A worked example: a classifier receives the feature vector x = [age=45, resting_heart_rate=92, BMI=31.2] and must predict y ∈ {low_risk, medium_risk, high_risk} for cardiovascular disease. It outputs one of the three categories — not a number on a spectrum.

Evaluating classification performance requires metrics suited to discrete outputs. The most common are: Accuracy: proportion of predictions that match the true label. Accuracy = (correct predictions) / (total predictions). Intuitive, but misleading when classes are imbalanced — a model that always predicts 'not fraud' achieves 99.9% accuracy if only 0.1% of transactions are actually fraudulent. Precision and Recall: precision measures how often positive predictions are actually positive. Recall measures how many actual positives the model caught. In medical diagnosis, high recall is often critical — you would rather flag too many potential cases than miss real ones. F1 Score: the harmonic mean of precision and recall. F1 = 2 × (precision × recall) / (precision + recall). Balances both concerns into a single number.

Regression: Predicting Continuous Values

Regression is the supervised task where the target variable y is a real number — it can take any value along a continuous scale. Examples: - House price prediction: y might be $347,000 or $891,500 — any positive number. - Temperature forecasting: y = 23.7°C tomorrow afternoon. - Stock return prediction: y = -2.3% daily return. - Age estimation from bone density scan: y = 42 years. A worked example: a regression model receives x = [square_footage=1850, bedrooms=3, distance_to_city_km=12, year_built=1998] and must predict y = sale price in dollars. The model output is a single real number. Evaluating regression performance: Mean Absolute Error (MAE): average of |ŷᵢ - yᵢ| across all test examples. Interpretable — same units as the target. A house-price model with MAE = $18,000 is wrong by $18,000 on average. Mean Squared Error (MSE): average of (ŷᵢ - yᵢ)². Penalizes large errors more heavily because squaring amplifies them. MSE = (1/N) Σ (ŷᵢ - yᵢ)². Root Mean Squared Error (RMSE): square root of MSE, restoring units to the original scale. Most commonly reported for regression.