Learning as Finding a Function

Before the first line of code, before a single neuron fires, there is a mathematical idea so clean it can be stated in one sentence: machine learning is a search for a function. Everything else — neural networks, training data, loss functions, optimizers — is machinery built in service of that search. Understanding this framing precisely will let you reason about any ML system you ever encounter.

What Is a Function, Precisely?

A function is a rule that accepts an input and produces exactly one output. Formally, a function f maps every element of a domain X to exactly one element of a codomain Y, written f: X → Y. In machine learning the domain X is the space of all possible inputs — every image a camera could capture, every sentence a person could write, every set of sensor readings a robot might record. The codomain Y is the space of all possible outputs — categories, numbers, probability distributions, sequences. A spam filter is a function f: Email → {spam, not_spam}. A housing-price predictor is a function f: (bedrooms, sq_ft, location, ...) → price_in_dollars. A language model generating the next word is a function f: sequence_of_words → probability_over_vocabulary. In every case, the job is the same: find a function that correctly maps inputs to outputs.

Why not just write the function by hand? For simple cases you can. A function that converts Fahrenheit to Celsius is f(x) = (x − 32) × 5/9 — exact, derivable from physics, no data needed. But consider the spam-filter function. The rule 'if subject contains free money then spam' fails immediately on 'I have free money to lend your nonprofit.' The space of all possible emails is astronomically large, the patterns that separate spam from legitimate mail are subtle and shifting, and the rule set would require millions of hand-crafted exceptions. No human can write this function. But given enough labeled examples (email, label), an algorithm can search for a function that approximates it remarkably well. This is the regime machine learning occupies: problems where the true function exists and is consistent, but is too complex for humans to specify explicitly.

Input Spaces and Output Spaces

The domain X and codomain Y are not vague — they have geometry. An input space might be a 784-dimensional space (one dimension per pixel in a 28×28 image), or a 300-dimensional word-embedding space, or a 12-dimensional sensor space. The function must be defined over every point in that space. This has a critical implication: when a user gives your model an input it has never seen — a photograph taken in fog, a sentence in a dialect not in the training set — the function still has to produce an output. The question of how well that output holds up is the question of generalization, which we will develop carefully in Lesson 5. For now, hold the picture: you are searching for a rule, written in the language of mathematics, that transforms any point in a high-dimensional input space into the correct point in an output space. The function you find will almost certainly be imperfect. Learning is the art of finding the best imperfect function you can.