Features: What the Machine Looks At

A detective solving a case does not look at everything in the room equally. They focus on the clues that matter — a footprint, a smudge, a time stamp on a receipt. A machine learning model works the same way. Before it can learn anything, someone has to decide: which pieces of information are the relevant clues? Those pieces are called features, and choosing them well is one of the most important decisions in building any machine learning system.

What Is a Feature?

A feature is any measurable property of an example that is given to the model as input. When a model tries to predict whether an email is spam, possible features include the number of exclamation points, the length of the subject line, whether the word 'FREE' appears, and the sender's domain name. Each of these is a single measurable piece of information. Features can be numbers (the temperature is 22°C), categories (the color is blue), or even binary flags (does the email have an attachment — yes or no). Before training begins, the raw data — a photo, a sentence, a spreadsheet row — is converted into a list of feature values. That list is what the model actually sees and learns from.

Consider a model trained to predict whether a patient has a high risk of heart disease. Possible features include: age, blood pressure, cholesterol level, whether they smoke (yes/no), family history of heart disease (yes/no), and body mass index. A doctor would recognize all of these as clinically relevant clues — and that recognition matters. Good features are chosen because domain knowledge says they are connected to the outcome. A model trained on irrelevant features — say, shoe size and favorite color — would fail to make useful predictions. It would still find some pattern in the training data (patterns exist everywhere if you look hard enough), but those patterns would not generalize to new patients. This is why feature selection — deciding which features to include — is as important as any step in the training process.

Good Features and Bad Features

Not all features are equal. Good features have three qualities. Relevance: the feature is actually connected to what you are trying to predict. Blood pressure is relevant to heart disease. Favorite cereal is probably not. Variability: the feature takes different values across your examples. If every patient in your dataset is exactly the same age, age carries no useful information — it cannot separate high-risk from low-risk patients. Availability: you can actually measure this feature for new examples after training. A feature that was only available in the training data and cannot be collected for real predictions is useless. There is also a dangerous type of bad feature: one that is correlated with the outcome in the training data by accident, not by any real connection. For example, in a dataset collected at one hospital, the ZIP code might correlate with heart disease — not because ZIP code causes heart disease, but because that hospital happened to serve a neighborhood with older, sicker residents. A model trained on this data might use ZIP code as a feature, and its predictions would be wrong and unfair when applied more broadly.