Estimating Effort

Have you ever thought a homework assignment would take 20 minutes and then looked up three hours later, still not finished? Or the opposite: spent all weekend dreading something that turned out to be a ten-minute task? Both of those experiences are failures of effort estimation — and they affect AI agents just as much as they affect people. A plan built on wildly wrong estimates will run out of time, skip critical steps, or waste resources on low-priority work.

What Does Estimating Effort Mean?

Estimating effort means predicting, before you start a task, how much work it will require. That work might be measured in time — how many minutes or hours? In computation — how many API calls, how much memory? In money — how many tokens, how much bandwidth? In human attention — how many decisions must a person make? Different kinds of tasks need different estimates. For a data processing step, the relevant estimate might be how many records to process. For a web search, it might be how many searches and how long each takes. For writing a report, it might be how many sections and how complex each is. Good estimates are not perfect predictions. They are informed guesses, made using past experience, known constraints, and explicit reasoning about the task's components. The goal is not accuracy to the minute — it is avoiding catastrophically wrong expectations.

Software engineers have developed a rich toolkit for effort estimation over decades. One widely used technique is story points: instead of guessing exact hours, developers rate each task on a relative scale. A tiny task gets 1 point. A moderate task gets 3 or 5. A huge task gets 13 or 21. The exact numbers do not matter — the relative relationships do. A 5-point task takes roughly five times as much effort as a 1-point task. Teams use historical data about how many points they complete per week to project when a project will finish. A simpler technique that AI agents can use is analogical estimation: compare the current task to a similar past task. The previous task took X resources — this one is slightly larger, so estimate 1.3X. This is how experienced engineers estimate too: by pattern-matching against memory.

Why Estimates Are Hard

Effort estimation is notoriously difficult for several reasons. Unfamiliar tasks have no analogies to draw on — you are estimating something you have never done before. Hidden complexity lurks inside what looks simple: a task that seems small might open up into a rabbit warren of sub-tasks once you start. And optimism bias causes humans — and poorly calibrated agents — to consistently underestimate how long things take, especially rare or complex tasks. The planning fallacy, described by psychologist Daniel Kahneman, is the well-documented tendency to assume everything will go according to the best-case scenario. Projects routinely take twice or three times as long as estimated. The cure is reference class forecasting: instead of estimating this specific task in isolation, ask how long similar tasks have taken historically, and use that as your baseline. AI agents can be designed with calibration in mind: they compare their estimates against actual completion times as they work and gradually adjust their models. This is a form of learning from experience built into the planning system itself.