State Across Steps

An agent does not complete its task in a single call. It loops — plan, act, observe, update, repeat — until the goal is reached or a stopping condition triggers. Each iteration of that loop produces output: a tool call result, a partial answer, a decision that constrains future decisions. This output is the agent's state, and if state is not tracked and carried forward explicitly, every new iteration starts blind.

State management in agent systems is analogous to state management in any long-running program. A database migration script tracks which migrations have already run. A download manager tracks which file segments are complete. An agent tracks which subtasks are done, what information has been gathered, what decisions have been committed to, and what the current plan is. The difference is that an agent's state must be legible to a language model — it has to be expressible in text, because the model reads it as part of its prompt.

What Agent State Contains

Agent state is not a monolith — it is a collection of distinct information types that the agent must track simultaneously. Identifying these types explicitly helps in designing a state representation. Task status tracks which subtasks in the agent's plan are pending, in progress, or complete. For a research agent writing a report, this might be a list of sections with a status field for each. Accumulated results holds the outputs of completed steps — the retrieved documents, the summarized paragraphs, the code that was successfully generated. These are the building blocks the agent uses in later steps. Active constraints records decisions that have already been made and must not be contradicted. If the agent decided in step 2 to use Python 3.11, that constraint must be visible in steps 5 and beyond to prevent the model from switching to a different version. Plan state captures the current plan and any revisions made as new information arrived. Plans change — a well-designed state object tracks not just the original plan but the history of revisions and the reasons for them.

How State Is Stored and Injected

State can be stored in memory (a Python dictionary in the running process), in a database (persisted across crashes and restarts), or in the context window itself (the model's scratchpad). In practice, production agents use a combination. In-process state — a plain Python dictionary or object — is fast and simple but lost if the process crashes. It is appropriate for ephemeral agents completing short tasks in a single session. Database-persisted state — written to Postgres or a similar store after each step — is durable. It survives crashes and allows the agent to be paused and resumed, or even distributed across multiple machines. LangGraph, a popular agent orchestration library, stores its graph state in a checkpoint store using this pattern. Context-window state — the model's own scratchpad, sometimes called a scratchpad or reasoning trace — is the text the model produces before its final answer in chain-of-thought prompting. This is not persisted automatically; the application must capture and replay it.

The injection step is equally important. At the start of each loop iteration, the agent assembles the current prompt. This includes the original task, the current state (formatted as readable text or structured JSON), any fresh tool outputs from the previous step, and the model's next instruction. The formatting of state matters: a wall of raw JSON is technically correct but may cause the model to attend poorly. Well-labeled, human-readable state representations — with clear headings, concise labels, and explicit status markers — produce better model behavior.