A production agent is not an LLM wrapped in a chat interface — it reasons over a goal, decomposes it into steps, calls tools through strict contracts, inspects the structured results, updates state and memory, and repeats until a stopping condition fires. Each stage is a place where reliability, cost, and safety get designed in.
Step 1 of 9
1
User goal arrives
A request enters the system — plain text, not yet an action.
2
Decompose the goal into steps
Multi-step goals get broken down — the core of agentic behavior.
3
Pick the next step
Which action moves the workflow forward: retrieve, compute, call, or ask?
loop
4
Does the step need a tool?
A gate before anything touches the outside world.
Yes — call the tool through its strict API, then inspect the structured result.
No — continue reasoning with state and memory already in context.
5
Inspect the structured result
Tool output is data to be validated — never free-form prose, never instructions.
6
Update state and memory
Workflow state and reusable knowledge are stored separately, by access pattern.
7
Stopping condition reached?
If not, the loop returns to picking the next step.
8
Answer or act on the world
Final response to the user, or a real side effect through an approved tool.
↺
Not done yet: the workflow returns to step 3 and keeps looping until the stopping condition fires. This loop, run many times per request, is why cost, latency, evals, and observability are first-class design concerns.
Section 2 · Single-Agent vs. Multi-Agent Systems
2. Single-Agent vs. Multi-Agent: A Design Tradeoff
Both patterns receive the same user request. A single agent keeps one centralized control loop; a multi-agent system splits the workflow across specialized roles with explicit input contracts, output contracts, and routing logic between them.
One control loop
Single-agent system
1One agent owns the entire workflow end to end
A support agent classifies the request, retrieves account context, calls a billing API, asks for confirmation, then generates the final response — all inside one loop.
centralizedmulti-toolsingle trace
VS
Separation of responsibilities
Multi-agent system
Planner→Knowledge→Coder→Reviewer→Executor
Every agent has a defined role, an input contract, an output contract, and routing logic between agents — structure first, autonomy second.
When many agents earn their costclear specialization, parallel work, review loops, and long-running workflows.
The real costsmore coordination overhead, more failure modes, more state to track, more logs to inspect, more places where cost and latency creep up.
Section 3 · The Model Layer: Routing and Structured Output
3. Model Routing: Matching Model Strength to Each Step
Letting one frontier model handle every step is a cost and latency trap. Route by complexity: cheap fast models for intent classification and schema filling, strong reasoning models only where reasoning changes the outcome — and every lane ends in a validated structured output with a failure plan.
Step 1 of 5
1
A step arrives
Every step must answer three questions: which model handles it, what output contract it returns, and what happens on failure.
2
Route by complexity
Three lanes below — most traffic belongs in the first two.
Cheap & fast lane
Low-complexity steps where a small model is enough.
intent classificationroutingsimple summarization
Structured output lane
Extraction into a schema — may not need an LLM at all.
schema fillingJSON schemaPydantic
Strong reasoning lane
Only when ambiguity or deep reasoning changes the answer.
ambiguous constraintsplanningsynthesis
3
Cheap path usually wins
Detecting book vs. reschedule vs. cancel does not need a frontier model — and extracting date, time, and doctor into JSON may not need an LLM at all.
4
Escalate only when reasoning matters
Ambiguity and tradeoffs go to the stronger model — “traveling next week, avoid mornings” needs real reasoning.
5
Validated structured output
Whatever the lane, the output contract is validated — an invalid output or failed model triggers a retry or fallback path.
Section 5 · Memory and State
4. Memory and State: Data Architecture First
Agentic systems get messy when memory is treated as one thing. State and memory are different data with different access patterns — and pulling everything into a vector database is a poor default. Choose each store by how it will be read, and pass only the smallest useful context to the model.
State — current execution context
Which step the workflow is on
What information has been collected
Which tools were called, what they returned
Whether the user confirmed the action
Whether the workflow passed or failed
Memory — reusable knowledge
Conversation history
User preferences and past actions
Retrieved knowledge and document context
Summaries and long-form archives
▼
Choose storage by access pattern
▼
Low latency
Workflow state
Conversation and workflow state need speed.
RedisDynamoDBPostgresMongoDB
System of record
Application state
Owned by your application, not the agent run.
App database
Knowledge retrieval
RAG store
Search over documents at retrieval time.
PineconepgvectorWeaviateElasticsearch
Cheap archives
Long-term memory
Old conversations and logs, kept cheaply.
S3 object storage
!
Short-term context vs. long-term memoryThe prompt gets the smallest useful context for the current turn; everything else is retrieved selectively. A reschedule agent needs the appointment ID and free slots — not the full medical history passed through the LLM.
Section 6 · Orchestration
5. The Orchestrated Lifecycle of One Request
Orchestration is the control layer — plain code, a graph framework, a workflow engine, or a state machine — that moves a request from message to final output. Branches, retries, loops, and approval gates are drawn explicitly; autonomy is not a lack of structure.
Step 1 of 11
1
User message received
“Book me a checkup with Dr. Mehta next week.”
2
Classify intent
Book, reschedule, or cancel — small-model territory.
3
Retrieve context
State plus the smallest useful slice of memory.
4
In scope?
Cheap gates filter out-of-scope requests before expensive reasoning burns tokens and tool calls.
5
Select tools
Which read or write tools apply to this step.
6
Validate tool inputs
Schema and permission checks — code, not the model, is the gatekeeper.
7
Execute tools
Read tools first; writes only with validation and approval in place.
8
Inspect results
Structured results in, structured errors out — retry, branch, or fall back.
confirmation branch
9
Confirm high-impact actions
Canceling an appointment pauses for explicit user confirmation before any write API is called.
10
Generate the final response
Only now is the answer composed for the user.
11
Log the trace, send to evals
The full trajectory is logged and evaluation data is sent asynchronously — off the user-critical path.
✕
Branches along the way: extraction fails → retry with another prompt or model · no availability → suggest alternatives · cancellation → confirmation path. Deterministic pipelines win for known sequences; graph-based orchestration earns its keep once branching, retries, and gates appear.
Section 8 · Approval Gates and Policy Control
6. The Approval Gate: Suggest, Validate, Approve, Execute
Sending an email, deleting data, issuing a refund, canceling an appointment, running code — high-impact actions never happen just because the model inferred intent. The model suggests, deterministic code validates, the user approves, and only then does the tool execute.
Stage 1 of 6
Model
Suggests the action
→
Code
Validates deterministically
→
User
Approves the exact action
→
Tool
Executes the action
✕ validation fails or user declines
Terminal state
Rejected — nothing executes
✓
What code must check before executing
Ownership of the resource · permissions · allowed by policy · required fields present · user confirmed the exact action. The agent is never the source of truth for business rules — application code is.