Handbooks / RAG & AI Agents / Chapter 4

Agent Loops and Memory

7 pages · ~5 min✓ Reviewed

Builds on Tool Calling. Next up: Guardrails and Evaluation.

RAG & AI AgentsPage 1

The ReAct Loop Mental Model

1. Agent = LLM + Tools + Loop

  • Agent = LLM + tools + loop
  • The model picks the next action
  • Code executes it; the result goes back into the context
  • Repeat until a final answer

2. The ReAct Cycle

  1. Thoughtreason about next step
  2. Actiontool call + arguments
  3. Observationtool result
  4. Final answerloop ends
Plain CoTReAct
Facts fromprompt onlyreal tool observations
Fetch data?noyes
Hallucinationhigherlower

3. Example: Weather in the capital of France?

  1. 1Thoughtneed the capital
  2. 2Actionsearch('capital of France')
  3. 3ObservationParis
  4. 4Actionget_weather('Paris')
  5. 5Final answerweather in Paris
One iteration

Thought → Action → Observation. The loop ends only on a final answer

4. Interview Point

  • Workflow vs agent? Workflow: code fixes the steps. Agent: the model picks them.
  • Why less hallucination than CoT? Each step is grounded in real tool observations.
  • One loop iteration? Thought, Action, Observation.
RAG & AI AgentsPage 2

Planning Patterns

1. Plan-and-Execute and Re-planning

  • Planner LLM writes the full step list up front
  • Executor runs each step in order
  • Fits long, predictable tasks
  • Re-plan after each step or failure: check plan against observations, then revise
  • A static plan breaks when a tool result surprises you
  1. Planall steps
  2. Executeone step
  3. Checkvs observation
  4. Reviseon surprise

2. ReAct vs Plan-and-Execute

ReActPlan-and-execute
Decidesone step at a timeall steps up front
Strengthflexiblecoherent
Weaknessmyopicrigid
Costmore LLM callscheaper
Use foradaptive taskslong, predictable tasks

3. Reflection Loop

  1. 1Draftfirst output
  2. 2Critiquemodel judges itself
  3. 3Reviseretry, then repeat
Cap the rounds

Limit reflection to 1-3 rounds. Every round adds LLM calls and latency.

RAG & AI AgentsPage 3

Short-term vs Long-term Memory

1. Two Memories

Short-termLong-term
IsContext windowExternal store: vector DB, KV, files
HoldsMessages, tool calls, resultsFacts, preferences, outcomes
LifetimeLost at session endPersists across sessions
AccessSize-limitedSimilarity or key lookup

2. Stay Within Budget

Sliding window

Keep recent turns

Summarize

Compress old turns

Trim

Cut bulky tool outputs

3. Long-term Write Policy

  • Write distilled facts: preferences, decisions, outcomes
  • Never raw transcripts
  • Read: retrieve top-k relevant items, inject into the prompt
  1. 1Query
  2. 2Search storesimilarity or key
  3. 3Top-k
  4. 4Prompt
Mistake

Storing raw transcripts bloats the store and buries useful facts.

4. Interview Point

  • Storage and lifetime? Context window, gone at session end, vs an external store that persists.
  • Window full? Slide the window, summarize old turns, trim tool outputs.
  • What to write, how to read? Distilled facts; retrieve top-k by similarity or key.
RAG & AI AgentsPage 4

Multi-Agent Patterns

1. Three Patterns

PatternShapeUse for
Orchestrator-workerLead splits, delegates, mergesMost common
Pipeline / handoffresearcher → writer → reviewerSequential stages
Debate / criticPropose, critique or voteHard, high-stakes answers
  1. Leadsplits task
  2. Workersspecialists
  3. Leadmerges results

2. Single vs Multi

  • Use multi-agent only if subtasks are parallel
  • Or they need separate contexts
  • Or they need different tools or prompts
  • Otherwise start with a single agent
Cost

Multi-agent multiplies token cost and adds coordination failures.

3. Interview Point

  • Single enough? Yes, unless subtasks are parallel or need separate contexts, tools or prompts
  • Sharing context? Lead sends each worker a task brief; workers return results the lead merges
  • Specific failures? Lost context at handoffs, conflicting outputs, runaway cost, error cascades
RAG & AI AgentsPage 5

Stopping Conditions, Cost and Latency

1. Stop Signals and Loop Failure

  1. Final answerno tool call
  2. Step capmax iterations
  3. Budgettokens or time
  4. Interrupthuman or guardrail
  • Same tool + same args repeated = never converges
  • Detect the repeat, then break out or force a different strategy
python
key = (call.name, json.dumps(call.args, sort_keys=True))
if key in seen:
    raise LoopDetected(key)  # or re-plan
seen.add(key)

key = tool name + sorted args

2. Cost

MitigationEffect
Prompt cachingre-sent prefix costs less
Smaller modelhandles easy steps
Trimmed tool outputsfewer tokens per step
Hard step capbounds total spend
Why cost snowballs

Every step re-sends the whole context, so step n pays for all earlier steps. Total cost grows faster than linearly.

3. Latency

LeverHow
Parallel callsrun independent tools together
Streamingshow tokens as they arrive
Fewer stepscut needless loop iterations
Fast modelroute simple requests to it
RAG & AI AgentsPage 6

Agents Cheat Sheet

1. The Loop

  1. Thoughtnext step
  2. Actiontool call
  3. Observationtool result
  4. Repeat
  • Stop on a final answer
  • Step cap reached
  • Budget (token or time) spent
  • Guardrail interrupts

2. Planning and Memory

TopicRule
ReActadaptive tasks
Plan-and-executelong, predictable tasks
Re-planon surprises
Reflect1-3 rounds at most
Short-termcontext window: window, summarize, trim
Long-termexternal store, distilled facts, top-k

3. Multi-Agent and Cost

  • Orchestrator-worker by default
  • Start with a single agent
  • Cost: step cap, prompt caching, small model for easy steps
  • Parallel tool calls cut latency
Stuck loop

Same tool call with same arguments never converges. Detect repeats, then break out.

RAG & AI AgentsPage 7

Check yourself

An agent calls search('Paris weather') with the same arguments five times in a row. What is wrong, and what do you do?
  • It is stuck: the same call and arguments give the same observation, so it never converges.
  • Detect repeats, then break out or force a different strategy. A step cap guarantees it stops.
Each step of a 10-step loop adds 1k tokens to the context. Is the total input about 10k tokens?
  • No. Every step re-sends the whole context: 1k + 2k + ... + 10k = 55k.
  • Cost grows faster than linearly. Prompt caching and trimmed tool outputs reduce it.
A plan-and-execute agent finds no flights in step 2, yet runs step 3, "book a hotel for those flights". What is the bug?
  • The plan is static and never checked against the observation.
  • Re-plan after a surprise or failure: revise the remaining steps.