Handbooks / Generative AI & LLMs / Chapter 7

Agents and Tool Calling

7 pages · ~5 min✓ Reviewed

Builds on Fine-Tuning vs Prompting. Next up: Guardrails and Red-teaming.

Generative AI & LLMsPage 1

Function Calling and the Agent Mental Model

1. Tool Definition

json

{"name": "get_order",
 "description": "Get order status by id",
 "input_schema": {"type": "object",
 "properties": {"id": {"type": "integer"}},
 "required": ["id"]}}

you send this list of tools with the messages

Three parts: name, description, input_schema
description is the model's main cue for when to use the tool
input_schema is JSON Schema: type object, properties, required

2. The Tool Call

Model returns {type: tool_use, id, name, input}
stop_reason is tool_use when it wants a tool run
The model never executes anything: your code runs the tool

3. Chatbot vs Agent

	Chatbot	Agent
Model calls	one	a loop
Tools	none needed	calls, reads results
Ends when	reply is sent	task is done

4. Schema Tips

enum limits a value to allowed choices
Add a description to every property
List every needed field in required

Vague schema

A vague schema gives wrong or missing arguments. Constrain it.

Generative AI & LLMsPage 2

The Agent Loop: Observe, Think, Act

1. The Loop

Observemessages + latest tool results
Thinkmodel picks next step
Actemit tool call
Repeatuntil no tool call

Loop ends when the model answers without a tool call
Your code runs the tool; the model only emits tool_use

2. Loop in Code

python

while True:
    resp = model(messages, tools)
    messages.append({"role": "assistant", "content": resp.content})
    if resp.stop_reason != "tool_use": break
    results = [run(b) for b in resp.content if b.type == "tool_use"]
    messages.append({"role": "user", "content": results})

fragment: model, tools and run are yours

3. History Rules

Stateless model

The full growing message list is resent every iteration, so context and cost grow per step.

Generative AI & LLMsPage 3

Tool Results Back into Context

1. The tool_result block

Sent as a user-role message, never assistant
Carries the matching tool_use_id
content is text or a JSON string

python

block = {
  "type": "tool_result",
  "tool_use_id": call.id,
  "content": json.dumps(data),
}
messages.append({"role": "user", "content": [block]})

call site only

2. Shape what goes back

Situation	Do
Failure	is_error: true plus a useful message, not a crash
Large output	Truncate, paginate or summarize
Raw API payload	Return only fields the model needs

A clear error lets the model retry or change approach
A 50k-token dump bloats context and degrades reasoning
Unfiltered payloads waste tokens and may leak secrets

3. Output is untrusted

Web pages and files can hide prompt injection text

Generative AI & LLMsPage 4

ReAct and Planning

1. The ReAct Loop

ReAct = Reason + Act
Each cycle: thought, action (tool call), observation
Real observations ground the reasoning

Thoughtneed order status
Actionget_order(id=42)
Observationshipped
Thoughtenough, answer

2. Plan-and-Execute vs ReAct

Plan-and-execute: write the full step list up front, then run it
Cheaper and more predictable
Brittle if early steps change the situation

	ReAct	Plan-and-execute
Adapts	step by step	needs re-plan on failure
Calls	more, can wander	fewer
Shape	loose	structured

3. Hybrid in Practice

Generative AI & LLMsPage 5

Stopping Conditions and Tool Permission Safety

1. How an Agent Stops

Natural stop: reply with no tool call (stop_reason is end_turn)
Never rely on it alone: a confused model may never stop
On a guard hit, return a partial result plus why it stopped

Hard guard	Typical limit
Max iterations	10-25 steps
Token or cost budget	fixed cap per run
Wall-clock timeout	fixed seconds per run

2. Detect Loops

Same tool, same arguments repeated
No progress across steps: results add nothing new
Then break out or ask a human for help

Tool callname + args
Seen before?or no progress
Break outpartial result or ask help

3. Permissions and Approval

Generative AI & LLMsPage 6

Agent Tool Calling Cheat Sheet

1. The Loop

Define toolsname, description, schema
Sendmessages + tools
tool_usestop_reason
Run toolyour code
Appendassistant turn + tool_result
end_turnloop stops

2. Tool Def and Result Block

python

tool = {
  "name": "get_order",
  "description": "Look up an order",
  "input_schema": {"type": "object",
    "properties": {"id": {"type": "integer"}},
    "required": ["id"]}
}

description is the model's cue for when to use it

tool_result field	Holds
`type`	`tool_result`
`tool_use_id`	id of the matching `tool_use`
`content`	text or JSON string
`is_error`	true on failure, so the model can retry

3. Guards and Safety

Max steps, token or cost budget, wall-clock timeout
Repeat-call detection: same tool, same arguments
On a hit, return a partial result with an explanation

Generative AI & LLMsPage 7

Check yourself

Your loop runs the tool, then appends only a user message holding the tool_result and calls the model again. What happens?

The API rejects the turn.
The assistant message with the tool_use must be appended first, so the result's tool_use_id has a match.

A tool's HTTP call returns 404 and your code raises an exception. What should the loop do instead?

Send a tool_result with is_error true and a clear message.
The model can then retry or change approach, and the loop keeps running.

An agent calls search("cats") with identical arguments 40 times and never answers. Which guards were missing?

A max-iterations cap and repeat-call detection.
On a hit, return a partial result with an explanation.