Anthropic puts it in six words in its guide to context engineering: agents are "LLMs autonomously using tools in a loop." The rest of this article unpacks that sentence.
In short
- The model decides. Tools let it act. Context is what it knows right now. Instructions define the job.
- An agent loops: think, call a tool, read the result, decide again, until it finishes or hits a limit.
- Agentic workflows are processes where the model iterates (plans, uses tools, checks its own work) instead of answering once.
- RAG is how an agent looks up your facts before answering.
- Agents shine on open-ended tasks where the steps can't be predicted. They're a poor fit for tasks a simple rule could handle.
- The main risks are compounding errors, cost, prompt injection and too many permissions. Human approval belongs on the actions that matter.
The four ingredients of an agent
1. The model (the LLM)
A large language model predicts text. That sounds small, but it means it can read an email, summarize a call, extract fields from a document or decide which of five options fits best, all described in plain language. In an agent, the model is the decision-maker. It doesn't store your data or run your systems. It reads what it's given and says what to do next.
2. Tools
On its own, a model can only produce text. Tools let it act on the world: search the web, query a CRM, send an email, run code, look up a calendar.
OpenAI's function calling guide describes how this works in practice:
- Your application sends the model a request and a list of tools it may use.
- The model replies with a tool call: which tool, with which parameters.
- Your code executes that call. The model never runs it directly.
- You send the result back to the model.
- The model gives a final answer or asks for another tool.
Step 3 is the most important detail for anyone building agents in business. The model requests. Your system decides what's allowed. That's where permissions, validation and approvals live.
3. Memory and context
Models don't remember anything between calls by themselves. Everything an agent "knows" during a task is what sits in its context: the instructions, the conversation so far, tool results and any documents you retrieved.
Anthropic's context engineering article makes a point that surprises beginners: context is a finite resource. As more tokens pile in, the model's ability to recall details can degrade, a problem the article calls context rot. More information isn't automatically better. The skill is giving the agent the right information at the right step.
In practice, "memory" comes in two kinds:
- Short-term memory: the history of the current conversation or task, kept in context.
- Long-term memory: facts stored outside the model (a database, a CRM, a document store) and pulled back in when relevant.
4. Instructions
The system prompt defines the goal, the tone, the rules, the tools to prefer and when to stop or escalate. Weak instructions are the most common reason an agent wanders.
RAG, in plain language
Retrieval-augmented generation (RAG) is how you get a model to answer with your facts. AWS defines it as making a model reference an authoritative knowledge base outside its training data before it responds.
Think of an open-book exam:
- The question arrives. "What's our refund policy for annual plans?"
- Retrieve. The system searches your documents for the most relevant passages. Often this uses a vector database, which finds text by meaning rather than exact keywords.
- Augment. Those passages are placed in the model's context with the question.
- Generate. The model writes the answer grounded in what it just read.
For an agent, RAG is often just one tool among others: "search the knowledge base." The agent decides when it needs to look something up.
RAG reduces made-up answers, but it doesn't eliminate them. If retrieval pulls the wrong passage, the model will confidently answer from the wrong passage.
Agentic workflows: why iteration matters
Andrew Ng has written about agentic workflows in his letters in The Batch. In one of them, he contrasts asking a model to produce an answer in one pass with letting it work iteratively: draft, review, revise. He describes four design patterns:
- Reflection: the model critiques and improves its own output.
- Tool use: the model calls tools to gather information or take action.
- Planning: the model breaks a goal into steps and executes them.
- Multi-agent collaboration: several agents with different roles work on one task.
In the same letter, Ng reports that on a coding benchmark, an older model wrapped in an agent loop outperformed a newer model answering in a single pass. The broader lesson for builders: how you structure the work can matter as much as which model you pick.
Workflows vs agents: a spectrum, not a switch
Anthropic's guide Building effective agents separates workflows (models and tools following code-defined paths) from agents (the model directing its own process). It describes common workflow patterns such as prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer, and it advises starting simple and adding autonomy only when needed.
It helps to see autonomy as levels:
| Level | What the model does | Example |
|---|---|---|
| 0. Single call | Answers once | Summarize this call transcript |
| 1. AI step in a workflow | Makes one decision inside a fixed path | Classify this reply, then rules route it |
| 2. Chained workflow | Several AI steps in a fixed order | Extract fields → draft email → check draft |
| 3. Bounded agent | Chooses tools within a narrow goal | Research this account using three allowed tools |
| 4. Open agent | Plans and acts over many steps | Handle this support case end to end |
Most revenue work is best served at levels 1 to 3. We dig into that choice in AI agents vs automation.
Human in the loop
The more autonomy an agent has, the more deliberately you place people. Useful patterns:
- Approval before action: the agent prepares the email or the CRM change; a person approves it.
- Escalation on uncertainty: when the input is unusual or the agent can't find an answer, it hands off with a summary instead of guessing.
- Sampling review: a person checks a sample of completed tasks every week and feeds corrections back into instructions.
This is the shift MitHub describes as moving from Doer to Director: you stop doing every task by hand and start directing systems and judging their output. More on that in Director vs Doer.
The limits of AI agents
Compounding errors. In a multi-step loop, a small misreading in step two becomes a wrong action in step six. Anthropic's guide warns that autonomy brings higher costs and the potential for compounding errors, and recommends testing in sandboxed environments with guardrails.
Unpredictability. The same input can produce a different path. That makes agents harder to test than workflows. You need a set of real test cases and you need to measure results, not eyeball one run.
Cost and speed. Each loop is another model call. An agent that takes eight steps to do what a rule does in one is slower and more expensive.
Prompt injection. Agents read outside content: emails, web pages, documents. OWASP lists prompt injection as the top risk for LLM applications, including indirect injection where instructions hidden in external content change the model's behavior. An agent with permission to send email that reads a malicious email is a real attack surface.
Too many permissions. Every tool you give an agent is something it can misuse. Give it the minimum.
Garbage context. An agent is only as good as what it can see. Bad CRM data means bad decisions, made faster.
MitHub's agent job description
Before building an agent, write a one-page job description, the same way you would for a new hire. If you can't fill it in, you don't need an agent yet. You need a process map.
- Goal: one sentence. What does "done" look like?
- Trigger: what starts the task?
- Tools: the exact list it may use, and which ones are read-only.
- Context: what information it needs, and where it comes from.
- Limits: max steps, max cost, what it must never do.
- Handoff: when it stops and passes to a human, and what summary it leaves.
- Approval: which actions require a person to say yes.
- Proof: the metric and the record that show it did the job correctly.
Example (hypothetical)
Imagine a "meeting prep" agent for a sales team:
- Goal: before each first meeting, produce a one-page brief on the account.
- Trigger: a meeting is booked in the calendar.
- Tools: CRM read, company website fetch, internal notes search. No write access.
- Context: meeting details, contact record, past emails.
- Limits: max 10 tool calls; never contact the prospect.
- Handoff: if the company can't be identified, flag it to the rep.
- Approval: none needed. It only writes an internal note.
- Proof: reps rate each brief useful or not; target a rising share of useful ratings over four weeks.
Notice the design choice: read-only tools and an internal output make this a low-risk first agent. That's the kind of build that earns trust before you give an agent the ability to act.
Where to go next
Agents are one part of a bigger discipline. Start with the foundations in What is AI automation?, then see how to build agents and AI steps visually in n8n for AI automation. If you want a structured path from concepts to a build you can prove, follow How to learn AI automation, and when you're ready to pick a first project, Prove value fast shows how to choose the quick win.