How AI Agents Work: The Roles of the LLM, Tools and Memory, Explained

The content of this article was checked against each vendor's official documentation and public materials on October 2, 2026. Product features may change. Check the sources at the end for the latest details.

How an AI agent works is easier to understand when you look at its parts. This article divides it into four components: the LLM, tools, memory, and the plan-and-act loop.

An AI agent is made of four parts

The AI Guidelines for Business (version 1.2) from Japan's Ministry of Internal Affairs and Communications (MIC) and Ministry of Economy, Trade and Industry (METI) define an AI agent as an AI system that senses its environment and acts autonomously to achieve a specific goal. Autonomy here includes not only a high degree of autonomy but also systems with some degree of it.

Anthropic describes the foundation of an agent as an LLM augmented with retrieval, tools and memory. It adds that agents are often LLMs using tools based on feedback from their environment, running in a loop.

In terms of underlying technology, three things form the foundation: a language model, tool calling and stored history. The fourth part, the loop, is the flow that connects the other three. The LLM decides the next step, a tool runs, the result goes into memory, and the LLM thinks again. The sections below look at each part.

LLM: the part that thinks and decides the next step

An LLM is a language model trained on large amounts of text. Inside an AI agent, its role is to read the instructions and the current situation and decide what to do next. The LLM also chooses whether to return an answer or call a tool.

OWASP materials explain that current agents use an LLM as the center of reasoning and let the LLM decide the app's processing flow. The official Dify documentation also recommends choosing a model for agents that reasons well and supports tool calling natively.

In Dify, how the agent runs depends on the model you choose. Models with native tool-calling support use the Function Calling approach and call tools directly. Models without it run with the ReAct approach, and Dify uses prompt techniques to make them use tools. When to use which tool and how to read the results depends on the model's reasoning ability.

LLM output also comes with cautions. Japan's Personal Information Protection Commission (PPC) warns that generative AI responses are produced from probabilistic correlations and may contain inaccurate content. In an AI agent, this output leads to the next action.

Tools: the part that runs search, files and external services

An LLM alone can only return text. Give it tools, and it can search the web, calculate, run code and call external service APIs. OWASP materials list this kind of tool use as one of the core abilities of agents.

• n8n: the AI Agent node needs at least one tool connected. HTTP Request, calling another workflow, running code and more can be tools
• OpenAI Agents SDK: Python functions can be used directly as agent tools
• MCP: a common protocol for connecting agents and tools. OWASP materials describe the agent side as the client and the tool side as the server

How a tool is used changes with how its description is written. Anthropic recommends putting as much effort into tool definitions as into the overall prompt. When it built a coding agent, it says it spent more time improving the tools than the overall prompt.

For example, after the agent moved out of the working directory, the model made mistakes with a tool that took relative file paths. When the tool was changed to always require absolute paths, the model used it without mistakes. The advice is to write descriptions that make usage clear when read from the model's point of view, and to design arguments that are hard to get wrong.

Memory: the part that carries over conversations and past results

Each vendor's documentation calls this memory. Without a memory mechanism, every exchange with the AI starts from scratch. The official n8n documentation describes memory as what keeps the history of earlier messages so the conversation can continue.

OWASP materials split memory in two: short-term memory used only within one conversation, and long-term memory kept after the conversation ends. Memory holds not just the conversation but also the tools called and the information retrieved.

In n8n, the components called chains cannot use memory. The official documentation says to use an agent when you want the conversation to continue.

Memory is useful, but you need to decide what it keeps. If personal information or confidential content comes up in a conversation, that is saved too. OWASP materials list attacks that slip false or malicious content into memory as one of the main threats to agents. They also point out that false information can spread through memory and tool use, and that errors can compound.

The plan-and-act loop: repeating until the work is done

The loop is the very flow by which an AI agent moves work forward. The official OpenAI Agents SDK documentation describes it as follows.

In the OpenAI Agents SDK, when there are no tool calls and text in the defined format is produced, it is treated as the final output. When the set number of turns is exceeded, the loop stops. Dify's Agent also has a setting for the maximum number of cycles of thinking, calling tools and processing results. Raising the limit handles more complex work, but increases wait time and usage fees.

The official n8n documentation also explains that during a single workflow run, the agent runs many times: initial setup, calling tools, evaluating tool results and replying to the user.

Anthropic recommends that agents check the actual situation at each step, using tool results and other feedback. Agents can also ask for human judgment at checkpoints or when they get stuck. In n8n, you can require human approval before a specific tool runs. If approved, it runs. If rejected, it is canceled.

Strengths and weaknesses that follow from how it works

Because the LLM decides the steps on the spot, AI agents are strong at work that differs a little each time. For the same reason, their behavior is harder to predict in advance.

Anthropic recommends predictable workflows for work with set steps. Because agents act autonomously, costs rise and errors can compound. It therefore recommends thorough testing in an isolated environment and adding safety mechanisms.

Anthropic also lists conditions for work where agents are especially useful: it needs both conversation and action, has clear success criteria, allows results to be reviewed and corrected, and allows meaningful human oversight. These four can be read as the conditions for the loop to work well.

The appendix to the AI Guidelines for Business notes that, while acting autonomously, an AI agent may order products or delete files that people did not intend. Added countermeasures are mechanisms that include human judgment, minimum necessary permissions and regular reviews of action history.

The difference between leaving the steps to AI and having a person set them is covered in Types of AI agents. For how agents differ from systems that search internal documents and answer, see AI agents vs RAG.

Choosing an AI employee on Employee Store

Employee Store is a marketplace where companies can adopt AI agents (AI employees) built by developers. Pricing is one-time, monthly, or an upfront fee plus monthly, and payments are processed by Stripe.

When choosing, check the following on the listing page for each of the four parts.

• LLM: which model it uses, and who pays the usage fees
• Tools: which services it connects to, and whether it only reads or also writes and sends
• Memory: what it keeps, and where it is stored
• Loop: the iteration limit, and where it stops for human approval

After purchase, you can message the seller on the page for each deal and review and accept the deliverables. The idea behind AI employees is explained in What is an AI employee.