What Makes an LLM an Agent

An LLM tells you what to do about a cancelled flight. An agent actually rebooks it.

My Amsterdam (AMS) to Delhi (DEL) flight is cancelled at 6 am. An LLM writes me helpful advice, but an agent actually helps me book the next flight. We can see it's the same model inside, but the result is very different. It's also why Claude chat today can search, run code and use connected tools, not just write a reply.

What an LLM does

An LLM takes text in and gives text back. You ask a question, it writes an answer, and that’s it. It doesn’t look anything up by itself, it doesn’t call anything, and nothing in the world changes because of its answer.

If my flight gets cancelled and I tell an LLM about it, I get a sensible paragraph about calling the airline and checking the next flights. Useful, but I still have to do all the work myself.

What makes it an agent

An agent is a program that works towards a goal and uses the LLM in a loop. It asks the LLM what to do next, does it using tools (search flights, check the booking etc.), looks at what came back and then goes again. The result of each tool call, including errors, decides what happens next. If the first flight it finds is sold out, it doesn’t stop there, it tries the next one.

It also doesn’t run forever. It stops when the goal is reached, or when it hits a limit like max steps, an error it can’t get past, or a human telling it to stop.

The LLM stops after one answer. The agent keeps going round the loop, and what the tools return (even a sold-out flight) changes what it does next.

An agent doesn’t need an LLM

This was the part that surprised me. An agent is anything that senses its environment, decides and acts towards a goal. The deciding part can be plain rules.

The thermostat at home is a good example. It senses the room is at 18°C, compares it with the 20°C I set, and switches the heating on. No LLM, just a rule, and it still works towards a goal on its own.

The same sense, decide, act loop as Figure 1, but a fixed rule does the deciding. Swap the rule for an LLM and the agent can handle cases nobody wrote a rule for.

So what does the LLM add? Flexibility. Writing rules for a thermostat is easy. Writing rules for every possible flight cancellation is not, because every case is a bit different. The LLM can reason, generalise and understand plain language, so the agent can deal with situations nobody wrote rules for.

Bounded autonomy

Agents don’t get unlimited freedom, and I don’t see that as a weakness. As a developer I work within standards and requirements, and agents need the same. The limits are what make them safe, reliable and aligned with what we actually want.

For the rebooking agent, a sensible boundary could be: search flights, hold a seat, rebook in the same cabin and accept up to €50 fare difference on its own. Anything beyond that, like a €900 upgrade or moving the passenger to another airline, goes to a human.

Inside the fence the agent decides on its own. Outside it, a human decides.

We can compare compares an LLM without tools to a developer without an IDE or internet access. You may know a lot, but you can't get much done. I like this metaphor because it fits agentic systems really well. It also means the tools around an agent have to keep improving, otherwise the agent is stuck with whatever the LLM learned during training.

I believe agents are not a new concept. In a sense, they are just another program that helps execute certain decisions by acting based on different parameters. An LLM on its own is not extraordinary, but when it is used as an input for the agent program, it can make decisions that would be impractical to write as fixed rules.