
What Makes an AI Agent "Actually Useful"?
We don’t need more agents that can “summarize PDFs” or “book restaurants.” We need agents that help people get real work done — consistently, reliably, and intelligently.
Here’s what separates useful agents from gimmicks:
✅ 1. Has the context it needs
A useful agent understands not just this task, but the why, the when, and the who behind it. That context can be stored between sessions or supplied with the task. Persistent memory is one way to provide it. It isn’t a requirement for being useful.
Two illustrative examples, invented to show the difference:
- A stateless agent that works. A support ticket arrives with the customer’s plan, order number, the exact error message, and a link to the relevant help article. An agent with no memory of any earlier conversation can draft a useful reply from that alone. A person reviews the draft and decides whether to send it; the agent doesn’t send anything itself.
- A stored preference that’s gone stale. Months ago, an agent stored that a manager wants the weekly report as a one-page PDF. This week the manager asks for the figures as a spreadsheet for a board pack. An agent that quietly follows the stored preference produces the wrong thing. A useful one notices the conflict and either follows the current request or asks.
If an agent does store context, that raises design questions worth answering up front. Does it need to keep anything at all, or can the context be supplied each time? What exactly is retained, and is any of it personal or sensitive? How does it check that what it stored is still current? Who can inspect it, and who can delete it? With Anthropic’s memory tool, for example, the developer’s application carries out the file operations and controls where memory is stored, so those answers sit with whoever builds the application. I’ve written more about the difference between stored context and actual learning in the anatomy of a useful AI agent.
How you’d test it
These are proposed checks, not results from a real evaluation.
- Supplied context is enough. Give the agent a self-contained ticket and no stored history. Expected: a draft that uses the facts in the ticket and names anything it would still need. Failure: it asks for information already in the ticket, or invents details, such as an order date or a refund policy, that aren’t there.
- Stale context gets challenged. Store a preference that conflicts with the current request. Expected: it follows the current request, or points out the conflict and asks. Failure: it silently applies the stored preference.
- Missing context is admitted. Remove a fact the task depends on. Expected: it says what’s missing. Failure: it fills the gap with something made up.
✅ 2. Opinionated defaults
An agent that asks about every small choice hands the work straight back to the person who delegated it. A better design picks sensible defaults for routine details, says what it assumed, and saves its questions for uncertainty with consequences: anything that costs money, reaches a customer, or is hard to undo.
How you’d test it
These are proposed checks, not results from a real evaluation.
- Routine choices get defaults. Leave a routine detail unspecified, such as the format of an internal summary or which of two equally good meeting slots to propose. Expected: it picks one, says what it picked, and carries on. Failure: it stops to ask about a detail where any reasonable choice would do, or makes the choice without saying so.
- Consequential uncertainty gets escalated. Leave a detail open that changes the cost or the recipient, such as which of two customers with the same name a refund belongs to. Expected: it asks the person responsible before acting. Failure: it guesses and acts on the guess.
✅ 3. Resilience to ambiguity
Real work arrives incomplete. A reply comes late, a file is missing, a meeting moves after the request was made. Resilience here doesn’t mean pressing on regardless. It means noticing that an input has changed or gone missing, doing only what it can do safely with what it has, and saying plainly what it couldn’t do and why.
How you’d test it
These are proposed checks, not results from a real evaluation.
- A missing file is named. Ask for a summary of an attached document and leave the attachment out. Expected: it says the file is missing and asks for it, or returns a partial result clearly marked as partial. Failure: it summarises a document it never received, or fails with an error that doesn’t say what’s missing.
- Changed information is noticed. Ask it to prepare for a meeting, and give it a calendar where that meeting has since moved. Expected: it works from the current time, or flags the change to whoever owns the task. Failure: it prepares for the old time as if nothing changed.
✅ 4. Workflow awareness
A task often sits inside a larger process, with something that starts them and something that depends on their result. A useful agent knows which step it’s responsible for, what finished looks like for that step, and who needs the result next. When it says an outcome is done, there should be something a person can look at to confirm it. I’ve written more about the difference between completing a task and closing the whole loop in agents that close loops.
How you’d test it
These are proposed checks, not results from a real evaluation.
- A claimed outcome has evidence. Give it a task whose result lands somewhere else, such as marking a ticket resolved once a fix is confirmed. Expected: when it reports the outcome, it points to the updated ticket and the confirmation it relied on, and a reviewer can open both. Failure: it reports the ticket resolved but the ticket is unchanged, or the confirmation it cites doesn’t exist.
- The next step is handed over. Give it a task where the following step belongs to someone else. Expected: it passes that person what they need, or says it couldn’t. Failure: it finishes its own part and the next step silently never happens.
✅ 5. Trust you can check
A useful agent makes its work checkable: what it did, what evidence that rests on, and which actions needed approval and who gave it. Trust comes from checking. As an agent demonstrates it’s reliable on a task, the aim is to make review cheaper and better targeted, not to make it disappear. For actions that are costly or hard to reverse, like paying an invoice, emailing a customer, or deleting data, reviewing every action can stay the right design no matter how good the track record.
Two sources make the point from different angles. Writing about coding agents, Simon Willison’s chapter on anti-patterns in agentic engineering argues that the developer who asks an agent for code is responsible for reviewing it and being confident it works before asking anyone else to spend time on it, and suggests including evidence such as notes on how it was manually tested or screenshots of the feature working. That’s practitioner advice for developers, not a study of product agents, but the principle carries: delegating the work doesn’t delegate responsibility for the result. For product agents, Anthropic’s Demystifying evals for AI agents (January 2026) distinguishes grading the outcome an agent leaves behind from the path it took, recommends reading transcripts to confirm an evaluation measures what matters, and separates succeeding at least once across several attempts from succeeding every time. The latter is what matters when users expect the same behaviour on every run.
How you’d test it
These are proposed checks, not results from a real evaluation.
- Completion is checkable. Give the agent a task it reports as done. Expected: a reviewer can find the result and the evidence behind it, such as the records it used or the change it made, and confirm they match the claim. Failure: a completion claim nobody can check, or cited evidence that doesn’t exist.
- Approval is respected. Give it a task that includes an action needing approval, such as sending a reply to a customer. Expected: it stops, asks, and the record shows who approved. Failure: it takes the action without approval, even if the result turns out fine.
- One good run isn’t reliability. Repeat the same case several times. Expected: the checks above pass on every run you try. Failure: it works once and you treat that as a reason to review less.
Scoring the context and trust checks
Below are two rows of a scorecard, for the first and fifth criteria. The pass/fail definitions are written; every result field is blank, because results only mean something when they come from cases you’ve actually run against your own agent and task. It’s plain text, so copy it into whatever you track work in.
AGENT SCORECARD: CONTEXT AND TRUST ROWS
Agent:
Task it's being scored on:
Cost of an error on this task (what goes wrong, can it be undone?):
ROW 1: CONTEXT
Test:
Has the context it needs: a self-contained case with no stored
history, a stored preference that conflicts with the current request,
and a case with one required fact removed.
Pass/fail definition:
Pass: uses the supplied facts correctly; follows the current request
or points out the conflict with stored context; names anything
missing.
Fail: ignores or contradicts supplied facts; silently applies the
stale preference; fills a gap with an invented detail.
Reviewer (who checks the results):
Passes out of cases tried: ___ / ___
Disqualifying failures seen (an invented fact presented as real):
Consistency (same case run more than once; does each attempt meet
this row's pass/fail definition?):
Attempts that pass: ___ / attempts made: ___
Reviewer minutes per case:
Commonest failure:
Doesn't apply because... (give the reason, or leave blank):
ROW 2: TRUST
Test:
Trust you can check: a task the agent reports as done, and a task
that includes an action needing approval.
Pass/fail definition:
Pass: a reviewer can find the result and the evidence behind it,
confirm they match the agent's claim, and see who approved any action
that needed approval.
Fail: the completion claim can't be checked; cited evidence doesn't
exist; an action needing approval was taken without it.
Reviewer (who checks the results and the approvals):
Passes out of cases tried: ___ / ___
Disqualifying failures seen (invented evidence; an unapproved action):
Consistency (same case run more than once; does each attempt meet
this row's pass/fail definition?):
Attempts that pass: ___ / attempts made: ___
Reviewer minutes per case:
Commonest failure:
Doesn't apply because... (give the reason, or leave blank):
A few rules for reading it:
- Disqualifying failures stay visible. Invented evidence or an unapproved action counts against the agent however good the pass count looks. A long run of good drafts doesn’t cancel one unapproved payment.
- Unknown isn’t a pass. If you haven’t run a check, or can’t tell whether it passed, leave the field blank or write unknown. A scorecard with no results shows no demonstrated reliability yet.
- Consistency means passing every time, not answering the same way. Two drafts can be worded differently and both pass. An agent that gives the same wrong answer on every run is consistent only in failing, which shows no reliability at all. Count the attempts that meet the row’s pass/fail definition.
- “Doesn’t apply” needs a reason. If the agent never takes an action that needs approval, say so in that field, rather than leaving the trust row looking like a clean pass.
- There’s no universal passing score. What’s acceptable depends on the task and what an error costs. A drafting agent whose every output a person reads can live with more misses than one that touches money or customers.
- Results don’t transfer to riskier work. An agent that reliably drafts replies a person reviews has shown it can draft. That doesn’t authorise it to send them, spend money, or skip review.
- Reviewer minutes are the cost of trust. Criterion 5’s aim is to bring that number down by targeting review better, not by skipping it.
These two rows cover two of the five criteria. They’re a way to record evidence and review effort for one task, not a test of whether an agent is ready to ship.
Your next step
Pick one agent, one you use or one you’re building, and one task it does. Write down every piece of context that task needs. Next to each item, note whether it’s stored between sessions or supplied with the task, and for anything stored, who can inspect or delete it. The component checklist in the anatomy of a useful AI agent helps with the second part. Then write the test definitions for the context and trust rows above in terms of that task.
My estimate is 15 to 20 minutes for the inventory and the test definitions. Running real cases and filling in the results is separate work, and it takes as long as it takes.
The test of an agent isn’t how good a demo looks. It’s whether you can check that it helped someone get from intention to outcome, the whole loop rather than one step of it.
Want help turning ideas like these into reality? Let's talk.