Knowing when your agent should ask.

Imagine an escalation arriving at the end of a busy day: “Your colleague promised me a £900 refund.” A helpful agent could make the queue look better by immediately saying yes. It could also create a commitment nobody authorised.

This is a fictional scenario. The operational question is real: what was the agent actually allowed to do?

An AI that drafts a refund email and an AI that issues a refund are not doing the same job.

They might share a chat interface. They should not share the same permission boundary.

A useful way to design an agent is to separate three levels of work: suggest, prepare and act.

Suggest

The agent explains what it thinks should happen. A person decides whether the suggestion is sensible.

For example: “This looks like a delivery complaint. Check the order status before promising a replacement.”

The answer can still be wrong, so show the evidence and unknowns. But the agent has not changed a customer record or made a commitment.

Prepare

The agent creates a draft action for review: a proposed email, a list of record changes, or a refund request.

The reviewer should be able to see the specific action, recipient, amount and supporting evidence before approval. “Approve agent” is much less useful than “Approve this £20 refund for this order.”

If the underlying details change, the old approval should not silently authorise the new action.

Act

The agent performs an operation using an explicitly granted permission. That permission should be enforced by the application, not only described in a prompt.

Keep it narrow. Reading an order should not automatically grant the ability to cancel it. Preparing a payment should not grant the ability to release it.

Four questions for the boundary

Can it be reversed? A draft is easy to discard. An external email cannot reliably be unsent.

Who or what could be affected? Money, access rights, personal data and customer commitments deserve particular care.

What evidence is available? Missing data should stay visibly missing. A high confidence score is not a substitute for evidence.

What happens after failure? If an action times out, check whether it already happened before trying again. A retry can create a duplicate charge, email or order.

Human review is a product feature

An approval step is useful only when someone can realistically understand it. Show what will happen, why it is proposed, and what the person needs to check.

Give them a clear way to edit, reject or investigate. A screen full of green ticks is not the same as meaningful oversight.

Try this

Pick one proposed agent action. Draw three columns: what it may read, what it may prepare, and what it may change.

Then mark the actions that affect money, permissions or other people. Those marks are a good place to start discussing approval rules with the accountable owner.

This is a design exercise, not a substitute for your organisation’s security, privacy or release requirements.

Make the call in the permission exercise. Three synthetic cases let you explore that boundary without sending a message or changing a record.