← All posts

ai:agents

Running AI agents in production: permissions, limits and rollback.

Running AI agents in production means the system acts: it opens tickets, updates records, sends messages. What makes a deployment safe is what sits between a wrong answer and a live system: which tools the agent may call, under whose identity, with what caps, and what happens after it gets something wrong.

This post is the control list for before an agent touches anything live: permissions scoped per tool, hard limits on actions and spend, a person in front of irreversible steps, an action log kept for the audit trail, and a stop switch with a way back. It is written for engineering and operations leads, and for the risk owner who asks what stops it.

What it takes to run AI agents in production.

An agent is a model in a loop with tools, and in OpenAI's and Anthropic's APIs the mechanics match: your code registers the tools, the model decides when one is needed and returns a structured call, and your application executes it and hands the result back. OpenAI's function-calling guide describes the loop in five steps, the third being to execute code on the application side with input from the tool call; Anthropic's tool-use overview describes client tools as ones where Claude returns a structured call that your application runs (both checked 16 November 2026).

That execution step is where every control in this post lives. Your code decides whether the call happens, under whose credentials, and after which checks. In production, those decisions are written down and enforced in code.

What changes when an AI system can act: excessive agency.

OWASP's Top 10 for LLM applications names the risk LLM06:2025 Excessive Agency: the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is making the model misfire. The common triggers it lists are confabulation from a poorly engineered prompt or a poorly performing model, and direct or indirect prompt injection.

The root causes it names are the three to check a design against: excessive functionality, excessive permissions and excessive autonomy. In practice: tools the agent does not need, credentials broader than the task, and high-impact actions nobody checks. The rest of this post is those three causes turned into controls, in the order a team deploys them.

Permissions: its own identity, least privilege, read before write.

Four decisions, made before the first deployment:

  • Its own service identity. The agent gets a dedicated account per downstream system, never a person's login and never a shared admin key. Its actions become traceable in every system it touches, and revocable in one place.
  • Least privilege per tool. A tool that reads tickets needs read on tickets, not administration of the service desk. OWASP's mitigations: limit what an extension is granted on downstream systems to the minimum necessary, enforced through the identity it connects with, and implement authorization in the downstream system rather than relying on the LLM to decide if an action is allowed.
  • Read before write. First deployment runs read-only tools against production data; write tools switch on once the team has reviewed those calls in the log. Where the agent acts for a user, OWASP's mitigation is to execute in that user's context with their rights, and in Microsoft Copilot Studio, tools authenticate with the end user's credentials by default and run in the user context (checked 16 November 2026).
  • No open-ended tools. OWASP advises avoiding open-ended extensions such as running a shell command, in favor of tools with more granular functionality. An agent with a 'run a command' tool holds a capability you cannot scope; an agent with an 'update ticket status' tool does not.

Limits: action caps, spend caps, approval before irreversible steps.

The team sets these limits before go-live:

  • Actions per run and per day. A hard ceiling on tool calls stops a loop from feeding itself; when the agent hits the ceiling it stops and reports. OWASP lists rate-limiting among the measures that limit damage: fewer undesirable actions per period gives monitoring more chance to find them before significant damage.
  • Spend caps. A ceiling per run and per day over model and tool costs, with a spike alert. An agent that fails expensively is a cost incident even when every action is harmless.
  • Approval before irreversible actions. Send, delete, pay, anything without an undo: a person approves before it happens. OWASP's mitigation is human-in-the-loop control requiring a person to approve high-impact actions before they are taken; in Copilot Studio this is a per-tool option, 'Ask the end user before running', set to No by default, so somebody has to turn it on (checked 16 November 2026). Who reviews, what they see and how escalation runs when the reviewer is away is the design our human-in-the-loop AI workflows post works through; for an agent, the reviewed unit is the tool call itself.
  • A start window. Business hours only for the first weeks, so the people who own the target systems are watching when it runs.

The action log: every call, input, output and result.

The log records every tool call: the arguments the model produced, the identity it ran under, the result, and the downstream object it touched, failures included. OWASP's mitigations include logging and monitoring the activity of extensions and downstream systems to identify where undesirable actions are taking place.

Three properties make the log usable: append-only, so the agent cannot edit its own trail; joinable, so each entry carries the run identifier and the user it acted for; and complete, so a failed call is as visible as a successful one. Set up the log before go-live. It is the debugging record the week after go-live and the audit trail the quarter after that.

Stop and undo: the switch, the reversal, the fallback.

Three ways back, in order of how fast they must work. The switch: a control that disables the agent's tools everywhere at once, reachable by someone who is not the developer, tested before go-live. The reversal: prefer tools whose effects can be undone, such as creating a draft or queueing a message, over tools that commit immediately; where the target system supports it, act through an API that records an attributable actor so a mistaken change can be traced and reverted. The fallback: the workflow the agent automates stays runnable by hand, by the team that owns it, in the first weeks.

After go-live: monitoring, loop and cost alerts.

Three alerts to set: a loop alert when a run exceeds its step budget or repeats the same call, an error alert on failed tool calls above the level the first weeks showed, and a cost alert against the ceiling set at go-live, with the ceiling itself derived from the numbers in our AI production costs post.

Someone also has to own the keeping-running part: models get deprecated, prompts drift, usage climbs, and a workflow that worked at launch quietly starts failing in a corner. For the agents we build, that ownership is managed AI operations: the same engineers who built them keep them alive in production, watching for failures and swapping models when a better or cheaper one ships.

Do it yourself, or bring us in.

None of this needs new software: a service account, a secrets store, a rate limiter, a log table, a feature flag and a rollback runbook. A team that already operates the target system can put all six in place itself, because every control above lives in code it already owns.

Outside help fits when the team has not run an agent in production before, or when nobody will own it after launch. The call below starts with the system you want the agent to touch.

Book a 30-minute call

Not ready for a call? Take the 10-question AI readiness check

Frequently asked questions.

What does excessive agency mean?

It is OWASP's name, LLM06:2025, for the vulnerability where damaging actions get performed because of unexpected, ambiguous or manipulated model outputs. The root causes are excessive functionality, excessive permissions and excessive autonomy: tools the agent does not need, credentials broader than the task, and high-impact actions with nobody approving them.

Should an AI agent ever get write access?

Yes, once read-only runs have earned it, and scoped to the one action it needs. Write access to a single endpoint with a log and an approval step on irreversible calls is a normal production control; broad write access with no log is the version that ends badly.

Who is accountable for an agent's action?

A named owner. The agent runs under its own service identity, so every system it touches shows which agent acted, and your AI register names the person who answers for it. The decision to deploy and the controls around it belong to the deploying team.

How do we test an agent before launch?

On real cases with the expected result written down first, against a sandbox or a copy of the target system, plus a few hostile inputs to see what it does when confused. The go-live review, and the evidence to show, is in our AI production readiness checklist.

When should we switch an agent off?

On its tripwires: repeated failed calls, activity outside its pattern such as a new tool or a new time of day, a cost spike, or a complaint traceable to its log. Halt first and investigate after, because it costs less.

Keep reading.

:

Human-in-the-loop AI workflows: running the review step in production.

:

AI production readiness checklist: what to prove before go-live.

:

What AI costs in production: budgeting beyond the pilot.