How Companies Govern Agentic AI

Part of our topic guide on AI Governance & Data Strategy.

By James Cotton, Founder, iO-Sphere

An agent does a process faster, not better. Point one at a task and it will carry that task out at machine speed, whether the task is right or wrong, hundreds of times before a person looks up. So the first move in governing agentic AI is not to choose a platform or draft a policy. It is to make the process the agent will run describable, measurable and open to inspection first, because a workflow no human can follow is one an agent can only speed up, mistakes included.

Keep one example in view for the rest of this page. Say you want an agent in your finance team that takes in supplier invoices, matches each invoice to its purchase order, and schedules it for payment. It is a good candidate: high volume, repetitive, and the kind of work people are glad to hand off. It is also a clear illustration of the stakes, because when it goes wrong it moves money.

Govern that agent the way you would hand any process to someone new: describe it, measure it, make it auditable, and only then let it run. The order is the whole point. Each section below walks the invoice agent through one part of it.

Describe the process before you choose a tool

Before anything else, put the process on one page: what the invoice agent should do, and every way it can go wrong. Not the software it uses, the work itself. Here is what that page looks like for our example.

The one page, worked: the invoice agent

• Process: receive the invoice; read the supplier, amount and PO number; match it to an open purchase order and the goods received; check it has not been paid before; confirm the payee bank details against the ones on file; schedule payment.
• Decision points: Is this a genuine invoice? Does it match an open PO? Is the amount right? Has it been paid already? Have the bank details changed? Is the supplier new?
• Failure cases: paying a duplicate; paying an invoice with no matching PO; paying an inflated amount; paying into a bank account a fraudster swapped in; paying a first-time supplier unchecked.
• Correct: the right supplier, paid the right amount, once, against a matched purchase order.
• Each audit-log line records: what the agent saw (the invoice, the PO it matched), what it decided (match, duplicate, hold for approval), what it did (scheduled, paused, escalated), and why (the rule or threshold that applied).

If you can write that page, you understand the process well enough to govern an agent doing it. If you cannot, that is the finding, and no tool closes the gap. Buying a governance product before you have the page just gives you a faster way to run a process you cannot describe.

This is why governing agents is a business-analysis task before it is a technical one. Describing a workflow, naming its decision points and its failure cases, is exactly what a business analyst does, and it is the part almost every organisation underinvests in.

Mark every point where the agent makes a decision

Look at the decision points on that page. Each one is a fork where a correct outcome and a costly one sit side by side, so governing the agent means ranking those forks by the worst thing that happens if it chooses wrong. Misfiling a reference number is recoverable. Paying a duplicate invoice, or paying a genuine invoice into a bank account a fraudster swapped in, is money gone, at the speed and volume the agent runs.

That ranking tells you where a person stays in the loop. Low-consequence decisions the agent can make on its own and log. The high-consequence ones sit behind a human who approves before the action takes effect: a payment above a set limit, a first payment to a new supplier, any change of bank details. The agent proposes; the person decides. The aim is to spend your scarce human attention on the decisions where the downside is largest and let the agent carry the rest.

Define what "correct" looks like, and how you would catch it drifting

The definition of correct on that page is what every control exists to protect. So the next question is how the process could drift away from it over time. Duplicate payments creeping in. Amounts drifting upward. Unmatched invoices slipping through because a rule was loosened. A fraudulent change of bank details getting paid because nobody flagged it. Drift is rarely a dramatic failure. It is a slow slide that a busy team stops seeing.

You catch drift through the audit trail, the log lines on that page. Every step the agent takes leaves a record a person can read, and that record is what makes your definition of correct enforceable. Without it you cannot prove the agent did the right thing, and you cannot reconstruct what happened when it did not. Build the trail first, before you build any autonomy on top of it.

And one person has to be answerable for reading it. Each agent needs a named owner, a person, not a committee, who owns what it does and has the authority to stop it. The organisations that get this wrong discover during an incident that everyone owned the agent and so no one did.

Hand it over assistive-first, then let it earn autonomy

With the process described, the decisions ranked, correct defined and the trail in place, the agent can start, and it should start as an assistant: it proposes, a person approves. That is the safe default for almost every organisation today.

Autonomy is then something the agent earns one decision at a time, as its trail shows it getting those decisions right. Once the record proves it has matched purchase orders correctly for months, you can let it clear low-value, well-matched invoices without a sign-off and tighten your attention onto the riskier calls. You loosen a check only where the trail has proved the agent can be trusted without it.

The gains here compound. A handful of small, well-audited automations add up to more than the single ambitious agent launched before anyone could describe what it should do, and each small one builds the same skill, describing and auditing a real process, that the larger deployment will need.

Why governing agents is a business-analysis problem first

Step back from the invoice agent and the pattern holds for any agent you might deploy. Almost all of the work of governing one is describing the process, defining correct, and building the trail. Almost none of it is the choice of tool. That pattern is not specific to invoices. It is where AI projects fail.

RAND's 2024 study of AI project failure found that more than 80% of AI projects fail, about twice the rate of ordinary IT projects, and named the leading root cause as misunderstanding or miscommunicating the problem the project was meant to solve (RAND, report RRA2680-1, August 2024; based on enterprise projects internationally). The failures cluster at the point this page starts: not knowing the process well enough to automate it.

The same signal shows up in adoption. MIT's NANDA research in 2025 reported that about 95% of enterprise generative-AI pilots delivered no measurable profit-and-loss impact, and located the barrier in how organisations learn to apply the tools.

This is why a governance policy or platform, on its own, governs nothing. A framework is made of real and necessary parts, and a good governance framework sets them out: risk tiers, human checkpoints, audit trails, named owners. But those parts are the output of the process work; they do not replace it. A policy no one in the building can operate is just a document, and a document controls nothing. The organisations that come unstuck are the ones that gave autonomy to a process they could not describe.

Where the rules fit: data protection and the EU AI Act

Regulation sits on top of this work, and mostly it asks you to show exactly what the describe-and-audit approach produces. Two reference points matter for most UK organisations.

First, data protection. When an agent handles personal data or makes decisions about people, your organisation stays accountable for what it does, and that accountability cannot be handed to the vendor or to the model. In the UK that runs through data-protection law (the UK GDPR and the Data Protection Act 2018) and the Information Commissioner's Office, whose guidance sets out the accountability and impact-assessment duties that apply when you use AI. Go there for the detail.

Second, the EU AI Act. It classifies AI systems by risk and can reach a UK organisation from outside the EU, most relevantly where an agent's output is used inside the EU. The detailed obligations and their timing are set out in the legislation and are still moving, so check the current position at source before you rely on a date or a threshold.

The practical point is that the describe, tier and audit work is most of what a regulator or a data-protection impact assessment asks you to demonstrate. Govern the process well and you are most of the way to being able to show your compliance.

Where to start, and who builds the capability

Start small and concrete. Pick one real process you already understand well, produce the one page for it in the shape of the worked example above, and stand up the trail before you automate anything. One governed agent, done properly, gives you a pattern you can reuse for the next.

The capability all of this depends on is buildable. It is business-analysis capability aimed at your own workflows: the ability to map a process, name where it makes decisions, define what correct looks like, and build the record that proves it. That is a skill your people can learn on your real processes. iO-Sphere is a training provider, so what it delivers is people who can run this, on your own work. It does not write the framework for you.

Two funded routes build that capability, depending on who is learning. AI Transformation is the Level 4 Business Analyst apprenticeship (standard ST0117), the route for the process-description and business-analysis capability that governing agents rests on; iO-Sphere delivers it in around 15 months plus a 3-month end-point assessment, against the standard's typical 18. Data & AI Governance is the route for practitioners who will own the policy, audit trails and compliance of an AI programme; the funding detail for that governance route lives on our funded data-governance training page.

Training is not the answer to every need on this page, and some of these are better handled elsewhere:

• If you need to certify an AI management system to ISO/IEC 42001, that audit is the job of an accredited certification body. Build the capability first, then engage one.
• If you need a legal opinion on your EU AI Act exposure, that is a question for a law firm with an AI regulatory practice.
• If you operate in a regulated sector such as financial services or healthcare, your sector regulator's own AI expectations come first, with this baseline underneath them.
• If your question is which governance platform to buy, this page is about the practice, not the product; a platform is only as good as the people reading what it shows.
• If you cannot fund training right now, the one-page test, the decision-point ranking and the audit-first order in this article work without any programme. Start there.

iO-Sphere delivers apprenticeships up to Level 5, so needs that sit above that level, or outside training altogether, are where the routes above point you elsewhere.

Frequently asked questions

What is agentic AI governance?

Agentic AI governance is how an organisation controls software that plans and carries out actions on its behalf, rather than software that only produces text or predictions for a person to use. In practice it means describing the process an agent will run, deciding which of its decisions a human keeps, defining what a correct outcome is, and keeping a record of what the agent did so you can inspect it. It governs actions and their consequences, which a standard AI-use policy was never built to do. For the wider foundations, see what AI governance is.

How is governing an AI agent different from having an AI-use policy?

An AI-use policy governs what people do with a tool: what they can put into a chatbot, what they can rely on its output for. Governing an agent has to go further, because the agent takes actions with consequences on its own, such as moving money, changing a record, or messaging a customer. That means ranking its decisions by their worst outcome, putting a person in front of the risky ones, and logging every step, controls a document about acceptable use never needed.

Where should you start with governing agentic AI?

Start with one real process, not with tooling. Produce the single page that captures what the agent should do, its decision points and its failure cases, and set down what a correct outcome is, before you evaluate any software. That one page is both the design for the agent and the test of whether you understand the work well enough to govern it.

Do you need to buy an AI governance platform?

A platform can help you run controls at scale, but it does not govern anything on its own. The tiering, the approval steps, the logs and the accountable owners it tracks only mean something once someone has described the process, defined what correct looks like, and can read what the agent did. A platform bought before that work gives you a dashboard nobody can interpret. The practice comes first, and the tooling supports it.

Should agents run autonomously, or with a human in the loop?

For almost every organisation today, start with the agent assisting rather than acting alone: it proposes, and a person approves the consequential steps. Let it take on more only as its audit trail shows it making those decisions correctly, and drop a human check once the record has earned it, not before. The record is what makes it safe to loosen the checks, so it has to exist before the autonomy does.

Does the EU AI Act apply to UK companies using agentic AI?

It can. The EU AI Act classifies AI systems by risk and can reach an organisation outside the EU, most relevantly where an agent's output is used within the EU. The detailed obligations and their timing are set out in the legislation itself and are still changing, so confirm the current position before you act on a specific date or threshold.

Who is accountable when an AI agent makes a bad decision?

The organisation deploying the agent is accountable, and that accountability cannot be passed to the software vendor or blamed on the model. Practically, each agent should have one named person who owns what it does and can stop it. Where personal data is involved, UK data-protection law and the Information Commissioner's Office set out the accountability and impact-assessment duties that apply, and the ICO's guidance is the place to confirm the detail.

If your task is to stand up governance for agentic AI, and you want the people who run your processes to be able to describe, rank and audit an agent themselves, that capability is what iO-Sphere trains. For an ISO certification, a sector-regulator overlay or a legal opinion, the routes above send you to the right door instead. Explore team training for your business or the Data & AI Governance apprenticeship.