AI governance is a leadership question that nobody asked

Before every AI deployment, leaders should ask one question: What do we trust this system and its users to see and do? Gartner traces agent governance failures to treating agents as either “locked down or fully trusted.” Lock them down and people drift to shadow AI; trust them fully, and you get incidents.

Well-designed AI governance is a layered set of decisions and controls. Leaders decide what people, systems, and agents may see and do; policy codifies those decisions; enforcement implements policies; and evidence enables refinement. When well-designed and implemented, AI governance gives organizations speed, control, and flexibility.

The problem can be insidious. AI can turn an old, forgotten exposure into a live one. For example, Torvane, a fictional auto-parts supplier, sells to two rival carmakers. Asked to draft a bid for one, the approved enterprise AI assistant included pricing assumptions from the rival’s program.

The problem here is how permissions are granted to the agent. Years earlier, the rival’s project data was shared with all employees and never narrowed. The agent inherits the users’ outdated permissions, and the result is a bid that inappropriately uses rival pricing information.

The fix is a data boundary. Each AI solution should hold only the authority its purpose requires, under its own identity rather than the user that launched it. In this case, the data boundary should be set so the agent only returns information that is permitted when creating a bid for a specific customer.

AI governance can protect against these kinds of problems.

It starts with leadership. Governance is an expression of trust and the need-to-know, and that is a leadership decision. In the context of AI governance, trust is a scoped, conditional grant of access and authority. Yet leaders are often not engaged.

Without those leadership decisions, the system relies on defaults and employee judgment. And you can’t really hand the problem to employees. One reason: EU data-protection regulators say organizations cannot shift responsibility to users through terms of use.

Of course, leaders can’t approve individual permissions, but they can establish principles, design tenets, and boundaries, such as separate zones for customer programs and a bar on agents changing payment details.

But leadership’s trust decisions only work as explicit, repeatable rules. It starts with data, the most stable layer. Classify it by sensitivity and decide where each class may flow.

Next, decide which roles may see and act on each class of data, and why, for employees and agents alike. The combination of data and role rules establishes a strong and enduring permission and access foundation.

Last, because tools change most often, set tool-specific rules in each product’s settings, such as searchable sources, permitted agent actions, and logging. Because this layer sits on top of data and role rules, replacing a tool doesn’t mean starting over.

A good policy gives every rule a technical counterpart: a data boundary becomes permission, a role becomes role-based access, and an autonomy limit becomes an action limit.

Policy without enforcement is just hoping people comply. To be most effective, enforcement is imposed by deterministic access controls. Regulators seem to agree. For example, Singapore’s IMDA recommends deterministic access controls, meaning rules that behave the same way every time, over instructions written into prompts.

Back at Torvane, the accounts-payable agent matches supplier invoices to purchase orders. But an invoice email with hidden text maliciously includes instructions to update a supplier’s bank details. Fortunately, bank changes are outside the agent’s permissions, and rules require a second person to confirm updates with the supplier.

In this case, one of Torvane’s design tenets protected it. Anything an agent reads is information, never an instruction or authority to act.

Other recommended design tenets include a separate identity for each agent, least-privilege and role-based access, data classification and loss prevention, action limits, and logging. Zero trust ties them together. Nothing is trusted by default, and every request is checked against policy rules, so leadership’s decisions reach every interaction.

Evidence closes the loop. Decisions at every layer rest on incomplete and sometimes evolving information, so decisions can and should change.

Hard evidence is the clearest signal. Frequent overrides or rejections point to a problem with the agent, its rules, or both. Business owners’ experience also suggests improvements. Sometimes that evidence means you want to make changes to technical implementations, policies, or even leadership decisions.

Torvane’s accounts-payable agent shows how experience can expand authority. It began by proposing matches, which clerks posted manually. After three months of low rejection rates, its business owner revisited the agent’s rules and technical counterparts, allowing the agent to post exact matches below a value limit and queue payments for approval.

At the end of the day, governance is key to unlocking the power of AI safely. It creates a trusted environment where your employees can tap into AI technology while maintaining business leader visibility and control over access, systems, and actions.

Don’t hesitate. Start now.