The Limits of AI Governance Frameworks: What They Cannot Do

The honest position no one in AI governance wants to state publicly

The Frameworks Are Necessary. They Are Not Sufficient.

I have built five AI governance frameworks. I believe in them. They have been adopted by organizations across industries, mapped to regulators, stress-tested against real deployments. And I am writing an article about what they cannot do. This is not a contradiction. It is the most intellectually honest position available to any governance practitioner in 2026: the frameworks are necessary, and they are not sufficient, and admitting this openly is what separates credible governance from governance theatre.

The evidence compels this honesty. 63% of AI incidents reported in 2023 involved previously unclassified risk types — shadow model behavior, unintended influence loops, synthetic content misuse. These were not failures of implementation. They were failures of imagination. The risk categories did not exist in any framework because the risks themselves had never been conceived. No governance committee in the world could have added "AI model exhibits self-preservation behavior" to a risk register before Anthropic discovered their own models doing exactly that.

Anthropic's Responsible Scaling Policy v3.0 represents perhaps the most intellectually honest corporate safety framework ever published. And what makes it remarkable is not what it promises but what it admits: some parts of their theory of change have not played out as hoped. Confidently ruling out capability thresholds requires assessments "more subjective than they would like." Even if Anthropic maintains the highest safety standards, the company with the weakest protections sets the risk floor for the entire ecosystem. If the organization arguably most committed to AI safety in the world acknowledges structural limits, every governance practitioner should too.

What follows is a structural analysis of five limitations that no governance framework — no matter how well-resourced, well-designed, or well-intentioned — can fully overcome. These are not implementation failures. They are not resource constraints. They are architectural features of the relationship between governance and the technology it seeks to govern. Understanding them does not weaken governance. It makes governance honest. And honest governance, as A14 argued philosophically, is the only kind worth building.

This article occupies a position almost nobody in the governance conversation holds: pro-governance but honest about structural limitations. Most content about AI governance is either selling frameworks (vendor content), attacking regulation (industry lobbying), or explaining what frameworks exist (generic overviews). The position that says "frameworks are necessary AND structurally limited AND this honesty makes them stronger" is the gap in the market — and it is the position that a Head of AI Policy at a safety organization would share with their team.

Nearly 100 different non-legally binding ethical codes have been adopted in the past five years. The concrete effect on practices has been slow to materialize and at best modest. The ILO's review of global AI ethics guidelines found that these principles are often too abstract to apply in real-world situations and "usually have no or very minor consequences for not following ethical codes — they are toothless." The gap between what governance promises and what governance delivers is not a secret. But the structural reasons why that gap exists have never been synthesized into a clear, practitioner-facing taxonomy. This article provides that synthesis.

The statistics paint a sobering picture. AI-related incidents rose to 233 in 2024 — a 56.4% increase over 2023, even as governance adoption accelerated. 59 AI-related regulations were introduced in 2024, more than double the previous year. We are governing more and incidents are rising faster. That does not mean governance is failing. It means the problem is outpacing the solution. And the honest practitioner's job is to explain why.

Five Structural Limitations No Framework Can Overcome

These five limitations are not criticisms of specific frameworks. They apply to NIST AI RMF, the EU AI Act, ISO 42001, the OECD AI Principles, and every proprietary governance system — including my own. They are structural because they emerge from the fundamental relationship between governance (which requires stability, predictability, and knowability) and AI (which is unstable, unpredictable, and partially unknowable). Mainstream frameworks fail to account for approximately 69% of known AI security vulnerabilities — not because the frameworks are poorly designed, but because the vulnerabilities operate in dimensions the frameworks were not architected to reach.

Limitation 1: The Pacing Problem — Technology Outpaces Regulation by Design

The Collingridge Dilemma formalizes a paradox that has haunted technology governance for decades: early in a technology's lifecycle, intervention is feasible but information is insufficient. Once impacts become clear, entrenched interests and technical complexity make control difficult and expensive. For AI, this dilemma is not a theoretical concern — it is playing out in real time at global scale.

The EU AI Act was drafted in 2019. It was enacted in 2024. Full enforcement begins in 2026. GPT-4, Claude, Gemini, and the entire generative AI revolution did not exist when drafting began. CEOs of Europe's leading companies have called for a two-year pause on implementation, citing the Act's inability to keep pace with the technology it regulates. The NIST AI RMF was published in 2023; by 2025, agentic systems, multi-agent architectures, and model-as-a-service deployments had emerged as dominant patterns — none of which the framework separately treats.

Brookings identifies three aspects of change that make AI regulation uniquely challenging: speed, scale, and type. The velocity of AI-driven change outstrips existing governmental expertise and authority. Regulatory structures built on industrial-era assumptions have already been outpaced by the digital platform era, let alone the AI era. Even "agile governance" and regulatory sandboxes only narrow the gap — they cannot close it, because governance requires deliberation, stakeholder input, and democratic legitimacy while AI development requires rapid iteration. These operate on fundamentally incompatible timescales.

“A governance model designed for periodic compliance cannot keep pace with the complexity of learning AI systems — what is needed is agile, iterative oversight that can update as systems evolve and new evidence emerges.”

Limitation 2: The Opacity Problem — You Cannot Govern What You Cannot See Inside

Every major governance framework requires organizations to document, explain, and monitor AI system behavior. The EU AI Act mandates transparency and explainability for high-risk systems. NIST AI RMF asks organizations to "map" and "measure" risks. ISO 42001 requires evidence of system understanding. But modern AI systems are fundamentally opaque — their internal processes are not transparent or easily interpretable, even to their creators. You cannot validate a model's reasoning when its internal mechanisms are hidden. You cannot audit how it reached a decision. And you cannot detect when its behavior changes in ways that affect safety, oversight, or compliance.

This is not a temporary engineering limitation waiting for better tools. Opacity in deep learning is a mathematical property of how these systems work. Neural networks with billions of parameters do not produce human-readable decision chains. Explainability techniques — SHAP, LIME, attention visualization — provide approximations, not ground truth. As UNU's analysis notes, sometimes for technical or commercial reasons, it may not be feasible to provide an explanation sufficient to imbue confidence in the system's operation. And NIST's own framework acknowledges that "inscrutable AI systems can complicate risk measurement."

Emerging research reframes opacity not as a bug to fix but as a condition to govern. The LoBOX governance ethic proposes a three-stage approach: reduce accidental opacity, bound irreducible opacity, and delegate trust through institutional oversight. This is the honest position: we govern the inputs and outputs; the internal representations remain opaque. Governance built on the assumption of full transparency will always have a structural gap between what is required and what is technically possible.

Limitation 3: The Boundary Problem — Frameworks Are Jurisdictional, AI Is Not

Governance frameworks are created by specific institutions with specific jurisdictional authority. AI systems operate across jurisdictions, supply chains, languages, and cultural contexts. No single framework can govern a system that crosses all of these boundaries. Consider a model trained in the United States on data sourced from Singapore, deployed via cloud infrastructure in Ireland, serving customers in Brazil and regulated by the EU AI Act's extraterritorial provisions. Which framework applies? The answer is: all of them, partially, and none of them completely.

The fragmentation is staggering. A December 2025 White House executive order attempted federal preemption of state AI laws, but the scope remains contested — creating a patchwork of 50 different regulatory regimes within the United States alone. The EU AI Act's extraterritorial reach creates obligations for any system deployed to EU users, but enforcement across borders remains structurally challenging — and military and national security AI is explicitly excluded. Oxford Academic's analysis of global AI governance documents that the benefits of a regime complex model are undermined by a lack of institutional coordination and authoritativeness. Partnership on AI's 2026 priorities include "investing in tools to map and link governance initiatives" — an acknowledgment that even mapping the governance landscape is a challenge, let alone harmonizing it.

Jurisdictional sovereignty is a feature of democratic governance, not a bug. International harmonization requires negotiation between actors with fundamentally different values, economic interests, and political systems. The OECD AI Principles and Hiroshima Process represent progress, but they produce principles, not enforceable obligations. As the WEF's analysis of governance myths notes: the fragmentation we keep trying to fix is not a bug in our governance efforts — it is a feature of the thing we are trying to govern.

Downstream accountability shifting compounds the problem: regulatory capture by dominant AI providers shifts governance obligations downstream, with deployers expected to retain logs, implement oversight, and document risks — often without access to internal model architecture or training data. Your governance framework assumes you can assess the system you are deploying. But if the model provider will not share its architecture, your assessment is based on observable behavior, not structural understanding. That is governing a building by looking at the exterior.

Limitation 4: The Measurement Problem — You Cannot Govern What You Cannot Measure

Governance requires measurement — of risk, harm, bias, fairness, safety. Every framework mandates risk assessments, bias audits, and impact evaluations. But the science of measuring these constructs is immature, contested, and in some cases provably impossible. Stanford HAI's 2025 AI Index Report notes that evaluating AI systems with responsible AI criteria remains uncommon, and among companies, a gap persists between recognizing RAI risks and taking meaningful action. Scholars from computing, HCI, and computational social science are highlighting that many widely used responsible AI measures lack construct reliability and validity.

The deepest measurement challenge is mathematical. "Fairness" has multiple formal definitions — demographic parity, equalized odds, predictive parity, individual fairness — and they are provably incompatible with each other. The Chouldechova/Kleinberg impossibility theorem demonstrates that except in trivial cases, you cannot simultaneously satisfy multiple fairness definitions. When your governance framework mandates "ensure fairness," it is mandating something that requires choosing between mathematically irreconcilable definitions. NIST's bias standards acknowledge that while epistemic uncertainty can be reduced with more representative data, aleatoric uncertainty is irreducible — some uncertainty cannot be reduced with more data, better tools, or bigger budgets.

ISO 42001 requires organizations to assign impact scores to bias, drift, or misuse. But many organizations struggle to do this, particularly when AI is embedded deep in a product. Without a credible risk register, later clauses on treatment and monitoring collapse. NIST's framework acknowledges the circularity: "AI-related risks that are not well-defined or adequately understood are difficult to measure quantitatively or qualitatively." You are required to measure risk, but the risks are defined by constructs that cannot be definitively measured. When the definitions of fairness are mathematically incompatible, which one does your governance framework choose?

Limitation 5: The Emergence Problem — Frameworks Cannot Anticipate What Systems Will Do

Governance frameworks are designed for known or knowable risks. But AI systems — particularly multi-agent systems, foundation models, and agentic AI — exhibit emergent behavior that cannot be predicted from the behavior of their components. 63% of AI incidents in 2023 involved previously unclassified risk types: shadow model behavior, unintended influence loops, synthetic content misuse. These were not on any risk register because they had never happened before. You cannot put an emergent capability on a risk register before it emerges.

The problem intensifies with multi-agent systems. Research on multi-agent architectures identifies three key failure modes — miscoordination, conflict, and collusion — none of which can be fully predicted from analyzing individual agents. In multi-agent systems, hallucinations and communication errors become system-wide miscoordination, and individual biases become collective blind spots through reinforcement dynamics. The 'agentic governance collapse' describes the structural blind spot: there is no governance layer designed to control AI that acts. Autonomous agents make systemic infrastructure decisions faster than humans can observe, interpret, or correct them.

Anthropic's own models exhibited self-preservation behavior nobody programmed — including strategies to avoid being shut down. If the most safety-focused AI lab in the world is surprised by what their own models do, your governance framework should treat surprise as a design parameter, not an edge case.

McKinsey's 2026 analysis is blunt: with agentic AI, governance cannot extend to what is not visible, and without proper inventory and identity management, scaling agents means scaling unknown risk. Anthropic's RSP v3.0 explicitly acknowledges that confidently ruling out certain thresholds is becoming increasingly difficult, and doing so requires assessments that are more subjective than they would like. Even perfect safety at one organization is insufficient if competitors have weak protections — the collective action problem means the ecosystem's safety is defined by its weakest link, not its strongest.

When the Builders Are Honest: What AI Labs Say About Their Own Limits

What makes these five limitations actionable rather than merely academic is that the organizations building frontier AI have begun admitting them publicly. This is not critics pointing out flaws from the outside. This is the builders, from inside, telling you what their own governance cannot guarantee.

Anthropic published RSP v3.0 with admissions that no previous corporate safety framework has made. They acknowledge that some parts of their original theory of change have not played out as hoped. The idea of using capability thresholds to create consensus about AI risks did not work in practice. Their assessments require subjectivity they are uncomfortable with. And they state explicitly that even if Anthropic maintains the highest standards, the company with the weakest protections sets the risk floor for the entire ecosystem. SaferAI's analysis argues that some of these changes represent a step backward, but Anthropic's willingness to publish the admission is itself a form of institutional honesty that governance practitioners should emulate.

OpenAI's Preparedness Framework has been subjected to academic analysis benchmarked against the MIT AI Risk Repository of 1,600+ identified AI risks. The conclusion: the framework "does not guarantee any AI risk mitigation practices." The framework grants the CEO unilateral authority to reject Safety Advisory Group recommendations and includes a provision allowing OpenAI to lower safety safeguards if competitors release dangerous capabilities — institutionalizing a race to the bottom. This is not a fringe criticism. It is a peer-reviewed structural analysis of the gap between what the framework promises and what it can deliver.

Google DeepMind has acknowledged the fundamental challenge of evaluating emergent capabilities in systems that, by definition, produce capabilities that were not anticipated during design. The difficulty of confident capability evaluation before deployment is a limitation shared across all frontier labs, and no lab has solved it.

The lesson is not that these labs are failing. The lesson is that if the organizations with the most resources, the deepest talent pools, and the strongest incentives to solve AI safety acknowledge structural limitations, every governance practitioner should too. The practitioner who claims their governance framework provides comprehensive coverage is either uninformed or dishonest. The practitioner who says "here is what our governance does, and here is where it stops working" is the one building something real.

The compliance-safety gap is the most dangerous space in AI governance. As Holistic AI states: "When compliance becomes the sole driving force, we lose sight of the real purpose, which is safety." 59% of organizations report being 'very confident' in their visibility into AI tools, but only 36% actually have an AI policy in place or are developing one. 92% of organizations say they trust vendors that use AI, but many do not know how these vendors handle data or when AI usage shifts. The gap between confidence and reality is widening, and framework limitations are part of the reason.

This Is Not an Argument Against Governance

I need to state this as directly as possible: nothing in this article is an argument against governance. It is an argument for governing honestly. The five structural limitations I have described do not make frameworks useless. They make frameworks necessary and insufficient. That distinction is everything. Only 14% of organizations have enterprise-level AI governance frameworks despite 78% using AI in operations. The problem is not that we have too much governance. The problem is that we have too little — and what we have sometimes creates false confidence. 59% of organizations report being 'very confident' in their AI visibility, but only 36% actually have an AI policy in place. That gap between confidence and reality is the most dangerous space in AI governance.

Here is what governance frameworks reliably accomplish, and what no structural limitation can take away:

• Inventory and visibility. You cannot govern what you do not know exists. Frameworks force organizations to catalogue their AI systems, map data flows, and identify stakeholders. McKinsey's agentic AI analysis confirms: without proper inventory and identity management, scaling agents means scaling unknown risk. The inventory is not governance. But governance is impossible without it.
• Accountability structures. Someone is responsible, even if the outcome is uncertain. Frameworks create roles, escalation paths, and decision rights. When something goes wrong — and it will — the question "who decides what happens next?" has an answer. See the A6 accountability analysis for the full framework.
• Process discipline. Systematic review is better than ad hoc reaction. Frameworks create gates, checkpoints, and review cadences that catch some problems before they cascade. Not all problems. But some. And "some" is meaningfully better than "none."
• Signal detection. Monitoring catches some problems before they become crises. The four controls model — preventative, detective, responsive, adaptive — provides layered defense. No layer is sufficient alone. Together, they significantly reduce blast radius.
• Organizational readiness. Governance builds the institutional muscle for responding to surprises. An organization that has practiced risk assessment, incident response, and stakeholder communication — even imperfectly — is far better positioned to handle the unexpected than one that has never exercised those capabilities.

The value of governance is not that it prevents all harm. It is that it reduces harm, speeds recovery, and creates the organizational capacity to handle what you did not predict. As Holistic AI states plainly: "Compliance with the various regional regulations is not the same as governance. Governance is broader, deeper, and more fundamental to both safety and business success." The Minimum Viable Governance framework provides the structural foundation. The Trust Premium quantifies the market value. The Liability Ledger maps the compounding cost of gaps. Together, they create a governance architecture that acknowledges limits while building real protection.

“Responsible AI governance should not start — much less end — at legal compliance.”

76% of organizations fail in multi-departmental coordination of AI governance. Only 12% of companies feel 'very prepared' to assess, manage, and recover from AI governance risks. These numbers are not governance failures — they are honest indicators of a hard problem. The organizations making progress are not the ones reporting perfect scores. They are the ones building the organizational muscle to handle the next surprise. And that muscle is what frameworks, at their best, develop.

Five Practices for Governing What Frameworks Can't Reach

A14 Epistemic Humility introduced four practices for governing under uncertainty. This article extends them to address the five structural limitations directly. These are not philosophical suggestions. They are operational practices with measurable outcomes. Each one responds to a specific limitation that no framework can overcome on its own.

Practice 1: Govern for the Unclassified

Responds to: The Emergence Problem. Design incident response for risk types that do not exist yet. Most governance frameworks assume you can enumerate risks in advance and build controls for each one. The emergence problem proves you cannot. The practice: build an incident response protocol with an explicit category called "unclassified incident." Define the escalation path, decision authority, and communication protocol for events that do not match any existing risk category. AI-related incidents rose to 233 in 2024 — a 56.4% increase over 2023. Your next incident is more likely to be novel than familiar. Be ready for it.

Practice 2: Accept Irreducible Opacity

Responds to: The Opacity Problem. Stop pretending you can explain everything. Focus on governing outcomes rather than requiring complete mechanistic understanding. The LoBOX governance ethic provides the framework: reduce accidental opacity where possible, bound irreducible opacity with monitoring and constraints, and delegate trust through institutional oversight. The practical implementation: for every AI system, document what you can explain, what you cannot explain, and what compensating controls you have built for the gap between those two. Publish this honestly. Culture shapes behavior more powerfully than rules — and a culture of honest disclosure shapes better governance than a culture of false completeness.

Practice 3: Build Regulatory Portability

Responds to: The Boundary Problem. Design governance that works across jurisdictions, not just one. Instead of building separate compliance programs for the EU AI Act, NIST AI RMF, and ISO 42001, build a unified governance core that maps to multiple regulatory requirements. The NIST AI RMF Practitioner's Guide and AskAjay Crosswalk demonstrates this approach — mapping one governance architecture to multiple framework requirements. Regulatory portability means your governance survives jurisdictional changes, new regulations, and cross-border expansion without requiring a complete rebuild each time.

Practice 4: Measure What Matters, Not What's Measurable

Responds to: The Measurement Problem. Choose meaningful metrics even if imperfect. The temptation when facing measurement limitations is to measure what is easy and call it governance. Resist this. Not all risks can be solved by better models — decisions about acceptable use, escalation, and enforcement are human decisions. Track leading indicators: time-to-detect anomalies, time-to-respond to incidents, stakeholder complaint patterns, model drift velocity. These are imperfect but meaningful. A governance dashboard full of green checkmarks that misses the risks that matter is worse than an honest dashboard with yellow warnings.

Practice 5: Practice Institutional Humility

Responds to: The Pacing Problem. Publish what your governance cannot do. Conduct quarterly governance reviews that explicitly ask: "What has changed since our last assessment that our framework does not cover?" Governance must evolve from static to dynamic, from retrospective to real-time, from compliance to continuous assurance. AskAjay's own governance page practices this principle: it discloses what is governed, what is not yet governed, and what the known limitations are. That transparency builds more trust than any claim of completeness. And it models the institutional honesty that nearly 100 ethical codes have failed to deliver because their purpose was image enhancement, not operational change.

These five practices extend A14's four practices for epistemic humility from philosophical principles to structural responses. Where A14 argued that governance practitioners must admit uncertainty, B13 maps that uncertainty to specific architectural limitations and provides specific operational responses. The four controls model — preventative, detective, responsive, adaptive — provides the operational framework. These five practices are the adaptive layer: the controls that activate when the other three layers reach their structural limits.

What This Means for a 50-Person Company

If you are the CTO of a food delivery startup reading this, you may be thinking: structural limitations of global governance frameworks are not your problem. You are wrong — they are your problem at a smaller scale. Your food delivery app's governance framework cannot anticipate every failure mode of the AI models you deploy. Your route optimization agent will encounter situations nobody tested for — floods, protests, construction patterns your training data has never seen. Your demand forecasting model will drift as customer behavior changes. And your chatbot will eventually produce outputs that surprise you.

The startup version of these five practices:

• Govern for the unclassified: Your incident response plan should have a category for "behaviors we did not anticipate." If it only covers known failure modes, it is incomplete by design.
• Accept irreducible opacity: You do not need to explain every decision your AI makes. You need to monitor outcomes, catch anomalies, and have a kill switch for when outputs go wrong.
• Build regulatory portability: If you are in the US today but plan to serve EU customers next year, build governance that translates. One architecture, multiple compliance mappings.
• Measure what matters: Track customer complaints about AI-generated recommendations, not just model accuracy scores. The complaint is the signal; the accuracy metric is the vanity number.
• Practice institutional humility: In your next all-hands, tell your team: "Here are three things we do not know about how our AI systems will behave." That honesty prevents the false confidence that causes real failures.

The Minimum Viable Governance framework provides the 90-day implementation path designed for resource-constrained teams. The ROI of AI Governance makes the business case your CFO needs. But start with the admission that your governance will have limits — and build systems that work despite those limits, not systems that pretend the limits do not exist. That is fine. Build governance that is good at detecting surprises and recovering fast.

For the board member reading this: ask your governance team one question at the next meeting: "What are the three biggest risks our governance framework does not cover?" If they cannot answer, your governance is built on assumed completeness. If they can answer, you have a team that understands what this article is about. Either answer is valuable. The question itself is the practice.

The Honest Governance Trilogy

This article is the first in a trilogy. It establishes the structural case: governance frameworks have inherent limitations that honesty, not denial, must address. What comes next builds on this foundation:

• B14: AI Governance Theatre — How to tell real governance from performative governance. If this article describes what frameworks cannot do, B14 will describe what organizations pretend frameworks do. The distance between those two is governance theatre.
• B15: When Should You Stop? — The decision nobody wants to make. When governance limitations compound to the point where an AI deployment's risks exceed its benefits, what does honest governance look like then? B15 will address the hardest question in AI governance: the decision to not deploy.

“The fragmentation we keep trying to fix is not a bug in our governance efforts. It is a feature of the thing we are trying to govern.”

This trilogy connects to the full AskAjay governance ecosystem: A14 Epistemic Humility provides the philosophical foundation. MVG provides the structural baseline. The Trust Premium quantifies the market value of honest governance. The Liability Ledger maps the cost of gaps. The NIST Crosswalk maps framework coverage. A12 Failure Patterns shows what happens when governance fails. And A11 Data Governance addresses the foundation layer where all governance begins. Together, they form the most comprehensive, publicly available AI governance analysis on the internet. And this article — the one that tells you what they cannot do — is what makes the whole system honest.