Governance of Agency

The Governance of Agency

Humans must explicitly define the scope, authority, escalation paths, and risk boundaries for every agent. No agent can safely or reliably determine its own limits. Without a clear, written constitution of delegation, agentic systems drift into misalignment, overreach, or collapse.

The Principle of Delegated Agency Scaling

Autonomy is not a binary setting but a variable slider dependent on verification capability. We scale AI agency in proportion to our ability to automatically validate the output. Low-risk or easily verifiable tasks allow for high autonomy; high-risk or subjective tasks require restricted agency (consultant mode). You cannot delegate authority where you cannot automate accountability. This scales agency according to The Principle of Graduated Agency by Structure and Risk and is constrained by The Principle of Verification Asymmetry.

Failure Scenario: Delegating complex build optimization to AI leads to short-term gains but introduces critical errors, increasing rework and risk.

The Corollary of Automated Guardrail Prerequisite

Before granting full autonomy to AI agents for a task, ensure robust automated safety nets (CI/CD, test suites) are in place. Automation must be checked by automation to prevent catastrophic failures.

The Corollary of Trust-Gated Orchestration

The velocity and scale of agent orchestration are strictly limited by the "Trust Latency" of the human operator. If a human must verify every intermediate "real task" within a workflow, the system degrades from an autonomous fleet into a synchronous, manual approval queue. True orchestration is only possible when the cost of verification is significantly lower than the cost of execution. Therefore, trust—built on robust provenance and observability—is not a sentiment, but a functional requirement for scaling.

Failure Scenario: A manager deploys a team of five agents to optimize marketing campaigns but requires manual approval for every keyword selection and ad copy variant. The "orchestration" becomes a bottleneck where the manager spends more time reviewing low-stakes decisions than if they had done the work themselves, stalling the entire workflow.

The Principle of Asymmetric Risk

The economics of automation are governed by convexity: the cost of verification is often linear (time spent reviewing), but the cost of failure can be non-linear (catastrophic data loss, security breaches). Agency must be capped not by the capability of the model, but by the bounds of the downside risk. When the "blast radius" of an error is infinite (e.g., production database access), autonomy must be zero, regardless of the model's intelligence.

Failure Scenario: An autonomous agent is given write access to the production environment to "fix a small bug." It hallucinates a command that drops a critical table. The 5 saved in developer time results in a \500,000 outage.

The Corollary of Bounded Edit Radius

An agent given write access to a region treats the entire region as eligible for modification; edits diffuse outward from the requested target because each local change is locally plausible. Because blast radius is what governs risk, the agent's editable surface—not just its executable actions—must be constrained: scoped to specific files, specific functions, or change budgets enforced by tooling. Without this, a one-line bug fix can return as a 200-line PR that is individually defensible and collectively unreviewable, silently expanding the asymmetric downside this principle exists to bound.

The Principle of Contextual Authority

An AI agent's effective capability is capped by the operator's ownership and mental model of the system. When the operator has deep knowledge (ownership), AI acts as an extension of will, amplifying intent. When the operator lacks deep knowledge (contracting), AI acts as a temporary shield against complexity, hiding implementation details the operator cannot evaluate.

You cannot safely delegate authority to an agent when you cannot predict the side effects of its output. This bounds agency even when the model is capable, because the operator cannot supply reliable review, escalation, or rollback judgment. This constraint interacts with The Principle of Delegated Agency Scaling, The Principle of Asymmetric Risk, and The Principle of Context Compressibility.

Failure Scenario: A contractor uses an AI agent to close a ticket in a legacy codebase they do not understand. The AI suggests a solution that works perfectly in isolation but relies on an internal API scheduled for deprecation. Because the operator lacks the Contextual Authority to know the API history, they accept the solution, solving the ticket today but creating a guaranteed failure for the next release.

The Corollary of Ownership Declaration

Before delegating meaningful agency, explicitly declare whether the operator is acting as an owner (has the mental model to predict side effects) or a contractor (does not). In contracting mode, restrict the agent to low-risk exploration and require an owner to review decisions with irreversible consequences.

The Corollary of Side-Effect Predictability Gates

Only allow an agent to execute irreversible actions (writes, merges, deployments) when the operator can articulate the likely side effects and the rollback path. If the operator cannot, force isolation (e.g., staging/sandbox, read-only tools), aligning with The Principle of Execution Isolation.