19 Ways To Set Boundaries For Autonomous AI Agents

Gopichand Mannava, Independent Researcher, Chief Data & AI Architect, State of Connecticut.

Retractable barrier tape labeled 'authorized access only'Alexander GrigorianEven when AI agents perform reliably in testing, certain actions warrant human oversight. For example, consider a semantic-integrity gate: automatically verify that each record’s meaning matches its metadata—not just its schema—before ingestion. An invoice date can be valid yet mislabeled as a payment date. Quarantine that mismatch before the model or vector store sees it. The failure this prevents is subtle: an AI answer that is perfectly grounded in the wrong business meaning.

Technology leaders must establish clear boundaries around agent autonomy. The challenge is enforcing those limits through system architecture rather than relying on prompts or trust alone. To help, I asked members of the Software Architects Group, a community that I lead through Forbes Technology Council, to share which actions they would reserve for human approval and how they build those safeguards into their systems.

Never let an AI agent expand its own authority without human approval. The bigger risk is not one bad action—it’s an agent quietly gaining the ability to take many consequential ones. The architecture for bounded autonomy is fixed permission ceilings, policy-based escalation and cryptographic auditability. This way, AI can act fast without redefining its own limits. - Swati Deepak Kumar (Nema), Citigroup

I am working on governance architecture for a workflow in our platform, using GPT-6 Astra. So far, I’ve learned never to accept governance architecture without verifying it and to make sure all incorrect cases are addressed. Also, keep it simple. An AI agent’s solution is complex and accommodates use cases that are not correct. One can evaluate different designs, but always verify and approve. Make your own judgment calls and never accept AI suggestions at face value. - Siva Nookala, Decision Lens

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?

Don’t allow an AI agent to independently make a binding commercial commitment, even with a flawless testing record. Agreeing to terms, accepting liability or committing funds requires human accountability. Architecturally, I would enforce this through restricted permissions, approval gates at the transaction layer and immutable audit logs. The agent can negotiate, recommend and prepare the action, but only an authorized human can commit the organization. - Prajkta Waditwar, Box Inc

Don’t let an AI agent change any privilege or authorization given, nor update the way it acts without any human supervision. Also, never let it update the way it works with data beyond the original responsibility granted. - Gregorio Alejandro Patiño Zabala, Pragma

We never allow database migrations to be performed by an AI agent autonomously. Many operations are irreversible, and if done without a human knowing, they can lead to permanent data loss and an SLA breach. - Jeff An, Momentic

I would never let an AI agent change a supplier’s bank details without human approval. The FBI documents scams that redirect legitimate invoice payments. My design would require verification through a known contact and server-side approval tied to the exact new account. Any subsequent edit would invalidate that approval. A routine data update can decide where the next million dollars goes. - Nikhil Jathar, AvanSaber Technologies

Don’t let an AI agent irreversibly destroy data, such as dropping a production database, force-pushing over shared history, or deleting backups, because a thousand clean test runs can’t pay for one unrecoverable mistake. That boundary lives outside the agent’s judgment. Deny rules and pre-execution hooks block those commands before they run, and the agent never has production-write or backup-delete credentials. A person should grant that access separately for each task. - Srinivas Chippagiri, Salesforce Inc

I would never let an AI agent change a ledger entry. Similarly, I wouldn’t allow it to cancel or add charges to an accounting system. Only humans can and should make those kinds of decisions. - Timmi Ryerson, Smart Property Systems

My non-negotiable boundary is that an AI agent must never independently authorize irreversible, high-consequence actions. Across high-risk defense-grade assurance environments, testing proves capability, not authority. You must enforce this through policy-gated execution, least-privilege identities, human approval and immutable audit evidence. Why? Reliability earns trust; it never grants permission to act. - Rajjie Sarmey, FutureProof CXO™

Never let an AI agent finalize safety controls for a hazardous job. It can identify missed hazards and recommend controls, but it cannot authorize them. We enforce that boundary in the application, not a prompt: AI output remains a recommendation until a human reviews and confirms it. Even a model that performs reliably in testing can miss the one hazard that changes someone’s life forever. Human judgment is not an optional safeguard. - Julia Penfield, PH.D., VelocityEHS

Don’t let an AI agent update its own behavior from live production data, such as changing prompts, memory, or retrieval sources, without human review. Small silent shifts can compound into a very different system from the one we tested. In our architecture, changes go through a versioned release pipeline with regression tests and a named approver, so production always runs what we validated. - Venkatesh Sankaran, Saguna Consulting Services LLC

Changing the architecture is something I would not let an AI do without human oversight. Best, if such changes are happening through a platform like CALM, make sure that the architecture is reviewable, drifts are detectable, and active oversight (which might possibly involve) can be expressed. The change in architecture might cause scalability issues, security lapses, and, really, any kind of “unknown unknowns” that you are trying to prevent the system from falling into. - Peter Smulovics, Morgan Stanley

Avoid letting an AI agent handle anything a customer cannot undo: deleting records, issuing refunds, changing someone’s access. Testing only tells you how the agent behaves on cases you thought of. Enforcement belongs in the permissions, not the prompt. The agent gets credentials that cannot make those calls at all, and the approval step sits in the service, so a manipulated agent has nothing to act with. - Kriti Faujdar, Yelp

I would never let an AI agent expand its own permissions or delegate authority without human approval. Reliability in testing does not establish authority. Architecturally, agents cannot modify their own access policies. A separate authorization layer enforces permissions, requires human approval for privilege changes and records the decision. - Vibhor Kumar, EnterpriseDB

I’d never let an AI agent make irreversible production changes—like finalizing financial payouts or deleting customer records—without human sign-off. We enforce this with air-gapped execution: The AI can draft and stage transactions, but our downstream APIs require a signed approval token from a verified human before processing the event. - Mahendran Chinnaiah

Don’t let an AI agent post a financial adjustment to a customer account or the general ledger without human approval. In large transformations, I have seen one adjustment ripple through billing, reporting and reconciliation. The boundary sits outside the model: The agent proposes, while a separate workflow verifies the approver’s role, issues a one-time posting token, and logs the trail. - Govinda Rao Banothu, Cognizant Technology Solutions

I would never let an AI agent elevate its own production privileges without human approval. I would enforce this through an independent authorization service, least-privilege access, and time-limited permissions tied to approved requests. The agent could propose changes but could not approve them or bypass the gate. All actions would be logged. - Vikram Pande, HCLTech

The right way is always verification before action. It often delivers a larger reliability improvement than using a bigger model at one go. - Santanu Boral, Cognizant Technology Solutions

I would never allow an AI agent to redefine the original human intent without explicit human approval. It can adapt how it executes a task, but changing the objective, scope, or constraints requires a human decision. I’d enforce that boundary through controls outside the agent, with approval required for changes to its authorized mandate.- Arun Kumar Elengovan, Okta, Inc.