Why Human Judgment Still Decides Whether Your AI Agents Are Safe
Petar Besalev is the EVP of Cybersecurity and Compliance Services at A-LIGN.
gettyAI agents are moving out of pilot programs and into production, taking on decisions in procurement, security and customer operations. As that shift accelerates, there is an irony at the center of it: The standards built to govern these systems still depend on humans to validate them.
The industry has spent the last year asking whether AI will replace compliance professionals. But that’s the wrong question. The right question is who’s accountable when the AI gets it wrong. And right now, the answer is still a human being.
Not long ago, most conversations in this industry started and ended with “we have AI.” Today, the more useful framing is: “We have people with experience and professional judgment, supported by technology.”
That shift reflects where the technology actually is, not where the hype says it should be. AI is a genuine enablement tool, making assessments faster and more efficient while removing a meaningful amount of manual work. But efficiency and judgment are not the same thing, and leaders who treat them as interchangeable are setting themselves up for a hard lesson.
Regulated industries already have a clear answer for what happens when something goes wrong: Someone signs their name to it.
Under frameworks like the Cybersecurity Maturity Model Certification (CMMC), the affirming official personally attests that an organization’s systems meet the required standard. That signature carries liability under the False Claims Act. No AI system can assume that liability. Only a person can.
Leaders should treat that as the test for any process they’re automating. If something goes wrong, who is exposed? And can that exposure sit with a model? In almost every regulated context today, the answer is no.
New technology tends to move faster than the frameworks meant to govern it, and AI governance is no exception. There is no single mandate for AI compliance yet, only an emerging patchwork that includes the NIST AI Risk Management Framework, ISO 42001, the EU AI Act and newer standards built specifically for AI agents.
That gap can look like a weakness in the system, but it’s actually the system working as intended, since regulators tend to wait for evidence before they mandate trust. But that doesn’t mean the market will wait.
According to recent industry survey data, 80% of companies using AI said customers are already asking about their AI risk management practices, yet one-third of organizations still have no AI compliance strategy in place at all. That’s why organizations should be adopting a recognized framework now, before it’s required.
As agentic AI takes on more autonomous work, from executing transactions to making real-time security decisions, the accountability question only gets more important. An agent can act faster than a person can review its actions, which makes the human role more important.
The organizations best positioned for this shift are the ones already building governance structures that assume a human owns the outcome, regardless of how autonomous the underlying tooling becomes.
Those building out governing systems for agentic AI need to make that structure explicit. They must separate independent technical testing on how the agent performs from a distinct governance audit of the policies, data handling and controls around it. That second layer is where human judgment lives. It’s evaluated on its own rather than folded into a single automated check, and it’s increasingly reassessed on a recurring basis, rather than treated as a one-time sign-off, since an agent’s behavior can drift over time.
The mechanism matters less than the principle: Autonomy in execution should never mean autonomy in accountability.
As AI governance and certification options multiply, a familiar problem is resurfacing: Not all validation is built the same.
Risk teams are already rejecting compliance reports that come from firms without the depth or track record to back them up, and that same scrutiny will apply to AI governance reviews. A superficial check against a framework creates false assurance, and false assurance is more dangerous than an honest gap because it hides the risk instead of surfacing it.
The frameworks gaining traction share a few traits: input from the security and risk leaders who have to live with the results, a regular update cadence rather than sitting static and dozens of specific, testable requirements rather than one broad checkbox.
As this space matures, the organizations that ask who is validating their AI systems, how rigorously they’re validating and how often they do it will be the ones that hold up under scrutiny.
This was never a question of people versus technology. It’s people, enabled by technology, which is a very different outcome. AI is a genuine competitive advantage when it’s paired with the judgment to use it responsibly. But the accountability, the liability and the professional judgment that regulators and customers are actually paying for will remain human, and that isn’t changing anytime soon.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
