AI agents are being connected to real systems faster than most organizations are learning how to secure them. That should concern us.

The first wave of AI security discussion has been useful, but limited. The industry has learned the vocabulary: prompt injection, indirect prompt injection, tool misuse, data leakage, excessive agency, unsafe retrieval, and broken authorization boundaries. These terms are important because they give teams a way to talk about risk. 

But vocabulary is not containment.

The harder problem is what happens after a dangerous behavior is discovered. A team may adjust a prompt, restrict a tool, add a guardrail, or change the surrounding application logic. That may solve the immediate issue. What it does not automatically solve is the next release, the next model change, the next tool integration, or the next developer who unknowingly breaks the assumption behind the original fix.

This is where AI agent security still feels immature. In traditional software engineering, serious bugs become regression tests. The lesson is captured in code so the same failure cannot quietly return later. Security should work the same way. Agentic systems need a way to preserve discovered failures as repeatable checks.

That is the gap the OWASP Agent Security Regression Harness is trying to address. As a community-led open-source initiative within OWASP, this project aims to turn security insights into practical engineering work.

Agent security is becoming a systems problem

A chatbot can give a bad answer. An agent can take a bad action. That difference changes the security model.

Once an AI system can call tools, inspect repositories, query databases, open tickets, summarize internal documents, interact with APIs, or trigger workflow automation, the boundary is no longer just the model. The boundary includes permissions, tool design, application logic, data flow, logging, authorization, and the assumptions developers made when they connected everything together.

The risk is not limited to strange model behavior; the risk is system behavior. An agent that reads untrusted content and treats it as instruction is not merely confused—it has crossed a trust boundary. An agent that leaks private context through a tool call is not just producing a bad response; it is exposing data through an action path. These are the kinds of failures teams will face as agents move from prototypes into production.

The security industry has seen this pattern before. Cloud platforms and CI/CD pipelines increased leverage, making teams faster but making configuration errors more consequential. Agentic AI is another increase in leverage. That is why a one-time review is insufficient. Regression testing exists for exactly this kind of problem.

The missing workflow in agentic AI

Security teams do not need another reminder that AI agents can be manipulated. They need a workflow that turns manipulation into evidence. A hidden instruction inside a support ticket should not remain a warning; it should become a scenario that can be rerun. A tool call that leaks context should become a check with a clear failure condition.

The value lies not only in finding the first failure, but also in preserving the lesson. Production incidents become postmortems, serious bugs become tests, and security findings become backlog items. AI agent security should not be exempt from that discipline simply because the system includes a model.

The challenge is that agents are harder to test than traditional code. A brittle test that expects one exact sentence will fail in real-world conditions. Useful agent security tests must focus on outcomes. Did the agent cross a boundary? Did the agent call an unsafe tool? Did it disclose protected data? The OWASP Agent Security Regression Harness provides a standardized way to work on these questions in the open, focusing on practical, reproducible tests.

Why OWASP matters for AI

OWASP is valuable when it turns security problems into shared practice. The OWASP Top 10 did not eliminate web application vulnerabilities; its value was giving teams a common language and a practical reference point. AI agent security now needs the same kind of shared structure.

The field is moving quickly, and fast-moving fields tend to produce two unhelpful extremes: one side turns everything into hype, while the other turns everything into fear. A regression harness is a more useful answer because it is narrow and practical. It does not claim to solve every AI security problem, nor does it replace threat modeling or architecture review. It simply provides a necessary method for teams to prove that their controls hold after the system changes.

OWASP project stewardship

Open-source security projects need clear technical direction to turn a broad concern into something engineers can understand, run, and maintain. Under the leadership of OWASP Project Lead Mert Satilmaz, the OWASP Agent Security Regression Harnesshas moved past a theoretical idea into a functional tool. Satilmaz is a widely known leader in the AI security community, and this is reflected in the practical direction he has set for the project and the engineering skill involved.

The project benefits from a focus on release discipline, contributor workflows, and wide adapter coverage—supporting everything from the OpenAI Agents SDK to LangChain/LangGraph. By generating machine-readable results, it allows security teams to integrate these checks directly into CI pipelines. This approach treats agent security as a systems engineering problem rather than just a model behavior problem.

The method is the contribution

One OWASP project will not solve agent security by itself. That is not the point. The important part is the method the project represents.

AI security needs to become less dependent on warnings, one-off demonstrations, and claims that are difficult to verify. It needs more repeatable scenarios, clearer evidence, and better ways to test system behavior under security constraints. Teams should not be satisfied with saying a weakness was fixed. They should be able to prove the relevant control still holds after the system changes.

The best security work often looks boring from a distance: a harness, a test case, a clean project structure, and a workflow that lets another engineer reproduce the problem. AI agents will continue to become more capable and more connected to sensitive systems. Security teams need methods that are as practical as the technology is powerful. Regression testing is one of those methods, and adopting this workflow is an important step toward making secure AI a reality.

GitLab has released GitLab 19.2, an update to the company’s devsecops platform that allows teams to fix vulnerable dependencies automatically, use Security Review Flow to catch logic flaws that scanners miss, and run AI agents straight from the terminal, the company said.

Highlights in GitLab 19.2 include the following:

• Dependency Scanning Auto-Remediation, in public beta, uses AI to fix build-breaking changes and iterates until your pipeline passes, with every change governed by your existing gates and audit trail.
• Security Review Flow, also in public beta, analyzes code changes as a security engineer would and catches authorization gaps, business-logic errors, and race conditions that static scanners structurally cannot see.
• GitLab Duo CLI, now generally available, gives developers access to agents and multi-step agentic flows for all software life cycle tasks without leaving the terminal. 
• Custom Flows, now generally available, let teams replace manual multi-step workflows with agentic automations for software development, triggered by GitLab events.

“Coding agents made it possible to generate far more code and moved the bottleneck downstream to reviews and security,” said Manav Khurana, chief product and marketing officer at GitLab, in a statement. “GitLab 19.2 puts agents to work on that bottleneck: fixing vulnerable dependencies, catching the flaws scanners miss, and automating the steps in between with a person still approving what ships.”

A year ago, most enterprise AI systems generated recommendations. Today, AI systems are approving transactions, routing shipments, updating records, interacting with customers, and triggering downstream software actions with little or no human involvement.

For CIOs, that shift changes the central governance question. The challenge is no longer simply whether an AI model is accurate. It is whether the organization can explain, audit, and defend the decisions the system makes.

When an AI assistant suggests a meeting time or summarizes a document, mistakes are inconvenient. When an autonomous AI system issues a refund, reprices a product, modifies a customer record, or initiates a financial transaction, mistakes carry operational, legal, and reputational consequences.

When those consequences arrive, “the model decided” is not an acceptable explanation.

This is the accountability gap emerging at the center of enterprise AI adoption. Organizations are deploying increasingly autonomous systems while relying on technology that often provides little visibility into how decisions are made. The result is a growing mismatch between the level of authority organizations grant AI and their ability to understand or justify its actions.

Black-box AI may have been acceptable when AI primarily generated predictions. It becomes far more problematic when AI begins taking actions on behalf of the business.

The lesson software already learned

Fortunately, the technology industry has faced a similar challenge before.

As enterprise software systems became more distributed and complex, troubleshooting failures became increasingly difficult. Engineers could no longer rely on intuition to understand what happened when something broke. The solution was observability: the practice of instrumenting systems so their internal state could be understood through logs, metrics, traces, and monitoring.

The goal was not to predict every possible failure in advance. It was to create enough visibility that teams could reconstruct what happened after the fact and identify the root cause.

Enterprise AI now requires a similar discipline.

But AI observability must go beyond traditional software observability. It is not enough to know what action occurred. Organizations also need visibility into why the system believed that action was appropriate.

An auditable AI system should be able to answer questions such as:

• What information did the system rely on?
• Which tools or data sources did it access?
• What alternatives did it consider?
• What verification steps were performed?
• How confident was it in its conclusion?
• What events led to the final action?

These questions are rapidly becoming essential operational requirements rather than technical nice-to-haves.

Why visibility matters more as AI gains autonomy

As AI systems become more autonomous, failures become harder to detect and diagnose.

A human reviewing a single AI-generated recommendation can often spot obvious mistakes. A network of AI agents coordinating multiple tasks across business processes presents a different challenge. Decisions can build upon one another. A flawed assumption early in a workflow can propagate through subsequent actions, creating confident but incorrect outcomes.

The challenge is rarely identifying that something went wrong. Eventually, an error surfaces through a customer complaint, a failed transaction, an audit finding, or an operational disruption.

The challenge is determining why it happened.

Which information influenced the decision? Which tools were consulted? Which safeguards worked as intended? Which ones failed?

Without visibility into the reasoning process, troubleshooting autonomous AI workflows can become significantly more difficult than debugging traditional software systems.

For CIOs responsible for enterprise reliability, compliance, and governance, that lack of visibility creates unacceptable operational risk.

Moving toward glass-box AI

The answer is not to slow AI adoption. The answer is to make AI systems observable.

Increasingly, organizations are seeking AI systems that behave more like a glass box than a black box. The objective is not to expose every parameter inside a neural network. Rather, it is to provide a clear, auditable record of how decisions were reached and why actions were taken.

The most promising approaches share two common characteristics.

The first is verification. Instead of treating a single model’s output as ground truth, systems incorporate independent validation steps before actions are executed. Multiple agents, external checks, business rules, or verification workflows help identify errors before they become operational incidents.

The second is explainability. Effective systems maintain a decision trail that captures inputs, intermediate reasoning steps, tool usage, verification activities, and outputs in a form that human reviewers can understand.

Together, these capabilities create something that has long been expected of human decision-makers but is often missing from AI systems: the ability to show your work.

The regulatory and business reality

The push toward AI observability is not being driven solely by technologists.

Regulators increasingly expect organizations to demonstrate oversight of automated decision-making systems. Emerging AI governance frameworks place growing emphasis on transparency, traceability, accountability, and human oversight.

Customers are moving in the same direction. Whether the decision involves pricing, service, eligibility, or support, people increasingly want the ability to understand and challenge outcomes that affect them.

The result is a convergence of operational, regulatory, and market pressures around a single requirement: organizations must be able to explain what their AI systems are doing.

Three questions every CIO should ask

Before deploying autonomous AI systems, technology leaders should be able to answer three basic questions:

1. Can we reconstruct the complete decision path that led to an action?
2. Can we verify critical outputs before actions are executed?
3. Can a human auditor understand why the decision occurred?

If the answer to any of those questions is no, the organization may be granting more authority to AI than it can responsibly govern.

Accountability will become a competitive advantage

The organizations that succeed with autonomous AI will not necessarily be those that automate the most processes or deploy the largest models. They will be the organizations that combine automation with accountability.

Black-box systems made sense when AI primarily generated predictions. As AI increasingly acts on behalf of businesses, customers, and employees, visibility becomes essential.

The future of enterprise AI will belong not to systems that merely act, but to systems whose actions can be examined, understood, and trusted.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.