I’ve had a version of the same conversation probably 30 times in the past year. A VP of engineering or a senior SRE describes their observability stack — the dashboards they’ve built, the alerting they’ve tuned, the Datadog or Grafana setup they’ve spent months getting right. And then they describe the last major incident. How the alert fired in seconds. How the dashboard showed exactly where the degradation was. And how their team still spent two to three hours figuring out what caused it before they could fix anything.

The framing I hear most often goes something like this: “We can see everything. We just can’t explain it.”

That gap between seeing and explaining is worth digging into, because it’s quietly become one of the more expensive problems in operations.

What observability was built to answer

Observability tools are excellent at what they were designed for. They surface what is happening inside systems — latency spikes, error rates, resource exhaustion, dependency failures — and they do so at scale, in real time. That capability is foundational. Without it, you wouldn’t know something broke until a customer told you.

But observability was built around a specific question, what is happening inside the system right now? It answers that question well. A different question, why is this happening and what triggered it, sits mostly outside its reach.

Those sound similar. They are not the same question.

“What is happening” lives in the infrastructure layer: metrics, traces, logs. The observability stack captures it. “Why is it happening” often lives outside the infrastructure entirely, in a deployment that went out two days ago, in a support queue that started spiking before the alert fired, in a change record that engineering approved last sprint. That context isn’t in the telemetry because it was never meant to be.

The investigation that precedes the fix

When a customer-facing incident occurs, the information that would explain it is rarely concentrated in one place.

Engineering is looking at the infrastructure signal. They know what the system is doing, but not which customers are affected or what those customers reported before the monitoring alert fired.

Support has the cases, the complaints written in plain language by people describing symptoms they can’t name technically. Support often knows something is wrong before engineering does, but the deployment history isn’t a system they can search.

Somewhere in Jira or a CI/CD pipeline is a record of what changed and when, with a timestamp, an author, and a scope. That record doesn’t know about the support cases or the anomaly.

So somebody has to connect all of it by hand. That means reading the customer complaints, cross-referencing the deployment history, holding multiple systems in their head simultaneously until the pattern surfaces. That is what the investigation actually is; not problem-solving, but reconstruction.

It takes hours. And it runs entirely before the resolution process can begin. Mean time to resolution (MTTR), the metric most teams are tracking and trying to improve, measures what happens after the investigation ends. The investigation itself is outside the frame.

Who actually does this work

This is the part I think gets systematically under-reported.

The manual correlation work does not fall to anyone with spare capacity. It routes to whoever knows the system well enough to read across these different views simultaneously. The engineer who remembers what shipped last week and why it was unusual. The support lead who can translate a customer’s description of a problem into a technical hypothesis. The person who has been around long enough to recognize the pattern.

That person gets pulled from whatever they were doing. Not because they are available, but because the investigation requires them specifically.

In my experience this is not an edge case. It is a structural feature of how incident response works when the relevant signals are distributed across systems that do not talk to each other. The investigation always finds the person with the most context. And that person is never the most available; they are the most consequential.

The hours they spend reconstructing a timeline are hours they are not doing the other things only they can do. That cost is real and it compounds. It shows up in velocity. In recurrence. In the quiet attrition of senior people who have been in the war room one too many times. It does not show up in MTTR or any incident metric most organizations track.

Two kinds of work

There is a distinction that rarely gets made in incident response conversations: the difference between investigation work that requires human judgment and investigation work that is, at its core, data assembly.

Deciding whether to patch or roll back requires judgment. So does knowing which customers are actually affected and which alerts are noise. These are high-context calls that require judgment that takes years to develop.

Cross-referencing timestamps across Jira, Salesforce, and a deployment dashboard to determine which change shipped before which customer complaint started; that is data assembly. It is tedious and time-consuming. It is what consumes the first two to three hours of almost every serious incident.

The problem is these two types of work have always arrived bundled together. The person who can make the judgment call also has to do the assembly, because no system was built to do the assembly for them. So your most experienced engineer — the one whose judgment you actually need — spends the majority of the investigation doing work that is far below their capability. Work that is numbing, not demanding. Work that clears their day and produces, at the end of it, a starting point for the actual decision.

That’s the waste that rarely gets named. Not the hours themselves, but the quality of what those hours contain.

What changes when they’re separated

The signals that would explain why an incident happened are already sitting in your systems. The customer cases, the deployment records, the infrastructure signals. They were there before the alert fired.

The investigation cost is a correlation problem, not a data problem. The data exists. It has just never been assembled automatically.

This is where a newer category of tooling is starting to take shape, sometimes described as operational intelligence: software that sits across observability, ticketing, and deployment systems and does the cross-referencing automatically, before a human ever opens a second tab.

When that assembly work is separated from the judgment work, the picture changes. The engineer in the war room arrives with a structured view of what happened, the deployment, the cases, the timeline. Their job becomes confirming it, refining it, and deciding what to do next. That’s the work they’re actually good at, and the reason they became engineers in the first place.

The organizations that get ahead of this won’t necessarily have better engineers or better observability stacks. They’ll have recognized that the investigation itself is a workflow, not an inevitability, and that data assembly and human judgment are separable. As AI-driven correlation tools mature, that separation is where the real leverage in incident response is likely to come from.

The observability stack tells you an incident is happening. What teams need next is the context that explains why. One of those capabilities has matured significantly over the past decade. The other is where the work is now.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

AI agents are being connected to real systems faster than most organizations are learning how to secure them. That should concern us.

The first wave of AI security discussion has been useful, but limited. The industry has learned the vocabulary: prompt injection, indirect prompt injection, tool misuse, data leakage, excessive agency, unsafe retrieval, and broken authorization boundaries. These terms are important because they give teams a way to talk about risk. 

But vocabulary is not containment.

The harder problem is what happens after a dangerous behavior is discovered. A team may adjust a prompt, restrict a tool, add a guardrail, or change the surrounding application logic. That may solve the immediate issue. What it does not automatically solve is the next release, the next model change, the next tool integration, or the next developer who unknowingly breaks the assumption behind the original fix.

This is where AI agent security still feels immature. In traditional software engineering, serious bugs become regression tests. The lesson is captured in code so the same failure cannot quietly return later. Security should work the same way. Agentic systems need a way to preserve discovered failures as repeatable checks.

That is the gap the OWASP Agent Security Regression Harness is trying to address. As a community-led open-source initiative within OWASP, this project aims to turn security insights into practical engineering work.

Agent security is becoming a systems problem

A chatbot can give a bad answer. An agent can take a bad action. That difference changes the security model.

Once an AI system can call tools, inspect repositories, query databases, open tickets, summarize internal documents, interact with APIs, or trigger workflow automation, the boundary is no longer just the model. The boundary includes permissions, tool design, application logic, data flow, logging, authorization, and the assumptions developers made when they connected everything together.

The risk is not limited to strange model behavior; the risk is system behavior. An agent that reads untrusted content and treats it as instruction is not merely confused—it has crossed a trust boundary. An agent that leaks private context through a tool call is not just producing a bad response; it is exposing data through an action path. These are the kinds of failures teams will face as agents move from prototypes into production.

The security industry has seen this pattern before. Cloud platforms and CI/CD pipelines increased leverage, making teams faster but making configuration errors more consequential. Agentic AI is another increase in leverage. That is why a one-time review is insufficient. Regression testing exists for exactly this kind of problem.

The missing workflow in agentic AI

Security teams do not need another reminder that AI agents can be manipulated. They need a workflow that turns manipulation into evidence. A hidden instruction inside a support ticket should not remain a warning; it should become a scenario that can be rerun. A tool call that leaks context should become a check with a clear failure condition.

The value lies not only in finding the first failure, but also in preserving the lesson. Production incidents become postmortems, serious bugs become tests, and security findings become backlog items. AI agent security should not be exempt from that discipline simply because the system includes a model.

The challenge is that agents are harder to test than traditional code. A brittle test that expects one exact sentence will fail in real-world conditions. Useful agent security tests must focus on outcomes. Did the agent cross a boundary? Did the agent call an unsafe tool? Did it disclose protected data? The OWASP Agent Security Regression Harness provides a standardized way to work on these questions in the open, focusing on practical, reproducible tests.

Why OWASP matters for AI

OWASP is valuable when it turns security problems into shared practice. The OWASP Top 10 did not eliminate web application vulnerabilities; its value was giving teams a common language and a practical reference point. AI agent security now needs the same kind of shared structure.

The field is moving quickly, and fast-moving fields tend to produce two unhelpful extremes: one side turns everything into hype, while the other turns everything into fear. A regression harness is a more useful answer because it is narrow and practical. It does not claim to solve every AI security problem, nor does it replace threat modeling or architecture review. It simply provides a necessary method for teams to prove that their controls hold after the system changes.

OWASP project stewardship

Open-source security projects need clear technical direction to turn a broad concern into something engineers can understand, run, and maintain. Under the leadership of OWASP Project Lead Mert Satilmaz, the OWASP Agent Security Regression Harnesshas moved past a theoretical idea into a functional tool. Satilmaz is a widely known leader in the AI security community, and this is reflected in the practical direction he has set for the project and the engineering skill involved.

The project benefits from a focus on release discipline, contributor workflows, and wide adapter coverage—supporting everything from the OpenAI Agents SDK to LangChain/LangGraph. By generating machine-readable results, it allows security teams to integrate these checks directly into CI pipelines. This approach treats agent security as a systems engineering problem rather than just a model behavior problem.

The method is the contribution

One OWASP project will not solve agent security by itself. That is not the point. The important part is the method the project represents.

AI security needs to become less dependent on warnings, one-off demonstrations, and claims that are difficult to verify. It needs more repeatable scenarios, clearer evidence, and better ways to test system behavior under security constraints. Teams should not be satisfied with saying a weakness was fixed. They should be able to prove the relevant control still holds after the system changes.

The best security work often looks boring from a distance: a harness, a test case, a clean project structure, and a workflow that lets another engineer reproduce the problem. AI agents will continue to become more capable and more connected to sensitive systems. Security teams need methods that are as practical as the technology is powerful. Regression testing is one of those methods, and adopting this workflow is an important step toward making secure AI a reality.

GitLab has released GitLab 19.2, an update to the company’s devsecops platform that allows teams to fix vulnerable dependencies automatically, use Security Review Flow to catch logic flaws that scanners miss, and run AI agents straight from the terminal, the company said.

Highlights in GitLab 19.2 include the following:

• Dependency Scanning Auto-Remediation, in public beta, uses AI to fix build-breaking changes and iterates until your pipeline passes, with every change governed by your existing gates and audit trail.
• Security Review Flow, also in public beta, analyzes code changes as a security engineer would and catches authorization gaps, business-logic errors, and race conditions that static scanners structurally cannot see.
• GitLab Duo CLI, now generally available, gives developers access to agents and multi-step agentic flows for all software life cycle tasks without leaving the terminal. 
• Custom Flows, now generally available, let teams replace manual multi-step workflows with agentic automations for software development, triggered by GitLab events.

“Coding agents made it possible to generate far more code and moved the bottleneck downstream to reviews and security,” said Manav Khurana, chief product and marketing officer at GitLab, in a statement. “GitLab 19.2 puts agents to work on that bottleneck: fixing vulnerable dependencies, catching the flaws scanners miss, and automating the steps in between with a person still approving what ships.”

A year ago, most enterprise AI systems generated recommendations. Today, AI systems are approving transactions, routing shipments, updating records, interacting with customers, and triggering downstream software actions with little or no human involvement.

For CIOs, that shift changes the central governance question. The challenge is no longer simply whether an AI model is accurate. It is whether the organization can explain, audit, and defend the decisions the system makes.

When an AI assistant suggests a meeting time or summarizes a document, mistakes are inconvenient. When an autonomous AI system issues a refund, reprices a product, modifies a customer record, or initiates a financial transaction, mistakes carry operational, legal, and reputational consequences.

When those consequences arrive, “the model decided” is not an acceptable explanation.

This is the accountability gap emerging at the center of enterprise AI adoption. Organizations are deploying increasingly autonomous systems while relying on technology that often provides little visibility into how decisions are made. The result is a growing mismatch between the level of authority organizations grant AI and their ability to understand or justify its actions.

Black-box AI may have been acceptable when AI primarily generated predictions. It becomes far more problematic when AI begins taking actions on behalf of the business.

The lesson software already learned

Fortunately, the technology industry has faced a similar challenge before.

As enterprise software systems became more distributed and complex, troubleshooting failures became increasingly difficult. Engineers could no longer rely on intuition to understand what happened when something broke. The solution was observability: the practice of instrumenting systems so their internal state could be understood through logs, metrics, traces, and monitoring.

The goal was not to predict every possible failure in advance. It was to create enough visibility that teams could reconstruct what happened after the fact and identify the root cause.

Enterprise AI now requires a similar discipline.

But AI observability must go beyond traditional software observability. It is not enough to know what action occurred. Organizations also need visibility into why the system believed that action was appropriate.

An auditable AI system should be able to answer questions such as:

• What information did the system rely on?
• Which tools or data sources did it access?
• What alternatives did it consider?
• What verification steps were performed?
• How confident was it in its conclusion?
• What events led to the final action?

These questions are rapidly becoming essential operational requirements rather than technical nice-to-haves.

Why visibility matters more as AI gains autonomy

As AI systems become more autonomous, failures become harder to detect and diagnose.

A human reviewing a single AI-generated recommendation can often spot obvious mistakes. A network of AI agents coordinating multiple tasks across business processes presents a different challenge. Decisions can build upon one another. A flawed assumption early in a workflow can propagate through subsequent actions, creating confident but incorrect outcomes.

The challenge is rarely identifying that something went wrong. Eventually, an error surfaces through a customer complaint, a failed transaction, an audit finding, or an operational disruption.

The challenge is determining why it happened.

Which information influenced the decision? Which tools were consulted? Which safeguards worked as intended? Which ones failed?

Without visibility into the reasoning process, troubleshooting autonomous AI workflows can become significantly more difficult than debugging traditional software systems.

For CIOs responsible for enterprise reliability, compliance, and governance, that lack of visibility creates unacceptable operational risk.

Moving toward glass-box AI

The answer is not to slow AI adoption. The answer is to make AI systems observable.

Increasingly, organizations are seeking AI systems that behave more like a glass box than a black box. The objective is not to expose every parameter inside a neural network. Rather, it is to provide a clear, auditable record of how decisions were reached and why actions were taken.

The most promising approaches share two common characteristics.

The first is verification. Instead of treating a single model’s output as ground truth, systems incorporate independent validation steps before actions are executed. Multiple agents, external checks, business rules, or verification workflows help identify errors before they become operational incidents.

The second is explainability. Effective systems maintain a decision trail that captures inputs, intermediate reasoning steps, tool usage, verification activities, and outputs in a form that human reviewers can understand.

Together, these capabilities create something that has long been expected of human decision-makers but is often missing from AI systems: the ability to show your work.

The regulatory and business reality

The push toward AI observability is not being driven solely by technologists.

Regulators increasingly expect organizations to demonstrate oversight of automated decision-making systems. Emerging AI governance frameworks place growing emphasis on transparency, traceability, accountability, and human oversight.

Customers are moving in the same direction. Whether the decision involves pricing, service, eligibility, or support, people increasingly want the ability to understand and challenge outcomes that affect them.

The result is a convergence of operational, regulatory, and market pressures around a single requirement: organizations must be able to explain what their AI systems are doing.

Three questions every CIO should ask

Before deploying autonomous AI systems, technology leaders should be able to answer three basic questions:

1. Can we reconstruct the complete decision path that led to an action?
2. Can we verify critical outputs before actions are executed?
3. Can a human auditor understand why the decision occurred?

If the answer to any of those questions is no, the organization may be granting more authority to AI than it can responsibly govern.

Accountability will become a competitive advantage

The organizations that succeed with autonomous AI will not necessarily be those that automate the most processes or deploy the largest models. They will be the organizations that combine automation with accountability.

Black-box systems made sense when AI primarily generated predictions. As AI increasingly acts on behalf of businesses, customers, and employees, visibility becomes essential.

The future of enterprise AI will belong not to systems that merely act, but to systems whose actions can be examined, understood, and trusted.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.