For years, the cloud market has made a simple promise: Move workloads to large-scale platforms, gain better resilience, and worry less about downtime. That promise was never entirely wrong, but it is becoming less complete. The latest findings from Uptime Institute’s seventh Annual Outage Analysis suggest that the outage landscape is changing in ways that should concern both cloud providers and cloud customers. The biggest risks are no longer limited to broken physical infrastructure. They are increasingly tied to the complexity of the systems used to run, coordinate, update, and recover that infrastructure.

The most alarming number in the report is that IT and networking issues accounted for 23% of impactful outages in 2024. Uptime Institute links these increases to growing IT and network complexity; the long-term shift toward colocation, cloud, and third-party digital services; and the resulting increase in change-management failures and misconfigurations. That number is more than a statistical footnote. It points to a structural change in how outages happen and why cloud outages are becoming such a stubborn problem.

Hardware redundancy can protect against component failures, but it doesn’t help much when the outage stems from a bad configuration, an automation error, a faulty network change, or an underappreciated control-plane dependency. In those cases, the infrastructure itself may remain intact while the system that governs it breaks down. The industry is learning that resiliency is less about duplicating equipment and more about managing complexity. Today’s increasingly distributed and software-defined environments cannot operate safely at scale.

Failures at the operational level

Uptime’s findings show that power remains the leading cause of major outages, underscoring that traditional infrastructure engineering still matters a great deal. But even as providers continue to improve physical resilience, outages can still arise from the digital and procedural layers above it. Cloud platforms are now dense stacks of services, APIs, orchestration systems, software-defined networks, identity controls, failover logic, and third-party dependencies. That complexity creates more possible points of interaction and more opportunities for an error in one layer to cascade into several others.

This helps explain why outages can feel more surprising today than they did a decade ago. In older data center models, an outage often had a more apparent root cause, such as a power event, a cooling failure, or a hardware fault. In cloud environments, the trigger may be a small configuration change that propagates across regions, a policy update that unintentionally blocks service communication, or a network control failure that affects seemingly unrelated services. These are not failures of raw infrastructure capacity. They are failures of complexity management.

The report’s language around change management and misconfiguration is especially important because it challenges one of the most common assumptions in the cloud market: that scale automatically produces better operational outcomes. The reality? Scale can magnify both strengths and weaknesses. Large cloud providers have more engineering talent, more sophisticated tools, and more redundancy than almost any enterprise customer. But they also run far more interconnected systems at far greater speeds with far more automation. A single process failure can have a wider blast radius.

Another important lesson from the Uptime analysis is that automation has not removed the human factor. If anything, it has changed its form. Even in highly automated environments, human error remains central to the problem. The report notes that in 2025, the share of outages caused by human failure to follow procedures rose by 10 percentage points compared with 2024. A related industry summary of the report notes that 58% of human error-related outages were caused by staff failing to follow established procedures.

That matters because cloud providers often position automation as the answer to reliability. Automation is essential, but it only works as well as the operational model that surrounds it. If teams deploy changes too quickly, rollback paths are weak, approval chains are bypassed, or procedures are incomplete, automation can accelerate failure rather than prevent it. In a modern cloud environment, a human mistake is rarely just a single keystroke. It is more often a design weakness in process, governance, testing, or accountability.

This is also why customers should resist the comforting notion that outages are somebody else’s problem once workloads move to the cloud. Provider-side mistakes remain real, but customer architectures are increasingly entangled with provider networking, identity, observability, and platform services. When an outage occurs, the customer may not have caused it, but they still bear the business impact. The shared responsibility model does not end with security. It extends to resilience planning as well.

Better change management

The Uptime data points to a clear conclusion: Cloud providers need to treat operational discipline as a first-class design requirement. That starts with better change management. High-risk changes should be tested more aggressively, staged more gradually, and accompanied by stronger rollback mechanisms. Providers also need better dependency mapping to understand how a change in one control layer can affect services far beyond its immediate scope. If the system is too complex to clearly explain, it is too complex to operate.

Providers also need to improve procedural quality. The rise in outages caused by failing to follow procedures suggests that procedures are being ignored under operational pressure or that they are too cumbersome, outdated, or unclear for real production conditions. Neither explanation is comforting. Stronger runbooks, better training, more realistic failure drills, and tighter operational guardrails are not glamorous investments; they are increasingly central to resilience.

Another pressure point is visibility. Uptime notes that software-based and distributed resiliency tools can improve availability, but they also introduce new risks and complicate root-cause analysis. Cloud providers need more transparent and faster incident diagnosis, not just more layers of abstraction. Customers cannot build trust in resilience if every major incident becomes a long exercise in reconstructing opaque service dependencies after the fact.

Design with outages in mind

What’s the financial impact of more frequent problems? Uptime’s 2024 analysis found that 54% of respondents reported that their most recent significant outage cost more than $100,000, and 20% said it cost more than $1 million. These are not edge-case losses. They show that outages remain costly even if they are less frequent than in earlier years.

Customers need to stop evaluating cloud resilience through uptime promises and start evaluating it through failure behavior. How does a provider isolate faults? How transparent is incident communication? How portable are workloads if a major service degrades? How dependent is the architecture on a single region, network path, identity service, or control plane? These are not just technical questions; they are now critical business questions.

The core lesson from Uptime’s data is simple. Outages are becoming a bigger problem for cloud providers and customers because the cloud’s biggest vulnerabilities are increasingly tied to complexity, process failures, and control-plane mistakes, not just broken infrastructure. In addition to adding redundancy, the next phase of cloud improvement will focus on building systems that are easier to understand, safer to change, and more disciplined to operate.

Microsoft has open-sourced an AI evaluation framework that converts natural-language requirements into executable tests, expanding its push into enterprise AI governance as organizations struggle to validate agent behavior before production deployments systematically.

The framework, called ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), generates evaluation scenarios, datasets, metrics, and scorecards from written specifications, product requirements, and governance documents, Microsoft said in a blog post announcing the release.

“Agents fail in ways that are hard to see,” Microsoft wrote in the blog post. “They drift from policy, produce unsafe outputs in edge cases, and behave differently in production than they did in testing. Generic benchmarks do not catch these failures because they are not built around your policies, your agent, or your use case.”

Rather than requiring developers to manually create evaluation suites, ASSERT translates written intent into reusable tests that can be integrated into AI development pipelines, the company said in the blog post.

With ASSERT, Microsoft is entering an increasingly competitive AI evaluation market that already includes platforms such as LangChain’s LangSmith, Braintrust, Patronus AI, Galileo, Arize AI’s Phoenix, and Promptfoo, which help enterprises benchmark, monitor, and validate large language model applications.

Behavioral testing remains immature

The release comes as enterprises rapidly expand AI agent deployments while formal evaluation practices remain the exception rather than the rule.

“Most organizations, in fact, 99% of them, do not evaluate any AI agents pre-production,” said Anushree Verma, senior director analyst at Gartner.

According to Verma, the industry’s next competitive advantage will depend less on advances in reasoning models than on how effectively organizations simulate and stress-test AI agents before deployment.

“The next competitive moat in agentic AI is not about the sophistication of reasoning models or the underlying architecture,” she said. “It will be about the depth and realism of the training environment through agentic simulation, particularly for mission-critical deployments.”

Gartner estimates that by 2029, more than 75% of domain-specific agents designed without agentic simulation in regulated industries will fail to deliver value.

Forrester sees enterprises moving toward behavioral evaluation but says most organizations have yet to make it a formal production requirement.

“Most enterprises are still in an intermediate stage where behavioral evaluation is inconsistently applied rather than treated as a formal production gate,” said Biswajeet Mahapatra, principal analyst at Forrester.

According to Forrester data, more than 45% of organizations are already using AI agents, and another 25% are piloting them, yet many continue to struggle with scaling because of immature governance and limited operational rigor.

“The net is that behavioral evaluation is becoming important, but for most organizations it is still ad hoc or tool-driven rather than a standardized release gate enforced across the lifecycle,” Mahapatra said.

AI judges still need human oversight

Microsoft said ASSERT uses large language models as judges, with model-generated evaluations agreeing with human reviewers 80% to 90% of the time in the company’s internal validation.

That level of agreement can help automate large portions of AI testing, but should not be treated as a standalone governance mechanism, Mahapatra said.

“An 80% to 90% agreement rate with human reviewers indicates strong alignment but is not sufficient as a standalone control for governance or compliance,” he said.

Instead, enterprises should adopt layered oversight where AI evaluates AI at scale while humans retain supervisory accountability for high-risk, regulated, or ambiguous scenarios. Buyers should also watch for bias, consistency issues, and overreliance on a single model acting as both generator and evaluator, he added.

Open source reduces lock-in, not governance risk

Microsoft released ASSERT under the MIT open-source license, allowing organizations to inspect, modify, and integrate the framework into existing AI development workflows.

But open sourcing a framework does not eliminate questions around evaluation neutrality, Mahapatra said.

“Open sourcing under an MIT license reduces lock-in concerns and enables broad interoperability across model ecosystems,” he said. “However, it does not fully eliminate trust or conflict-of-interest questions because the originating vendor still influences how evaluation criteria, scoring logic, and definitions of acceptable behaviour are encoded.”

Instead of relying on a single evaluation framework, enterprises should validate AI systems against multiple evaluation approaches and retain ownership of internal evaluation policies, he said.

Too many enterprises remain ignorant of the European Union’s 2024 Cyber Resilience Act, the first elements of which enter force on June 11, according to a new survey.

Two-thirds of respondents to the survey by Open Source Security Foundation said they were unfamiliar with the CRA, which aims to make hardware and software sold in the EU more secure.

As well as the CRA’s demands on vendors, it also has implications for users of open-source software, hence the Foundation’s interest in the topic. Among other measures, the CRA creates the role of open-source steward within the enterprise, with responsibility for ensuring that a security policy is in place for any software being used within the organization.

The first part of the CRA to enter force, on June 11, concerns the designation of conformity assessment bodies by member states. Then, from September 11, manufacturers will be required to begin reporting vulnerabilities in their products to the relevant authorities. The remaining obligations under the Act, which include substantial financial penalties, will apply from December 11, 2027.

The impending sanctions seem not to have concerned businesses: 56 percent of respondents to the OpenSSF survey were unaware that non-compliance fines could reach €15 million or 2.5 percent of global annual turnover.

The lack of knowledge about the implications of the Act surprised OpenSSF CTO Christopher Robinson. “We’ve been speaking on this topic for some time and we’re scratching our heads on why more companies are not aware of the implications of the Act,” he said.

Global concern

He surmised that some companies don’t think EU regulations on hardware and software security apply to them — but such concerns will soon be a global matter. “Other countries, like Japan, are considering similar laws,” he said.

One area of misunderstanding could be that the CRA applies to vendors, and their customers may think that the requirements under the Act didn’t apply to them. He said that this was a misguided approach, particularly when the CRA’s application to open-source software is taken into account.

“There are about 700 million projects in Git Hub. If you work for an organization like a bank, you have little idea which of those projects are being used,” he said.

Under the Act, software companies will have to supply a software bill of materials (SBOM) that has been passed as secure, he said.

Companies that supply US federal government organizations already face this requirement, he said: “If you’re selling to the US government — which is the largest customer on the planet – you should be providing an SBOM.”

Cybersecurity consultant Hans Study said that by addressing the supply chain issue, the CRA is a step in the right direction. “Almost every application has dependencies, whether that is free and open-source software, commercial packages, or some mix of both. The problem has always been responsibility, and the blame game that comes with it. What the CRA does is make it harder for companies to dodge that responsibility when they are building, selling, or placing products with digital elements on the market,” he said.

AI ignorance

According to Michael Callahan, VP of Cyber Strategy at Salt Security, one of the issues that could cause problems in the future is the growing use of AI in the software development process. “The Cyber Resilience Act assumes enterprises know what is in their software. That assumption breaks down when AI coding assistants are generating a significant share of code. An AI assistant has never read your organization’s security policies, your licensing obligations, or your open-source governance standards. The code it produces may contain dependencies, patterns, or vulnerabilities that your security team cannot easily trace back to a specific decision or a specific developer.”

Enterprises are quickly running out time to fix issues and many are pessimistic about their chances. According to the OpenSSF survey, only 41percent of manufacturers expect to be fully compliant by December 2027, while 39 percent do not know when they will be.

It may be that the proposed fines could concentrate minds. Robinson said that it could be like GDPR where a few heavy fines drew companies’ attention to the regulation. The upper limit on fines is per infraction, not per company, he said: “Something like that could wipe out an SME and seriously hit large corporations.” The legislation should be something that all businesses need to be aware of, but there is still a long way to go.

This article first appeared on CIO.

AIs struggle to understand documents designed for humans; the DocLang working group seeks to flip that imbalance with its specification for machine-readable business documents “built from the ground up for LLM tokenizers.”

The working group, founded by IBM, Nvidia, and Red Hat and hosted by the Linux Foundation’s LF AI & Data project, aims to create an open, universal, AI-native document format designed to improve how enterprises prepare, exchange, and govern document data for AI systems. ABBYY and Human Signal will also be involved in its development, and other contributors are welcome.

“Enterprises today work across a fragmented landscape of document formats, including PDFs, JPEGs, and other file types built primarily for human consumption rather than AI interpretation,” the group said in its launch announcement.

“This disconnect can introduce complexity, raise costs, and reduce reliability when extracting meaning from business documents,” as organizations increasingly rely on generative AI and agentic systems, it said.

Mark Collier, executive director of LF AI & Data, said the goal of the DocLang Specification Working Group is to “develop a vendor-neutral, interoperable standard that helps organizations prepare document data for AI more reliably, transparently, and at scale.”

DocLang defines a structured, machine-readable format for documents of any type, like JSON for data, that any tool can implement and any pipeline can consume. It builds on DocLing, a document processing toolkit hosted by LF AI & Data that can transform human-readable PDFs, word processor documents or spreadsheets into structured data.

Standards must evolve for AI

Something like DocLang is needed, said independent technology analyst Carmi Levy. “Existing document standards have done an admirable job allowing global stakeholders to confidently collaborate for decades, but it’s becoming increasingly clear that they are in desperate need of an update as AI reshapes the rules around how work gets done,” he explained.

Largely static document types, he said, “can be somewhat limiting when AI is redefining the very word, ‘document.’ In many ways. AI-age documents are far more iterative and dynamic than what they once were, and the definitions need to evolve with the times. The documents we currently live with simply weren’t designed for the AI age.”

Within that context, Levy said, “DocLang represents an early, best hope of achieving some kind of foundational baseline for document standards, one that will hopefully allow more intelligent, more efficient, lower-risk workflows than is currently the case.”

Taking an open-source, vendor-agnostic approach to the process ensures the collective will take precedence over the needs of specific vendors, he said, adding, “earlier standards-setting efforts around networking, documentation, the web, and the cloud powered the free-flowing digital landscape that defines modern life.”

An AI-centric documentation standard will carry that reality into the next generation of technology, said Levy.

A question of governance

The entire concept of LLMs, Jason Andersen, principal analyst at Moor Insights & Strategy said, “involves using natural human languages. The computer is supposed to understand us without us changing our syntax or language. Forcing a syntax on users is exactly what we have today with SEO and more advanced programming languages.”

With something like DocLang, where the standard can be applied to content ingestion, he said, “I would be OK with that being automated, which seems to be the intent. The use case I envision is that when I upload a document to an agent, a skill can be run to preprocess the document into the DocLang standard format, saving tokens.”

That makes sense, he said, adding that he thinks it’s good “if it can help generate outputs, like a visualization, that can be shared outside an AI tool. On that front, that is also why I am liking Web MCP, since you are just adding some code to the page, like CSS or JavaScript, and the consumer, in this case, an AI browser or skill, is better equipped to handle the site.”

The point, he said, is, “these standards need to preserve the fact that humans can still do what they want, and do not need to know any coding to be proficient. In terms of governance, I am not sure if it matters.”

But one analyst did foresee governance problems arising from DocLang’s use.

Yaz Palanichamy, senior research analyst at Info-Tech Research Group, said DocLang adoption will require organizations to implement and review controls in order to scale its use accountably and securely.

This article originally appeared on CIO.com.

Agentic AI has moved from conference hype to a budget line item. This is where the conversation gets more interesting and more uncomfortable. Unlike traditional AI systems that respond to a single prompt, classify a document, recommend an action, or generate a summary, agentic AI systems are designed to pursue goals. They plan, call tools, inspect results, retry failed steps, consult memory, hand off tasks to other agents, and sometimes critique their own work before producing an answer or taking an action.

That extra autonomy is the value proposition. It also introduces the cost problem.

A single chatbot interaction may consume a few thousand tokens. A useful agentic workflow can consume hundreds of thousands or millions of tokens per day because it does more than answer a question. It decomposes the problem, retrieves context, reasons through options, invokes APIs, checks the output, and often runs multiple passes before reaching a result. Therefore, the economics need to be understood at the level of “agent instances,” not just model calls.

For the estimates below, I am using a blended token cost of $3 dollars per million tokens. This is not intended to reflect a single vendor’s list price. It is a blended planning figure that assumes a mix of input and output tokens, reasoning steps, retrieval-augmented generation, summarization, tool calls, memory updates, and occasional use of larger context windows. Some enterprises will pay less through volume discounts or by routing work to smaller models. Others will pay more by using premium models, long-context prompts, web browsing, large document ingestion, and repeated reasoning loops.

The basic formula is straightforward. If an agent consumes 2 million tokens per day, it consumes 730 million tokens per year. At $3 per million tokens, that single agent costs about $2,190 per year in token burn. That number sounds surprisingly low until you multiply it by the number of agents, workflows, and users, plus the surrounding infrastructure required to run these systems safely.

What an agent really costs

In the model used here, the annual token-only cost per agent ranges from about $1,095 to $3,833, depending on the use case.

• A lightweight HR recruiting or onboarding agent that consumes one million tokens per day costs about $1,095 per year.
• A more demanding software engineering agent that consumes 3.5 million tokens per day costs about $3,833 per year.
• Customer support agents are roughly $2,190 per year.
• Legal contract agents cost about $2,409 per year.
• Security triage agents run about $2,738 per year.
• Research agents are around $3,066 per year.

These figures are useful but incomplete. They include only LLM token consumption and exclude orchestration platforms, vector databases, observability, model evaluation, security controls, workflow monitoring, human review, enterprise application integration, data pipelines, audit logging, prompt management, and the engineers needed to build and maintain the systems. In real deployments, I would expect the all-in operating cost to be two to five times the raw token cost. For regulated or mission-critical environments, the multiplier can be even higher.

This is where many agentic AI business cases become less clear. The model call may be inexpensive, but the system around the model is not. An agent that can update a customer relationship management system, approve a refund, generate a purchase order, or recommend a security containment action needs guardrails, permissions, logging, rollback mechanisms, and human escalation paths. These are not optional features. They are the difference between a demo and an enterprise system.

Use cases that make economic sense

Customer support is one of the more obvious use cases. A typical support automation deployment may use eight agents: an intake classifier, a knowledge retrieval agent, a response drafting agent, an escalation agent, a quality review agent, a CRM update agent, a sentiment detection agent, and an analytics agent. At two million tokens per agent per day, each agent costs about $2,190 per year in token burn, bringing the annual total to roughly $17,520. If that system deflects even a modest number of tickets or improves agent productivity, the economics can be attractive.

Sales development is another practical example. A five-agent system for account research, lead enrichment, email personalization, CRM updates, and follow-up scheduling may consume 1.2 million tokens per agent per day. That results in an annual cost of about $1,314 per agent, or $6,570 for the full agent team. This can be compelling if it improves pipeline quality, but it can also be wasteful if agents generate low-quality outreach at scale. The cost of brand damage is not measured in tokens.

Software engineering is more expensive but potentially more valuable. A 12-agent system covering requirements analysis, architecture, code generation, testing, review, security checks, documentation, CI debugging, refactoring, release notes, dependency analysis, and hot-fix support may consume 3.5 million tokens per agent per day. That works out to about $3,833 per agent annually or roughly $45,990 for the full system. Compared with engineering salaries, the token cost is small. The real question is whether the system reliably improves throughput without increasing defects, security vulnerabilities, or maintenance complexity.

Security operations also fit the agentic model because the work is repetitive, time-sensitive, and context-intensive. A 10-agent security triage system could include agents for alert triage, log analysis, threat intelligence, endpoint investigation, network investigation, incident summarization, ticketing, compliance evidence, escalation, and post-mortem. At 2.5 million tokens per agent per day, the annual token cost is about $2,738 per agent or $27,375 for the system. This is easy to justify if it reduces alert fatigue and accelerates response, but risky if the agents hallucinate causality or bury critical signals in confident summaries.

Finance, legal, healthcare administration, market research, HR, and supply chain are also viable:

• A six-agent finance close system may cost about $9,855 per year in token burn.
• A four-agent legal contract review system may cost about $9,636.
• A seven-agent healthcare administrative workflow may cost about $13,797.
• Competitive intelligence using six agents may cost $18,396.
• HR recruiting and onboarding using five agents may cost $5,475.
• Supply chain planning and exception management using eight agents may cost $21,024.

Across all 10 example use cases I’ve mentioned in this section, the model assumes 71 agents and a total annual token burn of about $175,638.

The comparison with traditional AI

The economics of agentic AI should always be compared with simpler approaches. Traditional AI, workflow automation, rule engines, robotic process automation, and non-agentic LLM calls are often cheaper, easier to govern, and more predictable. Agentic AI is usually overkill for tasks like classification, extraction, summarization, routing, or drafting within a narrow context. A deterministic workflow with a single model call can do the job at a fraction of the cost and risk.

Agentic systems make sense when the process requires judgment across multiple steps, dynamic planning, tool use, exception handling, and adaptation to incomplete information. They are valuable when the path to the answer cannot be fully scripted in advance. They are less valuable when enterprises use them as a trendy substitute for basic automation.

The best architecture is usually hybrid. Use traditional automation where the process is stable. Use non-agentic AI where the task is bounded. Use agentic AI only where autonomy creates measurable leverage. That means fewer agents, tighter scopes, explicit budgets, model routing, token monitoring, and human checkpoints for high-impact decisions.

The financial mistake many organizations will make is to treat agents as digital employees with near-zero marginal cost. They are not. They are probabilistic software components that consume tokens, trigger tools, create operational dependencies, and require supervision. The token bill may be manageable. The governance bill may not be.

Agentic AI can absolutely be worth the money. In many cases, the annual token burn for a useful agent team is less than the loaded cost of a single employee. But that is not the same as saying it is cheap. Companies must measure agent cost per completed business outcome, not cost per prompt or cost per model call. In the end, the question is not how much an agent costs. Ask yourself the right question: Does the autonomy an agent provides outweigh the complexity it introduces?

AI governance is an ongoing game of catch-up for enterprises. Model updates and iterations are rolling out at a rapid clip, often making governance frameworks obsolete before they’re battle-tested.

To evolve beyond this paradigm, OpenAI is introducing Active sessions. This new ChatGPT security feature allows users to review and log out of one or more sessions through a simple interface. The feature is now available across all ChatGPT accounts and workspace types, including personal and managed workspaces. Experts call it an important development for the model provider, which currently has 1 billion monthly active users.

Previously, organizations often had limited visibility into where users were logged in, and simply relied on password resets or broad account actions to force re-authentication, noted Ensar Seker, CISO at SOCRadar. “Granular session control is a more efficient and less disruptive approach. From a governance perspective, session transparency improves accountability and supports investigations,” he explained.

A holistic view across session activity

Active sessions allows admins to see known browser and app sessions across ChatGPT, Codex, and API Platform. Specifically, they are able to view device and browser information, approximate location, sign-in date and time, whether a device is trusted, and whether the session is current.

To access the feature, users can go to ‘Settings’ > ‘Security’ > ‘Active sessions.’ They can then log out of specific sessions and remove devices from trusted services. They also have the ability to log out of all sessions (thus ending sessions across devices), however, this action can take up to 30 minutes to complete.

However, OpenAI emphasizes that session details may be “approximate or incomplete,” and that the feature has limits. It does not show or manage connected apps or third-party app sessions, sign-ins through third-party services, Codex CLI sessions, or recently signed-out sessions.

Further, Active sessions cannot be used with accounts linked to an enterprise’s single sign-on (SSO), including security assertion markup language (SAML) and OpenID Connect (OIDC).

Better late than never

While Active sessions is an important security and governance development, experts note that the feature is basic, and was a long time coming.

“The reality of OpenAI offering the ability to end active sessions on ChatGPT by administrators is that it’s something that exists in lots of platforms,” said David Shipley of Beauceron Security. “They should’ve had it sooner, but better late than never.”

From a security standpoint, he noted, OpenAI could do a better job policing ChatGPT to prevent it being used by threat actors to host malware, which is the latest threat to enterprises.

SOCRadar’s Seker also pointed out that this type of visibility and oversight is something that enterprises have expected from SaaS platforms for years. “It allows administrators and users to quickly identify unauthorized access, terminate stale sessions, and reduce the risk of account compromise persisting undetected.”

Iterative upgrades disrupt governance

Last week, OpenAI updated GPT-5.5 Instant in both the ChatGPT app and API to “improve response style and quality,” the company said. It had rolled out GPT-5.5 Instant earlier in May as a successor to GPT‑5.3 Instant, calling it “generally smarter” and prone to fewer hallucinations.

According to OpenAI, the update makes GPT-5.5 Instant “easier to read, more natural in everyday conversations, and better paced in practical help tasks, with fewer overly long or bullet-heavy responses.”

But even with tools like Active sessions, enterprises continue to struggle with governance amidst seemingly continuous iterative model updates. It’s simply not sustainable, said Beauceron’s Shipley: “How do you build an appropriate testing plan with a nondeterministic system?”

SOCRadar’s Seker pointed out that many organizations perform security, compliance, and business validation testing before approving a model for production use. But, “when model behavior changes under the same version family, previously documented assumptions may no longer fully reflect actual performance,” he noted.

“The biggest governance challenge in AI is not model adoption, it’s model change,” Seker said. “Most organizations can evaluate a model once. Far fewer are prepared to continuously evaluate how that model evolves over time.”

This particularly creates challenges for regulated industries where auditability, repeatability, and change management are critical, he said. Even beneficial improvements can introduce governance concerns if organizations are not clearly informed about what changed and when.

Valence Howden, advisory fellow at Info-Tech Research Group, noted that organizations often can’t assess the implications of model iterations against their boundaries, and, worse, are often unaware of them.

While the biggest enterprise challenge was initially tied to which AI model was being used, what that model did, and who owned it, iterative updates can muddy those waters and increase reliance on third party practices and tools that organizations often don’t have the resources for, he noted. 

“Without the ability to opt out [of an update] before it’s incorporated, [enterprises] are basically red-teaming the updates with their clients,” said Howden.

The ongoing game

Security teams today are pushed to their limits because they are expected to manage rapidly evolving models, new features, and changing behaviors, while maintaining compliance, risk management, and business continuity, said SOCRadar’s Seker.

“Governance is difficult because organizations are no longer evaluating a static product,” he said. Rather, they are managing a “continuously evolving service” where capabilities, integrations, and user behaviors can change far faster than can traditional security review cycles.

Info-Tech’s Howden agreed, saying that enterprises’ existing governance practices, especially accountability, are poor, as are their risk practices.

“It’s hard to suddenly become good at things they’re already poor at doing,” he said. “They are also incentivized for speed and innovation, so they ignore governance as a constraint, or don’t want to do it at all.”

How enterprises should respond

Seker advised that, ultimately, organizations should treat AI models as living systems rather than fixed software releases.

Security and governance programs should include continuous validation, monitoring, and periodic re-assessment instead of sole reliance on one-time approval processes, he said. Enterprises should also establish clear vendor change management expectations, including requiring transparency around model updates, behavioral changes, and potential impacts to existing workflows.

“Effective AI governance increasingly depends on visibility into change, not just visibility into risk,” Seker said.

AI could soon lead to systems capable of improving their own performance faster than humans can effectively supervise them, reviving concerns about the industry’s longstanding “alignment problem,” ensuring AI systems reliably pursue human goals, senior Anthropic researchers have warned in a new blog post titled “When AI builds itself.”

Anthropic Institute lead Marina Favaro and Anthropic co-founder Jack Clark outlined three possible futures: growth in AI capabilities may flatten out; AI efficiency gains may continue to grow, but expose bottlenecks elsewhere in software development; or AI systems may become capable of full recursive self-improvement, and build their successors by themselves. It’s that third scenario that’s prompting them to suggest society be ready to hit the brakes on AI development.

“How the alignment problem gets solved — or not — in this future is something we are least certain about,” they wrote. Advanced, self-improving models could follow our needs and wants — or, they warned, “The rare occurrences of misalignment present in today’s models could compound as the models build their successors, growing more frequent but less understood until we lose control of them. It’s possible that we can’t build, integrate, and verify the tools that we’d need to understand which trendline we are actually on.”

While Anthropic’s warning is framed around future AI development, analysts say it highlights governance questions enterprises are already beginning to confront as autonomous AI agents move from answering questions to taking actions.

“The issue is no longer just whether AI gives the right answer, but whether autonomous systems take the right action, at the right time, within the right authority,” said Ashish Banerjee, senior principal analyst at Gartner.

From model governance to agent governance

The warning comes amid growing enterprise investment in agentic AI.

Gartner predicts that by 2028, 15% of day-to-day work decisions will be made autonomously through agentic AI and that one-third of enterprise software applications will incorporate agentic AI capabilities. The firm has also warned that governance shortcomings are already emerging, predicting that 40% of enterprises will demote or decommission autonomous AI agents by 2027 after governance failures become apparent in production environments.

Banerjee said many organizations continue to approach AI agents as advanced productivity tools when they increasingly resemble digital workers operating with delegated authority.

“CIOs should stop treating AI agents as smarter chatbots,” he said. “They are becoming digital workers with delegated authority — and must be governed like privileged users, not productivity tools.”

As agents gain the ability to conduct research, write code, invoke tools, trigger workflows, and make recommendations, enterprises face new risks around unauthorized actions, accountability gaps, data exposure, tool misuse, and insufficient auditability, Banerjee said.

“Human-in-the-loop is not a strategy if the human cannot keep up with the loop,” he said.

Charlie Dai, vice president and principal analyst at Forrester, said Anthropic’s concerns mirror challenges enterprises are already encountering as AI systems gain greater autonomy.

“Alignment becomes operational,” Dai said. “It is about ensuring agents consistently act within policy, not just model accuracy.”

Current governance approaches focus largely on models and data, but increasingly autonomous agents require oversight of runtime behavior, permissions, tool usage, and decision boundaries, Dai said.

Concerns about agent oversight are not limited to AI vendors and industry analysts.

In AI Agent Governance: A Field Guide, researchers from Institute for AI Policy and Strategy warned that “society is largely unprepared for this development” and said “the exploration of agent governance questions and the development of associated interventions remain in their infancy.” The paper argues that advances in autonomous AI agents are outpacing the governance mechanisms needed to oversee them.

Both analysts argued that governance frameworks originally designed for generative AI models may prove insufficient for increasingly autonomous systems. Dai said organizations will need greater oversight of runtime behavior, permissions, tool usage, and decision boundaries as agents become more capable.

Why Anthropic is worried

Anthropic’s researchers argue that those governance questions could become significantly harder if AI systems become increasingly involved in the process of AI research and development itself.

Favaro and Clark stopped short of predicting that fully autonomous recursive self-improvement is inevitable. Instead, they argued that the possibility warrants preparation and discussion among developers, policymakers, and other stakeholders. They also suggested the industry may eventually need mechanisms to slow development if capabilities begin advancing faster than safeguards, while acknowledging that such measures carry risks of their own.

“But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe,” they wrote in the blog post.

Forrester’s Dai said the practical implication for enterprises is that governance can no longer depend primarily on human review.

“Supervision becomes architectural, not manual,” he said. Organizations will increasingly need bounded autonomy, embedded guardrails, verifiable execution mechanisms, and fallback controls designed into agentic systems from the outset.

Over the next three to five years, both governments and the private sector will need to rapidly adapt identification and mitigation protocols as adversaries move from AI-assisted to AI-enabled sanctions evasion and proliferation financing (PF), a new research paper warns.

The report, Algorithms of Evasion: The Rise of AI-Enabled Proliferation Financing, from the Royal United Services Institute (RUSI), a UK-based defense and security think tank, defines PF as the use of funds or financial services to acquire, develop or otherwise deal in weapons of mass destruction (WMD). It states, “North Korea and Iran are now developing and deploying AI models to aid with sanctions evasion activities.”

Key findings include the fact that AI is now capable of mass producing high-quality fraudulent documents, as well as automating what the report describes as “the administrative minutia of managing extensive shell company  networks.” AI powered systems, it states, can also “analyze blockchain patterns in real time to dynamically adjust cryptocurrency mixing strategies, effectively evading detection tools.”

In addition, it says, “[tools such as generative AI] which can produce sophisticated fraudulent identification documents, for example, have helped North Korea perpetrate phishing attacks against Western companies.”

Dr. Aaron Arnold, senior associate fellow with the Centre for Finance and Security at RUSI, who authored the paper, said in an email that what prompted it was an uptick over the last year in North Korea’s use of AI to facilitate and enhance its cyber operations, in the form of phishing schemes designed to generate revenue for the country’s ballistic missile and nuclear weapons programs.

He advised enterprise IT managers who need to protect their organizations from becoming victims of sanction evasion activities that “[it] means largely adapting to a landscape where traditional human-focused security boundaries are being bypassed by automated technologies.”

For IT managers, said Arnold, “this might entail incorporating defensive AI, the use of behavior-based analytics, using ‘circuit breakers’ when there is heavy use of API or MCPs, updating personnel training, and hardening identity verification, especially for any remote hiring.” 

Distinction between AI-assisted and AI-enabled activity is ‘central’

Sanchit Vir Gogia, chief analyst at Greyhound Research, said that the RUSI report matters “because it names the right structural shift. AI is not creating sanctions evasion from thin air, it is compressing and scaling methods that already work.”

He pointed out that none of the sanction-evading techniques such as fraudulent documents, synthetic identities, shell companies, hidden beneficial ownership, crypto laundering, and others are new. “What changes is the speed, quality, volume and coordination with which these methods can now be assembled,” he said.

According to Gogia, “the distinction between AI-assisted and AI-enabled activity is central. AI-assisted evasion uses AI for discrete tasks: writing a better email, producing a cleaner document, generating a stronger false profile, translating a pitch, summarizing regulations or preparing a plausible job application. AI-enabled evasion is more serious.”

A ‘structural asymmetry’

This tactic, he said, “begins to coordinate the system itself. It links identity, documents, ownership structures, payment routes, cloud access, crypto wallets, API calls and timing. The difference is not whether AI helps someone fake a document. The difference is whether AI begins to orchestrate the deception.”

That is why the report’s findings should worry enterprise leaders, he noted: “Many organizations still assume the bad actor is mostly human, mostly linear and mostly slow. That assumption is expiring. AI lets adversaries run more attempts, with fewer errors, across more channels, in more languages, with better paperwork and greater patience than most enterprise review processes can absorb. This is not a tale of genius criminals discovering magic. It is the story of ordinary controls meeting industrialized plausibility.”

The evidence today, he pointed out, is strongest around tactics such as identity fraud, document fraud, synthetic personas, remote-worker deception, phishing, social engineering, crypto obfuscation and workflow abuse. “Fully autonomous evasion networks sit on the horizon,” he said. “They are serious, but they are not yet the everyday baseline.”

This distinction matters, said Gogia: “If enterprises obsess over cinematic autonomous agent scenarios while leaving remote hiring, vendor onboarding, payment approvals, and document review full of holes, they will lose in the most prosaic way imaginable.”

The report, he said, also gets the “asymmetry” right. “Offensive actors can learn across the ecosystem,” he said. “They can scrape open information, reuse leaked records, study enforcement patterns, test onboarding forms, inspect public procurement data, watch court filings, probe compliance thresholds and [use the information to] refine their behavior.”

Defenders, by contrast, are hemmed in by privacy rules, fragmented data, explainability requirements, jurisdictional boundaries, conservative operating models and siloed technology estates. “Offensive AI learns broadly,” he said. “Defensive AI often learns from fragments. That is the structural asymmetry.”

He explained that the regulatory landscape also amplifies the problem, in that regulatory bodies “still speak in separate dialects. [For example] the EU AI Act pushes organizations toward stronger obligations for high-risk AI. NIST-style frameworks push risk management, transparency, and governance.”

A trust architecture problem

Financial Action Task Force (FATF) expectations push national risk assessment and counter-proliferation controls, he noted, while banking regulators focus on model risk, accountability and operational resilience. “None of these streams is irrelevant. The trouble is that criminals do not organize themselves around regulatory workstreams. They organize around outcomes.”

What that means, said Gogia, “is that enterprise cannot wait for a clean global rulebook. It will not arrive in time. CIOs, CISOs, compliance officers and boards need a working governance model now. They need privacy-preserving analytics, controlled data environments, audit trails, legal safeguards and clear model-risk accountability.”

He said that enterprise IT managers should treat the situation as a trust architecture problem rather than a narrow sanctions-screening problem. “The uncomfortable truth is that AI is not simply helping bad actors write better phishing emails or forge tidier documents,” he noted. “It is helping them manufacture legitimacy across a chain of enterprise workflows.”

Likely outcome an ‘AI arms race’

Report author Arnold also noted that there are signs that cyber criminals have discovered new AI technologies and abilities that legitimate enterprises could adopt for legitimate applications.

History, he said, “is replete with [criminals] developing novel solutions to tough problems, [which are] later adopted by law enforcement. Much of our anti-financial crime policy is effectively a response to bad actors exploiting systems or using technology in novel ways to perpetrate crimes. In this scenario, I think an ‘AI arms race’ between enforcement authorities and bad actors is the most likely outcome.”

Gogia added, “the baddies are not teaching enterprises how to invent AI. They are teaching enterprises where trust is leaking. That is the lesson worth taking seriously.”

This article originally appeared on CIO.com.