Too many enterprises remain ignorant of the European Union’s 2024 Cyber Resilience Act, the first elements of which enter force on June 11, according to a new survey.

Two-thirds of respondents to the survey by Open Source Security Foundation said they were unfamiliar with the CRA, which aims to make hardware and software sold in the EU more secure.

As well as the CRA’s demands on vendors, it also has implications for users of open-source software, hence the Foundation’s interest in the topic. Among other measures, the CRA creates the role of open-source steward within the enterprise, with responsibility for ensuring that a security policy is in place for any software being used within the organization.

The first part of the CRA to enter force, on June 11, concerns the designation of conformity assessment bodies by member states. Then, from September 11, manufacturers will be required to begin reporting vulnerabilities in their products to the relevant authorities. The remaining obligations under the Act, which include substantial financial penalties, will apply from December 11, 2027.

The impending sanctions seem not to have concerned businesses: 56 percent of respondents to the OpenSSF survey were unaware that non-compliance fines could reach €15 million or 2.5 percent of global annual turnover.

The lack of knowledge about the implications of the Act surprised OpenSSF CTO Christopher Robinson. “We’ve been speaking on this topic for some time and we’re scratching our heads on why more companies are not aware of the implications of the Act,” he said.

Global concern

He surmised that some companies don’t think EU regulations on hardware and software security apply to them — but such concerns will soon be a global matter. “Other countries, like Japan, are considering similar laws,” he said.

One area of misunderstanding could be that the CRA applies to vendors, and their customers may think that the requirements under the Act didn’t apply to them. He said that this was a misguided approach, particularly when the CRA’s application to open-source software is taken into account.

“There are about 700 million projects in Git Hub. If you work for an organization like a bank, you have little idea which of those projects are being used,” he said.

Under the Act, software companies will have to supply a software bill of materials (SBOM) that has been passed as secure, he said.

Companies that supply US federal government organizations already face this requirement, he said: “If you’re selling to the US government — which is the largest customer on the planet – you should be providing an SBOM.”

Cybersecurity consultant Hans Study said that by addressing the supply chain issue, the CRA is a step in the right direction. “Almost every application has dependencies, whether that is free and open-source software, commercial packages, or some mix of both. The problem has always been responsibility, and the blame game that comes with it. What the CRA does is make it harder for companies to dodge that responsibility when they are building, selling, or placing products with digital elements on the market,” he said.

AI ignorance

According to Michael Callahan, VP of Cyber Strategy at Salt Security, one of the issues that could cause problems in the future is the growing use of AI in the software development process. “The Cyber Resilience Act assumes enterprises know what is in their software. That assumption breaks down when AI coding assistants are generating a significant share of code. An AI assistant has never read your organization’s security policies, your licensing obligations, or your open-source governance standards. The code it produces may contain dependencies, patterns, or vulnerabilities that your security team cannot easily trace back to a specific decision or a specific developer.”

Enterprises are quickly running out time to fix issues and many are pessimistic about their chances. According to the OpenSSF survey, only 41percent of manufacturers expect to be fully compliant by December 2027, while 39 percent do not know when they will be.

It may be that the proposed fines could concentrate minds. Robinson said that it could be like GDPR where a few heavy fines drew companies’ attention to the regulation. The upper limit on fines is per infraction, not per company, he said: “Something like that could wipe out an SME and seriously hit large corporations.” The legislation should be something that all businesses need to be aware of, but there is still a long way to go.

This article first appeared on CIO.

AIs struggle to understand documents designed for humans; the DocLang working group seeks to flip that imbalance with its specification for machine-readable business documents “built from the ground up for LLM tokenizers.”

The working group, founded by IBM, Nvidia, and Red Hat and hosted by the Linux Foundation’s LF AI & Data project, aims to create an open, universal, AI-native document format designed to improve how enterprises prepare, exchange, and govern document data for AI systems. ABBYY and Human Signal will also be involved in its development, and other contributors are welcome.

“Enterprises today work across a fragmented landscape of document formats, including PDFs, JPEGs, and other file types built primarily for human consumption rather than AI interpretation,” the group said in its launch announcement.

“This disconnect can introduce complexity, raise costs, and reduce reliability when extracting meaning from business documents,” as organizations increasingly rely on generative AI and agentic systems, it said.

Mark Collier, executive director of LF AI & Data, said the goal of the DocLang Specification Working Group is to “develop a vendor-neutral, interoperable standard that helps organizations prepare document data for AI more reliably, transparently, and at scale.”

DocLang defines a structured, machine-readable format for documents of any type, like JSON for data, that any tool can implement and any pipeline can consume. It builds on DocLing, a document processing toolkit hosted by LF AI & Data that can transform human-readable PDFs, word processor documents or spreadsheets into structured data.

Standards must evolve for AI

Something like DocLang is needed, said independent technology analyst Carmi Levy. “Existing document standards have done an admirable job allowing global stakeholders to confidently collaborate for decades, but it’s becoming increasingly clear that they are in desperate need of an update as AI reshapes the rules around how work gets done,” he explained.

Largely static document types, he said, “can be somewhat limiting when AI is redefining the very word, ‘document.’ In many ways. AI-age documents are far more iterative and dynamic than what they once were, and the definitions need to evolve with the times. The documents we currently live with simply weren’t designed for the AI age.”

Within that context, Levy said, “DocLang represents an early, best hope of achieving some kind of foundational baseline for document standards, one that will hopefully allow more intelligent, more efficient, lower-risk workflows than is currently the case.”

Taking an open-source, vendor-agnostic approach to the process ensures the collective will take precedence over the needs of specific vendors, he said, adding, “earlier standards-setting efforts around networking, documentation, the web, and the cloud powered the free-flowing digital landscape that defines modern life.”

An AI-centric documentation standard will carry that reality into the next generation of technology, said Levy.

A question of governance

The entire concept of LLMs, Jason Andersen, principal analyst at Moor Insights & Strategy said, “involves using natural human languages. The computer is supposed to understand us without us changing our syntax or language. Forcing a syntax on users is exactly what we have today with SEO and more advanced programming languages.”

With something like DocLang, where the standard can be applied to content ingestion, he said, “I would be OK with that being automated, which seems to be the intent. The use case I envision is that when I upload a document to an agent, a skill can be run to preprocess the document into the DocLang standard format, saving tokens.”

That makes sense, he said, adding that he thinks it’s good “if it can help generate outputs, like a visualization, that can be shared outside an AI tool. On that front, that is also why I am liking Web MCP, since you are just adding some code to the page, like CSS or JavaScript, and the consumer, in this case, an AI browser or skill, is better equipped to handle the site.”

The point, he said, is, “these standards need to preserve the fact that humans can still do what they want, and do not need to know any coding to be proficient. In terms of governance, I am not sure if it matters.”

But one analyst did foresee governance problems arising from DocLang’s use.

Yaz Palanichamy, senior research analyst at Info-Tech Research Group, said DocLang adoption will require organizations to implement and review controls in order to scale its use accountably and securely.

This article originally appeared on CIO.com.

Enterprises are moving aggressively into generative AI. On the surface, that seems like the right call. The technology is powerful, accessible, and increasingly embedded in how businesses build applications, automate processes, and support decision-making. A development team can connect an application to a large language model in days. A product team can add AI features in weeks. Business leaders see quick wins, faster innovation, and a path to modernizing nearly every part of the company.

These are the upsides everyone is talking about. The part we don’t discuss enough is the economic trap forming underneath all this convenience.

Most enterprises think of tokens as a technical billing detail. They are not. Tokens are the unit of economic dependency in generative AI. Every prompt, response, summarization, retrieval step, workflow action, and agent decision is measured and monetized through tokens. Tokens are not just part of the plumbing. They are the tollbooth between your enterprise and a provider’s intelligence platform. The more AI becomes central to your operations, the more power that tollbooth holds over your future costs.

Tokens are not just a pricing unit

A token is usually described as a chunk of text processed by a model. That is accurate enough for developers, but it misses the bigger issue for CIOs, architects, and corporate boards. In the enterprise, tokens are the mechanism by which AI capabilities are rented. They are the meter attached to the intelligence itself.

That distinction matters because token usage grows faster than most companies anticipate. A simple user prompt rarely remains simple in production systems. It can trigger retrieval from internal knowledge stores, multiple model calls, tool use, post-processing, policy checks, and agent loops. What appears to be a single transaction to the user may involve several layers of token consumption behind the scenes. As a result, enterprises often underestimate the true operating cost of AI-enabled systems, especially as those systems mature and spread across departments.

Today, those costs still feel manageable. In many cases, they feel surprisingly low. That is exactly why the trap is so dangerous.

The market is in a subsidy phase

Current token pricing is giving enterprises a false sense of comfort. Many remote LLM providers are aggressively competing for market share. They want developers building on their APIs. They want enterprise applications tightly coupled to their platforms. They want AI agents, copilots, workflows, and customer experiences to depend on their models. To make that happen, pricing remains highly attractive relative to the value delivered.

That does not mean the economics of generative AI are stable. It means the market is still being shaped by investor capital, strategic pricing, and growth expectations. Providers are racing to establish position, and enterprises are benefiting from that race. But no market stays in that phase forever. At some point, investors will expect durable profitability. At some point, weaker providers will disappear, consolidate, or retreat. At some point, the survivors will have more leverage and much less reason to price primarily for adoption.

That’s when the token trap closes.

Enterprises that build deep dependence on remote models during the subsidy phase may find that what seemed inexpensive at pilot scale becomes punishing at enterprise scale. The application that costs $1,000 per month today may cost 10 or 20 times that amount a few years from now, not only because usage has increased, but also because the market has repriced the dependency.

Easy to adopt, expensive to exit

Cloud computing followed a similar path, with many enterprises mistaking short-term convenience for long-term economics. In the early years, the case was compelling and largely accurate. Move faster, reduce friction, avoid capital spending, and scale with ease. Those benefits were real. Many organizations made architectural decisions that prioritized speed over leverage. They became dependent on managed services, provider-specific tools, and operating models that were easy to adopt but expensive to unwind.

Years later, many enterprises discovered that their cloud bills were much higher than expected and their exit options much narrower than advertised. That was not because the cloud failed. Architectural dependency eventually became financial dependency.

Generative AI is repeating that pattern, only faster. The integration barrier is lower, the pressure to adopt is higher, and the pace of enterprise experimentation is far greater. As a result, companies are wiring remote LLMs into applications, workflows, and agentic systems with very little thought about how these costs will behave in the next five to 10 years.

Agentic AI makes things worse

The more enterprises move from simple prompt-response systems to agentic architectures, the more dangerous the token trap becomes. Agents are not single-call systems. They plan, deliberate, retrieve information, invoke tools, evaluate results, retry steps, and often coordinate with other agents. Each of those actions consumes tokens. Costs no longer rise in a neat linear fashion. They compound.

This matters because agentic AI is increasingly being presented as the future of enterprise automation. It’s true in many cases. But if an enterprise builds agentic systems primarily on remotely hosted intelligence, it is also building future business processes on top of someone else’s pricing model. That is a major strategic risk. The more successful those systems become, the harder they are to replace. The harder they are to replace, the more pricing power shifts to the provider.

This is how businesses end up operationally dependent on a cost structure they do not control.

The appeal of AI sovereignty

The answer is not to reject public models or pretend that external providers play no role. They clearly do. There will always be cases where renting frontier AI capabilities makes sense. But enterprises need to stop assuming that renting is the default for every workload.

AI sovereignty is the alternative that deserves much more attention. That means building, tuning, deploying, and governing models inside the enterprise for use cases where long-term control matters more than access to the absolute frontier. Enterprises need to recognize that most business applications do not need a world-class general-purpose model. They need a model that is good enough for a specific purpose, aligned to the enterprise’s data, governed by the enterprise’s rules, and operated at a predictable cost.

It’s a very different way of thinking.

A self-hosted or enterprise-controlled model may not match the rich feature set of the largest public offerings. It may lack the same breadth, polish, or marketing appeal. But for many internal business tasks, those factors do not matter.

Here’s the most critical question to guide your architectural direction: Can a sovereign AI model solve the problem reliably, securely, and economically over time? If the answer is yes, owning that capability may be far more strategic than forever renting something with more power than you need. In effect, the enterprise becomes its own provider for the workloads that matter most.

Prepare for changing markets

Too many companies still treat generative AI architecture as a tactical IT issue. It is not. These decisions directly affect cost structure, operating flexibility, data control, and long-term competitiveness. If AI becomes a force multiplier across the business, the economics of AI become strategic to the business itself.

The companies that get this right will not necessarily be the fastest adopters. They will understand the difference between experimentation and dependency. They will use external models when it makes sense, but they will also invest in sovereign capabilities where ownership matters. They will think like architects, not consumers.

Here’s the takeaway: Cheap tokens come with strings. They are a gateway to a dependency model that will typically look very different once providers stop pricing for growth and start pricing for leverage. Enterprises cannot keep mistaking today’s bargain for tomorrow’s reality. Boards and executive teams need to act now to get ahead of this issue. The key question is not whether generative AI creates value. It clearly does. The real question is whether the enterprise can still afford and control the value it creates once the market matures.

Agentic AI has moved from conference hype to a budget line item. This is where the conversation gets more interesting and more uncomfortable. Unlike traditional AI systems that respond to a single prompt, classify a document, recommend an action, or generate a summary, agentic AI systems are designed to pursue goals. They plan, call tools, inspect results, retry failed steps, consult memory, hand off tasks to other agents, and sometimes critique their own work before producing an answer or taking an action.

That extra autonomy is the value proposition. It also introduces the cost problem.

A single chatbot interaction may consume a few thousand tokens. A useful agentic workflow can consume hundreds of thousands or millions of tokens per day because it does more than answer a question. It decomposes the problem, retrieves context, reasons through options, invokes APIs, checks the output, and often runs multiple passes before reaching a result. Therefore, the economics need to be understood at the level of “agent instances,” not just model calls.

For the estimates below, I am using a blended token cost of $3 dollars per million tokens. This is not intended to reflect a single vendor’s list price. It is a blended planning figure that assumes a mix of input and output tokens, reasoning steps, retrieval-augmented generation, summarization, tool calls, memory updates, and occasional use of larger context windows. Some enterprises will pay less through volume discounts or by routing work to smaller models. Others will pay more by using premium models, long-context prompts, web browsing, large document ingestion, and repeated reasoning loops.

The basic formula is straightforward. If an agent consumes 2 million tokens per day, it consumes 730 million tokens per year. At $3 per million tokens, that single agent costs about $2,190 per year in token burn. That number sounds surprisingly low until you multiply it by the number of agents, workflows, and users, plus the surrounding infrastructure required to run these systems safely.

What an agent really costs

In the model used here, the annual token-only cost per agent ranges from about $1,095 to $3,833, depending on the use case.

• A lightweight HR recruiting or onboarding agent that consumes one million tokens per day costs about $1,095 per year.
• A more demanding software engineering agent that consumes 3.5 million tokens per day costs about $3,833 per year.
• Customer support agents are roughly $2,190 per year.
• Legal contract agents cost about $2,409 per year.
• Security triage agents run about $2,738 per year.
• Research agents are around $3,066 per year.

These figures are useful but incomplete. They include only LLM token consumption and exclude orchestration platforms, vector databases, observability, model evaluation, security controls, workflow monitoring, human review, enterprise application integration, data pipelines, audit logging, prompt management, and the engineers needed to build and maintain the systems. In real deployments, I would expect the all-in operating cost to be two to five times the raw token cost. For regulated or mission-critical environments, the multiplier can be even higher.

This is where many agentic AI business cases become less clear. The model call may be inexpensive, but the system around the model is not. An agent that can update a customer relationship management system, approve a refund, generate a purchase order, or recommend a security containment action needs guardrails, permissions, logging, rollback mechanisms, and human escalation paths. These are not optional features. They are the difference between a demo and an enterprise system.

Use cases that make economic sense

Customer support is one of the more obvious use cases. A typical support automation deployment may use eight agents: an intake classifier, a knowledge retrieval agent, a response drafting agent, an escalation agent, a quality review agent, a CRM update agent, a sentiment detection agent, and an analytics agent. At two million tokens per agent per day, each agent costs about $2,190 per year in token burn, bringing the annual total to roughly $17,520. If that system deflects even a modest number of tickets or improves agent productivity, the economics can be attractive.

Sales development is another practical example. A five-agent system for account research, lead enrichment, email personalization, CRM updates, and follow-up scheduling may consume 1.2 million tokens per agent per day. That results in an annual cost of about $1,314 per agent, or $6,570 for the full agent team. This can be compelling if it improves pipeline quality, but it can also be wasteful if agents generate low-quality outreach at scale. The cost of brand damage is not measured in tokens.

Software engineering is more expensive but potentially more valuable. A 12-agent system covering requirements analysis, architecture, code generation, testing, review, security checks, documentation, CI debugging, refactoring, release notes, dependency analysis, and hot-fix support may consume 3.5 million tokens per agent per day. That works out to about $3,833 per agent annually or roughly $45,990 for the full system. Compared with engineering salaries, the token cost is small. The real question is whether the system reliably improves throughput without increasing defects, security vulnerabilities, or maintenance complexity.

Security operations also fit the agentic model because the work is repetitive, time-sensitive, and context-intensive. A 10-agent security triage system could include agents for alert triage, log analysis, threat intelligence, endpoint investigation, network investigation, incident summarization, ticketing, compliance evidence, escalation, and post-mortem. At 2.5 million tokens per agent per day, the annual token cost is about $2,738 per agent or $27,375 for the system. This is easy to justify if it reduces alert fatigue and accelerates response, but risky if the agents hallucinate causality or bury critical signals in confident summaries.

Finance, legal, healthcare administration, market research, HR, and supply chain are also viable:

• A six-agent finance close system may cost about $9,855 per year in token burn.
• A four-agent legal contract review system may cost about $9,636.
• A seven-agent healthcare administrative workflow may cost about $13,797.
• Competitive intelligence using six agents may cost $18,396.
• HR recruiting and onboarding using five agents may cost $5,475.
• Supply chain planning and exception management using eight agents may cost $21,024.

Across all 10 example use cases I’ve mentioned in this section, the model assumes 71 agents and a total annual token burn of about $175,638.

The comparison with traditional AI

The economics of agentic AI should always be compared with simpler approaches. Traditional AI, workflow automation, rule engines, robotic process automation, and non-agentic LLM calls are often cheaper, easier to govern, and more predictable. Agentic AI is usually overkill for tasks like classification, extraction, summarization, routing, or drafting within a narrow context. A deterministic workflow with a single model call can do the job at a fraction of the cost and risk.

Agentic systems make sense when the process requires judgment across multiple steps, dynamic planning, tool use, exception handling, and adaptation to incomplete information. They are valuable when the path to the answer cannot be fully scripted in advance. They are less valuable when enterprises use them as a trendy substitute for basic automation.

The best architecture is usually hybrid. Use traditional automation where the process is stable. Use non-agentic AI where the task is bounded. Use agentic AI only where autonomy creates measurable leverage. That means fewer agents, tighter scopes, explicit budgets, model routing, token monitoring, and human checkpoints for high-impact decisions.

The financial mistake many organizations will make is to treat agents as digital employees with near-zero marginal cost. They are not. They are probabilistic software components that consume tokens, trigger tools, create operational dependencies, and require supervision. The token bill may be manageable. The governance bill may not be.

Agentic AI can absolutely be worth the money. In many cases, the annual token burn for a useful agent team is less than the loaded cost of a single employee. But that is not the same as saying it is cheap. Companies must measure agent cost per completed business outcome, not cost per prompt or cost per model call. In the end, the question is not how much an agent costs. Ask yourself the right question: Does the autonomy an agent provides outweigh the complexity it introduces?

AI governance is an ongoing game of catch-up for enterprises. Model updates and iterations are rolling out at a rapid clip, often making governance frameworks obsolete before they’re battle-tested.

To evolve beyond this paradigm, OpenAI is introducing Active sessions. This new ChatGPT security feature allows users to review and log out of one or more sessions through a simple interface. The feature is now available across all ChatGPT accounts and workspace types, including personal and managed workspaces. Experts call it an important development for the model provider, which currently has 1 billion monthly active users.

Previously, organizations often had limited visibility into where users were logged in, and simply relied on password resets or broad account actions to force re-authentication, noted Ensar Seker, CISO at SOCRadar. “Granular session control is a more efficient and less disruptive approach. From a governance perspective, session transparency improves accountability and supports investigations,” he explained.

A holistic view across session activity

Active sessions allows admins to see known browser and app sessions across ChatGPT, Codex, and API Platform. Specifically, they are able to view device and browser information, approximate location, sign-in date and time, whether a device is trusted, and whether the session is current.

To access the feature, users can go to ‘Settings’ > ‘Security’ > ‘Active sessions.’ They can then log out of specific sessions and remove devices from trusted services. They also have the ability to log out of all sessions (thus ending sessions across devices), however, this action can take up to 30 minutes to complete.

However, OpenAI emphasizes that session details may be “approximate or incomplete,” and that the feature has limits. It does not show or manage connected apps or third-party app sessions, sign-ins through third-party services, Codex CLI sessions, or recently signed-out sessions.

Further, Active sessions cannot be used with accounts linked to an enterprise’s single sign-on (SSO), including security assertion markup language (SAML) and OpenID Connect (OIDC).

Better late than never

While Active sessions is an important security and governance development, experts note that the feature is basic, and was a long time coming.

“The reality of OpenAI offering the ability to end active sessions on ChatGPT by administrators is that it’s something that exists in lots of platforms,” said David Shipley of Beauceron Security. “They should’ve had it sooner, but better late than never.”

From a security standpoint, he noted, OpenAI could do a better job policing ChatGPT to prevent it being used by threat actors to host malware, which is the latest threat to enterprises.

SOCRadar’s Seker also pointed out that this type of visibility and oversight is something that enterprises have expected from SaaS platforms for years. “It allows administrators and users to quickly identify unauthorized access, terminate stale sessions, and reduce the risk of account compromise persisting undetected.”

Iterative upgrades disrupt governance

Last week, OpenAI updated GPT-5.5 Instant in both the ChatGPT app and API to “improve response style and quality,” the company said. It had rolled out GPT-5.5 Instant earlier in May as a successor to GPT‑5.3 Instant, calling it “generally smarter” and prone to fewer hallucinations.

According to OpenAI, the update makes GPT-5.5 Instant “easier to read, more natural in everyday conversations, and better paced in practical help tasks, with fewer overly long or bullet-heavy responses.”

But even with tools like Active sessions, enterprises continue to struggle with governance amidst seemingly continuous iterative model updates. It’s simply not sustainable, said Beauceron’s Shipley: “How do you build an appropriate testing plan with a nondeterministic system?”

SOCRadar’s Seker pointed out that many organizations perform security, compliance, and business validation testing before approving a model for production use. But, “when model behavior changes under the same version family, previously documented assumptions may no longer fully reflect actual performance,” he noted.

“The biggest governance challenge in AI is not model adoption, it’s model change,” Seker said. “Most organizations can evaluate a model once. Far fewer are prepared to continuously evaluate how that model evolves over time.”

This particularly creates challenges for regulated industries where auditability, repeatability, and change management are critical, he said. Even beneficial improvements can introduce governance concerns if organizations are not clearly informed about what changed and when.

Valence Howden, advisory fellow at Info-Tech Research Group, noted that organizations often can’t assess the implications of model iterations against their boundaries, and, worse, are often unaware of them.

While the biggest enterprise challenge was initially tied to which AI model was being used, what that model did, and who owned it, iterative updates can muddy those waters and increase reliance on third party practices and tools that organizations often don’t have the resources for, he noted. 

“Without the ability to opt out [of an update] before it’s incorporated, [enterprises] are basically red-teaming the updates with their clients,” said Howden.

The ongoing game

Security teams today are pushed to their limits because they are expected to manage rapidly evolving models, new features, and changing behaviors, while maintaining compliance, risk management, and business continuity, said SOCRadar’s Seker.

“Governance is difficult because organizations are no longer evaluating a static product,” he said. Rather, they are managing a “continuously evolving service” where capabilities, integrations, and user behaviors can change far faster than can traditional security review cycles.

Info-Tech’s Howden agreed, saying that enterprises’ existing governance practices, especially accountability, are poor, as are their risk practices.

“It’s hard to suddenly become good at things they’re already poor at doing,” he said. “They are also incentivized for speed and innovation, so they ignore governance as a constraint, or don’t want to do it at all.”

How enterprises should respond

Seker advised that, ultimately, organizations should treat AI models as living systems rather than fixed software releases.

Security and governance programs should include continuous validation, monitoring, and periodic re-assessment instead of sole reliance on one-time approval processes, he said. Enterprises should also establish clear vendor change management expectations, including requiring transparency around model updates, behavioral changes, and potential impacts to existing workflows.

“Effective AI governance increasingly depends on visibility into change, not just visibility into risk,” Seker said.

Cloud computing has reached a crossroads. The high cost and data sensitivity of AI workloads are raising the appeal of private clouds, even as neoclouds and sovereign clouds shake up the cloud provider landscape. New cyberthreats, shifting compute requirements, and management complexity are adding to cloud complications.

Download the June 2026 issue of the Enterprise Spotlight from the editors of CIO, Computerworld, CSO, InfoWorld, and Network World, and learn how to navigate the latest cloud strategy developments.

AI could soon lead to systems capable of improving their own performance faster than humans can effectively supervise them, reviving concerns about the industry’s longstanding “alignment problem,” ensuring AI systems reliably pursue human goals, senior Anthropic researchers have warned in a new blog post titled “When AI builds itself.”

Anthropic Institute lead Marina Favaro and Anthropic co-founder Jack Clark outlined three possible futures: growth in AI capabilities may flatten out; AI efficiency gains may continue to grow, but expose bottlenecks elsewhere in software development; or AI systems may become capable of full recursive self-improvement, and build their successors by themselves. It’s that third scenario that’s prompting them to suggest society be ready to hit the brakes on AI development.

“How the alignment problem gets solved — or not — in this future is something we are least certain about,” they wrote. Advanced, self-improving models could follow our needs and wants — or, they warned, “The rare occurrences of misalignment present in today’s models could compound as the models build their successors, growing more frequent but less understood until we lose control of them. It’s possible that we can’t build, integrate, and verify the tools that we’d need to understand which trendline we are actually on.”

While Anthropic’s warning is framed around future AI development, analysts say it highlights governance questions enterprises are already beginning to confront as autonomous AI agents move from answering questions to taking actions.

“The issue is no longer just whether AI gives the right answer, but whether autonomous systems take the right action, at the right time, within the right authority,” said Ashish Banerjee, senior principal analyst at Gartner.

From model governance to agent governance

The warning comes amid growing enterprise investment in agentic AI.

Gartner predicts that by 2028, 15% of day-to-day work decisions will be made autonomously through agentic AI and that one-third of enterprise software applications will incorporate agentic AI capabilities. The firm has also warned that governance shortcomings are already emerging, predicting that 40% of enterprises will demote or decommission autonomous AI agents by 2027 after governance failures become apparent in production environments.

Banerjee said many organizations continue to approach AI agents as advanced productivity tools when they increasingly resemble digital workers operating with delegated authority.

“CIOs should stop treating AI agents as smarter chatbots,” he said. “They are becoming digital workers with delegated authority — and must be governed like privileged users, not productivity tools.”

As agents gain the ability to conduct research, write code, invoke tools, trigger workflows, and make recommendations, enterprises face new risks around unauthorized actions, accountability gaps, data exposure, tool misuse, and insufficient auditability, Banerjee said.

“Human-in-the-loop is not a strategy if the human cannot keep up with the loop,” he said.

Charlie Dai, vice president and principal analyst at Forrester, said Anthropic’s concerns mirror challenges enterprises are already encountering as AI systems gain greater autonomy.

“Alignment becomes operational,” Dai said. “It is about ensuring agents consistently act within policy, not just model accuracy.”

Current governance approaches focus largely on models and data, but increasingly autonomous agents require oversight of runtime behavior, permissions, tool usage, and decision boundaries, Dai said.

Concerns about agent oversight are not limited to AI vendors and industry analysts.

In AI Agent Governance: A Field Guide, researchers from Institute for AI Policy and Strategy warned that “society is largely unprepared for this development” and said “the exploration of agent governance questions and the development of associated interventions remain in their infancy.” The paper argues that advances in autonomous AI agents are outpacing the governance mechanisms needed to oversee them.

Both analysts argued that governance frameworks originally designed for generative AI models may prove insufficient for increasingly autonomous systems. Dai said organizations will need greater oversight of runtime behavior, permissions, tool usage, and decision boundaries as agents become more capable.

Why Anthropic is worried

Anthropic’s researchers argue that those governance questions could become significantly harder if AI systems become increasingly involved in the process of AI research and development itself.

Favaro and Clark stopped short of predicting that fully autonomous recursive self-improvement is inevitable. Instead, they argued that the possibility warrants preparation and discussion among developers, policymakers, and other stakeholders. They also suggested the industry may eventually need mechanisms to slow development if capabilities begin advancing faster than safeguards, while acknowledging that such measures carry risks of their own.

“But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe,” they wrote in the blog post.

Forrester’s Dai said the practical implication for enterprises is that governance can no longer depend primarily on human review.

“Supervision becomes architectural, not manual,” he said. Organizations will increasingly need bounded autonomy, embedded guardrails, verifiable execution mechanisms, and fallback controls designed into agentic systems from the outset.

Cloud computing has reached a crossroads. The high cost and data sensitivity of AI workloads are raising the appeal of private clouds, even as neoclouds and sovereign clouds shake up the cloud provider landscape. New cyberthreats, shifting compute requirements, and management complexity are adding to cloud complications.

Download the June 2026 issue of the Enterprise Spotlight from the editors of CIO, Computerworld, CSO, InfoWorld, and Network World, and learn how to navigate the latest cloud strategy developments.

Enterprise IT leaders have always struggled with AI pricing, especially the need to pay for AI in a way that delivers ROI. But the typical IT exec may not be right person to decide how a company uses AI — and how it tries to deliver ROI — because so many line-of-business workers and partners are now experimenting with the technology on their own.

And if IT leaders don’t have a grip on how they want to use AI over the next year or two, it’s impossible to figure out how they want to pay for it. They likely hate the current method of paying per token. And other options, such as SAP’s push to charge per AI task completed, aren’t any better. 

To use a sales analogy, IT doesn’t want to pay a lot of money for leads, because there’s no way to know if those leads will generate any revenue — let alone how much. What IT leaders want is the tech equivalent of paying commission, where they only pay when a lead converts into a paying customer. And even then, they only pay a percentage of the final sale. That guarantees ROI for the enterprise.

The problem: no AI vendor would ever go for it because that approach puts too much risk on them. 

Finding a pricing model that works for both enterprise IT and AI vendors is all but impossible as long as IT is trying to deliver ROI.

Irfan Khan, president of SAP Data & Analytics, said the problem is challenging for both sides. “Everyone is scrambling to justify their investments,” and “the day one cost is not necessarily the day one value,” he said.

The problem is one of sequence. Pricing has to be negotiated and locked in long before a project starts. But with technology as new and experimental as agentic AI, there’s almost no solid information about what benefits it will (or will not) actually deliver. 

Beyond that, generative AI (genAI) and agentic AI systems might well deliver benefits that are harder to jot down in a spreadsheet. Let’s say the CFO wants to see a sharp rise in order fulfillment. But what if AI “manages to fulfill those orders more efficiently,” Khan said. “And what are the likely ripple effects of bringing more efficiencies into the process?”

Justin Greis, CEO of consulting firm Acceligence, frames the AI pricing disconnect in terms of market economics:

“The market is trying to force-fit AI into infrastructure-era pricing models, when AI is fundamentally closer to labor augmentation and business process transformation than compute consumption,” Greis said. “The core disconnect is: Enterprise IT buyers want pricing aligned to realized business value. AI vendors want pricing aligned to resource consumption and platform utilization. Those are very different economic models. 

“Token pricing is attractive to vendors because it is measurable, scalable, and predictable. But from the enterprise perspective, tokens are almost meaningless as a business metric. Nobody on the CFO side cares how many tokens were consumed if the process improvement never materialized.”

The competing pricing strategies overwhelmingly rely on just two factors: what delivers the most profit and which is the easiest to execute. Given human nature, the latter is usually the path most often taken.

It’s like one of my favorite jokes. A guy is heading to his car when he sees a man with a flashlight intently looking at the ground right next to a streetlight pole. 

“Can I help you? Are you looking for something?” the guy asks.

“Yes, I lost my car keys.”

“Silly question, but where do you last remember having them?”

“I was standing over there in that dark alley up the street. A cat screeched and I dropped my keys.”

“Wait a second — if you lost your keys over there, why are you looking here?”

“The light’s better over here.”

The lesson: taking the easy route usually beats realizing the actual objective.

Greis argued that not only would it be hard to persuade AI vendors to accept ROI pricing, but if they did  somehow agree, the unintended results could prove disastrous. 

“AI vendors cannot realistically absorb unlimited downstream business risk tied to variables they don’t control — poor internal adoption, broken processes, bad data, organizational politics, weak change management, or unclear KPIs. But the moment vendors are compensated primarily on outcomes, you create strong incentives for increasingly autonomous optimization behavior. That sounds great until organizations realize that AI systems may pursue the metric rather than the intent behind the metric,” Greis said. 

“We’ve already seen versions of this in recommendation engines, ad targeting systems, and engagement algorithms. The system learns to maximize the measurable outcome even if the methods become operationally risky, ethically questionable, reputationally damaging, or strategically misaligned. In enterprise environments, that could become dangerous very quickly. An AI system incentivized around reducing service costs might aggressively deflect legitimate customer issues. A model rewarded for sales conversion could push manipulative messaging or optimize for short-term wins at the expense of customer trust. A procurement optimization engine might lower costs while quietly increasing supplier concentration risk or degrading operational resilience.

“The more autonomous these systems become, the harder it is to separate ‘successful outcome’ from ‘acceptable behavior.’”

The best way to resolve this is potentially the most difficult. Every AI project must be approved by an AI committee whose members must ask the hard questions. What are you hoping to accomplish? If it works, specify and quantify your best-case scenario benefits. What are the most likely ways it could fail? What are the costs and disruptions most likely to happen if it fails in that way? Quantify those. 

The committee should have at least a couple of members who know exactly what these models can and cannot do to serve as a reality check. 

Next, require the LOB chief, or whoever the most senior exec involved in the project is, to share in the pain. Tie gains or losses to executive bonuses. Give those execs a reason to make sure their people are honestly and creatively thinking the project all of the way through. 

Only once that happens can a CIO know how to negotiate a fair and reasonable AI pricing deal.

Over the next three to five years, both governments and the private sector will need to rapidly adapt identification and mitigation protocols as adversaries move from AI-assisted to AI-enabled sanctions evasion and proliferation financing (PF), a new research paper warns.

The report, Algorithms of Evasion: The Rise of AI-Enabled Proliferation Financing, from the Royal United Services Institute (RUSI), a UK-based defense and security think tank, defines PF as the use of funds or financial services to acquire, develop or otherwise deal in weapons of mass destruction (WMD). It states, “North Korea and Iran are now developing and deploying AI models to aid with sanctions evasion activities.”

Key findings include the fact that AI is now capable of mass producing high-quality fraudulent documents, as well as automating what the report describes as “the administrative minutia of managing extensive shell company  networks.” AI powered systems, it states, can also “analyze blockchain patterns in real time to dynamically adjust cryptocurrency mixing strategies, effectively evading detection tools.”

In addition, it says, “[tools such as generative AI] which can produce sophisticated fraudulent identification documents, for example, have helped North Korea perpetrate phishing attacks against Western companies.”

Dr. Aaron Arnold, senior associate fellow with the Centre for Finance and Security at RUSI, who authored the paper, said in an email that what prompted it was an uptick over the last year in North Korea’s use of AI to facilitate and enhance its cyber operations, in the form of phishing schemes designed to generate revenue for the country’s ballistic missile and nuclear weapons programs.

He advised enterprise IT managers who need to protect their organizations from becoming victims of sanction evasion activities that “[it] means largely adapting to a landscape where traditional human-focused security boundaries are being bypassed by automated technologies.”

For IT managers, said Arnold, “this might entail incorporating defensive AI, the use of behavior-based analytics, using ‘circuit breakers’ when there is heavy use of API or MCPs, updating personnel training, and hardening identity verification, especially for any remote hiring.” 

Distinction between AI-assisted and AI-enabled activity is ‘central’

Sanchit Vir Gogia, chief analyst at Greyhound Research, said that the RUSI report matters “because it names the right structural shift. AI is not creating sanctions evasion from thin air, it is compressing and scaling methods that already work.”

He pointed out that none of the sanction-evading techniques such as fraudulent documents, synthetic identities, shell companies, hidden beneficial ownership, crypto laundering, and others are new. “What changes is the speed, quality, volume and coordination with which these methods can now be assembled,” he said.

According to Gogia, “the distinction between AI-assisted and AI-enabled activity is central. AI-assisted evasion uses AI for discrete tasks: writing a better email, producing a cleaner document, generating a stronger false profile, translating a pitch, summarizing regulations or preparing a plausible job application. AI-enabled evasion is more serious.”

A ‘structural asymmetry’

This tactic, he said, “begins to coordinate the system itself. It links identity, documents, ownership structures, payment routes, cloud access, crypto wallets, API calls and timing. The difference is not whether AI helps someone fake a document. The difference is whether AI begins to orchestrate the deception.”

That is why the report’s findings should worry enterprise leaders, he noted: “Many organizations still assume the bad actor is mostly human, mostly linear and mostly slow. That assumption is expiring. AI lets adversaries run more attempts, with fewer errors, across more channels, in more languages, with better paperwork and greater patience than most enterprise review processes can absorb. This is not a tale of genius criminals discovering magic. It is the story of ordinary controls meeting industrialized plausibility.”

The evidence today, he pointed out, is strongest around tactics such as identity fraud, document fraud, synthetic personas, remote-worker deception, phishing, social engineering, crypto obfuscation and workflow abuse. “Fully autonomous evasion networks sit on the horizon,” he said. “They are serious, but they are not yet the everyday baseline.”

This distinction matters, said Gogia: “If enterprises obsess over cinematic autonomous agent scenarios while leaving remote hiring, vendor onboarding, payment approvals, and document review full of holes, they will lose in the most prosaic way imaginable.”

The report, he said, also gets the “asymmetry” right. “Offensive actors can learn across the ecosystem,” he said. “They can scrape open information, reuse leaked records, study enforcement patterns, test onboarding forms, inspect public procurement data, watch court filings, probe compliance thresholds and [use the information to] refine their behavior.”

Defenders, by contrast, are hemmed in by privacy rules, fragmented data, explainability requirements, jurisdictional boundaries, conservative operating models and siloed technology estates. “Offensive AI learns broadly,” he said. “Defensive AI often learns from fragments. That is the structural asymmetry.”

He explained that the regulatory landscape also amplifies the problem, in that regulatory bodies “still speak in separate dialects. [For example] the EU AI Act pushes organizations toward stronger obligations for high-risk AI. NIST-style frameworks push risk management, transparency, and governance.”

A trust architecture problem

Financial Action Task Force (FATF) expectations push national risk assessment and counter-proliferation controls, he noted, while banking regulators focus on model risk, accountability and operational resilience. “None of these streams is irrelevant. The trouble is that criminals do not organize themselves around regulatory workstreams. They organize around outcomes.”

What that means, said Gogia, “is that enterprise cannot wait for a clean global rulebook. It will not arrive in time. CIOs, CISOs, compliance officers and boards need a working governance model now. They need privacy-preserving analytics, controlled data environments, audit trails, legal safeguards and clear model-risk accountability.”

He said that enterprise IT managers should treat the situation as a trust architecture problem rather than a narrow sanctions-screening problem. “The uncomfortable truth is that AI is not simply helping bad actors write better phishing emails or forge tidier documents,” he noted. “It is helping them manufacture legitimacy across a chain of enterprise workflows.”

Likely outcome an ‘AI arms race’

Report author Arnold also noted that there are signs that cyber criminals have discovered new AI technologies and abilities that legitimate enterprises could adopt for legitimate applications.

History, he said, “is replete with [criminals] developing novel solutions to tough problems, [which are] later adopted by law enforcement. Much of our anti-financial crime policy is effectively a response to bad actors exploiting systems or using technology in novel ways to perpetrate crimes. In this scenario, I think an ‘AI arms race’ between enforcement authorities and bad actors is the most likely outcome.”

Gogia added, “the baddies are not teaching enterprises how to invent AI. They are teaching enterprises where trust is leaking. That is the lesson worth taking seriously.”

This article originally appeared on CIO.com.