OpenAI is selling its first hardware — without any help from Jony Ive. It describes the Codex Micro as a “command center for agentic work” but it’s really a 13-switch wireless keyboard customized to help developers keep tabs on what their Codex agents are doing. It costs $230.

The keyboard has 13 mechanical switches (one keycap covers two of them by default), a rotary encoder, joystick, and RGB backlighting around the whole keypad and individual keys. It comes with 32 customizable icon keycaps.

OpenAI claims that the Codex Micro is a serious business tool: The command keys enable Codex users accept changes, reject outputs, push-to-talk, start new chats, and trigger custom actions. The rotary encoder can be used to dial up the “brainpower” allocated to tasks — or in more conventional terms, how many tokens to allocate to reasoning on a task. And the RGB lighting can provide feedback on how various tasks are progressing. And the RGB lighting under the “agent” keys can provide feedback on how various tasks are progressing.

The Codex Micro’s manufacturer, Work Louder, already has a similar device on the market, the Creator Micro, which offers similar functionality, but without the colorful keys. It costs $56 less than the Codex Micro, however.

The Codex Micro will fit in snugly with OpenAI’s other merchandise, where using Codex is as much about a fashion statement as a technological choice.

Linus Torvalds has a complicated relationship with AI, seeing both its good and bad points. But his latest remarks on the usefulness of AI may have raised a few eyebrows in open-source circles.

Just a few weeks after the Linux founder complained that a “continued flood” of AI-generated vulnerability reports had made the Linux kernel security mailing list “almost entirely unmanageable”, he has come to see the advantages of the technology.

“Linux is not one of those anti-AI projects,” Torvalds wrote in an email response to Linux Kernel senior engineer Roman Gushchin, archived at Kernel.org.

“It can also be a somewhat painful tool, both for maintainer workloads and just from a ‘it keeps finding embarrassing bugs’ standpoint,” he said of the use of AI in security scanning. “The solution is to make sure those LLM tools help maintainers instead of just causing them pain.”

Developers should be free to choose whether they use AI, he said. “We’re not forcing anybody to use it, but I will very loudly ignore people who try to argue against other people from using it.”

Torvalds’ measured support for AI does not come completely out of the blue: Around the same time that he was complaining about AI distorting security maintenance, he also spoke about its usefulness, claiming that it could improve programmer productivity by a factor of 10.

Thinking Machines Lab, the San Francisco startup founded by former OpenAI CTO Mira Murati, has released Inkling, its first general-purpose AI model. The launch adds another US-developed entrant to an open-weight market where Chinese developers produce several leading coding and reasoning models.

Inkling uses a mixture-of-experts architecture with 975 billion total parameters, of which 41 billion are active during processing. It supports a context window of up to 1 million tokens and was pretrained on 45 trillion tokens spanning text, images, audio, and video. Thinking Machines said it also trained the model for coding, tool use, and multimodal tasks.

The release follows the October 2025 launch of Tinker, Thinking Machines’ first product and an API-based platform for customizing AI models. Developers can fine-tune Inkling through the platform.

In a June 2026 assessment, AI model routing platform OpenRouter highlighted DeepSeek V4 Flash, GLM 5.2, MiniMax M3, and Nvidia Nemotron 3 Ultra as four notable open-weight models. Nemotron was the only US-developed model in the group.

Performance and developer access

Thinking Machines Lab’s benchmark table shows mixed results. Inkling scored 77.6% on SWE-Bench Verified, behind DeepSeek V4 Pro and GLM 5.2 but ahead of Nvidia Nemotron 3 Ultra. It also recorded 74.1% on MCP Atlas, 77.1% on BrowseComp with context management, and 79.8% on IFBench.

Thinking Machines said Inkling’s result used a bash-only harness, while the comparison figures were reported by the competing models’ developers.

The model includes a reasoning-effort setting that developers can adjust from 0.2 to 0.99. Thinking Machines said the setting allows users to balance performance against the number of generated tokens. In the company’s testing, Inkling matched Nemotron 3 Ultra’s Terminal Bench 2.1 score while generating about one-third as many tokens.

Developers can fine-tune Inkling through Tinker using context lengths of 64,000 or 256,000 tokens and test it through the Inkling Playground. The model is available through APIs from Together AI, Fireworks, Modal, Databricks, and Baseten. It is also supported by inference software, including SGLang, vLLM, TokenSpeed, llama.cpp, and Hugging Face Transformers.

Inkling’s full weights are available on Hugging Face as the original checkpoint and as a quantized NVFP4 checkpoint. Thinking Machines also previewed Inkling-Small, which has 276 billion total parameters and 12 billion active parameters. The company said it would release the smaller model’s full weights after completing testing.

Enterprise impact

Inkling’s differentiation lies in its open weights, multimodal capabilities, controllable reasoning, and integration with Tinker, rather than benchmark leadership, according to Biswajeet Mahapatra, principal analyst at Forrester.

“Enterprises are most likely to benefit in workloads where domain adaptation matters more than generic model performance, including knowledge-intensive copilots, multimodal customer service, document understanding, operational workflow automation, and agentic tasks that require organization-specific data, policies, and processes,” Mahapatra said.  

Inkling’s US origin could also influence adoption among Western enterprises, according to Pareekh Jain, CEO of Pareekh Consulting. He said many Western organizations face regulatory or procurement barriers when considering Chinese-developed AI models.

“Inkling gives those organizations a US-developed open-weight option that they can deploy on their own infrastructure,” Jain said.

However, the benefits will need to be weighed against the cost of deploying the full model.

Running Inkling on private infrastructure requires a GPU cluster with at least 2 TB of aggregated VRAM for the BF16 checkpoint, according to the model card. Thinking Machines lists configurations of eight Nvidia B300 GPUs or 16 H200 GPUs. A quantized NVFP4 checkpoint lowers the requirement to at least 600 GB and can run on four B300 GPUs or eight H200 GPUs.

“Because Inkling is a massive model with 975 billion total parameters, running the full model still requires significant GPU infrastructure, making closed-model APIs more economical for many organizations,” Jain said.

Jain said Inkling-Small may be a more feasible option for many enterprises because it could reduce infrastructure costs and latency while retaining useful performance across key workloads.

Safety and governance

Thinking Machines said it trained Inkling for calibration, instruction following, and resistance to censorship. The company said the model showed “strong patterns of censorship non-compliance” when evaluated by Cognition on its Propaganda and Censorship Eval.

Inkling scored 98.6% on StrongREJECT, which Thinking Machines described as a test of whether models refuse unambiguous harmful requests.

The model’s safety behavior should be retested after an enterprise customizes it, according to Jain. “Model fine-tuning can weaken safety filters, so companies should retest safety after customizing the model rather than assuming it stays safe,” Jain said.

He added that self-hosted and modified versions could diverge from Thinking Machines’ official model over time without receiving automatic updates.

“CIOs need to ensure every AI agent action is logged, auditable, and governed by human approval for high-risk tasks,” Jain said.

OpenAI’s recent update to its Codex CLI has introduced a new protocol that appears to shift more orchestration decisions from user-defined configuration to the runtime, prompting developers to request greater visibility into the instructions exchanged between AI agents.

In a detailed GitHub merged request, users stated that the Multi-Agent V2 protocol-infused architecture of the CLI no longer exposes the instructions passed between parent and sub-agents, making it difficult to inspect how work is delegated across the system.

“Multi-agent v2 currently routes agent instructions through normal tool arguments and inter-agent context. That means the parent model can emit plaintext task text, Codex can persist it in history/rollouts, and the recipient can receive it as ordinary assistant-message JSON,” the request read.

“This changes the v2 path so agent instructions stay encrypted between model calls: Responses encrypts the message argument returned by the model, Codex forwards only that ciphertext, and Responses decrypts it internally for the recipient model,” it added.

Other users, commenting on the thread, also said that the lack of visibility into agent instructions can be attributed to the recently introduced Multi-Agent V2 protocol, with one user stating that reverting to the previous version of the CLI restored visibility, but only as a temporary workaround.

Separately, Ignat Remizov, CTO at payment service Zolvat, filed a GitHub feature request to offer what can be described as a permanent fix after stating that OpenAI may have introduced the change in efforts to harden security.

“A possible shape is to keep the encrypted message field for model delivery, but add a separate non-encrypted audit field for the readable task text. The audit field should be persisted in rollout/history/trace metadata so users and maintainers can inspect what was delegated without needing to decrypt model-delivery ciphertext,” Zolvat wrote.

Enterprise governance concerns are likely to emerge

While an OpenAI contributor said the protocol remains under development and declined further changes to the request, analysts warned that the issue would create debugging, governance, and operational challenges for development teams and their enterprises if the issue persists or becomes a long-term characteristic of multi-agent systems.

“Hidden agent instructions reduce observability in multi-agent systems. Developers can no longer see whether failures stemmed from incorrect task delegation, poor orchestration, or model reasoning, making debugging, prompt optimization, and root-cause analysis significantly harder. Agent instruction traces are becoming as essential as application logs in modern software,” said Pareekh Jain, principal analyst at Pareekh Consulting.

For CIOs, Jain pointed out, opaque agent interactions create governance challenges.

“Without visibility into how agents delegated and executed tasks, it becomes harder to audit decisions, investigate incidents, demonstrate compliance, and build trust in AI systems. Enterprises will increasingly expect secure but auditable agent communication rather than completely hidden orchestration,” Jain said.

“Any big enterprise, especially in regulated industries such as banks and hospitals, needs to be able to prove what their AI systems did and why, especially if something goes wrong. If a sub-agent does something bad, like touching private data, the company needs to show here’s exactly what it was told to do. If that record doesn’t exist, it is a serious problem for trust and legal accountability, not just an annoyance,” Jain added.

Further, the analyst pointed out that issues around the visibility of agent operations could even slow production deployments of mission-critical AI.

“Enterprises, just like we are seeing with developers on GitHub, are likely to demand stronger observability, audit trails, and governance before trusting autonomous multi-agent systems. It is nearly as important as model performance,” Jain added.

An email sent to OpenAI enquiring about planned changes to the protocol went unanswered.

These days, application developers can take their pick from a vast menu of architectural solutions. We can choose from the well-understood to the experimental, and from blended solutions in between. Several powerful middle-ground technologies that emerged during the cloud revolution have really come of age. Here we’ll take a look at putting together three of the most impressive: GitHub, Vercel, and Firestore.

Each of these is an important tool in its own right that can be used to attack specific problems. In combination, they not only meet the needs of several important application scenarios, but they have a superpower—the ability to dramatically shorten the distance between development and deployment.

There is nothing quite as gratifying as putting your hands on just the right mix of tools for a given need.

A ‘no-ops’ stack built for speed

If your primary goal is sheer development velocity, you would be hard-pressed to top this architecture. This “no-ops” stack collapses the distance between your local IDE and a globally distributed production environment. You are essentially trading the overhead of managing VMs and load balancers for the sheer speed of committing code and watching it deploy automatically.

While each component is highly flexible, adopting them requires a specific, event-driven mindset. There are a few finicky bits to manage, mostly around routing environment variables securely and designing around stateless back-end functions. But the constraints are obvious and well-documented.

Before we look more closely, let’s quickly identify the kinds of apps that are a perfect fit here, along with those that are workable and those that really merit a different approach.

• The sweet spot (deploy and go): AI-mediated applications, asynchronous game back ends, and real-time collaborative B2B dashboards. This architecture perfectly absorbs the unpredictable latency of LLM APIs and instantly syncs state across multiple clients without requiring you to build custom WebSocket infrastructure.
• The middle ground (workable, with trade-offs): Headless e-commerce, moderate IoT telemetry, and apps requiring scheduled batch processing. You will encounter friction if your catalog relies on deeply relational SQL constraints, or if your background reporting jobs take longer than a few minutes and hit serverless execution limits.
• The danger zone (look elsewhere): High-frequency trading, fast-paced action multiplayer games, heavy data ETL pipelines, and core financial ledgers. Serverless architectures cannot natively hold open the persistent WebSockets required for twitch-reflex data, and heavy compute tasks will abruptly time out.

We should mention that these categories are not mutually exclusive. Many enterprise applications, such as a full-scale e-commerce platform, straddle these lines. You might use Vercel and Firestore to build a lightning-fast, reactive storefront that handles ephemeral user state like shopping carts, while simultaneously “stitching in” a managed SQL database like Supabase or PlanetScale. This hybrid approach allows you to maintain the relational integrity required for back-office inventory and financial ledgers and pair it with the front-end velocity this stack provides.

GitHub: the bedrock

I don’t need to introduce you to GitHub. It is a central element of the development landscape. I still remember CVS and SVN with a certain nostalgia, but the enhancements of Git speak for themselves. When combined with the orchestration powers of GitHub, it is no wonder that virtually the whole industry has adopted this type of platform.

Git plus GitHub gives you an enormous amount of power already, in terms of how you can organize and automate your projects. But there is a next-level experience in combining GitHub and Vercel. For JavaScript-based projects, you can take simple GitHub pushes and turn them into instantly deployed clients and serverless functions. It is one of the cleanest and least fiddly ways to move from raw code on your local machine to a globally deployed, full-stack architecture.

Vercel: the nexus

Vercel is more than just a deployment host. It is a control plane that ties this high-velocity, no-ops architecture together. Alongside GitHub and Firestore, Vercel’s deeper strength is its ability to act as an orchestration layer between your reactive front end and external stateful services.

Vercel has a great amount of facility in fine-tuning what branches go to what environment and helpful features like instant rollback. You can just log into Vercel’s dashboard for your project and see the history of deployments and any errors and logs. It’s a simple menu choice to roll back to a historical version or compare one version against another.

When you “stitch in” third-party services (such as a managed SQL database like Supabase or a payment processor like Stripe), Vercel’s serverless functions become the lightweight interface, and Vercel’s the adapters handle the communication. You offload the integration logic (the service layer) to Vercel’s global Edge Network, keeping your UI and back end clean, responsive, and decoupled. 

In short, Vercel allows you to get the speed of the “no-ops” development life cycle without sacrificing the complex transactional integrity required for some applications like enterprise inventory systems.

Firestore: the datastore

Firestore is an extremely lightweight, NoSQL, cloud datastore. It has a great deal of add-on power, but its core value proposition is that it accepts virtually any data you stuff into it and it provides event-driven subscriptions to data changes.

These two capabilities together make Firestore about as straightforward a solution to a managed back end as you can imagine. You subscribe to collections or even fields and then you simply stick “unstructured” data (read: JSON with variable fields) in and the client waits for the changes it is interested in.

This is so streamlined that one can just point the browser (or native mobile app) directly at Firestore and listen for events. Which immediately raises the question of identity, for auth and for data visibility, but hold on—Firestore’s third superpower is that it has an authentication module that actually works. What I mean is, it is actually pretty simple and yet confidently secures your app.

Sometimes auth solutions seem either too simple (and yet opaque) or too mired in the nitty gritty. Firestore auth will let you do some basic configuration and start using a reasonable auth almost immediately. 

Not to belabor the point, but having a realistic and attainable auth solution elevates your stack to a production grade—one that can handle many real-world applications. Firestore auth plays nicely with other important APIs, like Stripe. Typically, auth is a major feature that feels like off-roading in a Honda Civic, but Firestore’s approach to auth, added to this particular stack, feels like a normal speed bump. It’s just another component you plug in, rather than a tentacled alien you weave into the your code.

The limits of the velocity stack

This architecture combines components that are optimized for flexibility. That same character also introduces distinct limitations. Understanding these is essential before committing production workloads.

The serverless life cycle

Serverless functions are spun up to handle requests. They close out soon afterward and lose any state. For that reason, they cannot natively hold open persistent WebSockets. If your system requires continuous, sub-millisecond, bidirectional streams—like a real-time multiplayer action game or a high-frequency trading dashboard—pure serverless will fight you all the way. You are forced to introduce a third-party managed WebSocket service to route messages back to your stateless endpoints via HTTP webhooks.

The execution time ceiling

Vercel (like all serverless platforms) enforces strict timeouts on operations. While enterprise tiers might grant you up to 15 minutes, standard functions often time out after 10 to 60 seconds. Long-running tasks like video transcoding, database scripts, or orchestrating multi-step AI agent workflows, which might take 20 minutes to resolve, will run up against these limits. Heavy-lifting tasks must be offloaded to a dedicated, long-running service like Google Cloud Run, or broken into smaller, asynchronous chunks via message queues.

The cold start reality

While the industry has made massive strides in minimizing initialization times—particularly with lightweight edge networks—traditional Node.js-based serverless functions still experience cold starts. If a function has not been invoked recently, or if traffic spikes require a new instance to spin up concurrently, the first request will take a noticeable latency hit as the container provisions and the code loads.

API instead of RAM

In a traditional server environment, you can store transient data in global RAM, allowing subsequent requests to access shared context instantly. In the serverless model, every request might hit a fresh container. Therefore, all shared context must be externalized. Although Firestore serves brilliantly as the state manager, relying on a database for high-frequency, sub-millisecond, ephemeral caching introduces network latency and per-operation costs. That said, using a shared RAM state on a server is non-trivial also, unless you are using a single app server and VM (because high-availability or fail-over requirements will lessen the RAM win on a traditional server).

Tuning for velocity and control

Every architectural decision is a trade-off. There are no cost-free choices. By adopting the GitHub, Vercel, and Firestore stack, you are explicitly maximizing feature velocity over fine-grained control.

You lose the ability to tweak the underlying operating system, hold open persistent sockets, or run hour-long back-end scripts. In exchange, you gain an architecture that scales from zero to global distribution instantly, requires virtually no devops maintenance, and perfectly absorbs the asynchronous, event-driven realities of modern application development.

For the right application—whether it is a fast-moving prototype or an enterprise AI copilot—this stack doesn’t just save time; it fundamentally changes how quickly a small team (or a single person) can impact the market. You stop worrying about build chains, load balancers, and server patches, and you focus on the central mission: shipping features.

The first time I heard someone say their AI now wrote 80% of their tests, I asked the obvious question. Eighty percent of what?

After 20 years building and leading test automation for consumer-scale platforms, my honest answer turned out to be eighty percent of the typing, not eighty percent of the engineering. The remaining twenty was where the work still lived. Budgeting for two percent of leftover effort was the mistake. When the real number was closer to thirty, that gap was the difference between a pipeline that shipped and one that quietly built up a queue of half-trusted features nobody could rely on.

This piece is about that gap. As an independent research project on LLM-augmented testing methodology, I built a six-stage agentic pipeline that takes a design in Figma and produces running tests in WebDriverIO, connected end to end over the Model Context Protocol. It works. It has been useful. And the parts that broke surprised me, because they were not the parts the hype cycle tells you to worry about.

How I wired a six-stage pipeline over one protocol

The pipeline runs six stages in sequence, each owned by a different agent, with every handoff crossing MCP.

Six-stage agentic test pipeline: design capture → requirements writer → ticket opener → code generator → test-case writer → automation generator. Each stage carries an MCP handoff and a provenance stamp.

The end-to-end trace links a pull request back to a Jira ticket, a requirements section and a Figma frame. Each artifact is stamped with the agent that produced it, the model it used and the inputs it was given.

MCP is the boring middle that makes any of this work. The cliché is that MCP is “USB-C for AI”: one open protocol, any tool. Like most analogies, it is about eighty percent right. The part that matters is the eighty: I do not have to write a custom adapter for every system the agent talks to. One MCP server per tool and every agent talks to all of them the same way.

Typed handoffs between agents are my own architecture, layered on top of MCP rather than provided by it. Each agent writes a typed artifact the next agent reads. Each handoff is logged with provenance. When something went wrong six stages in, I could replay the chain. Without that discipline, a multi-agent pipeline is a debugger’s worst day. You know the test plan is wrong. You cannot tell whether the mistake came from the Figma read, the requirements interpretation or the ticket scaffolding. With it, I could point at exactly which stage went sideways and which inputs it was looking at when it did. The pattern lives in a public MIT-licensed reference implementation for any reader who wants to run it.

The sixteen-minute number is the marketing number. I ran the full chain end to end in about sixteen minutes on a synthetic net-new screen, Figma in, automation suite out. That repeated across my runs; it is not a demo trick. But sixteen minutes is the part of the story most fun to tell and least useful to learn from. It is what gets quoted in the all-hands. The hours that come after, when a human reviews each handoff, are where the work actually lives.

What actually broke in production-style runs

The failures that stalled my pipeline were rarely the ones I expected.

I expected hallucinated APIs. I got them: the agent confidently called endpoint names that sounded right but did not exist. I expected sparse-spec-in, sparse-spec-out, where a Figma frame with no annotations produced a requirements doc with vague acceptance criteria, every time. I expected locator drift, the common UI-automation failure mode where a renamed component silently breaks an entire test suite. There is solid outside writing on non-determinism in tests covering this whole family of failure modes, and the agent inherited every one.

What I did not expect, and what kept the pipeline down longer than any of the above, was the plumbing.

The model backend timed out under load. It lost credentials silently and started returning empty strings, which the agent then read as confidence. A duplicate consumer on a shared long-poll API endpoint produced an HTTP 409 conflict that broke delivery without throwing anything visible. One unguarded exception inside one agent aborted a whole shared scheduler run and took the other agents in the registry down with it. The single worst incident cost me three hours to find. An environment variable had silently rotated overnight; every agent in the fleet was returning structurally valid but semantically empty requirements docs; the downstream stages were dutifully generating tests against nothing.

None of those are model bugs. They are infrastructure. The agent literature, which is what I went looking through when I started this work, mostly does not talk about them.

The fix was not better prompts. It was circuit-breaker-style review checkpoints between stages and what I now call the four-guard discipline: four small guards I consider non-negotiable on any unattended agentic pipeline. The bulkhead pattern from microservices is the most consequential. An unhandled exception inside one agent can no longer abort the shared run; the offending agent fails fast with a structured error and the others keep going. Paired with that, a pure-data fallback ensures a model timeout produces a deterministic output explicitly marked as degraded mode, rather than an empty string the next stage will misread as confidence. A single-owner lease sits on every shared external endpoint, the cure for the duplicate-consumer incident that ate one of my Sunday afternoons. The cheapest guard was the last to arrive: a one-line synthetic canary every agent has to produce a known correct response to before any real work begins, so a credentials rotation or silent backend failure trips an alert before downstream stages have generated artifacts against garbage.

None of these guards is novel. They are textbook stability patterns at a new boundary: the seam between the LLM agent and the rest of the system, which most of the existing agent literature still treats as a solved problem.

The 20% you don’t see, and when not to do this

Here is the part the demo videos leave out. Even when the pipeline works, the human time per stage does not go to zero.

Human review time per ticket across five pipeline stages: code review 60-180 min, automation review and flaky-fix loop 30-90 min, ticket architecture and sequencing 30-60 min, test data and environment 15-30 min, requirements review 20-30 min. Net: the human still spends 20-30% of the original effort, almost all of it reviewing rather than creating.

Net of all that, the human still spends twenty to thirty percent of the original effort, almost all of it reviewing rather than creating. The pipeline saves seventy to eighty percent, not ninety-eight. The trap is budgeting for the two percent you do not save.

When does this kind of pipeline make sense? In my experience, when the Figma is richly annotated and acceptance criteria are clear up front; when there is review capacity to absorb the work the pipeline shifts onto humans; when the stack is well represented in the training data; and when the feature is net-new rather than a deep edit of legacy code. When does it not? When the design lives on a whiteboard. When the integration touches old code with hidden contracts. When the path is regulated or safety-critical. When there is no senior reviewer who can hold the line. When the work is exploratory and writing the spec is the actual point of the exercise.

Teams I have seen succeed with agentic pipelines budget for the rework explicitly, staff the review queue and treat the saved hours as capacity for harder problems rather than headcount they can release. Teams I have seen struggle did the opposite: declared victory at the demo and quietly accumulated a backlog of half-trusted features the next quarter had to clean up.

The right unit of measurement is not how much the pipeline generates. It is how much of what it generates a human still has to touch before you would ship it. Call it the 80/20 rework rule: measure the rework, not the generation. The teams that get the rework number right are the ones whose AI investments compound. The teams that stop counting at the headline percentage are the ones that own the cleanup six months later.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Port has rolled out Port AI Builder, a vibe coding platform designed for software development and platform engineering teams.

Announced July 14, Port AI Builder lets teams create and run agentic workflows in natural language, with built-in human-in-the-loop review and approval. The platform lets organizations apply AI agents across the SDLC (software development life cycle), drawing on domain skills spanning site reliability engineering, devops, architecture, security, AI governance, data modeling, and UX, while maintaining governance and visibility, the company said. Port AI Builder is available through free and paid subscriptions.

Port AI Builder works on top of Port’s Agentic SDLC Platform, which provides the context lake, workflow orchestration, agent management, and governance that enterprises need to operationalize AI SDLC, according to the company. The new AI Builder lets teams build production-grade workflows in minutes, for use cases such as autonomous resolution, AI cost management, and engineering performance tracking, without losing control, Port said.

Oracle on Tuesday expanded its AI Agent Studio for Fusion Applications with new pro-code development tools, including a CLI-based capability called AI Studio Skill, allowing developers to build agentic applications using familiar environments such as VS Code, Codex, and Claude Code.

The AI Studio Skill is the CLI that provides the Fusion-specific context and tooling that AI coding assistants need to build Fusion-native applications. It provides access to the project structure, APIs, templates, validation, packaging, and deployment workflows required for Fusion Agentic Applications, Natalia Rachelson, SVP of product for Fusion Applications at Oracle, told InfoWorld.

“Think of it as Oracle’s development harness for popular AI coding assistants. Developers can use models like Codex or Claude Code to generate code, while the AI Studio Skill connects those models to Oracle AI Agent Studio and the Fusion runtime,” Rachelson said.

Familiar tools and workflows to simplify development

The access to familiar IDEs and harnesses, according to analysts, will make it easier for developers to build and maintain agentic applications for business workflows.

“The AI Studio Skill provides developers a way to build Fusion agents like a new software function versus configuring them like application extensions,” said Scott Bickley, advisory fellow at Info-Tech Research Group.

“Enterprise developers expect source control, code review, repeatable deployments, testing or debugging, and integration into their existing toolchains. Connecting the various IDEs and code assist products will make it easier to build, validate, and maintain agentic applications using already familiar tools and processes. This ostensibly will result in agents that are easier to maintain, govern, and align with enterprise development practices,” Bickley added.

For Robert Kramer, managing partner at KramerERP, the move is likely to drive more adoption of the Studio itself: “Oracle is meeting developers where they already work and making Fusion a more attractive place to build agentic applications.”

Native runtime could aid governance in production deployments

However, the CLI and IDE integrations, for Bickley, extend beyond developer productivity into tackling the governance and execution challenges that often prevent AI prototypes from reaching production.

“One of the most painful barriers to production AI is that many prototypes are built outside the enterprise systems where identity, permissions, workflow approvals, and overall system governance are already built in,” Bickley said.

In contrast, the integrations will allow enterprises to run agentic applications from inside Oracle’s platform, leveraging existing business context, identity, approvals, and governance rather than recreating those capabilities through external orchestration layers when moving them into production, Bickley added.

That shift, the analyst further added, will prove beneficial for CIOs because it will accelerate business outcomes while operating within a trusted environment.

Governance, observability, and lifecycle management matter more to CIOs after agentic applications move into production, Kramer echoed.

Governance gains come with strategic trade-offs

The approach of building and running agentic applications natively inside Oracle Fusion, though, is not without trade-offs, analysts cautioned.

CIOs should pay close attention to vendor lock-in as more business processes become agentic, Bickley pointed out.

“In the case of Oracle Fusion, ensure the ATLAS framework provides an accurate validation layer at a low cost of overhead. Consider the levers that Oracle may avail itself of contractually or commercially in the future,” Bickley said.

“ROI should be modeled against a progressive monetization schema as AI agents operate upon a consumption-based infrastructure.  As such, ensure provisions limiting cost overlays and uplifts are agreed upon prior to locking in,” Bickley added.

These considerations, the analyst further added, are becoming increasingly relevant because most enterprise software vendors, including the likes of SAP and ServiceNow, are introducing offerings and features to become the runtime and orchestration layer for enterprise AI.

Earlier in May, SAP expanded its AI strategy with the Autonomous Enterprise vision, introducing a unified Business AI Platform, Joule Studio 2.0, and AI Agent Hub to let enterprises build, govern, and run AI agents within a managed runtime.

In June, ServiceNow expanded its AI transformation by adding new features to its Context Engine and AI Control Tower, in order to better embed governance, enterprise context, and observability into AI workflows across enterprise systems.

During the same month, Salesforce, via its Informatica acquisition, added features to tie AI agents more closely to trusted enterprise data and operational workflows.

For developers and enterprises willing to try out the new CLI-based experience, it can be accessed from within the Studio without any additional cost, Oracle said.

The company is also adding a public GitHub repository that it said will provide templates, starter projects, sample applications, reusable assets, and reference architectures to help developers build and validate agentic applications faster.

Visual Studio Code 1.128, the most recent edition of Microsoft’s code editor, adds support for multi-chat agent sessions using Anthropic Claude agents.

Introduced July 8, VS Code 1.28 can be downloaded for Windows, Linux, or Mac from code.visualstudio.com.

With VS Code 1.28, multi-chat agent sessions now support using Anthropic Claude agents. With multiple chats, a session can contain related chats, allowing users to compare approaches, branch from an earlier turn, and run work in parallel. Users can add chats, fork a chat from an existing turn, switch between peer chats, and send turns concurrently. Each chat keeps its own history, title, and model selection, and restores with the parent session after restart, Microsoft said.

VS Code 1.28 also makes Copilot Vision generally available. With Copilot Vision, VS Code users can attach images and PDFs by pasting them into chat, dragging and dropping them, or using the context menu. Images also may be read by the agent via a tool call, Microsoft said.

Also in VS Code 1.28:

• Quick chats allow users to ask a question in the Agents window without opening a workspace first.
• VS Code can contribute OS-level keyboard shortcuts that take effect even if VS Code is not in focus. Users can add systemWide to a keybinding definition in keybindings.json to make it OS-level.
• Developers can configure a location for browser tabs to open via a new setting, workbench.browser.newTabPlacement. The setting provides three options: to open browser tabs in the active editor group, in a dedicated group to the side, or in a dedicated auxiliary window.
• VS Code can contribute OS-level keyboard shortcuts. These shortcuts take effect even if VS Code is not in focus.
• Administrators can configure where GitHub Copilot sends OpenTelemetry (OTel) data, allowing organizations to enforce telemetry flows to an approved collector without each developer setting OTEL_* environment variables. The managed configuration applies to both the Copilot Chat extension and the agent host process.

GitHub Copilot has introduced an interactive canvas that makes it easier to follow, review, and steer the actions of the upgrade agent as it modernizes .NET applications.

The GitHub Copilot upgrade agent assesses the application, generates a structured upgrade plan, creates implementation tasks, and executes the work, according to a July 9 bulletin from Microsoft. Now developers can follow that workflow in the GitHub Copilot app through an interactive upgrade canvas. The upgrade canvas provides a live view of the modernization workflow as it unfolds, Microsoft said.

This agent upgrade agent starts by assessing the .NET application and identifying what needs to change:

• What version of .NET is this application targeting?
• Which NuGet packages need to be updated?
• Are there breaking API changes?
• Which projects can be upgraded independently?
• What should happen first?

From this point, the agent generates a structured upgrade plan and breaks the work into actionable implementation tasks. While the GitHub Copilot app provides the interactive upgrade canvas, GitHub Copilot upgrade also is available in the following tools:

• Visual Studio – Built directly into Visual Studio. Right-click a solution or project in Solution Explorer and select Modernize to start a .NET upgrade.
• Visual Studio Code – Install the GitHub Copilot upgrade extension, select the Upgrade agent from the agent picker dropdown menu, and prompt the agent to modernize your .NET application.
• GitHub Copilot CLI – Install the GitHub Copilot upgrade plugin to assess, plan, and execute .NET upgrades directly from the terminal.

Developers can get started with GitHub Copilot and the GitHub Copilot upgrade agent by visiting the GitHub Copilot upgrade marketplace.

Microsoft has been pushing hard to make Visual Studio Code a major way to consume its AI services, mostly in the form of GitHub Copilot. GitHub Copilot’s deep integration with VS Code brings many conveniences — inline autocomplete, for instance — but it’s frustrating for those, like me, who would rather use another model provider, or even a locally hosted LLM, for those functions.

Visual Studio Code 1.122 introduced a new feature, “Use BYOK [Bring Your Own Key] without a GitHub sign-in,” that allows you to “use chat, tools, and MCP servers in air-gapped or restricted environments where GitHub sign-in isn’t possible.” More importantly, it “enables fully offline workflows with local models like Ollama.”

In other words, you can now use locally hosted LLMs for chat, tools, and Model Context Protocol servers inside Visual Studio Code. The one thing you still can’t do is use a local LLM for inline and next-edit suggestions — at least, not without additional tooling.

Choosing a model for BYOK mode

If you want to use a local LLM with VS Code’s bring-your-own-model system, the first thing you need is a way to host the model. VS Code lacks a model-hosting mechanism of its own, although it’s conceivable that a VS Code extension may offer something like that in the future. That said, hosting models is complicated enough that a dedicated app is really needed for the job.

One easy way to host models is via a product like LM Studio, a convenient GUI for standing up, serving, and managing LLMs on one’s own hardware. The model host does not have to be the same system you run VS Code on, either. It can be on a server box you control, or on a cloud instance.

The choice of model is also important. Many models are powerful but won’t run well on commodity hardware because they’re simply too big. A good rule of thumb is to choose a model that fits into existing VRAM, along with the memory needed for a sizable token context (the more, the better). Also, the model should be suited to coding and development work. Some models in this vein that fit comfortably into 8GB VRAM include:

• Gemma4 (effective 2 billion parameters version)
• Qwen3.5 9B
• Codestral 22B v.0.1 (proprietary license)

Setting up BYOK mode in VS Code

Once you have a model up and running, you can integrate it with Visual Studio Code. If you’ve disabled VS Code’s AI features, you will need to turn them on. Make sure the setting chat.disableAIFeatures is turned off. You can find it in Settings | Chat | Miscellaneous.

Third-party language models are managed through Visual Studio Code’s language model list. Press Ctrl-Shift-P and type Manage Language Models to open the list of existing language models.

First you will see a list of the built-in models, which are all externally hosted. To add a new model, select Add Models at the top right and select Custom Endpoint.

You’ll then get a series of prompts:

• Group Name: This is “Custom Endpoint” by default, but you can choose any name you want. The name is strictly for organizing the model list and doesn’t affect things like model recognition or connectivity.
• API Key: If you’ve configured LM Studio to use an API key for serving models, provide it here. If you’re hosting the model locally and you haven’t explicitly set up API keys, you can leave this blank.
• API Type: The options here are Chat Completions, Responses, and Messages. Most of the time you’ll want to use Responses, as it’s the most general-purpose option of the three.

Once you finish providing those answers, you’ll be dropped into a modal editor for a JSON file that holds the details about the endpoint you’re configuring.

You’ll need to provide a few more details by typing them into the labeled fields:

• id: A text field that uniquely identifies this particular entry. The choice of ID is pretty much arbitrary; if you’re using only a single model, the ID could be the model name.
• name: The name of the model that is used to identify it on the model server. In LM Studio, you can get this name by clicking on My Models in the main interface, then selecting the three-dot icon for the model in question and clicking Copy Default Identifier. For Qwen 2.5, for instance, name might be something like qwen2.5-coder-7b-instruct.
• url: The URL to the server’s endpoint. On LM Studio, this defaults to something like http://127.0.0.1:1234/v1. The /v1 at the end is important because that endpoint is used for autodiscovery of models and their capabilities.

The other fields generally don’t need editing. Most models have tool calling functionality. If you know for a fact that the model you’re using doesn’t have vision support, then set vision to false.

Once you have these fields filled in, you can close the modal editor to save the changes. If you reload the Manage Language Models page, you’ll now see your new endpoint:

You should now be able to launch the chat window and use the defined model for conversation and utilities:

One current, and major, limitation of Visual Studio Code’s BYOK functionality is that it only works for chat and utility tasks. It doesn’t allow you to use a local model for inline suggestions or code completions. The only way to take advantage of local models for expanded functionality with VS Code is to use a third-party tool like Continue.

It isn’t clear if Microsoft will eventually lift this restriction. GitHub Copilot integration in VS Code is a large part of how Copilot as a service reaches its target audience. For the time being, you can certainly use third-party and local models for a significant part of your AI-assisted development work in VS Code, and you can close the functionality gap with additional tooling.

AI coding agents can tend to isolate research, running experiments and generating ideas that are then forgotten when context windows reset. This can waste tokens, as models then repeat the same mistakes and hit the same dead ends.

But new research argues that it’s not the model itself, but the overarching ‘tree,’ that needs tweaking. To that end, data scientists from the Gaoling School of Artificial Intelligence, Renmin University of China, and Microsoft Research have introduced Arbor, a “persistent hypothesis tree” that helps agents remember and refine learnings over long research sessions.

A long-lived coordinator manages research strategy across the tree, while short-lived executors spin up isolated worktrees to test different hypotheses. As results come back, the tree updates, narrowing and refining throughout experimentation.

In practical tests, this technique delivered more than two-fold performance gains over standard AI coding agents across real-world engineering tasks, for the same budget.

This is because, said Mahmoud Ramin, a research director at Info-Tech Research Group, “Arbor accumulates information over time and allows agents to build upon prior discoveries just as humans do, through learning, adaptation, and eventually building upon what they have learned in the past.”

How Arbor grows

Arbor’s builders argued that longer execution on its own does not guarantee research progress. The challenge is maintaining a state that turns many individual attempts into “cumulative hypothesis refinement.”

Further, progress should not depend on human overseers regularly stepping in to dictate logical next steps or interpret the meaning of previous trials, they noted. To be truly autonomous, agentic research frameworks must maintain connections between experiments, data, results, and failures over time.

Arbor is built to fulfill three system requirements. First, it must be able to branch as sub-trees test out competing hypotheses that are all potentially plausible. At the same time, unrestricted branching can degenerate the whole framework, so that must be controlled to remain organized. The researchers call this “branching with coherence.”

Second, the infrastructure must separate local execution from overarching strategy. Testing out single hypotheses requires short-horizon tasks like editing, debugging, and evaluation. But these should not “obscure” the larger tree making decisions based on evidence gathered across the whole run.

Finally, the systems must be able to distinguish exploratory improvement from verified improvement. This prevents AI from overfitting during trial-and-error instead of iteratively learning from underlying patterns.

Persistence is at the core; the tree links hypotheses and ideas, the code or configuration artifacts used to test them, experimental evidence (results, metrics), and distilled insights (such as “this data filter helped, but this learning rate scheduler didn’t”).

Once a project kicks off, shorter-execution work trees run code, log their work, and collect metrics. The long-lived coordinator above them serves as the de-facto head of research, keeping an eye on the process, updating nodes, selecting “promising leaves,” pruning or merging branches, propagating reusable lessons, and deciding which hypotheses to pursue next.

“The tree therefore acts as the operational research state of the system,” Arbor’s builders wrote. “It is simultaneously the search frontier, the memory of past attempts, and the audit trail for verified artifact improvement.”

Outperforming Codex and Claude on new data

To test how well this process works, the researchers evaluated Arbor in an autonomous optimization (AO) setting: the agent was given an initial research artifact (a data pipeline, harness, or training script) and was tasked with improving its “held-out performance” through iterative experimentation, without human steering. Held-out performance is a machine learning (ML) metric that evaluates how well models are able to generalize on data they haven’t seen before.

The tree-based architecture was tested on several real research tasks across model training (its ability to improve training recipes and hyperparameters), harness engineering (how well it can upgrade evaluation or training harnesses), and data synthesis (its capacity to generate better data for training or evals).

Ultimately, Arbor outperformed the average held-out gains of Codex and Claude Code by 2.5x, for the same resource budget.

The takeaway, said the researchers: Keeping a structured, evolving hypothesis tree yields greater performance improvements than running the same models as ‘memoryless’ coding agents.

Arbor’s most innovative feature is its ability to maintain the agent’s memory and retain relevant data from prior attempts and hypotheses, Info-Tech’s Ramin pointed out, and, he said, “the next step for autonomous agents may be accumulating evidence over time.”

However, this does raise concerns about the auditability of robust research environments at a large scale, he noted. “As autonomous agents become more capable of performing work without human operators overseeing them, enterprises will need transparency into how and/or why an agent took a specific action or reached a certain conclusion.”

In hardware, when you ship something broken, the consequences are severe and often irreversible. That’s the world I worked in for years, in verification roles at Mellanox and later at Alibaba. The stakes forced the industry to build a rigorous verification culture. You proved designs worked before they left the building.

In software, verification disciplines look like CI/CD pipelines, static analysis, canary deployments, and observability. But those systems were built around code written at human speed, with human comprehension baked into the process. AI code generation has broken that assumption. The writing process can no longer be trusted to carry institutional knowledge and judgment into the codebase. The industry is being pushed toward the kind of rigorous verification culture that hardware engineers have practiced for decades.

Enterprises are generating code faster than at any point in history. Google recently disclosed that 75% of the company’s new code is now AI-generated. Meta has set internal targets requiring most of its engineers to generate the majority of their committed code with AI tools by mid-2026. The velocity gains are significant. But a growing body of evidence suggests the industry is accumulating a new form of technical debt, one that is less visible than the traditional kind and harder to unwind. It’s also preventable, and the organizations that get ahead of it will have a meaningful advantage over those that don’t.

Understand what makes this debt different

The standard narrative frames this as AI writing bad code. That’s not quite right. The more precise problem is cognitive debt: the loss of understanding of how and why software was built the way it was.

When a human writes code, something else happens alongside the typing. They simulate edge cases, reason through dependencies, and make judgment calls grounded in organizational context, including the business requirements behind a feature, the best practices the team has established, and the reasoning behind past architectural choices. That cognitive loop is how institutional knowledge gets built. When AI writes the code, you can get output that is syntactically correct, passes CI, ships cleanly, and leaves no one holding the mental model. The code works until something changes or breaks, and then the team is excavating a black box.

This is distinct from traditional technical debt, which is messy code. Cognitive debt is invisible code that functions but that nobody truly owns. And it compounds faster, because the same velocity that makes AI generation attractive is what prevents anyone from stopping to build the understanding that maintainability requires.

GitClear’s analysis of 211 million changed lines of code across major repositories found that during 2024, duplicate code blocks of five or more lines increased eightfold, while refactoring dropped from 25% to under 10% of all code changes. Refactoring is the slow, unglamorous work that keeps codebases healthy, and developers are doing far less of it. Google’s 2024 DORA report found that a 25% increase in AI adoption correlates with a 7.2% decrease in delivery stability. DORA analysts note that the root cause isn’t flawed code per se; AI inflates batch sizes, and larger changesets have always been riskier to ship.

These findings aren’t indictments of AI-assisted development. They’re diagnostics, and they point toward a specific set of fixes.

Close the context gap first

In a survey of 609 developers we conducted last year, 65% said AI misses relevant context during critical tasks like refactoring, writing tests, or reviewing code. Context is the primary driver of AI code quality, and it’s where most enterprise organizations are underinvesting.

When an AI tool generates code without access to your organization’s architectural decisions, historical pull requests, security policies, or existing module patterns, you get solutions that are locally correct but globally incoherent. Closing that gap requires context engineering: ensuring the tools and agents you use have access, at the right moment, to the right organizational knowledge, and the judgment to determine what is actually relevant for a given task. A retrieval system that surfaces too much irrelevant context can degrade output quality as readily as one that surfaces too little. The specific tooling matters less than the discipline. Context infrastructure needs to be actively maintained, not indexed once and forgotten.

Build this infrastructure before you scale AI generation. Retrofitting is significantly harder. Treat it the way you treat your CI pipeline, as a prerequisite for safe production deployment.

Consider what happens when a team has built this context infrastructure well. Their code review tooling knows about a deprecated internal API, because that deprecation decision lives in months of past pull request discussions that have been indexed and surfaced. When generated code references the old API, the review flags it. Without that context layer, the same mistake gets waved through every time. That’s the kind of institutional knowledge that evaporates when humans stop writing every line of code, and that you have to actively work to preserve.

Build a verification layer that matches your generation velocity

Almost all of the investment in AI-assisted development has gone into generation. Very little has gone into verification. That imbalance is where the tech debt accumulates.

I think of these as the blue team and the red team. The blue team covers code generation, autocomplete, and agentic coding. It’s getting the headlines, the budgets, and the product launches. The red team covers integrity checks, behavior coverage, and alignment with organizational standards. In most organizations, it’s an afterthought. A CI pipeline catches obvious failures. A code review might happen, but reviewers are overwhelmed by the volume of AI-generated output and cannot meaningfully evaluate all of it. The result is code with a veneer of having been reviewed without anyone having actually understood it.

The Crowdstrike outage of 2024 is worth keeping in mind here. AI didn’t generate the problematic code, but the incident illustrated what happens when a single software error propagates through production systems without sufficient verification. That exposure multiplies when code is being generated faster than humans can understand it.

A real verification layer means automated analysis that evaluates whether generated code aligns with your organization’s best practices, architectural standards, and compliance requirements. It means test coverage that reflects intended behavior, not only the happy path the AI chose to generate tests for. And it means traceability: a connection between the requirement and the implementation, so that six months from now, someone can understand what the code does and why it exists.

The numbers support investment here. In the same developer survey, teams that integrated AI into their code review workflow saw quality improvements in 81% of cases, compared to 55% for comparable teams without it. 

Make ownership non-negotiable

Every piece of AI-generated code in production needs an accountable human who understands it well enough to maintain it. This is harder than it sounds, and it’s where most organizations are falling short.

The same velocity that makes AI generation attractive also creates pressure to skip the slow work of genuine comprehension. A developer reviews a 500-line pull request that an AI generated in three minutes and faces a real choice: spend two hours actually understanding it, or approve it because it looks right, passes the test, and “LGTM” (looks good to me). 

Real ownership means slowing down generation velocity enough to allow for meaningful review, and being explicit with your team that this is the right trade-off. When that doesn’t happen, you’ve started building your next legacy system.

What to do this quarter

The good news is that none of this requires a multi-year transformation. The structural problems are real, but they have concrete solutions, and engineering leaders can make meaningful progress on all three fronts without waiting for the next budget cycle.

• Audit your context infrastructure. If your AI tools are generating code without access to your organization’s architectural decisions, deprecated APIs, and security policies, fix that before expanding generation velocity further. The quality of context determines the quality of output.
• Invest in the red team. Establish an automated code quality and governance layer that evaluates generated code against your specific organizational standards, beyond functional correctness. This is a distinct investment from your code generation tooling and needs to be treated as such.
• Establish ownership explicitly. Every AI-generated feature in production should have a human owner who genuinely understands it. Make that a formal requirement.

The organizations that get this right will find that AI generation becomes far more reliable once it has a verification layer underneath it. The ones that don’t will keep shipping faster while understanding their systems less, until the accumulated debt forces a reckoning. That’s a solvable problem. The question is whether you solve it now or later.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

The problem with software development today may no longer be writing code. With AI coding assistants generating code faster than ever, the bigger challenge is reviewing, testing, and safely releasing it.

AWS is betting that software teams need help with that part of the process, adding release management features to its DevOps Agent.

The new features, currently in preview, automatically assess code changes against organizational standards, identify potential release risks, and generate tests tailored to individual changes before they reach production.

The release readiness feature, in particular, runs the code in an AWS-managed isolated environment, executing lightweight user journey tests to verify the software builds, runs, and passes basic functional checks before the change enters the pipeline, the company wrote in a blog post.

The findings of these tests can be viewed through the DevOps Agent console, as comments on pull requests in GitHub or GitLab, or can be invoked directly through IDEs via Kiro or the Claude Code plugin, it added.

Targeting AI-era software delivery bottlenecks

Running code in isolated environments and delivering the results directly through developer tools helps address two longstanding challenges in software delivery, said Pareekh Jain, principal analyst at Pareekh Consulting.

It enables teams to validate how code changes behave before deployment, catching issues that static analysis may overlook, while reducing context switching by embedding findings into existing workflows, in turn accelerating fixes, Jain said.

The analyst pointed out that the release readiness capability addresses a key bottleneck in AI-driven software development: “While AI coding agents can generate code quickly, reviews, compliance checks, dependency validation, and release approvals still slow deployment.”

“By automatically checking code changes against internal standards, security policies, and dependency impacts, AWS helps developers, DevOps teams, and SREs identify issues earlier, reduce manual review effort, and improve release confidence,” Jain added.

These gains in productivity for developers could also translate into tangible business benefits for CIOs, according to Jain.

“Release readiness as a feature could help enterprises capture more value from AI-generated code while reducing operational overhead by eliminating the need for additional QA and DevOps resources. This means that they can accelerate software delivery without sacrificing reliability,” the analyst noted.

Autonomous testing before merging code

While release readiness reviews focus on assessing whether a code change is safe to move through the delivery pipeline, AWS is also adding a separate feature aimed at validating how those changes behave in production-like environments.

Named autonomous release testing, the new capability generates and runs change-specific test plans for web and API-based applications in customer-provisioned, production-like environments before the change actually merges, the company wrote in the blog post.

For Jain, autonomous release testing is “even more” important for developers and SREs as it “automates one of the most time-consuming parts of software delivery.”

“Developers spend less time creating and maintaining tests, while SREs benefit from fewer rollbacks and improved system reliability,” Jain said.

These benefits stem from the feature’s ability to automatically generate tests tailored to individual code changes, covering functional correctness, behavioral regressions, and integration scenarios that might otherwise require significant manual effort, Jain added.

However, AWS is not alone in trying to bring AI deeper into the software delivery lifecycle.

Microsoft-owned GitHub has been expanding Copilot’s code review capabilities, allowing the service to automatically review pull requests, suggest fixes, and provide feedback directly within developer workflows.

Google, meanwhile, has been steadily broadening the scope of Gemini Code Assist beyond code generation to support software development tasks such as code review and developer assistance.

AWS’s differentiation, though, according to Jain, lies in tying those capabilities to release management and operational workflows that span development and production environments.

Availability and pricing

For development teams interested in evaluating DevOps Agent’s new capabilities, AWS said both features are available in preview at no additional cost in the US East (N. Virginia) region.

AWS DevOps Agent, billed per agent-second, is included in the AWS Free Tier for new customers.

Additionally, new AWS DevOps Agent customers receive a 2-month free trial starting with their first operational task after general availability.

Each trial month includes up to 10 agent spaces, 20 hours of investigations (incident response), 15 hours of evaluations (incident prevention), and 20 hours of on-demand SRE tasks (chat), the company said.

Once those limits are exhausted, customers are charged based on consumption, with investigations, evaluations, and on-demand SRE tasks each priced at $0.0083 per agent-second, AWS added.

A prerequisite for using the new release management features includes connecting at least one GitHub or GitLab repository to an AWS DevOps Agent Space.

Z.ai has released GLM-5.2, an MIT-licensed open-source AI model designed for long-running software engineering tasks, as the Chinese company seeks to challenge proprietary coding models on cost and performance.

The company said GLM-5.2 ranked just behind Anthropic’s Claude Opus 4.8 on FrontierSWE, a long-horizon coding benchmark, trailing it by 1%. Z.ai said the model also edged out OpenAI’s GPT-5.5 by 1%.

Z.ai said GLM-5.2 supports a one million-token context window with up to 131,072 output tokens, positioning it for agentic coding workflows that require reasoning across large codebases.

The company is also making an efficiency argument. It said GLM-5.2 uses a technique called IndexShare, which reduces per-token compute by 2.9 times at a one million-token context length. It also said changes to the model’s multi-token prediction layer increased the acceptance length for speculative decoding by up to 20%.

The changes are aimed at a practical problem for developers: long-context coding agents can be expensive to run when they are asked to work across large repositories.

Enterprise appeal

GLM-5.2’s clearest appeal is that it pairs stronger coding capabilities with the cost advantages of an open-source model. But capability alone will not be enough to make it a credible alternative.

“Western enterprises will want independent benchmark validation, successful deployments at global enterprises, strong security and governance controls, and long-term support commitments,” said Pareekh Jain, CEO of Pareekh Consulting.

Jain said the fastest route to enterprise credibility would be hosting by a major cloud provider like AWS. That would allow customers to use the model under standard enterprise terms, with service-level commitments and compliance certifications.

Tulika Sheel, senior VP at Kadence International, said GLM-5.2 would also need to prove it can operate as a stable enterprise product.

“Demonstrated success in real-world deployments and transparent governance will be just as important as benchmark scores,” Sheel said.

The performance and cost claims will also need to hold up against established models.

“Enterprise leaders generally consider two major factors when evaluating new models,” said Lian Jye Su, chief analyst at Omdia. “First, they look at overall performance against competitors, where GLM-5.2 performs well in long-horizon agentic coding and software engineering. Second, they look at the cost of adoption. As an open-source model, GLM-5.2 has clear cost advantages.”

Su said the model could appeal to engineering teams under pressure to control AI costs. It may also attract open-source advocates and companies with significant operations in Asia-Pacific.

But the claims still need wider validation, particularly around hallucination control and coherence during extended tasks. These are critical issues for enterprises considering AI coding agents, which may need to work across large codebases and multi-step software engineering workflows.

Jain said the one million-token context window could be useful for large codebase analysis. It could also help with legacy modernization projects and complex engineering documentation.

He said long-context capability may also help with audit logs or legal contracts, where splitting material into smaller chunks can create errors across document boundaries. But for everyday coding tasks, effective retrieval systems may matter more than very large context windows, making some of the benefits more limited in practice.

Governance risks

The governance question depends largely on where the model runs.

Sheel said enterprises should evaluate GLM-5.2 as they would any strategic technology partner, rather than as a standalone model. That means looking at where data is stored and whether the model can be used in environments customers control.

That deployment choice is central to the risk calculation, according to Jain. Because GLM-5.2 is available under an MIT license, companies can download the weights and run them on their own infrastructure, reducing the need to send sensitive data to Z.ai.

“The risk flips completely if you use Z.ai’s hosted API instead,” Jain said.

He said Chinese national security rules could require domestic companies to cooperate with government requests, making hosted use difficult for regulated industries or workloads involving sensitive data.

Su said the issue is not limited to Chinese vendors. Recent restrictions affecting access to some Anthropic models have also highlighted the risk that enterprises may have limited control over the availability of AI services from foreign providers.

“Selecting solutions from American and Chinese AI vendors does expose non-US Western enterprises to additional risk of having zero control over the availability and uptime of these models,” Su said.

When SpaceX on Tuesday officially announced its plan to purchase AI coding startup Cursor for $60 billion in stock, as it had predicted it would do in April, it presented CIOs and developers with a little good news, a little bad news and a massive pile of uncertainty. 

The details of the proposed acquisition were virtually identical to the terms announced in April, even retaining the $10 billion consolation prize for Cursor should SpaceX back out of the deal.  

But for CIOs and developers, the increasing probability of the deal happening is forcing them to make long-term decisions without many long-term answers about what is likely to happen to Cursor, which says its coding agents are used by 64% of Fortune 500 companies.

But whether the deal is good news for Cursor’s customers, or even those of its rivals, is an open question.

Arnal Dayaratna, research VP for software development at IDC, is firmly in the good news camp. He argued that the main element holding back the company, which he estimated brings in about $2 billion in annual revenue, was access to GPUs. And, he said, SpaceX’s xAI has the ability to resolve that problem.

It’s all about the GPUs

“The implication is that Cursor can get better because it will get better access to more compute, in the form of GPUs. That was what Cursor was struggling to obtain,” Dayaratna said, adding that Cursor joins both Anthropic and Google in aligning with xAI. 

“It is bad news for Anthropic,” he said, pointing out, “[Elon] Musk figured out that GPUs were the limiters, the bottleneck. Not human talent: compute. And it’s not only about GPUs. It’s about the data centers,” as well as the mechanisms needed to support data centers, including electricity access, cooling apparatus, and zoning rights.

“Zoning rights are very difficult to obtain in the United States, and Elon has been able to leverage his political capital” to obtain those rights, Dayaratna said.

Dayaratna also commented on the highly unusual tactic of a company pre-announcing an acquisition and then announcing it “officially” two months later. The reason, he suggested, is that Musk didn’t want the acquisition to delay the SpaceX IPO, but he did want potential investors to know about it. “It gave the IPO more legs, even though it was not formally a part of the IPO. He wanted to socialize it before it happened,” he said.

Jason Andersen, principal analyst at Moor Insights & Strategy, took a far more cautious approach and predicted, “enterprise customers will be very concerned.”

Time for CIO due diligence

“xAI’s models and treatment of guardrails are very different than what Cursor has stood for,” Andersen said, and the key issue involves model choice. “Will Cursor be able to point at models other than Grok? If that is the case, it could go towards helping customers be OK with things for a while,” he said. 

But Andersen stressed the words “for a while.” He argued that Cursor rivals have gotten more sophisticated, which could give enterprise CIOs more alternatives.

“The tooling options from the classic enterprise vendors have gotten really good, and also have a number of features that enable better governance and teaming. For example, Kiro from AWS is already a worthy competitor to Cursor, but there have also been major improvements from Google and Microsoft this year,” Andersen said. “I’d venture to say that Cursor was already facing a substantial competitive challenge in the enterprise, especially as the vibe coding movement lost some steam towards agents that enable so much more developer capability.”

Andersen added, “A lot of SpaceX’s valuation is pinned to xAI and there wasn’t enough IP there for it to be long-term competitive with Anthropic or OpenAI without rapidly injecting new technology.”

Shashi Bellamkonda, principal research director at Info-Tech Research Group, said the current period of due diligence gives CIOs opportunities to ask difficult questions.

“I do believe that enterprise customers will start to ask harder questions now, especially if more of the processing starts moving through Grok or xAI infrastructure,” Bellamkonda said, but he stressed that he does see a SpaceX deal helping Cursor.

“Cursor is already carrying a lot of LLM cost and if SpaceX and xAI can give it access to more compute and a faster, improved Grok model, customers could eventually get better performance at a lower cost. Grok is getting faster and more accurate, and under SpaceX there may be more enterprise appetite for xAI than xAI had on its own,” Bellamkonda said.

Questions about data privacy

But Bellamkonda also pointed to the trust issue, specifically asking whether Cursor would be allowed to continue its zero-data-retention policy under the new ownership.

“Enterprise users with sensitive codebases will want to understand exactly where their data is going, who has access to it, and whether any prompts, code, metadata, or embeddings are touching SpaceX or xAI systems,” Bellamkonda said. “Zero data retention is only valuable if customers believe the controls still apply across the new infrastructure. If they can preserve Cursor’s privacy posture while using xAI and SpaceX compute to improve the product, this could be a very strong outcome for customers. If the data governance is unclear, enterprise buyers will hesitate.”

Justin Greis, CEO of consulting firm Acceligence, echoed Bellamkonda’s concerns.

“For many enterprise customers, Cursor’s zero-data-retention policy was not simply a security feature. It was a foundational part of the procurement and approval process,” Greis said, pointing out that security teams, legal departments, compliance officers, and executive leadership all became more comfortable adopting AI-assisted development because they believed their source code, prompts, and proprietary intellectual property would not be retained, stored, or repurposed.

“Whenever ownership changes, customers naturally revisit those assumptions,” he said. “I would expect enterprise customers to immediately seek clarity on whether existing zero-data-retention commitments remain unchanged, whether those commitments continue to be contractually enforceable, and whether any future modifications could be introduced through changes to terms of service or product strategy.”

Reassessing platform risk

Another concern is whether the move will create greater vendor concentration, and whether that will increase risk exposure. 

“Many organizations are already concerned about becoming overly dependent on a small number of AI providers. If Cursor becomes more tightly integrated into a broader SpaceX technology ecosystem, some enterprises may worry about reduced flexibility and fewer alternatives. I suspect this will be a continued question with SpaceX’s recent IPO and growth plans,” Greis said.

Additionally, he pointed out, “AI coding platforms are rapidly becoming part of critical software delivery infrastructure. When organizations standardize on these tools, they are making a long-term platform decision that influences developer productivity, software quality, security processes, and engineering workflows. A change in ownership naturally prompts a reassessment of platform risk, roadmap alignment, and long-term strategic direction.”

As well, Sanchit Vir Gogia, chief analyst at Greyhound Research, noted, “zero data retention survives only where it stays contractual, auditable, enforceable and fenced from affiliate use. The deeper point is what Cursor has become. It is no longer a developer convenience. It sits inside the act of software creation, close to the intellectual-property bloodstream of the enterprise. That is a control plane, and control planes change hands rarely and consequentially.”

Gogia added that even before an acquisition, Cursor was already weakening its data guarantee. “Even now the promise is layered. Cursor’s own disclosures show that standard Privacy Mode still permits some code data to be stored for product features, while only the stricter legacy setting retains nothing. Zero is the exception, not the default.”

Developers are caught between the joy — or pressure — of using agents to ship 10x faster today and the dread of how they will maintain that code tomorrow. The gap between “vibe” code and code that can be deployed to millions of users is vast and easy to underestimate. Closing the gap requires care, expertise, and effort, with the payoff coming later. Agents are able to complete increasingly complex programming tasks but without the quality we need. What’s missing, and how can we fill the gap?

Why agent-generated code degrades: the bloat problem

Enterprise code has to clear three bars: it must be maintainable, reliable, and secure. Out-of-the-box AI agents can miss all three. Let’s focus on the biggest and most visible maintainability issue, which is bloat: redundant validation, defensive checks that cannot fire, near-duplicate functions, dead code that nothing removes. A None check on a parameter typed as dict. A try/except around a call that never throws. Two functions, identical except for the negation in their return statement.

Bloat varies dramatically by model. Sonar’s LLM Leaderboard runs every frontier model through 4,400+ Java tasks and analyses the code generated. To complete the benchmark, GPT-5.4 High generated 1,159,000 lines of code at an 81.05% pass rate, while Claude Opus 4.7 Thinking generated only 336,000 lines of code to return a better than 82.52% pass rate. Different models generate dramatically different code to achieve similar outcomes.

Bloat is not just messy. Carnegie Mellon researchers studied 807 open-source projects that had adopted Cursor, matched against 1,380 controls, measured by SonarQube. A short-term velocity gain disappeared by month three, while static analysis warnings rose 30% and code complexity rose 41% — both persistent. The harder it became to change the codebase and the more bugs it contained, the more the velocity was dragged down. Any experienced developer will know how this goes: quality problems compound until the code feels impossible to change and the only option is the dreaded rewrite.

Three forces produce bloat once a model is in use:

1. Agents do not feel the maintenance burden. Armin Ronacher, the creator of Flask, made the point on the Pragmatic Engineer podcast in late April. Humans feel the cost of bad code over time, and as Ronacher put it, “if the pain gets too big, you as a human are incentivized to fix the cause of your pain” — so we refactor. Agents do not. They obliviously extend bad structure indefinitely. A senior engineer’s job is to say no to unnecessary abstraction. The agent has no equivalent reflex.
2. Training rewards apparent completeness. Pretraining corpora are full of explanatory material — Stack Overflow answers, tutorials, README snippets — deliberately self-contained and verbose. Post-training compounds the effect: human raters prefer outputs that look thorough, so models learn that “comprehensive” reads as better. When uncertain which edge case matters, the safe move is to handle all of them. Each guard is locally defensible. The aggregate is bloat.
3. Iterative generation has no deletion pressure. Agents add but rarely delete. Removing dead code does not make any test go green, so superseded functions accumulate alongside their replacements. SlopCodeBench, a March 2026 benchmark across 11 coding models, found rising structural complexity in 80% of trajectories and rising verbosity in 89.8%. Agents continue to patch bad code, treating every task as if it’s their last.

AC/DC: the loop that compensates

What closes the gap is a loop around each iteration of agent work. The agent does what it is good at — generating code — and our job is to wrap that with three steps the agent cannot reliably do on its own. At Sonar we call this the Agent Centric Development Cycle, or AC/DC: guide, verify, solve.

Guide

Many teams overcorrect on context. They paste the style guide, three years of architectural decisions, and the entire onboarding doc into the agent’s instructions and expect output to improve. ETH Zurich researchers tested this and found the opposite: large context files often reduced task success against no context at all, and added 20% or more to inference cost.

Keep agent-facing context short — under 200 lines is a useful heuristic — and restrict it to fundamentals that can’t easily be inferred from the code: naming conventions, architectural invariants, what has been tried and failed. However, this will only get you so far, so make sure to provide specific context for each task. If you have architectural expectations, don’t expect the agent to guess them. Software architecture tools can be used to provide additional context in the guide phase.

Task shape matters too. Break the work into steps and agree on a plan; ask the agent to provide three solutions and evaluate the impact on quality of each. There is no perfect software architecture, and you understand the trade-offs in your codebase best, so think critically about the changes before they happen. Without this, the agent will confidently pick an option, seemingly at random, and the further it goes the harder it is to “unpick.” If you want to test this, ask three instances of your preferred agent to complete a task that involves some polymorphism and watch each one confidently suggest a different solution.

Verify

The most expensive verification mistake is doing it last. Reviewing 200-line pull requests (PRs) after the agent is done is the dynamic behind the Faros/DORA figures Addy Osmani highlighted: 98% more PRs merged in high-adoption teams, review times up 91%. Verification inside the loop is different. Unit test runs, static analysis, and security scanners produce output the agent can act on. This is where AI-native tooling belongs: purpose-built for the agent to invoke, not just for humans to consult through a UI.

Human reviewers cannot keep up. When agents merge twice as many PRs per week and each one takes nearly twice as long to review, doubling the review staff still leaves you behind. Automated verification is the only response that scales. Fast feedback has always been a fundamental tenet of good software engineering. Feeding it directly back to the agent protects the developer from simple mistakes and leaves them headroom to work on the harder ones.

Solve

If verification happens within the agentic loop, the agent can fix any issues whilst the code is being generated without expensive remediation steps. Static analysis tools can guide the agent on how to resolve the issue quickly. Some cases need human judgment — a None check at a system boundary may document a real precondition. But most of the work is mechanical. Automate the obvious fixes and let engineers spend their attention on the cases that are not.

The investment that compounds

Better models will keep arriving. They may not change the mechanism of bloat or the dynamics of compounding decay. The loop is what does — bounded tasks, sharp context, in-loop verification, and a deliberate “solve” step to clear bloat before it accumulates.

The same logic governs how autonomy should expand. Reduce human interventions only when the agent’s guide, verify, and solve cycle is making them redundant. Our biases can sting us here: an agent’s ability to write code can lead us to agree with it more than we should. Don’t trust blindly; wait for the evidence.

The teams that will be shipping enterprise-quality code with AI agents in 18 months are not the ones running the “best model.” They are the ones treating workflow as the engineering investment, with the seriousness once given to build systems and CI. The model is the tool, the workflow is the discipline. That is where the durable advantage compounds.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

CIOs and CISOs have many strategic and operational fears when it comes to unleashing fully-autonomous agents on tasks and hoping that everything works out. Will the agent start to delete critical files? Will the agent go off on a mission tangent and generate a massive token bill for the team when they return the next morning? Will it be tricked by a state actor and engage in malicious actions?

To help alleviate those concerns, OpenAI announced on Thursday that it has agreed to acquire Ona, a 79 person cloud development environment (CDE) provider formerly known as Gitpod, to accelerate its efforts to make agentic AI enterprise-friendly. 

An OpenAI statement said Ona’s technology “provides secure, persistent environments where agents can access the tools, systems, and context they need to make progress over time. By bringing Ona to OpenAI, we will expand Codex beyond work tied to a single device or active session and help more organizations deploy agents securely in production.”

An Ona statement attributed to CEO Johannes Landgraf shared similar sentiments.

“Ona brings the building blocks agents need for enterprise work: trusted, customer-controlled cloud environments where work continues across devices, inside the systems where software actually lives,” Landgraf said. “OpenAI brings frontier intelligence, product polish, and a scale of research and distribution we could never reach alone.”

Landgraf’s statement did not provide any annual revenue numbers, but did hint, without naming, at some large customers. “Since the beginning of the year, weekly Ona agent sessions have grown 13x in production across some of the world’s most demanding institutions: the oldest bank in the US, one of Europe’s largest pharma companies, one of Asia’s largest sovereign wealth funds and many others,” he wrote. “The largest enterprises out there love the platform and are expanding more rapidly than ever before.”

Arnal Dayaratna, research VP for software development at IDC, said IDC’s figures for Ona put its annual revenue for 2025 at “roughly $7 million.” He speculated that Ona’s revenue for 2026 would be higher: “Let’s say it’s $15 million. I am being generous. Maybe it’s really $10 million or $12 million.”

Dayaratna said if he uses a standard acquisition price of roughly a multiple of 30 times revenue, then depending on the actual 2026 figure, “that comes to $450 million or $500 million or so.”

But IDC sees this being a potentially good move for OpenAI, regardless of the specific acquisition price, given that OpenAI had the classic “buy or build” challenge. 

OpenAI has a substantial Codex effort, Dayaratna said, but what they lack is a safe area to protect enterprise autonomous agent efforts. “This is outside of what OpenAI has now. These are secure environments where agents can have memory and operate securely,” he said. “This is the kind of technology that one would expect to be needed, but I don’t know how good it is, to be honest.”

Gartner’s First Take, published today, noted that the acquisition will bring Codex “the essential scaling capability it lacked,” but also pointed out it forces some difficult decisions on enterprises: “Software engineering leaders must weigh the benefits of a vendor-specific integrated stack against the flexibility of staying vendor-agnostic.”

In addition, Gartner wrote, “This acquisition appears to be OpenAI’s response to Anthropic supporting self-hosted sandboxes in Claude Managed Agents, starting May 2026.”

Tom Findling, CEO of Conifers.ai, said he also sees OpenAI’s fear of Anthropic playing a meaningful role in this deal. 

“It feels like a move to keep pressure on Anthropic, especially as Claude Code gains traction with developers and enterprise buyers,” he said. “So I’d read this less as OpenAI taking out a small competitor and more as OpenAI trying to make sure Codex is enterprise-ready before Anthropic gets too far ahead. In the enterprise market, the battle is not just who has the smartest coding model, but who can make AI agents safe and useful enough for big companies to actually deploy.”

He added, “I don’t think this means OpenAI suddenly needs help making Codex better at writing code. The bigger issue is making Codex work inside real enterprise environments, where security, access controls, persistent cloud workspaces, audit trails, and integration with existing developer workflows matter just as much as the model itself. Ona gives OpenAI some of that missing plumbing.”

Jason Andersen, principal analyst for Moor Insights & Strategy, echoed the concerns about Anthropic.

“To be honest, I think it reinforces what I think, which is that OpenAI and Codex have given a lot of ground to Anthropic and Claude Code, who are winning right now,” he said. “But again, this is not about the market today, I think it’s about how OpenAI will need to position itself as more than just a model as we see the incumbent players, particularly Microsoft, bolster their enterprise coding infrastructure story.”

Andersen said that Moor doesn’t have any strong basis for a guess on the financials, but added, “I am going to assume it was a fairly high multiple, but on a small base. I would not speculate on an amount, but given the enterprise customers that Ona did have, it may be more than we think.”

He also reinforced the idea that OpenAI is going to need help to achieve its own objectives.

“We continue to see that AI adoption is strongest in coding, and other use cases are not as far along,” he said. “So, if you are a general-purpose AI company like OpenAI, you need to double down on development use cases. The meaningful investment and spending on development is happening at the enterprise level, and those customers have more demands for governance, security, etc. than Codex or Claude code can handle.”

That said, he noted, “what you’re seeing is traditional software and cloud plays building out the coding and ops infrastructure around the popular models. That increased competition, while good for selling tokens, is still keeping OpenAI and Anthropic on the outside looking in. So, OpenAI and Anthropic need a stronger enterprise dev story, or they are just another model that could be easily replaced.”

Jeremy Roberts, senior director at Info-Tech Research Group, said that he also sees this as likely a good move for OpenAI.

“OpenAI is growing up a little bit,” and they may be falling behind Anthropic, Roberts said. “I see Ona as a boring company, but not in a bad way. They are not flashy, but absolutely necessary.”

Ona is delivering a workspace for Codex that an enterprise can run in its own virtual private cloud, with governance and persistence and an environment where the company can apply their own controls including log management, credential management and resource access, he said. “It is a bucket for the agents to operate in” where IT can “make sure that access is properly credentialed and is controlled effectively to prevent the model doing what it shouldn’t be doing,” which includes managing read/write protections.

Microsoft has open-sourced an AI evaluation framework that converts natural-language requirements into executable tests, expanding its push into enterprise AI governance as organizations struggle to validate agent behavior before production deployments systematically.

The framework, called ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), generates evaluation scenarios, datasets, metrics, and scorecards from written specifications, product requirements, and governance documents, Microsoft said in a blog post announcing the release.

“Agents fail in ways that are hard to see,” Microsoft wrote in the blog post. “They drift from policy, produce unsafe outputs in edge cases, and behave differently in production than they did in testing. Generic benchmarks do not catch these failures because they are not built around your policies, your agent, or your use case.”

Rather than requiring developers to manually create evaluation suites, ASSERT translates written intent into reusable tests that can be integrated into AI development pipelines, the company said in the blog post.

With ASSERT, Microsoft is entering an increasingly competitive AI evaluation market that already includes platforms such as LangChain’s LangSmith, Braintrust, Patronus AI, Galileo, Arize AI’s Phoenix, and Promptfoo, which help enterprises benchmark, monitor, and validate large language model applications.

Behavioral testing remains immature

The release comes as enterprises rapidly expand AI agent deployments while formal evaluation practices remain the exception rather than the rule.

“Most organizations, in fact, 99% of them, do not evaluate any AI agents pre-production,” said Anushree Verma, senior director analyst at Gartner.

According to Verma, the industry’s next competitive advantage will depend less on advances in reasoning models than on how effectively organizations simulate and stress-test AI agents before deployment.

“The next competitive moat in agentic AI is not about the sophistication of reasoning models or the underlying architecture,” she said. “It will be about the depth and realism of the training environment through agentic simulation, particularly for mission-critical deployments.”

Gartner estimates that by 2029, more than 75% of domain-specific agents designed without agentic simulation in regulated industries will fail to deliver value.

Forrester sees enterprises moving toward behavioral evaluation but says most organizations have yet to make it a formal production requirement.

“Most enterprises are still in an intermediate stage where behavioral evaluation is inconsistently applied rather than treated as a formal production gate,” said Biswajeet Mahapatra, principal analyst at Forrester.

According to Forrester data, more than 45% of organizations are already using AI agents, and another 25% are piloting them, yet many continue to struggle with scaling because of immature governance and limited operational rigor.

“The net is that behavioral evaluation is becoming important, but for most organizations it is still ad hoc or tool-driven rather than a standardized release gate enforced across the lifecycle,” Mahapatra said.

AI judges still need human oversight

Microsoft said ASSERT uses large language models as judges, with model-generated evaluations agreeing with human reviewers 80% to 90% of the time in the company’s internal validation.

That level of agreement can help automate large portions of AI testing, but should not be treated as a standalone governance mechanism, Mahapatra said.

“An 80% to 90% agreement rate with human reviewers indicates strong alignment but is not sufficient as a standalone control for governance or compliance,” he said.

Instead, enterprises should adopt layered oversight where AI evaluates AI at scale while humans retain supervisory accountability for high-risk, regulated, or ambiguous scenarios. Buyers should also watch for bias, consistency issues, and overreliance on a single model acting as both generator and evaluator, he added.

Open source reduces lock-in, not governance risk

Microsoft released ASSERT under the MIT open-source license, allowing organizations to inspect, modify, and integrate the framework into existing AI development workflows.

But open sourcing a framework does not eliminate questions around evaluation neutrality, Mahapatra said.

“Open sourcing under an MIT license reduces lock-in concerns and enables broad interoperability across model ecosystems,” he said. “However, it does not fully eliminate trust or conflict-of-interest questions because the originating vendor still influences how evaluation criteria, scoring logic, and definitions of acceptable behaviour are encoded.”

Instead of relying on a single evaluation framework, enterprises should validate AI systems against multiple evaluation approaches and retain ownership of internal evaluation policies, he said.

Modern AI systems have evolved beyond the simple chatbots that quickly became popular. Now they use semantic tools to manage workflows and link machines to machines, providing a flexible and effective framework for the next generation of business automation. What you used to build in Microsoft’s Power Platform or construct inside Biztalk is now an agent, built around large language models (LLMs) that can parse both your data and the APIs that you want to use your data with, orchestrating workflows with a level of autonomy that traditional tooling can’t match.

That shift has offered new opportunities, much like those that came with business platforms like Microsoft Dynamics and Salesforce. Here, tools built to solve one set of business problems could be turned into applications that could be sold to other companies. What worked for you to solve one of your problems could now be an added revenue stream, sold through platform marketplaces that helped customers manage installations and customizations.

Agents are business applications now

Modern agents are much like those business applications. Often developed to solve a specific need, but quicky adopted by organizations and refactored to apply enterprise standards (using tools like the Agent Governance Toolkit and frameworks like Microsoft’s Agent Framework), they’re rapidly maturing and are ready to be shared more widely. The process of sharing needs to be curated and controlled, and, if possible, tied to a revenue stream.

There’s certainly some urgency here. Until recently, subsidized tokens have kept costs artificially low. Now companies like GitHub and Anthropic are moving to a more sustainable (for them) pricing model, increasing the cost of inferencing and squeezing companies’ AI budgets. As a result, switching AI projects away from a cost to a revenue source is high on CIOs’ agendas. If those tuned and trained agents can be sold on a marketplace, then that token budget can be justified.

Microsoft has always been a company built on partner relationships, starting with individual developers and working all the way up to the largest software companies and consultancies. That reach is key to helping partners extract as much value as possible from their agents, as it allows Microsoft to integrate its partner sales tools into its own products and services, as well as into other platforms.

Extending the Microsoft Marketplace for AI developers

We’re already familiar with many of Microsoft’s marketplaces, built into individual tools like Teams, into platforms like Microsoft 365 or Visual Studio, or into the Windows Store. Now the company is doing the same for AI developers, extending Microsoft Marketplace to software agents. Announced at Build 2026, the updated Microsoft Marketplace provides ways to publish code—apps and agents—developed across all of Microsoft’s development platforms, including Copilot Studio, opening the marketplace up to traditional and non-traditional developers alike.

Perhaps the most important aspect of this new Marketplace is its own intelligence, using context to expose your code to the right audience. If you’ve developed an agent for use with Microsoft 365, it will be exposed inside the Microsoft 365 Copilot Agent Store; for Visual Studio, in the Visual Studio Marketplace; or for Teams, in the Microsoft Marketplace. All of these are different views on the same back end, using AI to ensure that the relevant agents are displayed.

Replacing search with Intelligent Discovery

This is extended by another new service, Intelligent Discovery, which adds natural language support to search, using AI to infer user intent and highlight the most relevant tools. Building on the familiar metaphor of the search bar, the initial smart search model offers a freeform way to explore the Marketplace. While there are suggested prompts, they’re not necessary. The search tooling allows you to generate comparisons between tools using your own criteria, with the Marketplace AI generating views based on your requirements.

Microsoft’s aim here is to shift discovery from keywords to use cases, so that buyers can quickly get the tools they need without having to evaluate different solutions, before completing a purchase. By handing that aspect of the buying process over to Marketplace’s AIs, customers can go straight to trials or even to buying agents and applications.

For a tool like this to be successful it needs to be trustworthy. By building it on top of the same development frameworks as your agents, Microsoft can take advantage of the AI guardrails built into Microsoft Foundry as well as low-level tooling like the Agent Governance Framework. Restricting the intelligent search to the Marketplace catalogue reduces the risk of hallucination, as output is grounded in Marketplace data and metadata.

A developer-friendly marketplace

Microsoft is providing tooling to help developers get their listings right. According to Cyril Belikoff, Microsoft’s vice president of Commercial Cloud and AI, “We actually have a separate AI tool that we give to software companies to optimize their listings, called a listing optimizer, funny enough, and that listing optimizer reviews their listing and then provides them with particular guidance on how to best improve it, so that it can be best discoverable in today’s search world.”

You can expect the listing optimizer to be tuned to work with the new Marketplace tooling, but for now it still focuses on traditional search. As Marketplace is a B2B platform, there’s a lower risk of spam applications, but even so, Microsoft remains aware of the possibility of a new system being gamed, and will be rolling out Intelligent Discovery carefully, monitoring its performance as more customers get access over time.

Having a new discovery method is one thing; getting quality AI applications in the Marketplace is another. Microsoft is validating all code submitted, though the criteria will differ between target platforms. An agent built for Teams will be treated differently than one built on Microsoft 365’s WorkIQ. It’s an approach that allows Microsoft to support new standards as they become available.

Alongside its agent development tooling, Microsoft is rolling out a new set of guidelines and processes to help developers get ready to sell their agents. Hosted on GitHub, these offer code templates as well as a link to the App Advisor guidance tools.

Still gaps to fill

This first release of Intelligent Discovery is promising, but some key features are missing. With agent token costs an increasing problem for businesses, it would be nice to see tools that help predict costs, integrating with finops tooling. We’re living in an age of shadow AI, and putting the AI we used to buy with credit cards in Microsoft Marketplace is one way to shine a light on those shadows — bringing the necessary control and governance to AI purchases, and maybe even providing support for site licensing.

Microsoft Marketplace is becoming a useful resource for AI application developers. It encompasses the entire development life cycle: offering tools that can help you build agents, the models that you need to power your agents, and finally a way to monetize your work. There’s a longer-term opportunity here, for both the Marketplace and Intelligent Discovery to offer Model Context Protocol (MCP) interfaces, ensuring that tooling and tool discovery become part of the developer workflow, and making developers aware of new tools that might help solve a problem or simplify a task.