Career paths for software developers have always been a touch problematic.
Early on, you started out as a junior developer, and maybe there were ladder rungs that got you raises until you became a senior developer. But there was typically a ceiling to that, leaving you with little choice but to move into a management position if you wanted to continue to advance and make more money.
On top of that, promoting developers to be managers is often less than optimal for the company or the developer in the long run. Many of the qualities that make a good developer don’t always translate into being a good people manager. And certainly, many software developers want to continue to write code and not have to worry about the bureaucracy and paperwork that often comes with management. Plus, it hardly seems optimal to take your best coders and tell them to stop coding.
The industry dealt with this by creating technical positions such as “Architect” and “Staff Developer” and “Chief Scientist.” That works well, but then these positions often lead to doing less programming and attending more meetings.
No matter what the title or role, the simple fact of the matter is that the more senior you get, the more you’ll be dealing with other people. “Tech Leads” have to supervise the work of other developers. Maybe you aren’t responsible for managing their careers or writing their performance evaluations, but you are responsible for making sure their code is up to snuff.
You’re the boss
It will come as no surprise that agentic coding changes this entire dynamic.
We don’t write much code anymore. And we are all too aware that coding agents aren’t perfect. Like humans, they make bad decisions, write bugs, and do things they weren’t asked to do. They need clear instructions, wise guidance, and the right working environment. You need to keep an eye on your agents to make sure they are on track and headed in the right direction.
One might even say that you need to, ahem, “manage” your coding agents.
And in that sense, we are all managers now. “A coding agent is most useful when you treat it more like an employee than an autocomplete tool,” says Jim Bourke, founder and CEO of Zide Software, producer of a development tool that treats developers more like managers than coders. “It needs a clear assignment, the right context, a safe place to work, checkpoints, and a way to evaluate the result.”
How you deal with coding agents isn’t all that different from how you would deal with a real live developer. You have an array of choices to make about who/what will write the code, what they will be building, and when their work can be called finished. You have to guide them, give them clear instructions, review their code, and decide when that code is ready to be checked in.
Career twists and turns
All of that should sound pretty familiar to managers and individual contributors alike. In fact, one might argue that this is the job now — managing coding agents. Even the most junior of developers will be managing a “developer” in the future.
What does that mean for the career path of a person who manages coding agents all day? I hate to say it, but I don’t know and I don’t think anyone can say for sure. The set of skills required to successfully lead a “team of coding agents” compared to those needed to manage a team of real live people overlap, but they are not congruent.
It may be that managing coding agents is an innate talent, leading to some junior developers suddenly outproducing seniors. Or maybe it will be a skill that needs to be carefully nurtured and grown under the mentorship of an experienced manager. It’s hard to say. And how the career ladder will look going forward, and exactly what skills will be rewarded, are questions I don’t think we can answer right now.
Everything is so new and changing so fast that it’s all a blur at this point.
Wherever we may be headed, I don’t envy the person who will have to rewrite all those job descriptions.
Now that many devops organizations are using AI code generators, vibe coding, and applying spec-driven development practices, the questions increase. How does the AI know your organization’s standards? What processes monitor and enforce these standards? What are the developer responsibilities for owning the outcomes?
“Generative AI will happily hand you the movie set of an app with looks of a house from the street. The doors and windows open, and the demo runs clean,” says Mike Toole, director of security and IT at Blumira. “Behind the facade, there’s no plumbing and no wiring: no input validation, no auth boundaries, nothing actually holding it up. If your only gate is ‘does it run and look right,’ you’re shipping a set, not a building.”
One report shows that 92% of developers use AI coding tools daily in 2026 and that 41% of all global code is now AI-generated. Will all that code deliver value, operational issues, or mounting AI debt? I spoke to several experts about how to keep AI-generated code aligned with organizational standards and how to define an AI code generator’s context beyond just functional requirements.
Document like a construction specification
A general contractor in the construction industry will review building information models that provide a view of what they are developing. But many of the details are in the construction specifications, including component stipulations and performance requirements. Devops teams without equivalent documentation for their organization, architectures, and applications may have gotten away with this omission in the pre-AI era. But institutional knowledge and compliance requirements are the context AI code generators need before you share the first functional prompts with them.
“AI-generated code stays aligned with engineering standards only when those standards are explicit, available, and difficult to bypass,” says Eddie Zhou, founding engineer at Glean. “Put architecture, approved libraries, data-handling rules, security controls, naming conventions, observability, test expectations, and non-functional requirements into specs and automated checks. Then, ensure you give the model that context but treat its output as untrusted until it passes the same reviews, tests, scans, and production monitoring as human-written code.”
Overcoming tribal knowledge was always challenging as devops teams had too much pressure to push features and little incentive to maintain documentation. Two of my top AI leadership skills are change management and agile collaborative leadership, because devops teams need practice alignment while introducing AI capabilities.
Zhou adds, “AI amplifies ambiguity and weak governance as efficiently as good engineering. If standards live only in tribal knowledge, AI will expose that problem, not solve it.”
Encode non-functional requirements as automated tests
The problem with requirement documents and reference architectures is they aren’t directly enforceable with human or AI developers. Once a devops team agrees on standards and documents them, the next step is to develop approaches to encode them as automated acceptance criteria in CI/CD pipelines.
“AI code generators and human developers optimize for functional correctness—making the feature work—and quietly skip the non-functional requirements where most of your real standards live, including performance, security posture, error handling, observability, and auditability,” says Michael Bevilacqua, vice president of AI product management at Adeptia. “Encode your NFRs as executable acceptance criteria that generated code must pass before it merges, so ‘aligned with our standards’ becomes a gate the pipeline enforces on every change.”
An example: Performance requirements stating that all web pages should have a total page weight under 2MB can be validated during the pull request, and requiring the time to first byte (TTFB) to be under 800ms can be automated in CI/CD in a staging environment.
Requirements for stakeholders and AI
Architecture and automated non-functional requirements covering performance, security, and other organization-level compliance requirements should become the base-level context standards shared with AI-code generators. The next set of considerations for setting accurate contexts starts with application- and agent-specific requirements, their intended use cases, and specific compliance guardrails.
“CIOs are mandating AI to build apps faster, but many are realizing speed isn’t the bottleneck because more code doesn’t automatically create business value,” says Matt Healy, senior director of product strategy and marketing at Pegasystems. “Before developers write any code, organizations must understand customer needs, assess legacy systems, stay current with regulations and best practices, map enterprise architectures and integrations, and align business and IT.”
Consider three stages of understanding requirements. Once stakeholders set business value and intent, developers must first align on an implementation strategy and discuss performance, cost, and other trade-offs. Then, developers should reconvene with stakeholders and review implementation trade-offs. Healy adds, “On large projects, defining the solution and success metrics can take as much time as building it. Leaders aren’t just using AI to accelerate development, but to help executives, business analysts, product owners, and IT teams collaborate more effectively.”
The third stage is gathering the goals, user requirements, and implementation approach into a development-specific context. But we’re not done yet, because data requirements and operations considerations must also be included in the context fed to AI code generators.
Build governance into the specs
How the application or AI agent being developed can use the data it has access to requires clear specifications. Data governance programs should have standards based on data sets, defined roles, entitlements, and usage specifications.
Kash Mehdi, vice president and field CTO at Reltio, says the teams that are pulling ahead treat data governance as an input to AI, not an audit after it. “They ground their tools in trusted, semantically consistent data models, approved specs, and security policies, so the AI inherits the organization’s standards instead of inventing its own. Without that governed context layer, you don’t scale from a handful of AI-assisted workflows to thousands; you just accumulate inconsistent code faster,” Mehdi says.
Here are two options to consider in creating standards: First, establish repurposable data sets as data products with defined integrations and usage rules. Organizations with many data sets and integration points should consider data fabrics. The second (and key requirement) is to standardize the format of data contracts, using human- and machine-readable formats so that data owners, users, and AI code generators operate from a consistent set of principles.
“AI writes code that compiles and passes tests but quietly ignores your data contracts, so the failures show up in production as corrupted state, not build errors,” says Ed Huang, cofounder and CEO of TiDB. “The fix is to make governance machine-checkable, including schema constraints, access policies, and data-quality rules enforced at the database layer where the AI can’t route around them, not documented in a wiki the model never reads. Treat the database as the enforcement point for your standards, and it stops mattering whether a human or an agent wrote the query.”
Observability and testing first
When AI generates the code, a best practice is to use static application security testing (SAST) and other code analysis tools to perform automated code reviews. Another best practice is to put applications through penetration testing and other application security validations. But many of these tools are outside in view of what the AI developed. To get an inside view, observability and testing requirements must be included as a standard, with specific requirements around the key transactions you’re asking the AI to develop.
Sanjay Gidwani, CEO and founder at Kosmos, says, “More code volume just means more time spent investigating, and the fix isn’t reading the code more carefully before it ships. It’s connecting the observability signal, the thing that broke, back to the specific change that caused it. Without that link, you’re doing the same manual investigation as before, just with a bigger haystack.”
Testing requirements must go beyond generating automated tests, using synthetic data to expand the test patterns, and implementing continuous testing. “Instead of testing one input against one output, test whether an invariant holds across a wide range of generated inputs, which is a good fit for AI’s unbounded input space and non-deterministic outputs,” says Lior Gavish, cofounder and CTO at Monte Carlo. “I treat it as one layer in a stack with evals mapping the input space, property-based testing stress-testing it for edge cases and brittleness, and observability on live traffic checking whether production is drifting from the properties you defined as non-negotiable.”
Package and evolve the standards
Ed Frederici, CTO at Appfire, recommends codifying standards as machine-readable rules, including linters, formatters, and type checks. Merge the organizational and application standards into one context shared with the AI code-generating tool. His other recommendations:
Enforce standards in continuous integration so violations block merges.
Give the AI persistent context, such as CLAUDE.md, style guides, and architecture docs.
Keep changes small and reviewable.
Require human review of every pull request because AI-generated code must get the same scrutiny as any contributor’s. No exceptions.
Back this approach with solid test coverage to catch regressions.
Periodically audit for drift as your standards evolve.
It’s also important to remember that as code must be extended and maintained, so must the underlying development standards. “AI-generated code only stays aligned with your standards when it’s powered by a context engine that keeps learning, not a static snapshot of your rules,” says Itamar Friedman, CEO and cofounder at Qodo. “That means an engine that continuously adapts to your codebase’s evolving standards, your team’s PR history, and the prior decisions your engineers have already made, so the AI isn’t just following instructions but actually understands how your team works.”
As more devops organizations rely on AI code-generating tools, developers have an important opportunity to develop the standards and automated validations required to ensure what is built meets engineering and compliance requirements.
A swarm of hundreds of OpenAI agents uploaded “malicious packages” to RubyGems and tried to steal API keys, the Ruby community gem hosting service revealed Friday.
OpenAI confirmed part of the disclosure, saying, “our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.”
The agents’ goals were unclear, as was whether they engaged in the swarming activity for research, and even whether they were explicitly sent by OpenAI staffers, but an analysis published by RubyGems strongly suggests malicious intent.
“Once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys, though we are unsure if they succeeded or not,” the RubyGems post said. “The agents clearly regarded what they were doing as hacking. Agents used file names like hack[.]rb, evil[.]rb, inject[.]rb, exploit[.]rb, and ssrf[.]rb. SSRF stands for ‘Server-Side Request Forgery,’ a type of security vulnerability. They also dubbed packages conspicuous titles like pwnp999, exfiltestwand3, hacksvn1778554764 and lambproxyhackabcxyz. Comments such as “# malicious probe” or “#hack” are littered across the campaign.”
The post also said that the agents attempted to trick defensive systems.
“At some points, the agents attempted to be covert. We found multiple packages that would disarm themselves to hide their payload in the next version,” the post said. “They uploaded one package with the comment ‘# disable evil in the next version and bump version,’ which after execution would modify the package to remove the malicious code initially inserted.”
OpenAI should be accountable
Analysts and consultants said the attack was concerning because if such efforts happen often enough, it could slow down security operations center (SOC) responses.
Nader Henein, a Gartner VP analyst, said he was highly concerned about an upcoming bot swarm trend.
“What we know is that this is the kind of standard attack, now AI-augmented, that will become commonplace over the coming months,” Henein said. “It’s less so a rogue agent, more so an attacker, potentially using compromised credentials, weaponizing an agent swarm, in the same way that attackers used compromised endpoints to mount DDoS attacks for the better part of the last decade. The difference here is the fact that these are not individually compromised bots. OpenAI’s guardrails should have not allowed this to happen.”
Frank Dickson, principal analyst at Dickson Research, added, “OpenAI needs to be held accountable. They seem to want to create ‘Dr. Frankenstein’s monster,’ but don’t seem to want to accept blame for the outcomes. OpenAI hasn’t denied its agents used RubyGems. It has disputed the word ‘malicious’ and called the activity ‘benign,’ while separately acknowledging that, in that same stretch of weeks, its agents escalated to cluster-admin access at Hugging Face and compromised accounts at four other third-party services. Those two characterizations are hard to square. The behavior is still unacceptable.”
However Erik Avakian, technical counselor at Info-Tech Research Group, stressed that it’s not necessarily the case that OpenAI launched these agents with explicit instructions. The agents might have easily charted this destructive path all on their own.
The OpenAI agents “absolutely could have acted autonomously. We’ve already seen that capable agents can pursue various unexpected paths to accomplish a task when they have enough autonomy and access,” he said. “A human may have authorized the evaluation or given the agents access to tools, but that doesn’t mean a human approved every action they subsequently took.”
Could delay SOC responses
Dickson added that he fears the ultimate cybersecurity risk is that SOC staffers see so many of these attacks that they start to experience alert fatigue.
“If the vendor whose agents did this is the one downgrading the language, a SOC analyst reading headlines instead of the underlying report has every reason to underreact,” Dickson said. “Security operations aren’t fit for purpose if they run on the assumption that an AI agent label makes an intrusion less real. A stolen API key or a remote code execution path behaves identically whether the actor is a ransomware crew or an unsupervised model chasing a reward signal.”
Mike Wilkes, enterprise CISO at Aikido Security, also noted that the fact that the agents self-identified as OpenAI should mean nothing, as all agents can persuasively pretend to be representing anyone, especially if they think it will slow down a response, even for a brief period.
“A User-Agent string is a nametag written by the visitor, not a passport,” he said. “If SOC tooling begins suppressing alerts because traffic claims to be an OpenAI, Anthropic, Google or other AI agent, attackers will adopt those identities immediately if they haven’t already.”
Thus, he said, “if a human researcher or employee delegates authority to an autonomous agent, there should be a verifiable chain showing who delegated that authority, which organization they represent, what agent was authorized, what scope it was given, and for what period of time.”
Plan for similar attacks
Consultant Brian Levine, executive director of FormerGov, encouraged CISOs to anticipate more such attacks and plan accordingly.
“Organizations that depend on open source, which is nearly all of them, should assume registries are an active battleground [and should] rotate and scope API keys tightly, monitor for anomalous package publishing and credential access, and pin and verify dependencies rather than trusting a name,” he said. “The economics have shifted. Automation lets an attacker try thousands of variations cheaply, so defenders have to make the payoff of any single success as small as possible.”
Justin Greis, CEO of consulting firm Acceligence, agreed.
“If legitimate AI research activity increasingly generates behavior that looks like hostile scanning, exploitation, credential access or persistence, SOC teams can become conditioned to treat those signals as noise,” Greis said. “Attackers will understand that very quickly. The dangerous phrase becomes ‘that is probably just an AI agent.’”
Attackers have opened a new front in their war on software developers: Vite servers, which they are probing for sensitive data including cloud credentials, infrastructure configuration and environment files.
Vite was created as a build tool for Vue, a JavaScript framework for building user interfaces and web applications, but has now become a widely used development server and build tool across the JavaScript ecosystem.
F5 Labs reported that attackers sent more than 32,000 attempts to scan exposed Vite servers on its honeypot network, grouped into 807 attacks (or sessions), during August, a sharp increase from just 1,732 attempts over the previous three months.
“Rather than target a single file, the scanning fleet systematically cycled through extensive wordlists of environment files, AWS keys, Azure tokens, and Infrastructure-as-Code state files,” F5 threat researcher, Adam Metcalfe-Pearce, wrote in a blog post on F5’s blog.
F5 noted that Vite normally binds to localhost, but developers can expose it through the “–host” option, server configuration, container port mappings or other deployment mistakes.
Scans targeted a file-access bypass
The activity targeted a recently disclosed vulnerability that allows unauthenticated attackers to bypass Vite’s file-access restriction and retrieve files from the host system. Tracked as CVE-2026-39364, the flaw allows attackers to bypass the “server.fs.deny” deny-list protection used to prevent access to sensitive files.
“When specific parameters such as ?raw, ?import&raw, or ?import&url&inline are appended to a request, the server fails to enforce deny-list filtering and serves the target file with an HTTP 200 response,” Metcalfe-Pearce wrote.
Some requests also used double-encoded path traversal, which F5 said indicated an attempt to evade security controls such as reverse proxies and web application firewalls (WAFs).
Assigned a severity rating of CVSS 8.2 ,the flaw affects Vite 7.1.0 through versions before 7.3.2 and Vite 8 versions before 8.0.5.
F5 recommended updating Vite to a patched version, rotating potentially exposed secrets, ensuring development servers do not bind to external interfaces, and auditing Docker, Kubernetes and cloud configurations so development ports are not exposed to the public internet.
The blog also shared the curated directory and credential wordlists the attackers used during these attacks.
Vite was part of a broader scanning pattern
F5 also observed attackers combining CVE-2026-39364 with older Vite file access vulnerabilities, including CVE-2025-30208, CVE-2025-31125 and CVE-2024-45811. The same scanning infrastructure also probed for a Next.js middleware bypass, indicating that the activity is not confined to a single framework.
In its blog post the company also noted that, apart from CVE-2025-31125, none of these CVEs are yet listed in CISA’s Known Exploited Vulnerabilities (KEV) catalog.
While F5 saw a sharp increase in August in attacks on recently uncovered flaws in Vite, it didn’t make the top three CVEs attacked on the company’s honeypots, all of them much older. CVE-2017-9841, an almost decade-old critical remote code execution flaw in PHPUnit, remained top of the table with 4,201 recorded attacks, followed by CVE-2018-14028, a failure to verify WordPress plugins as valid ZIP files (4,102), and CVE-2018-20062, a ThinkPHP remote code execution in NoneCms (3,482).
Cockroach Labs, the company behind distributed SQL database CockroachDB, is betting that the rise of agentic applications will make traditional database provisioning, where systems are sized for peak demand, increasingly difficult to justify as agents create highly concurrent, unpredictable and often short-lived database workloads that can leave provisioned compute idle between bursts.
Its solution to that problem is Cockroach Continuum, a database fleet management platform that pools compute and storage across isolated CockroachDB databases in an enterprise to allow capacity to be shared across workloads rather than reserved for individual systems.
That shared model can help enterprises accommodate sudden bursts of agent activity without maintaining dedicated infrastructure for every database, while paying only for consumption rather than peak capacity, the company said Tuesday.
Continuum does this by combining database virtualization with Cockroach Plenum, a feature that separates compute from storage so that the two capacities can scale independently.
It also draws on Cockroach Aegis, which uses an AI-assisted operations layer to diagnose issues and recommend actions across an enterprise’s database estate to help database administrators (DBAs) reduce the manual work involved in monitoring, troubleshooting and tuning databases while retaining control through policies, the company said.
What Continuum could mean for CIOs
That reduction in manual work for DBAs is likely to benefit CIOs in terms of operational cost.
“Enterprises don’t just pay for database capacity. They pay people to provision, monitor, tune, troubleshoot and scale these environments. If Cockroach can automate a meaningful portion of that work, there is value there and a chance to offset costs,” said Ashish Chaturvedi, executive research leader at HFS Research.
For more direct infrastructure-related cost savings via Continuum, though, CIOs will need to look at how their database estates are provisioned.
Stephanie Walter, practice lead of AI stack at HyperFrame Research, said the greatest savings will come from large fleets of independently provisioned databases that spend much of their time underused: “Sharing capacity across workloads with different demand patterns could substantially improve utilization,” she said.
Not all enterprises will benefit: “Cost savings will be less meaningful for databases that already run near capacity or require tightly dedicated resources,” she said.
One caveat is that, because Continuum shifts the calculation of database costs from provisioned capacity to consumption, unexpected bursts of activity could make it harder for CIOs to forecast database costs.
CIOs will therefore need to assess their own workload patterns and test whether the higher utilization and reduced idle capacity actually translate into lower overall database costs, rather than assuming that consumption-based pricing will automatically deliver savings, Walter said.
While Continuum may simplify database management, it could also deepen dependence on CockroachDB and make workloads harder to move elsewhere, she said — with the added risk that, by coupling independent databases together onto one platform, it creates a new single point of failure.
What Continuum could mean for developers
For developers though, the shift to Continuum could reduce the time and effort required to build applications, especially agentic ones.
Development teams using Continuum can design applications without having to worry as much about the database compute and storage capacity those applications will need, when that capacity needs to scale, or the operational work required to provision and manage it, said HFS’s Chaturvedi.
But, said Walter, those advantages don’t come without challenges.
“Pooling infrastructure as Continuum does is more likely to create questions around workload isolation, performance predictability, and debugging,” she said. “Enterprise teams will want to know whether one demanding workload can affect others and whether they can trace a performance problem through the shared environment.”
Who is likely to adopt Continuum
Existing CockroachDB customers with large fleets of distributed transactional databases are likely to be among the earliest adopters of Continuum, Walter said.
SaaS companies, financial services firms, digital platforms and AI-native businesses could also be strong candidates, particularly where applications generate unpredictable demand and database capacity needs to scale quickly, she said.
For those enterprises, though, one of the capabilities that could make Continuum most useful for unpredictable workloads is yet to arrive.
Automatic compute scaling, which will help scale compute up or down as necessary to maximize cost savings without compromising availability, is “coming soon,” the company said.
Java Development Kit 27, an Oracle-driven, planned update to standard Java, is now generally available. This latest update to standard Java has features ranging from making the Garbage-First (G1) garbage collector the default collector and adding structured concurrency to simplify concurrent programming.
Overall, there are nine features listed on what is described as a reference implementation of JDK 27. The first five features include making G1 the default garbage collector, post-quantum hybrid key exchange for TLS 1.3, lazy constants, structured concurrency, and primitive types in patterns, instanceof, and switch. The next four include compact object headers, JFR (JDK Flight Recorder) in-process data redaction, a vector API, and PEM (Privacy-Enhanced Mail) encodings for cryptographic objects.
With JDK 27, G1 becomes the default garbage collector in all environments, rather than just in server environments. Goals are to ensure that the HotSpot JVM will always select G1 and that performance including throughput, latency, memory footprint, and startup time does not degrade significantly.
With post-quantum hybrid key exchange for TLS 1.3, the goal is to enhance the security of Java applications that require secure network communication by implementing hybrid key exchange algorithms. Such algorithms defend against future quantum computing attacks by combining a quantum-resistant algorithm with a traditional algorithm.
Lazy constants introduces an API for objects that hold unmodifiable data. Lazy constants are treated as true constants by the JVM, enabling the same performance optimizations that are enabled by declaring a field final. Compared to final fields, however, lazy constants offer greater flexibility in the timing of their initialization. This feature is in a third preview.
The goal of primitive types in patterns, instanceof, and switch is to enhance pattern matching by allowing primitive types in all pattern contexts, and to extend instanceofand switch to work with all primitive types. This feature is in a fifth preview.
Structured concurrency simplifies concurrent programming by introducing an API that treats groups of related tasks running in different threads as single units of work. Goals include streamlining error handling and cancellation, improving reliability, and enhancing observability. Structured concurrency is in its seventh preview.
JDK 27 makes compact object headers the default object header layout in the HotSpot JVM. Compact object headers reduce object headers from 96 bits down to 64 bits on 64-bit architectures, reducing heap size, improving deployment density, and increasing data locality.
JFR data in-process redaction allows users to redact command-line arguments and the initial values of environment variables and system properties in JFR recordings. JFR recording files may contain events that include sensitive data, such as secrets in command-line arguments, access tokens in environment variables, and passwords in system properties. This data is (optionally) redacted before it leaves the process, so that sensitive information does not leak.
The vector API introduces an API to express vector computations that reliably compile at run time to optimal vector instructions on supported CPUs, thus achieving better performance than equivalent scalar computations. This is an incubating API in its 12th incubation.
PEM encodings of cryptographic objects introduces an API for encoding objects that represent cryptographic keys, certificates, and certificate revocation lists in the PEM transport format, and for decoding from PEM back into objects. This is a preview API.
JDK 27 is a short-term feature release (non-LTS) that will receive six months of support from Oracle. Its predecessor, JDK 26, also a short-term release, became generally available on March 17.
If you like the idea of running an LLM on your own computer but tried awhile ago and were disappointed, it may be time to give it another chance.
“A few months ago, any LLM that I could run on my Macbook scored 0% on an agentic coding eval I put together,” Simon P. Couch, senior software engineer at Posit, posted on Bluesky this spring. “[The April] Qwen 3.5 and Gemma 4 releases both scored 90%.”
A model running on your laptop still won’t come close to what a state-of-the-art LLM from Anthropic or OpenAI can do in the cloud. But for defined tasks like answering coding questions, writing functions, or summarizing documents, they can be surprisingly capable.
“Laptop-available models, while a lot weaker than the frontier, have started wildly outperforming expectations,” open-source developer Simon Willison, who follows the AI industry closely, said in his PyCon US 2026 lightning talk in May.
There are many ways to run local models on a PC or Mac. Ollama, while perhaps not the fastest, is among the most popular and easy to set up. It’s also supported out of the box by many mainstream programming tools such as Visual Studio Code, JetBrains AI Assistant, Zed, and Posit Assistant. Ollama also can launch Claude Code or Codex with the option to use a local LLM.
I’ll be focusing on Ollama here, but many other tools are available for running LLMs locally, such as LM Studio, Jan, Unsloth, Simon Willison’s LLM, and llama.cpp.
You can download Ollama and install it as a conventional software program for Windows, Mac, and Linux. There’s both a chat application with a GUI and a terminal CLI.
From choosing a model and setting its parameter options to tweaking its runtime configuration, here are some tips for getting the most from your Ollama setup.
Choosing a model
Choosing a model is obvious but not always easy, given that the Ollama model catalog has hundreds of listings. What’s best for you depends on both your task and available hardware. For coding, the Gemma 4 family from Google and the latest Qwen models from Alibaba are both worth considering. I’m partial to Gemma, because it writes R code well. In fact, Gemma 4 26B in the cloud is the only open-weight LLM that Posit offers in its Posit AI subscription service that might fit on a laptop. I also lean toward US models for summarizing text, because they’re more likely to be trained in nuances of American English.
Next is selecting a size. The Gemma 4 12B version introduced in June fits comfortably in a lot of systems. Google says a 4-bit version (the Ollama catalog default) requires 6.7 GB of VRAM or Mac unified memory. And, Google’s internal coding benchmarks show it’s close to the larger 26B MoE (mixture of experts) Gemma 4 version from April in some tasks. It’s a decent place to start if you don’t have a ton of VRAM.
Another possible option: Meta recently announced Muse Glimmer, a 30B model unveiled in August, which is designed for end-to-end agentic work, reliable tool use, and multi-step reasoning. It needs around 24 GB of VRAM to run at a usable clip, though.
If none of these suggested models work for you, one of the easiest things to do these days is ask a frontier model such as ChatGPT or Claude. Tell it how much GPU VRAM and regular RAM your Windows PC has, or how much unified RAM your Mac has, what LLM family you’re interested in running if you’ve got a favorite, and what you want to do. Summarize short documents? Long documents? Answer coding questions in a terminal? Run a coding agent? Ask for its three top suggestions and start with those. Then try larger or smaller options as appropriate.
For Windows PCs, if an LLM can’t fit entirely into GPU VRAM, it may run so slowly as to be effectively unusable.
Note, though, that an LLM’s architecture matters along with its size. A so-called mixture-of-experts model like Gemma 26B only uses some of its parameters for a task. While the whole LLM needs to be loaded, it may run fine on a PC with less GPU memory than you’d expect—depending on what you’re doing. Gemma 4 26B is pretty snappy for some tasks in a terminal window on my PC with 12 GB of VRAM, even though more than half of the model spills over into RAM.
And you don’t always need the largest model that fits in your hardware; smaller ones may be fine, especially for simple, non-mission-critical tasks.
The takeaway? Selecting a good model is a bit more complicated than “this hardware can run LLMs with that number of parameters, and bigger is always what I want.”
One more note: If you have a Mac with Apple silicon, choosing an MLX version of an LLM if available will likely give you better performance. MLX is a framework that’s optimized for Apple hardware.
Downloading and running your model
Once you decide on an LLM, the Ollama model listing should show you how to download and run it. For example, the terminal command for Gemma 4 12B is ollama run gemma4:12b or ollama run gemma4:12b-mlx. If the model is already on your system, the ollama run command just loads and launches it; if the LLM isn’t on your system yet, Ollama downloads it first. The ollama pull command will download a model without running it. Use ollama list to list all the LLMs Ollama has already downloaded on your system. And use ollama rm [model-name] to delete a model.
Setting your model’s parameter options with a Modelfile
There are ways to tweak an LLM’s performance with parameter options such as temperature (how “random” its responses are), top_k (if its choices should be restricted to a certain number of best-matching tokens), top_p (how many next tokens the model considers when generating its response), and maximum context window size.
Ollama suggests 64000 tokens as a minimum context window size for web search, agents, and coding tools. However, the Ollama default is just 4000 for systems with less than 24 GB of VRAM, even if an LLM has a much larger possible context window.
Instead of specifying parameters on the command line each time you run a specific LLM, you can create a “copy” of the model with your desired parameters using a Modelfile. Below is an example Modelfile that creates a version of the Gemma 4 12B LLM with some suggested parameters and a context size of 64000 tokens:
FROM gemma4:12b
PARAMETER temperature 1
PARAMETER top_p 0.95
PARAMETER top_k 64
# For the context window
PARAMETER num_ctx 64000
Google advises setting temperature to 1.0, top_p to 0.95, and top_k to 64 for all the Gemma 4 LLMs. (That’s already what the default Ollama versions are, but it doesn’t hurt to specify again.)
You can also set a default system prompt in a Modelfile with SYSTEM instruction.
Save your Modelfile as a plain text file, such as Modelfile-gemma4-12b-64k-optimized.txt, and then create the LLM copy with a terminal command such as
The ollama list command will show your newly created copy as having the same size as the original, but it’s not actually taking up all that additional space on your hard drive. ollama show [model-name] will display a number of settings that have been baked in.
Tweaking how long an LLM stays loaded
Ollama defaults to unloading a model out of memory after just five minutes of session inactivity. That may be helpful if you’ve ended your session, but not so great if you want to resume a chat that’s been idle for seven minutes. Having to re-load an LLM into an existing session can make a model feel a lot more sluggish than it should.
To change how long an LLM stays in memory, update your system’s OLLAMA_KEEP_ALIVE environment variable. On Windows, you can change the default in a terminal window with setx OLLAMA_KEEP_ALIVE "15m" or setx OLLAMA_KEEP_ALIVE 900 (a number without a qualifier such as m for minutes is treated as seconds).
On a Mac, you can change the default with launchctl OLLAMA_KEEP_ALIVE 15m.
Restart Ollama for new settings to take effect.
You can unload a model manually with the terminal command ollama stop [model-name]. To see how much time is left on a running LLM if there’s no additional session activity, use the ollama ps command. Note that the results of this command include an “UNTIL” column.
Sample results from an ollama ps command.
Foundry
Rethinking other Ollama settings
Another important Ollama option is the default context window. If you’re not using a Modelfile-customized LLM, you can set Ollama’s default context window for any LLM you run with setx OLLAMA_CONTEXT_LENGTH 64000 on Windows or launchctl OLLAMA_CONTEXT_LENGTH 64000 on a Mac. (If the LLM has a smaller maximum than this setting, Ollama should ignore it.)
If you’re on a PC, you may also want to change Ollama’s default for KV cache, which controls how a model stores chat history in your current session. The default is f16 quantization, which gives high precision—but with high memory use. Changing to q8_0 (8-bit quantization) “uses approximately 1/2 the memory of f16 with a very small loss in precision, this usually has no noticeable impact on the model’s quality,” according to Ollama’s documentation. Yes please!
setx OLLAMA_KV_CACHE_TYPE "q8_0"
This usually works with NVIDIA GPUs that should auto-enable a memory optimization called Flash Attention. For other setups, Ollama docs suggest force-enabling Flash Attention by setting OLLAMA_FLASH_ATTENTION to 1 in an environment variable. If you’re using an MLX model on a Mac, you probably don’t need to worry about this.
To check which Ollama variables you’ve set on your PC, you can run these terminal commands:
Running ollama ps in a new terminal window while Ollama is running an LLM elsewhere also shows you how much of a currently running model is in your GPU. If your LLM is slow but it all fits in your VRAM, the problem may be elsewhere.
Turning off a model’s ‘thinking’ capability
Gemma 4 is a “thinking” model family, which means its models can go through what look like reasoning steps before generating their final response. This “thought process” can improve results but also slow things down—a lot.
Thinking is on by default in Ollama for all LLMs that have the option. You can turn it off from the Ollama command line with --think=false. For example:
ollama run gemma4:12b --think=false
Other applications like IDEs may have their own ways of switching thinking on and off, such as a lightbulb icon in the chat window. Unfortunately, you can’t turn thinking off from a Modelfile.
In a test summarizing documents with Gemma 4 E4B, turning thinking off boosted speed five-fold and result quality was about the same. For Gemma 4 12B, the speed improvement was eight-fold.
How thinking affects coding will depend on the complexity of the task. You may want to test some real-world work to see whether time spent “thinking” pays off with better results.
Running models outside the Ollama catalog
One of the largest sources of open-source LLMs is Hugging Face, and many models there have a “use this model” button that gives the command you need to download and run it on Ollama. Just be aware that not all of those models are as plug-and-play as the Ollama-optimized versions on Ollama.com, especially new ones.
For example, I tried running the Unsloth project’s 2-bit quantized version of Meta’s new Muse Glimmer. The command was
ollama run hf.co/unsloth/Muse-Glimmer-30B-GGUF:UD-Q2_K_XL
However, this errored out of the box. ChatGPT helped me fix the issue (with a Modelfile to create a modified version of the LLM) after I told it the error messages.
Hugging Face model pages often show the command needed to run the LLM on Ollama.
Foundry
The bottom line
Local LLMs have come a long way, but a model that can fit inside a consumer desktop or laptop won’t be able to do the kind of work that top cloud-based models can. Gemma 4 12B may write good basic functions and document summaries, but don’t expect it to do complicated jobs from a single prompt like “create an R package to use the National Weather Service API.” As with frontier LLMs of old, you’ll need to go step by step, be specific, and have some patience. If you never want to go back to those days, stay with the state-of-the-art models in the cloud.
Gemma works well for basic data analysis, “importing, tidying, and visualizing data,” and “information retrieval and short factual questions,” according to Posit AI’s documentation. “It is less reliable than the Claude models on longer, multi-step tasks. You might see a strong response on one turn and a misread on the next. We do not recommend Gemma 4 for long-running agentic coding or complex package development.”
That was pretty much the status of paid cloud-based LLMs a couple of years ago.
Even if limited compared to current state-of-the-art models, though, free coding help that’s 100% local and private can still be compelling for use cases like exploring sensitive data, summarizing personal documents, doing simple tasks, working on a plane with bad Internet, and saving paid tokens for when they matter. Plus it can feel empowering to have everything on your own system.
And, local model capabilities will likely keep getting better.
There is a quiet war happening underneath the AI boom, and most people are not paying attention to the right front. While headlines focus on which model is winning, which startup is raising money, or which chatbot is gaining users, a parallel race is playing out in data centers and corporate boardrooms across the world. The race is for AI infrastructure: the GPU clusters, power contracts, cooling systems, networking fabric, orchestration tools, and inference services that make modern AI actually work. The companies fighting in that race are called neoclouds.
Understanding the importance of neoclouds matters for the same reason understanding cloud computing mattered in 2008. The companies that first saw the shift to cloud and positioned themselves around it correctly ended up controlling a massive layer of the technology industry. The same dynamics are playing out in AI infrastructure right now. But the neocloud market is more complicated than it looks. Not all of these companies do the same thing. Grouping them together as if they are all just GPU rental businesses misses the point entirely. The more useful way to understand the space is to recognize that neoclouds are evolving into four distinct categories, each with a different focus, a different customer, and a different role in the AI stack.
What a neocloud actually is
Before getting into categories, it helps to establish what the term means. A neocloud is a cloud infrastructure provider designed specifically for AI workloads. That sounds simple, but it covers a lot of ground. General-purpose hyperscalers like AWS, Azure, and Google Cloud were engineered to support a wide variety of workloads across many industries. They built architecture around flexibility, breadth of services, and broad enterprise appeal. Neoclouds took a different approach. They built around one mandate: make AI compute as fast, dense, and scalable as possible.
That means designing facilities around high power density, integrating fast interconnects between GPUs, investing in liquid cooling and energy-aware cooling, and building orchestration around tools like Kubernetes and Slurm that are purpose-built for distributed AI training. It also means competing aggressively for access to the latest Nvidia accelerators, because in AI infrastructure, GPU access is often the single most important competitive advantage a company can have.
The result is a fragmented but deeply competitive market. Some neoclouds are focused on physical capacity. Some are focused on making GPU compute accessible to developers. Some are building enterprise-grade infrastructure that can compete directly with hyperscalers in specific AI domains. And some are focused on one of the most important long-term bets in AI: inference. Let’s take a closer look at the four categories of neoclouds.
Full-stack AI infrastructure
The first and most prominent category is full-stack AI infrastructure providers. These companies are trying to become the AWS of the AI era, yet they are built from the ground up around GPUs rather than general-purpose compute. Their goal is to own as much of the AI infrastructure experience as possible: raw GPU clusters, networking, storage, orchestration, managed Kubernetes, Slurm scheduling, private clusters, enterprise support, training workloads, and inference services.
The companies in this space include CoreWeave, Nebius, Lambda, Crusoe, Nscale, and Fluidstack. CoreWeave has emerged as the clearest example of this category. It built its identity around massive GPU clusters optimized for large-scale AI training, paired with Kubernetes-native orchestration and enterprise contracts worth billions. Nebius took a different path, combining deep engineering expertise with a sovereign and European-oriented strategy while expanding heavily into US data centers. Lambda started as a developer and researcher-friendly GPU cloud for machine learning but has moved deliberately upmarket, adding superclusters and private cluster services.
Developer and self-serve GPU clouds
The second category is developer and self-serve GPU clouds. These platforms are not competing for billion-dollar enterprise contracts. They are competing for developers, startups, researchers, and smaller teams who need GPU access quickly without negotiating long-term deals. The product experience here is self-serve: pick a GPU, spin up an instance, deploy a model, run a notebook, test a workload, or launch a small cluster.
The brands that fit here include Vultr, Runpod, Civo, Lambda, and Together AI. Vultr brings GPU compute into a broader cloud platform with a global footprint and self-service deployment. Runpod has built a loyal developer following by keeping the experience simple and fast. Civo targets developers who prefer a Kubernetes-first environment. Lambda still offers strong self-serve GPU instances and 1-Click Clusters alongside its enterprise offerings. Together AI goes beyond raw GPU rental to combine inference, fine-tuning, model hosting, and dedicated clusters in one developer-accessible platform.
This category matters because not every AI team needs thousands of GPUs and a dedicated account manager. A lot of important work happens at the research bench, the startup prototype stage, and the individual developer level. These clouds serve that market, and they often grow into larger accounts as teams scale.
Inference-first AI clouds
The third and increasingly important category is inference-first AI clouds. Training gets the dramatic headlines, but inference is where AI products actually live. Every chatbot response, coding assistant completion, voice agent interaction, and image generation request is inference. And unlike training, which happens in bursts, inference runs all day, every day, for every user.
The companies to watch include Together AI, Groq, CoreWeave Inference, Nebius Token Factory, Crusoe Managed Inference, and Nscale Serverless Inference. Together AI has built one of the clearest inference-first platforms in the market, especially for open source and open-weight models, while also supporting fine-tuning and dedicated GPU clusters. Groq stands apart because its identity is tied to ultra-fast inference running on its own language processing unit (LPU) architecture, which inverts the traditional GPU paradigm. CoreWeave Inference extends CoreWeave’s infrastructure story into serverless and dedicated model-serving. Nebius Token Factory and Crusoe Managed Inference represent how full-stack players are adding inference layers to monetize their GPU capacity differently. Nscale Serverless Inference is Nscale’s move into on-demand model serving without infrastructure management.
What makes this category so compelling is the economic direction. As AI moves from a handful of large frontier labs into millions of production applications, the demand for inference infrastructure will likely dwarf training demand. Companies that control inference capacity may end up controlling the more durable long-term revenue stream.
Data-center and capacity-heavy players
The fourth category is the most physically grounded. These are the companies that are building and controlling the real estate of AI. This goes beyond GPU clouds in the traditional sense. These companies are acquiring land, securing power contracts, designing data centers, developing campuses, deploying high-density racks, and racing to control capacity that genuinely cannot be replicated quickly.
The brands that fit here include Fluidstack, Applied Digital, Core Scientific, Voltage Park, Crusoe, and Nscale. Some of these companies also offer cloud services, but their strategic advantage is fundamentally about commanding physical infrastructure. Fluidstack specializes in delivering large-scale custom GPU cluster capacity and custom data center solutions for enterprises and AI labs. Applied Digital provides hosting infrastructure for AI and high-performance computing at scale. Core Scientific transitioned from crypto-mining data centers into AI infrastructure. Voltage Park is focused on large-scale GPU capacity for AI and research organizations.
Crusoe and Nscale appear in this category as well as in others, which gets to the heart of one of the most important points about the neocloud market overall. These categories are not clean boxes. Companies routinely sit across multiple categories, and competitive pressure is pushing most of them to expand their scope. Crusoe is both a cloud provider and a data-center developer. Nscale is both a cloud platform and a major infrastructure builder. The companies that ultimately win may be those that can span the four categories most effectively.
Why the category framing matters
Understanding these categories is not just an academic exercise. It has real consequences for how companies make buying decisions, how investors evaluate the space, and how technology leaders think about AI infrastructure strategy.
A company focused on physical capacity is playing a fundamentally different game than one selling managed inference APIs. They may appear in the same technology articles and compete for some of the same headlines, but their economics, customer relationships, and long-term differentiation look very different. From an enterprise buyer perspective, understanding which layer you are actually buying from matters before you sign a contract. Some neoclouds are building long-term platform relationships. Others are offering capacity to fill an immediate gap. The category determines which is which.
The neocloud market is young, fast-moving, and genuinely complicated. New entrants, mergers and acquisitions, and platform expansion are all reshaping the landscape in real time. But the underlying logic remains constant. AI infrastructure is becoming a critical layer of the technology stack, and whoever controls that layer will have enormous influence over how AI gets built, deployed, and monetized.
The companies in this market are not all the same. Paying attention to what they actually do, rather than just what they call themselves, is how you separate the ones that will shape AI’s future from the ones that were simply first to send a press release.
Steve Yegge is not the shy and retiring type, so when he wants recommendations on how others are managing 10 to 20 (or more) coding agents, he posts the question on X. Specifically, he’s looking for an IDE because “I use Emacs, and I wouldn’t wish it on you.”
The replies (600 and counting) offer plenty of alternatives: Herdr, cmux, Conductor, the Claude and Codex desktop apps, and an impressive assortment of homemade software. For anyone hoping the industry had settled on a sensible way to work with agents, clearly we haven’t. Hence, Yegge’s question.
The most interesting data hidden in the replies, however, isn’t which IDE or IDE stand-in developers are using. Rather, it’s what developers have added to these tools to accommodate the technology’s shortcomings. These include shortcuts to find the agent that needs a decision, ways to recover an earlier conversation, and groupings that explain how a task fits into a project. In other words, despite the incredible intelligence we now have to help us write code, developers are still struggling with the simple task of automating their remembrance of what they asked AI to do.
We seem to have made it easier to generate work without making it easier to absorb and finish work.
Of course, developers have always customized their tools. Emacs users alone could supply scads of evidence. But, again, it’s not really about the fact that developers are tweaking their tools, but why. So much of the tweaking is intended to help cohere the developer’s intentions with the work that’s happening in several places at once. They’re not asking for better AI: They’re asking for AI to help them express their humanity, as it were.
The work comes back
Consider a developer who asks one agent to fix a bug, another to investigate a performance problem, and a third to update a dependency. While the agents work, she can do something else, which is what makes LLM-driven development so appealing—at least, until the agents return.
None of those agents does work that is fully isolated from the rest of the system. Think about it: The bug fix changes behavior someone may depend on. OK … we can address that! But wait, now we have the performance investigation offering three options, each with different costs. Hmm. Hold on a minute while I tackle this … except the dependency update isn’t waiting. It passes its tests, but the agent has also rewritten a configuration file. Better check that!
As this simple example shows, before the developer can move forward on any decision, she first needs to recover what she asked for, what the agent discovered, and what remains uncertain. And, no, adding more agents doesn’t solve the problem. It could instead compound the problem.
The agents may have saved hours. Hurray! But a list of completed sessions doesn’t tell the developer where to begin. Nor does a green test result answer the question of whether the change belongs in the application.
This is why the number of agents running tells us so little, even if it makes us feel cool. I mean, what are those agents doing, anyway? Yegge’s commenters provide some clues. For example, Anton describes using 30 terminal tabs with one active agent per project, so as to keep their work from overlapping. Jerry Combs says he generally keeps no more than three conversations going, with agents managing their own subagents as needed. Even those three conversations are difficult enough to follow.
Both approaches can make sense. The demands depend on how independently the work can proceed and how much human judgment it requires along the way. A developer with 20 agents may have fewer decisions to make than someone with three.
Gergely Orosz recently listed several observations worth considering together: developers spending less time in IDEs, code reviews becoming theater, and people working more despite AI’s productivity promises. His observations don’t establish that agent tools cause overwork or superficial review. They do make it worth asking whether we’re shifting effort into places our productivity stories neglect. Watching an agent produce a change is satisfying, but spending the afternoon reconstructing why six changes were made is less so.
But that work, however unsatisfying, is necessary if those changes are going to be understood and maintained.
Teach the tools to wait
Some of the plumbing already exists to alleviate these issues. For example, Herdr tracks agents as working, blocked, or idle and carries those states into its tabs and workspaces; cmux offers notifications and a shortcut to the workspace with the most recent unread notification. These are useful starts, but they do highlight just how much remains to be done. After all, the most recent notification may concern the least important task, and an agent asking a question may be perfectly capable of waiting while the developer finishes something else.
The next improvement should help developers make those distinctions. A task should retain its original purpose, the relevant decisions, and the evidence supporting its result. When it needs a person, it should explain the decision required and what happens if that decision waits. A summary can help, but it must lead back to the actual changes and test results. Otherwise, we’ve made it easier to approve work without understanding it.
Again, the best tools for agentic AI will be the ones that bring the human back into the loop and provide the information necessary to make decisions.
I’d like a tool that can tell me a task is ready for review because the requested behavior has been demonstrated, while keeping a separate, less urgent question out of my way. I’d also like it to remember that I rejected an approach yesterday, so I don’t have to discover it again in today’s proposed fix. Whether that tool calls itself an IDE seems secondary. Google introduced Antigravity 2.0 as a standalone agent application without an IDE, while recommending that developers use it alongside their IDE of choice. In other words, the editor still has a job, even when it is no longer where every task begins.
In 2022, I wrote about cloud convenience and the tendency to misunderstand what developers want from their tools. They have work to do and want fewer obstacles to doing it. We should apply that same standard to agentic development across the whole task, including the effort of returning to it.
Not that we can put all the onus on AI and its tools. Teams have a part to play here. If every task is urgent, the tool has nothing useful to prioritize. If nobody defines what counts as finished, the tool can only report that an agent stopped. Agreeing on those things and limiting the work that needs simultaneous human decisions will do more than adding another status badge.
Yegge’s readers are already building pieces of this future, one shortcut and homemade workspace at a time. The vendor opportunity is to make that convenience available without requiring everyone to maintain a side project. Give developers more work they can confidently put behind them and fewer conversations they need to keep alive in their heads.
Yet another security vulnerability has been discovered in GitLab infrastructure, this one a perfect 10 in severity.
CVE-2026-85706, the second flaw GitLab has disclosed in just a month, is a maximum-severity vulnerability that allows attackers to read arbitrary files in a single HTTP request. The path traversal flaw results from improper confinement and lack of authentication enforcement in GitLab’s repository commits API, the company reported.
Threat actors could exploit it “under certain conditions” and read arbitrary files (credentials, secrets, and other sensitive data) on vulnerable GitLab servers.
The company has fixed the vulnerability, which impacts GitLab Community Edition (CE) and Enterprise Edition (EE), and has advised customers with public-facing self-hosted GitLab instances to patch their servers immediately, or remove public access.
Experts say the flaw is alarming because GitLab’s DevSecOps platform is used by roughly 50% of the Fortune 100 and has more than 50 million estimated registered users.
“GitLab is not simply a source-code repository,” noted Safayat Moahamad, advisory director at Info-Tech Research Group. In many enterprises, it is connected to build pipelines, deployment processes, application security workflows, and other trusted systems.
As a result, unauthorized access to configuration files, secrets, or credentials on the GitLab server could “create consequences well beyond the affected instance,” he said.
Don’t wait for the normal patch cycle
GitLab has been a favorite hacker target of late: In January, it patched a high-severity flaw that allowed attackers possessing a target’s account ID to bypass two-factor authentication, and in August, it fixed a critical vulnerability that could give unauthenticated users the ability to make modifications inside code repositories or even completely delete them with a single HTTP request.
The current maximum severity bug, CVE-2026-85706, was reported through GitLab’s HackerOne bug bounty program. It affected CE and EE versions 18.7 before 19.1.8, 19.2 before 19.2.6, and 19.3 before 19.3.2.
The US Cybersecurity and Infrastructure Security Agency (CISA) has added CVE-2026-85706 to its Known Exploited Vulnerabilities catalog, noting that this type of vulnerability is a frequent attack vector for malicious cyber actors, and particularly poses significant risks to the federal enterprise; watchTowr Intel has reported that it is already observing “in-the-wild probes.” “Based on recent GitLab vulnerabilities, we know the time until indiscriminate exploitation is likely not far away,” it warned.
This is not a vulnerability that organizations should leave for the normal patch cycle, Moahamad emphasized. “It offers an unauthenticated path to arbitrary files on a platform that frequently sits at the center of source code, build, and deployment workflows,” he said.
Enterprises should patch immediately, hunt for suspicious repository-commits API activity, and investigate whether exposed files contained credentials or secrets that may now require rotation, he advised.
Beyond patching, the watchTowr Intel team said defenders should try to identify exploitation attempts by hunting through log files for HTTP POST requests to “/api/v4/projects/{id}/repository/commits/” URIs containing “file.path” parameters.
CI/CD platforms are critical trust infrastructure
Organizations running affected self-managed GitLab CE or EE instances should be most concerned, Moahamad noted. Risk increases where GitLab is connected to sensitive repositories, CI/CD pipelines, cloud environments, or production-deployment processes.
The information and/or access that attackers could obtain depends on what the GitLab service can read and what organizations store on the server, he explained. It could include configuration files, secrets, credentials, and other sensitive server-side data. If those files happen to contain usable tokens, keys, or credentials, an attacker could attempt to access connected infrastructure
“The flaw directly creates an unauthorized file access risk,” he said. Credential theft, lateral movement, source code exposure, and supply chain compromise are possible follow-on scenarios.
Source code and CI/CD platforms must be governed as “critical trust infrastructure,” he said. While patching remains essential, resilience will depend on knowing where platforms can be exposed, limiting what they can access, detecting abnormal API behavior, and having a tested process for investigating and rotating credentials.
David Shipley of Beauceron Security described two factors colliding to generate “maximum pain” for GitLab users.
First is the vulnerability. “It’s a 10 for a reason: Unauthenticated read access to GitLab client source code,” he said.
Second, this especially hurts because developers still have bad habits: way too much code continues to ship or is in production with embedded SSH keys, cloud secrets, tokens, and other valuable data that attackers can use to get into infrastructure, Shipley said.
“So for attackers, you’ve got the classics,” he noted: Data exfiltration and extortion, ransomware, cloud infrastructure hijacking for crypto mining, and the stealing of AI tokens and infrastructure to enable other criminal activity.
“This level of vulnerability is the code and cloud equivalent of being able to get the credit card number, expiry and security number,” Shipley said. “It opens up a world of criminal data shopping.”
The bottom line: “Get those secrets out of production code and adopt modern coding authentication best practices,” he advised.
For CIOs and CDOs heading into Microsoft Fabric and SQL Community Conference Europe, the main question is not which data technology sits at the center. It is whether the data platform improves revenue, productivity, and decision speed while creating a trusted foundation for AI at scale. That should drive the data strategy.
Enterprise AI is forcing companies to tackle a familiar problem, with data spread across ERP, CRM, operational systems, databases, applications, warehouses, and analytics platforms built over years. That was manageable for reporting and point-to-point integration. Copilots and agents raise the stakes because they need to reason across customers, orders, inventory, financial implications, operating constraints, permissions, and business rules at the same time.
That moves the discussion from architecture into CIO and CDO strategy. What outcomes are we trying to improve? What data and context are required? What needs to move versus connect? Where do Microsoft Fabric, Azure Databases, and other platforms fit? How will governance and authority follow the data? Where should the data be stored and what sovereignty or regulatory restrictions apply? And how will we measure results?
Microsoft is positioning its broader data platform around AI business value. That framing is useful when it translates into an operating strategy. As I wrote in Forbes about Microsoft Build in June, the next phase of AI depends less on the model and more on data readiness, governance, interoperability, and operational execution. The conference should give customers a closer view of how those pieces come together. The question for CIOs and CDOs is how that direction fits their existing data estate.
The AI problem is usually a data problem
Many AI initiatives stall on data readiness rather than the model itself. The issue is not one statistic. It is the combination of fragmented data, inconsistent definitions, and the effort needed to prepare and govern information before AI can use it reliably.
Copilots and agents need trusted context around customers, financials, inventory, operations, permissions, and business rules. AI exposes existing weaknesses. Duplicate customers, poor product data, and inaccurate inventory become more consequential when AI is recommending or taking action. AI does not create the data quality problem; it makes the operating cost harder to ignore.
The goal is to make more of the existing data estate useful for AI without turning modernization into a wholesale rebuild.
Microsoft Fabric and Azure Databases play different roles
For customers, the question is how Microsoft’s broad portfolio translates into architectural choices, including what should move, what should connect, and what can stay in place without limiting the result the business is trying to achieve.
Azure Databases support transactional and operational workloads, each with distinct performance and application requirements. Fabric provides a unified SaaS experience across data integration, engineering, analytics, real-time intelligence, data science, and business intelligence. Fabric IQ adds a semantic intelligence layer that provides shared business context for Copilot and agents. Some workloads may belong in Fabric, others in Azure Databases, and some on third-party platforms; moving them adds cost without creating enough business value.
The architecture should follow workload, data maturity, economics, and business value, not a single destination.
Connected data has to improve the decision
Microsoft’s broader data platform can connect systems of record and operational applications with analytics and AI. Azure Databases, Fabric, OneLake, Power BI, and governance can play different roles. What matters is whether those connections shorten the path from operational data to a decision, not how neatly the products line up.
A customer record may live in CRM, orders in ERP, inventory in operational databases, and demand models in an analytical environment. Fabric Real-Time Intelligence can add live events to that picture. A manufacturer can stream equipment events or production events, detect a developing issue, and alert teams while there is still time to change the outcome. The value comes from reasoning across that context so the business can understand what is happening, why it matters, and what action makes sense next.
Trusted context turns data into AI business value
I recently wrote in InfoWorld that trusted context is becoming the currency for enterprise AI. That is especially relevant here because once copilots and agents rely on enterprise data, then quality, permissions, lineage, and security become operating requirements, not background data issues.
Knowing inventory is 8,000 units is data. Knowing whether those units are available, committed elsewhere, in the wrong distribution center, or on quality hold is business context. As AI moves from answering questions to taking action, that context determines whether the result can be trusted.
Microsoft frames AI business value around growth, insight, and scale. Each needs an operating definition. Growth can mean new revenue, better customer responsiveness, or growing without adding cost at the same rate. Insight means combining context across Microsoft 365, Fabric, and operational systems to make better decisions faster. Scale means expanding AI without rebuilding governance for every use case.
A reusable data foundation can also reduce pipeline and management effort while improving decisions around pricing, inventory, customers, supply chain, and finance. Microsoft cites coverage of more than 100 standards. The practical question is whether AI can scale without risk and complexity growing at the same pace. For multinational or regulated organizations, that also includes where data is stored and processed and which sovereignty requirements apply.
Agentic workloads raise the bar. Organizations need to know what information informed a decision, who or what had authority to act, which policies applied, and how a wrong action can be reviewed or reversed.
That is where the data platform becomes part of what I describe as decision architecture. Context explains what is happening. Decision logic determines what should happen next. Authority defines what a person or agent can do. Orchestration coordinates systems and people. Execution completes the action and provides a path to review or reverse it. Trusted data feeds that architecture. Without trusted data, automation will be difficult to trust at scale.
Microsoft’s breadth is most useful when it reduces complexity
Microsoft’s advantage may be its enterprise footprint. Azure Databases, Fabric, OneLake, Power BI, Microsoft 365, business applications, developer tools, identity, security, and partners touch many of the places where information is created and work gets done.
Most customers will keep data across multiple systems. Microsoft’s opportunity is to use that breadth to create a simpler path from data to workflow to business outcome without requiring everything to live in one place.
Customer proof points matter because data strategies can get abstract. The KPMG Australia example is useful because KymChat moved beyond an internal productivity tool into a client-facing offering, creating a path from internal AI investment to new business. Microsoft also reports that improvements to the data foundation increased KymChat search quality from 50% to 91%, with results delivered in under a second.
BMW Group provides a more recognizable operating example. Microsoft says Azure helped BMW make vehicle data delivery and analysis 10 times faster, cutting the lead time for engineering insights from days to hours or minutes. Audi adds an AI deployment example. It used Azure AI Foundry, Azure App Service, and Azure Cosmos DB to launch a secure HR assistant in two weeks and is extending the same framework to eight additional agents across the enterprise. Levi Strauss & Co. offers another angle. It consolidated nine ERP systems on Azure, reported a twofold improvement in latency and a 60% improvement in IOPS and now uses Fabric IQ for companywide cost reporting. These are the kinds of proof points that connect platform architecture to time, cost, productivity, and decision quality.
Interoperability matters across the enterprise estate
Microsoft sits in a market with Snowflake, AWS, Google Cloud, Oracle, and others, each with different strengths across infrastructure, databases, analytics, openness, and AI. Databricks also plays an important role, but the connection is different because Azure Databricks is provided as a first-party service directly on Azure. Most large enterprises will remain heterogeneous, making interoperability, cross-platform governance, and incremental modernization central to strategy.
The opportunity is to make Microsoft’s data platform useful across the mix of systems that customers already run. Most enterprises are not starting from a blank sheet, so the value is in helping them connect, govern, and modernize over time without forcing everything into one place.
CIO and CDO strategy should start with the business outcome
This is where the conversation becomes most useful at the Microsoft Fabric and SQL Community Conference Europe. For CIOs and CDOs, an architecture diagram matters more when it is backed by a decision model. A useful data and AI strategy should work backward from the business outcome through five decisions.
Start with the business outcome. Define what needs to improve, including revenue, margin, productivity, customer experience, decision speed, working capital, or time to value. Let the technology choice follow the objective.
Map the data and context required. Identify the operational, transactional, and analytical information, semantics, and business rules that influence the outcome.
Decide what moves and what connects. Determine where Fabric, Azure Databases, and non-Microsoft platforms fit. Let workload, economics, and value drive modernization rather than a single destination.
Design governance, authority, and economics together. Define how security, permissions, lineage, and controls span operational, analytical, and AI workloads, and identify opportunities to simplify pipeline, integration, and management efforts.
Measure the operating result. Set KPIs before scaling. Technology adoption is not the outcome. CIOs and CDOs should know which business measures should change, by how much, and over what period.
That is how the data platform becomes enterprise strategy, tying architecture, governance, AI, and economics to operating outcomes and bringing business leaders into the discussion before the architecture is set.
What customers will still want to understand
Microsoft has many of the pieces required to connect enterprise data, analytics, and AI. Customers will keep looking for guidance on the roles of Azure Databases, Fabric, and the broader data platform, along with cost, licensing, migration, and governance across heterogeneous environments. They will also want more proof points from KPMG, BMW Group, Audi, and Levi Strauss & Co. that start with a business result.
That is where Microsoft’s positioning around a data platform for AI business value comes together. The value is in helping customers connect platform choices to better decisions, faster action, and measurable operating results.
After three decades in enterprise technology, I keep coming back to the same point. The technology is only part of the answer. For CIOs and CDOs, the harder part is deciding how the data platform fits into the way the business operates, how decisions get made, and where AI can improve the outcome. That is really the question behind how Microsoft connects your data across the enterprise. The conference should give customers a useful view of how Microsoft is bringing those pieces together and where that can create better business outcomes.
Before the rise of all-in-one frontier models, artificial intelligence and machine learning libraries tended to focus on highly specialized jobs. Even today this is still a powerful and useful approach: the more specialized the tool, the better the job it will do for its narrowly defined purpose.
One category for such tooling is computer vision: real-time analysis of video or still images. Facial recognition, OCR, motion tracking, object detection, augmented reality, and gesture detection are all common examples of computer vision jobs.
The OpenCV library is easily the most commonly used and long-lived toolkit for computer vision work. OpenCV saves you the trouble of writing from scratch the software needed for any of the above-mentioned tasks. It also gives you a handy bundle of image and video processing tools that can be used in most any project that might need them. It’s also powerful way to take existing or newly trained machine learning models for vision tasks and put them to work for you.
The basics of OpenCV
OpenCV’s original release dates back to the year 2000, when it was originally an Intel Research project. Today the project is under the stewardship of a non-profit foundation, and has gone through several major point revisions. The most recent, OpenCV 5.0, is a significant cleanup and reorganization of the toolkit, with a modernized API structure and better compatibility with the kinds of models and workflows used in machine learning today.
OpenCV is at core a collection of Apache 2-licensed C++ libraries. You can link against those libraries in your own C++ application, or work a different language and use OpenCV through an interface. Python, Java, and JavaScript are all supported, and a separate wasmCV project provides a WebAssembly interface.
OpenCV is designed to run across a broad range of architectures. It also uses a separate hardware acceleration layer, so every platform OpenCV runs on can use the acceleration techniques available for it (e.g., AVX-512). Programs written with OpenCV need not be aware of any hardware acceleration techniques; OpenCV automatically selects the best option on the current system. It’s also possible to build your own acceleration layer if you want to deploy OpenCV on custom silicon or tweak an existing deployment.
OpenCV modules and task sets
OpenCV’s modules are organized by major tasks in the general field of computer vision. Most of them are self-explanatory. For example, the objdetect module is the best place to start if you’re putting together a straightforward object-detection system.
Some high-level OpenCV modules might appear to overlap, but they are meant for different tasks. For instance, the video I/O module gives you a convenient interface for reading and writing video files or image sequences. But there’s also a video analysis module, which handles a different high-level use case: performing motion analysis or object tracking on video or image sets. If you want to generate video (with or without a large language model), use the video I/O module; if you want to analyze video (i.e., track the movement of objects), use the video analysis module.
Many of OpenCV’s functions can be hooked into and customized. For instance, object detection is a high-level module that includes barcode detection. Support for common barcodes (EAN-8, EAN-13, UPC-A, and UPC-E) is already available. But the barcode detector can optionally use an ONNX-format model for detection, so you could have it detect a bespoke variety of barcode, or attempt to decipher damaged or distorted barcodes based on a model.
OpenCV, pretrained models, and inference
OpenCV can work with external models for many other tasks, too, and without having to set up extra software to do it. The DNN module lets you load third-party, pretrained models for tasks like object detection and use them to serve predictions from still-image or video input. You can use models created with TensorFlow or PyTorch, or any other framework that can export a model in the common ONNX format.
OpenCV also curates its own collection of useful models, called the OpenCV zoo, on Hugging Face. The zoo is another good source for models and examples for common tasks, such as deblurring images, pose estimation, and object detection.
One major advantage of OpenCV for serving inferences from these models is that you don’t need anything except the model and what OpenCV itself provides. You do not have to set up a separate runtime for the model to serve inferences from it. This makes it easier to stand up a focused pipeline for a particular task. This pipeline could in turn later be attached to, or built out into, a larger and more complex pipeline if needed. But for basic inference, you can start with OpenCV.
OpenCV extras
OpenCV also comes with a large catalog of extra modules for other, highly specific tasks: direct access to encoding/decoding of video, adding text to images, video stabilization, text detection in scenes, and many more. One of these modules, datasets, provides a way to load well-known, pretrained datasets for common tasks (face recognition or gesture recognition, pedestrian detection, etc.) that can be used to make predictions or to generate performance benchmarks for those tasks.
Another convenient OpenCV feature is the high-level GUI module, which lets you create simple GUIs—windowing, buttons, images, mouse operations—with nothing but OpenCV. This is handy not only for quick prototypes, but even for creating a simple application without having to reach for an entire UI framework like GTK to create the front end. High-level GUI is no substitute for GTK or app frameworks like Electron, but it isn’t meant to be.
Fast and flexible computer vision
OpenCV’s big benefits go beyond giving you a prepackaged set of algorithms to perform computer-vision work. You can use OpenCV also to serve predictions from models for related tasks, like object recognition, without having to spin up extra infrastructure. And you can use it to build basic GUIs for those tasks with no extra libraries—quite handy to have when working with images or video. Finally, OpenCV’s decades of development and ongoing advancement make it a solid base to build on going forward.
Postgres just turned 30. Over those three decades, I’ve watched it pick up hot standby replication, logical replication, asynchronous IO, just-in-time compilation, and dozens of other features built for enterprise workloads. Every one of them solved a real problem and raised the floor on what a DBA needs to know just to do the job well.
I don’t think most people outside this world fully register how much that floor has risen. In the early days, improvements were small and mostly about stability. Now, 30 years in, the focus is squarely on enterprise functionality, and that functionality comes with complexity attached. The more you build, the more there is to understand and manage, and that increases every year.
Hiring more people doesn’t solve this
Most organizations treat a scaling database problem as a staffing problem. Hire enough DBAs and you’re covered. I think that’s the wrong way to look at it entirely.
It’s not far off from the old “too many cooks spoil the broth” problem. You don’t make better soup by adding cooks. You make better soup with a better chef. A 100-terabyte database with serious performance issues doesn’t get fixed by putting 10 DBAs on it instead of one. It gets fixed by someone who has seen that failure before and knows exactly where to look.
Experienced DBAs are expensive and hard to find, and the problem is only getting worse. A few years ago, it was IoT driving demand for time series expertise at scale. Now it’s AI, and organizations are building enormous databases to support retrieval pipelines and large language model (LLM) training, often without the experience on hand to run them safely.
Where human error comes from
Humans are unpredictable. We make assumptions. Sometimes, we think the process is getting in the way, so we take a shortcut around it. And we make mistakes because that’s simply human nature.
I don’t think that’s something we’re ever going to fully engineer away as long as a person is in the loop, and honestly, we want a person in the loop making the real decisions. So it becomes a balancing act. You have to understand the mistakes people tend to make, build the discipline to follow good processes consistently, and still know when a situation calls for judgment rather than a checklist.
Why AI helps more than people expect
I think this is where AI has earned its place. It’s not as a replacement for the DBA, but as something that follows a process more consistently than a person under pressure tends to.
Modern AI systems still have problems with hallucination and getting the full picture right. However, with the right guardrails, auditing, and tracking in place, they are able to follow the process very well. It is much easier for me to say, “Forget it, I know a shortcut, I’ll just push this through,” than it is for a well-constrained model to deviate from its instructions. As long as the instructions are clearly defined, LLMs tend to stick to them.
Another advantage worth naming is if an LLM is doing the diagnostic work, you can put much stronger gating around its ability to actually make changes. You can let it follow a tight, well-defined process to diagnose a problem while still requiring a human to approve anything that touches the live system, whether that’s creating an index, dropping one, or changing a configuration value.
That combination gets you to a solution far faster than the old way, using the model’s training plus whatever supplemental knowledge you feed it through retrieval-augmented generation (RAG), while keeping a person at the one point where it actually matters. Did the model get the diagnosis right? Does the proposed fix line up with the actual problem? That’s the question a human needs to answer before anything gets applied.
The industry hasn’t figured out the limits yet
I am not sure we are far enough along the path as an industry to really know what we’re getting wrong. I am working through these questions myself right now, looking at autonomous agents for Postgres operations and where the real risk is.
What’s clear to me already is that strong guardrails matter in two places, not one. You need guardrails on the prompts so the model verifies its findings and doesn’t hallucinate into a bad recommendation. And you also need guardrails on what the model is actually allowed to do. Letting an agent create an index carries a manageable risk. Letting it truncate your orders table does not. Auditing matters just as much; when something does go wrong, you need to see exactly what happened and undo it. The fallback needs to stay in place until the industry broadly trusts these systems more than it trusts a human.
Picture a load spike that hits overnight, the kind where batch processing that normally runs fine suddenly does not. The old way means spending hours trawling logs, index stats, table stats, and query performance before you land on a likely cause and you can apply a fix.
Put an LLM on that same problem and the hours-long process compresses into minutes. It’s the same productivity shift we’ve already seen in software development, where an experienced user with the right skills and guardrails can get LLM output that matches or beats a junior developer’s work. The quality comes down to this process: Structure the work, define a clear spec, build the implementation plan, then verify each stage before moving on.
The same discipline applies to database operations. The worst thing you can do is rush in and say to the model, “This happened, go fix it.” You need to guide the tools properly to get good results from them. When it’s done right, it works.
Where this goes from here
A few things need to be true for this to scale across the enterprise. Frontier models already understand Postgres well, but they don’t know everything, so you need to feed them supplemental knowledge alongside that training. Full documentation matters here, so do internal runbooks. The process has to require the model to actually consult that material rather than coast on what it already knows.
I expect the same kind of structured skills that are common in AI-assisted development, the step-by-step approach to planning and verification, to show up specifically for Postgres operations. That will make the job easier for DBAs, and easier is what matters.
We’re probably not far from agents that sit and watch a Postgres instance around the clock and rarely need a human to step in. The slower part is trust. For example, look at how long it took people to actually believe AI-written code was any good. I would guess it is the same pattern here, but I don’t doubt for a second that we will get there.
In the meantime, you do not need to hand an LLM the keys to your database tomorrow. Start smaller and begin with read access to your monitoring data. It’s a low-risk place to start.
An agent watching constantly can catch the kind of slow, compounding anomaly that threshold-based alerting tends to miss, a gradual rise in IO alongside a gradual rise in memory, and flag it before it becomes the kind of outage that costs real money every hour it runs.
That’s not a small thing. The DBA shortage is not going away on its own, and waiting for the industry to fully trust autonomous agents before using AI at all is the same as falling behind. The teams that figure out where the human belongs in this process are the ones who’ll be ready when the rest of the industry catches up.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.
Anthropic has owned up to a fourth security incident involving its AI model, Claude, escaping onto the open internet and attacking other organizations during a test of cybersecurity abilities on what was believed to be a closed system.
After this discovery, the company instigated a wider search of 481 million transcripts, covering all those from its Frontier Red Team, some non-cyber evaluations, reinforcement learning environments, and more, to see if any other incidents had occurred. So far, this search has only identified the four already-known incidents, it said.
It has also reported details of all the previous incidents to the non-profit lab Model Evaluation and Threat Research (METR), which has agreed to conduct an independent investigation.
Anthropic is not revealing too many details of its latest discovery. It has contented itself with saying that it was due to a misconfiguration which mistakenly connected to the open internet, when the simulation was meant to be without such access. It also said that it all four faults were with the same evaluation partner. It has asked METR to investigate all the incidents. The company said that this latest revelation was not connected to the Mythos incident reported by the UK’s AI Security Institute last month.
OpenAI on Wednesday introduced a new Agents API that brings the agent harness and infrastructure behind Codex to developers, potentially giving enterprises a way to build custom AI agents while removing much of the orchestration and infrastructure management traditionally required to make such agents work.
That reduction in engineering complexity is possible because the Agents API is a managed service, with OpenAI hosting and maintaining the underlying harness and infrastructure.
Previously, developers building a custom agent typically had to assemble the components needed to support its work, including an agent runtime, context and session management, tools and external data connections, execution environments, and associated infrastructure.
OpenAI itself already offers several of those building blocks through products such as its Responses API, which developers could use to combine models with built-in capabilities including web search, file search and computer use, and its Agents SDK for defining and orchestrating agent workflows.
Agents API, which is currently in public beta, in contrast, can help developers design a custom agent in a single API call after they specify the task, model, tools, and the environment, the model provider wrote in a blog post.
For that execution environment, developers can choose to run agents in an OpenAI-managed sandbox, on their own infrastructure, or through supported sandbox providers, including Blaxel, Cloudflare, Daytona, DigitalOcean, E2B, Modal, Oracle, Runloop, and Vercel, it added.
These options give enterprises flexibility to choose between fully managed environments and deployments within their own VPCs, as well as different approaches to file and secret storage and compute configurations based on their workloads, it further explained.
The API could reduce time to production
The Agents API “significantly reduces” engineering work, helping developers spend more time building the actual business application instead of the agent infrastructure, said Pareekh Jain, principal analyst at Pareekh Consulting.
“The main advantage with the Agents API is fewer moving parts. A long-running agent built by hand needs a job queue, a state database, a sandbox fleet, a compaction routine and a retry policy, and someone has to own and page for each of them,” echoed Amit Kumar Jena, AI development head at IT consulting firm Kanerika.
That reduction in engineering work, in turn, could help CIOs reduce the time required to roll out an agent in production and subsequently scale it.
“Enterprises may need fewer engineers to build the infrastructure around each agent, which should reduce development time,” said Phil Fersht, CEO of HFS Research.
That time advantage, Jena said, could be critical for most enterprises as agent development usually stalls between a working demo and a system capable of running unattended for hours, with infrastructure-related issues often becoming the key bottleneck to moving an agent into production.
Convenience at the cost of lock-in?
The reduction in engineering effort and potentially faster path to production, however, does not eliminate the tradeoffs CIOs must consider, including greater reliance on OpenAI for both the underlying agent infrastructure and the models powering it.
“Lock-in is the biggest concern. If OpenAI provides the model, context management, tools, orchestration, and execution environment, moving to another platform becomes harder,” Jain pointed out.
That dependency could also weaken an enterprise’s negotiating position on pricing and terms, Jena said.
There are also issues around data privacy and governance as the new API “doesn’t support Zero Data Retention” even when enterprises use their own sandbox, Jena pointed out.
That, in turn, could limit its adoption, especially in industries in the regulated sectors, such as healthcare and BFSI, Fersht said.
Rather, Startups, SaaS companies, and enterprises already using OpenAI are more likely to adopt the new API more readily as it reduces development effort, Jain said.
“Enterprises pursuing multi-model strategies may also prefer their own independent harness or are likely to take a hybrid approach,” Jain added.
Agents API’s adoption could be further limited due to the competition it faces from hyperscalers and rivals, analysts pointed out.
“This is a crowded category. Anthropic’s Claude Managed Agents has been in public beta since April. AWS also offers Amazon Bedrock AgentCore, whose managed harness became generally available in June and does the same category of work, meaning the loop, tool execution, context, state, and recovery, while letting you use any model and switch providers mid-session without losing context,” Jena pointed out.
Other competing offerings include Microsoft’s Foundry Agent Service and LangGraph, Jena added.
For 15 years, public cloud providers have defined the industry for most enterprises: on-demand, automatically metered services delivered over the open internet that give developers immediate access to storage, databases, compute, application development platforms, and AI capabilities. That model remains dominant because it is simple, well-instrumented, and universally supported. Yet an adjacent reality has always existed. Large-scale, off-market capacity deals where technology companies with surplus GPUs, storage, and compute sold blocks of capacity in bulk to other firms under nondisclosure agreements were once common. Those were cloud transactions in substance, since you were using somebody else’s servers. However, they lacked the automation, metering, and governance that define public cloud services, and they operated entirely outside the frameworks enterprises rely on for budgeting, compliance, and accountability.
A shadow market becomes visible
These arrangements are now becoming more visible and more formalized. Some are even openly auctioned. Meta’s entrance into the cloud capacity space, essentially offering its excess compute infrastructure to outside buyers, represents the maturation of a market that has existed in back rooms for years. Large, multiyear capacity commitments are no longer whispered about during boardroom lunches. They are being announced, financed, and tracked by analysts. This shift is creating a distinct layer in the cloud landscape, one that sits between traditional hyperscalers and true private clouds, and one that enterprises can no longer afford to ignore.
The practical implication is that the cloud market you thought you understood now has a parallel track. When a large enterprise needs maximum GPU capacity for model training or inference, it has more options than a year ago, and those options come with genuinely different trade-offs. You can go to a hyperscaler and pay the published rate for metered GPU as a service, receiving a fully managed platform with all the tools, governance, and integration you need. Or you can reach out to a technology provider that happens to have idle or excess GPU capacity and negotiate a bulk deal at a fraction of the published rate. The first option provides predictability and depth of service. The second offers raw cost savings if you have the operational capability to handle less-managed infrastructure.
This bifurcation is muddying procurement decisions. The gap in pricing between bulk off-market capacity and public cloud offerings can be substantial. Some analysts cite differentials ranging from 10 to 100 times. But headline GPU rates are only part of the evaluation. Bulk deals tend to come with minimal metering, limited monitoring, and a do-it-yourself culture around incident response and optimization. You are, in effect, purchasing wholesale energy. It is robust and capable, but it requires your own wiring and maintenance. Organizations accustomed to the instrumentation and service-level agreements of hyperscalers may find such a different operational reality a bit jarring. Those with mature cloudops teams may find the economics compelling.
Who benefits and who adjusts
The hyperscale incumbents will not be displaced overnight. They offer the complete ecosystem, not just GPU frameworks but development tools, hundreds of managed services, identity, compliance, and integration layers that are genuinely difficult to replicate. However, they are now facing a market where customers with intensive computational needs can source raw GPU capacity from alternative suppliers, particularly for bursty or cost-sensitive workloads. This is not necessarily a threat to hyperscaler dominance, but it is a complication. The hyperscalers will likely respond through partnerships, capacity-coverage agreements, and possibly acquisitions of promising capacity players. Effectively, they will become customers of the same off-market ecosystem they once ignored.
The cloud market is adding a new axis of choice: managed breadth versus specialized efficiency. Enterprises will increasingly construct hybrid operating models, keeping steady-state, security-sensitive, and toolchain-dependent workloads on hyperscaler platforms while offloading cost-intensive training runs and elastic inference to bulk-capacity providers. This is not a binary choice. It is an expanding continuum of sourcing options that demands more sophisticated procurement and architecture planning.
How to take advantage
Know your actual workload cost before you negotiate. If your AI workloads are compute-intensive, model training-heavy, or require burst capacity at unpredictable intervals, the economics of bulk capacity deals can be compelling. Run a pilot that captures total cost of ownership, including data movement, integration effort, operational overhead, and potential downtime during self-managed troubleshooting before you assume bulk is simply cheaper. Compare fully loaded costs, not GPU-hour sticker prices.
Build portability into your AI stack from the start. Bulk-capacity providers often lack the governance, compliance tools, and model life-cycle management enterprises need. Mitigate this risk by standardizing on containerized inference runtimes, open model formats, and CI/CD pipelines capable of deploying across multiple targets. This type of discipline turns vendor diversity into negotiating leverage rather than operational liability. It also ensures you can shift workloads as capacity markets fluctuate.
Treat capacity sourcing as a strategic procurement function. Bulk cloud capacity is not a niche quirk. It is becoming a recognized market that rewards supply-chain thinking. Diversify your supplier base, negotiate multi-vendor commitments where possible, and develop a reserve strategy for critical workloads. Keep your hyperscaler relationship for reliability and breadth, but maintain at least one secondary capacity arrangement for cost-sensitive or elastic workloads. The enterprises that treat capacity as a variable asset, rather than a fixed infrastructure decision will be better positioned as this market continues to expand and formalize.
The cloud market is no longer a simple fork between public and private. A defined middle ground of negotiated bulk capacity is now attracting billions in investment and reshaping how AI infrastructure costs are allocated. Enterprises that understand this landscape and build flexible, portable architectures that can span it will capture more value from AI investments than those who default to the familiar hyperscaler menu alone.
When I talk to people about agents interacting with websites, the conversation almost always starts with perception: how does an agent “see” a page? Is it screenshots or the DOM or the accessibility tree? In my previous InfoWorld article,“Accessibility is the first-class interface for AI agents”, I argued that the accessibility tree is becoming an increasingly important interface between websites and AI agents. From there the conversation shifts to actuation, and to ways of making it less brittle, which let a site expose its own actions as callable tools instead of forcing an agent to guess at buttons and forms.
That conversation matters. But it leaves out a more consequential architectural question, one that determines who is accountable when an agent gets something wrong: where does the agent live?
There are three plausible homes for an agent that interacts with your business: on-site, in-browser and off-browser. Each one trades control for reach differently. Right now, much of the industry’s attention is focused on the two homes furthest from the business: browser extensions that co-browse alongside a user, and autonomous agents running in the cloud that talk to your APIs or drive a browser on their own.
I think that is the wrong place for most enterprises to start.
If you run a business with real compliance exposure, or a brand you cannot afford to have misrepresented, the pragmatic move is to build the agent that lives on your own site first and treat reach as a problem you solve later, on your own terms.
Reach without governance is a liability
The appeal of in-browser and off-browser agents is obvious. They promise discovery, scale and automation you do not have to build or maintain yourself. An off-browser agent can compare your prices against every competitor and keep working after your support team has gone home. A co-browsing extension can act inside a user’s already authenticated session without requiring any special integration on your side.
But an agent you do not control is still speaking for your business.
An off-browser agent talking to your APIs or scraping your DOM runs on a model you did not choose, follows guardrails you did not write and represents your brand in a conversation you cannot see. If it misquotes a price, misstates a policy or initiates a transaction that creates a customer dispute, the fact that the agent belonged to somebody else may not make the resulting problem disappear.
There is also a deeper security problem. Computer-use agents interact with untrusted content by design. Anthropic’sguidance for browser and computer-use agents notes that webpages and application interfaces can contain adversarial instructions and recommends scoped permissions, human confirmation for high-stakes actions, and monitoring of agent behavior. OWASP’sAI Agent Security Cheat Sheet goes further, identifying agent-specific risks including prompt injection, tool abuse and privilege escalation, data exfiltration and memory poisoning.
In-browser agents are a better deal, because a human is watching the same screen and can catch a mistake before it lands. But the model, the tone and the decision logic are still outside your hands. For or any other regulated industry, that is not a risk to take lightly just to pursue reach you have not yet learned how to govern.
What you actually gain by building at home
An on-site agent is less glamorous than the idea of your product being discovered by roaming agents across the open web. But it is the only home where the business keeps full governance.
You choose the model. You define the guardrails. You decide the tone, the escalation paths and exactly what the agent is and is not allowed to promise a customer. Because the agent has direct access to your systems, it does not need to reverse-engineer your checkout flow as an outside agent would. It can be given your actual domain knowledge and your actual APIs.
That matters because enterprises have solved this problem before. They did not expose databases directly to the internet; they introduced APIs. APIs were not just about integration. They created contracts, authentication, auditing and policy enforcement. On-site agents represent a similar evolution. Rather than forcing an external agent to infer behavior from a user interface, a business can expose governed capabilities through tools with explicit contracts.
WebMCP is a promising step in that direction. It makes it easier for a site to publish actions an agent can invoke, instead of forcing the agent to guess at the page. The important shift is not that the agent can click a button. It is that every action becomes observable, policy-governed and replayable.
The Chrome team’s WebMCP Sports demo is a good preview of what this can look like done well: an on-site assistant calling the site’s own WebMCP tools, with every call visible in the chat log. That is not a black box you are hoping behaves. It is an auditable interaction, with named tools and defined contracts.
The three homes
The cleanest way to think about this is as a spectrum.
On-site. The agent lives on your website. You ship it. You own the model, the tools, the tone, the policies and the telemetry. It has first-party access to your systems and your workflows. The trade-off is reach: it is coupled to your site, and it is not meant to carry the user’s journey across the web.
In-browser. The agent lives inside the user’s browser session, usually as a co-browsing extension or a browser-integrated assistant. It benefits from the user’s logged-in state and can act with the user present. That gives it real reach and a strong human safety net. But the business does not control the model or the guardrails, and the agent can still misrepresent the brand.
Off-browser. The agent runs elsewhere — in the cloud, on a server or on a user’s local machine and reaches your business by talking to APIs or driving a browser remotely. This is the widest-reach model, but also the least governed from the business’s point of view. It may compare competitors, cross workflows and act at scale, but you do not control the experience.
None of these homes is inherently bad. They solve different problems. But if your first priority is governance, accountability and brand integrity, on-site is the right starting point.
Start with the governed interface
For teams deciding where to put their first investment, three moves make the difference between a real interface and a chatbot glued to a homepage.
First, ship the on-site agent as the authoritative interface, not a fallback. It should have first-class access to your own APIs and domain knowledge — the same systems your support team and checkout flow already use — rather than being handed a scraped version of your own page.
Second, instrument every tool call it makes. If your agent calls a WebMCP tool to check inventory or apply a discount, log it the way you would log an API call from any other client. That log is your audit trail, your debugging tool and eventually your evidence when a regulator or customer asks what happened.
Third, design for human escalation from day one. Even a well-governed agent should not make every decision autonomously. Build explicit handoff paths to human operators for ambiguous or high-risk situations.
Taken together, those choices turn an on-site agent into a governed interface rather than a novelty feature. That distinction matters. The first is something an enterprise can safely operate; the second is something it will eventually have to defend.
When agents start talking to agents
There is another reason to build the on-site agent first: the three homes are unlikely to remain isolated.
Imagine that a user’s browser agent arrives at an airline website. It already knows the traveler’s preferences, calendar constraints, loyalty programs and perhaps their budget. The airline’s on-site agent knows something completely different: routes, seat availability, fare rules, upgrade eligibility and the airline’s operational policies.
Why should the visiting agent reverse-engineer all of that from a website?
A more natural architecture may be for the two agents to communicate.
This is no longer purely hypothetical. TheAgent2Agent (A2A) protocol is explicitly designed to allow independent agents built by different vendors and frameworks to discover capabilities, exchange information, delegate work and coordinate tasks.
In that world, the user’s agent brings intent and user context. The site’s agent brings domain knowledge and governed execution.
And suddenly the on-site agent’s limited reach does not look quite so limiting. It does not need to travel across the web if other agents can come to it.
The incentive problem nobody is pricing in yet
There is a harder question underneath the technical one: who does the on-site agent actually work for?
A visiting agent, co-browsing or otherwise, is acting on the user’s behalf. An on-site agent, however well-built, is the business’s employee. And the moment the two negotiate a discount, an upgrade or a substitution, that exchange stops looking like a neutral handoff and starts looking like a sales conversation with an AI on each side of the table.
That tension is going to matter.
Businesses building on-site agents now should design their guardrails with that scrutiny in mind, rather than treating “the agent decided” as a shield. Control is worth having. It is also a responsibility. And building the agent that answers to you is the only way to exercise it.
The first agent should be the one you can govern
The history of enterprise software suggests a familiar pattern. Organizations adopt new interaction models by first bringing them under their own governance. Internal systems become APIs. APIs become platforms. Platforms become ecosystems.
AI agents will likely follow the same path.
That is why I think the first agent your business builds should not be the one that can reach the entire web. It should be the one whose behavior you can understand, measure and improve. Once that foundation exists, expanding outward becomes an architectural choice instead of a leap of faith.
Reach is tempting. But for regulated and brand-sensitive businesses, control has to come first.
Every AI feature you’re building is based on a price that isn’t real. Current AI services are heavily subsidized as model providers seek to expand their market share, and subsidies inevitably end. Just think back to the early days of Lyft and Uber, when a single ride often cost less than taking public transit. As the costs associated with building on cheap AI infrastructure slowly start to reflect reality, the architecture decisions you made to get to market quickly may be the ones hemming you in.
This isn’t unique to AI. IT leaders have always had to weigh speed-to-value against the cost of dependency. What’s different now is how fast those dependencies are being built, and how little visibility most teams have into them.
Across IT, scalability is contingent on portability. And architecting with portability in mind is the best way to minimize lock-in as AI pricing models evolve, giving you leverage in future contract negotiations, and a fighting chance of switching platforms if the costs of one vendor get too high.
The challenge is that AI providers are incentivized to make leaving as painful as possible, which they achieve through deep integrations and proprietary features. You can control some of this through architectural decisions. Other parts are structural dependencies you should at least be aware of going in.
Swapping platforms will never be painless, but it’s less painful than being locked in when your vendor decides it’s time to recoup its losses, as every AI provider eventually will.
The part you control
The foundation for portability is the abstraction layer, an internal service your applications call instead of hitting Claude, GPT, or Gemini directly. It uses a request format defined by you, and translates that to whatever the vendor behind it expects. If you need to change vendors later, it’s closer to a configuration change than a code rewrite.
You don’t need to build this from scratch; open-source gateways like LiteLLM or Portkey already handle the routing and interface. Scoped to what you actually need, it’s a small project that one or two engineers can complete in a few weeks.
From there, audit the parts of your codebase that call a vendor’s API directly and route them through your proxy layer one at a time (have fun with that). Skip the features you’re not using yet, like fine-tuning or batch APIs, and focus on the 80% case. Think: text in, text out, and maybe tool calls.
The parts you don’t control
The API call is the easy part to abstract. Other dependencies may be tougher to account for given the pressure to deploy quickly, but you should at least be aware of them.
Prompts get tuned to a specific model’s characteristics, and those instructions won’t necessarily make sense to another model. If you do switch vendors, expect to re-tune each prompt. Fine-tuning a model will box you in, unless you use an open-weight model that you operate internally.
Vectors generated by one provider’s model won’t be compatible with another’s, so switching means re-embedding your knowledge base. Tool calling and structured output conventions differ by vendor, so code written around one model’s schema will need rewriting for another. Caching and batching mechanics carry their own vendor-specific rules and cost implications that don’t map across providers.
In short, code portability is cheap. Behavioral portability is the real cost, and no gateway tool solves that for you.
Build with your eyes open
Most teams don’t realize how locked into a model they’re becoming until they’re forced to move, which is the worst time to find out. How many parts of your codebase import a vendor SDK directly, versus calling a shared internal function?
If you want a real answer instead of a guess, pick one workflow that isn’t business-critical and try porting it to a different vendor. See what breaks and how long it takes to fix. That will uncover your dependencies better than a checklist.
The bottom line is that you should treat AI providers like any other infrastructure vendor where concentration risk matters. Hold AI providers to the same standard you would apply to any IT or technology vendor. There will likely always be some level of dependency, but you should know where the hooks are and choose them deliberately. That way, you’re prepared when AI price hikes inevitably arrive.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.
Frontend teams have become pretty good at monitoring what happens after an application reaches production. We track JavaScript errors, API failures, latency and Core Web Vitals. When something changes after a release, these signals help us determine whether the application is healthy.
But there is one gap I think frontend teams often overlook. An application can look completely healthy in those dashboards while an important workflow is unusable for someone navigating with a keyboard or assistive technology.
Imagine a checkout flow where a recent UI change makes the final button unreachable by keyboard. The page still loads. The API responds successfully. There are no new JavaScript exceptions, and performance metrics look normal. Every traditional signal says the application is working, yet some users can no longer complete the purchase.
That is why I have started thinking about accessibility as more than a compliance or pre-release testing concern. If an accessibility regression prevents someone from completing a critical workflow, that is also a production reliability problem.
Where traditional frontend observability falls short
Most frontend observability focuses on technical health. We want to know whether pages load quickly, requests succeed and deployments introduce errors. These are useful signals, but they mainly tell us whether the software is functioning as the system expects.
Accessibility failures are different because they often do not produce obvious technical errors. Removing an accessible name from an icon button will not necessarily throw a JavaScript exception. Replacing a semantic button with a clickable div might leave mouse interaction working normally. A focus-management regression inside a modal can make the interface extremely difficult to navigate while having no effect on page performance.
This creates a blind spot. Our monitoring can tell us that the button rendered and that clicking it successfully triggered an API request. It may not tell us that a keyboard user cannot reach that same button.
Automated accessibility checks in CI are an important first line of defense, and I use them as an early guardrail. But I would not treat a passing CI check as proof that the production experience is accessible. Production applications contain changing data, feature flags, third-party scripts and combinations of components that may not be represented in isolated tests.
Automated tools also have limits. They can identify many structural problems, but they cannot detect every accessibility or WCAG conformance failure. CI should therefore be an important checkpoint, not the end of accessibility verification.
Monitor accessible journeys, not just healthy pages
One way I think frontend teams can close this gap is by applying the same idea behind synthetic monitoring to accessibility.
Instead of checking only whether a page loads, we can test whether important user journeys remain usable. Can someone navigate the primary controls using only a keyboard? When a dialog opens, does focus move somewhere sensible? Can a form be completed without relying on a mouse? Do important controls still expose useful names and roles?
Consider the checkout example again. Traditional synthetic monitoring might verify that the page loads, products appear and the checkout API responds. An accessibility-aware journey could go further by navigating through the same flow with keyboard interactions and verifying that focus reaches the controls required to complete the purchase.
I would not try to automate every accessibility requirement this way. The goal is to identify a small number of critical workflows and ask a simple question: Can a user still complete the task?
For applications with a design system, I would extend the same thinking to shared components. A regression in a button, dialog, form field or navigation component can spread across many screens at once. Testing keyboard behavior, semantics, accessible names and focus expectations at the component level can catch problems closer to their source instead of discovering the same regression page by page.
This creates two useful layers of protection. Design-system checks help protect the building blocks, while synthetic journeys help verify that those building blocks still work together in real application flows.
Make accessibility part of release health
Frontend teams already look at signals after deployment to decide whether a release is healthy. We watch error rates, latency, failed requests and performance changes. I think accessibility signals can become part of that same conversation.
That does not mean creating another huge dashboard or turning every automated accessibility warning into a production incident. The useful approach is to focus on impact.
If a release introduces a minor issue in secondary content, the team may address it through the normal development process. If the same release prevents keyboard users from completing checkout, submitting a form or navigating a primary workflow, that deserves a different level of attention.
This is where accessibility-aware release health becomes valuable. Teams can define a small set of expectations around critical journeys and shared components, then watch for regressions just as they would watch other frontend reliability signals.
The need for this approach is becoming harder to ignore. The2026 WebAIM Million analysis found an average of 56.1 detected accessibility errors per home page, a 10.1% increase from 2025. It also found that the average home page contained 1,437 elements, a 22.5% increase in a single year. As interfaces become larger and more dynamic, relying only on occasional accessibility audits becomes increasingly difficult to scale.
If users can’t complete the workflow, the system isn’t healthy
Frontend observability has given us increasingly sophisticated ways to understand what applications are doing in production. But our definition of application health should not stop at whether JavaScript executed, APIs responded or pages rendered quickly.
A system can pass every one of those checks and still fail the person trying to use it.
Accessibility testing during development remains essential, but production-oriented signals can help close the gap. Synthetic user journeys can tell us whether critical interactions still work. Design-system regression tests can protect shared components. Accessibility-aware release checks can help teams recognize when a deployment has affected a critical workflow.
For me, the principle is simple. If users cannot complete an important task because of an accessibility regression, I do not think we should describe the application as healthy just because the traditional dashboards are green.
That is not only an accessibility problem. It is a frontend reliability failure.
Databricks on Wednesday introduced Adaptive Instructed-Retriever, a new retrieval model designed to improve enterprise AI search by taking additional steps for complex queries while stopping early on simpler ones in order to help its customers balance answer quality, latency, and cost.
The new model builds on research behind Databricks’ earlier Instructed-Retriever-1 but takes a different approach to handling complex queries.
While Instructed-Retriever-1 uses parallel, single-step search and incorporates enterprise data schemas and custom instructions to improve retrieval, Adaptive Instructed-Retriever can combine parallel retrieval with sequential, multi-step search when additional evidence gathering is needed.
That distinction matters because applying multi-step search to every query can increase latency and the computational resources required for retrieval, while limiting every query to a single search step can hurt results for complex, multi-hop questions.
Adaptive Instructed-Retriever, the company said, is designed to take additional steps only when they are likely to improve retrieval quality, allowing it to return earlier on simpler requests.
To achieve that capability, Databricks trained the model using synthetic enterprise retrieval environments and an agentic data synthesis process, reusing training data from Instructed-Retriever-1 while adding synthetic multi-hop questions designed to benefit from multiple search steps.
It then used online reinforcement learning to train the model to weigh the potential improvement in retrieval quality against the cost of additional search steps, rewarding high-performing search trajectories while penalizing steps that did not produce corresponding gains.
That training approach, the company said, can produce multiple versions, or checkpoints, of the model with different quality-latency trade-offs, allowing one trained with a lower penalty for additional search steps to spend more time improving results and another with a higher penalty to limit search steps and reduce latency.
For customer enterprises deploying the model in production, these versions or checkpoints would allow retrieval performance to be matched more closely to the requirements of different applications rather than applying the same search strategy across all workloads, it added.
Adaptive retrieval could help control AI search costs
That flexibility could become increasingly valuable as enterprises move AI agents and other applications into production, where improving answer quality often requires additional inference, compute and retrieval resources that can raise the cost of running them at scale, analysts say.
“Repeated searches can improve answers to complex questions but add latency and compute costs, while a single search can be faster and cheaper but fall short on harder queries. The ability to determine when additional search is worthwhile could therefore help enterprises avoid spending resources without improving answer quality,” said Ashish Chaturvedi, executive research leader at HFS Research.
For CIOs, that ability to limit unnecessary search could eventually offer greater predictability over compute consumption and costs.
“The problem with agentic AI at scale is that consumption is hard to forecast, agents searching and re-searching create compounding, unpredictable cost and latency, and finance teams hate these variable bills. Knowing your agents will search within a defined ceiling, and that you can set that ceiling per workload, is what makes agentic search safe to run at scale rather than a runaway meter,” Chaturvedi noted.
The economics could become even more compelling, Chaturvedi added, if the specialized model can deliver the claimed retrieval quality of larger general-purpose models with lower latency.
Adaptive Instructed-Retriever, according to Databricks, matched or exceeded the retrieval quality of Claude Sonnet 5, GPT-5.6 Luna and DeepSeek-V4-Flash in its internal evaluations while completing requests in 5.8 seconds, or more than twice as fast as those models.
That difference could matter economically because retrieval is a high-frequency operation for data agents, Chaturvedi said, potentially allowing enterprises to avoid applying the cost and latency of larger models to every retrieval request.
Developers could shed some retrieval orchestration
There are advantages for developers too, especially in terms of reducing orchestration-level work.
Traditionally, developers handling complex, multi-hop questions often have to design the control flow for multiple search rounds themselves, including deciding when to refine a query, search again, or stop, but Adaptive Instructed-Retriever’s learned search policy shifts those decisions into the model, potentially reducing the amount of bespoke retrieval orchestration developers need to build and maintain, Chaturvedi said.
That, in turn, could make sophisticated retrieval capabilities more accessible to teams that lack the resources, expertise, and time to build a search agent from scratch, echoed Stephanie Walter, practice leader of AI stack at HyperFRAME Research.
More flexibility brings new trade-offs
However, there are trade-offs that developers should consider, analysts pointed out.
While the model’s checkpoints can help prevent individual queries from consuming an unpredictable amount of compute, teams will still need to test and monitor different settings to determine which one of these delivers the best balance of cost, latency and retrieval quality for each application or use case, Walter said.
That, in turn, adds another layer of performance and cost optimizations developers need to manage, Chaturvedi echoed.
More so because being able to choose the right model checkpoint means working through real business questions and documenting where answers are coming from in your database or other knowledge bases, said Mike Leone, principal analyst at Moor Strategy and Insights.
“Very few enterprise development teams have built such a curated list. In practice, that means most teams are likely to simply select a setting once and leave it unchanged,” Leone added.
Even with the right checkpoint in place, however, Adaptive Instructed-Retriever does not address every challenge that can affect retrieval quality. Teams would still need to prepare and organize their data, enforce permissions, evaluate retrieval quality, and verify final answers, Walter said.
Echoing Walter, Leone added that a smarter search policy cannot compensate for poorly structured underlying data or a lack of agreement over what key terms mean, meaning it could still retrieve the wrong answer even if it does so more efficiently.
There are also caveats that CIOs should take note of.
Adaptive Instructed-Retriever’s performance and quality claims, Chaturvedi said, are based on Databricks’ own benchmarks against competitors selected by itself, meaning that these claims will need to be tested against an enterprise’s actual workloads rather than accepted at face value.
Even if those results hold up, however, CIOs will still need to decide whether the potential gains in retrieval performance, latency, and cost justify adding a specialized model to their AI stack.
Rather, using a larger general-purpose model for both retrieval and reasoning could offer a simpler alternative by reducing the number of models enterprises need to deploy and manage, Chaturvedi said. The economic case for Adaptive Instructed-Retriever for CIOs, therefore, will depend on whether its claimed performance holds up on real enterprise workloads and whether the potential savings outweigh the additional complexity of using a specialized model, Chaturvedi added.
Login
