OpenAI has signed a partnership agreement with Visa that allows the company’s AI agents to use the payment card for e-commerce transactions. The agreements lets users shop for everything from groceries and diapers to airline tickets without having to manually enter a lot of information.

“As AI agents become active participants in the economy, Visa’s focus is on ensuring that transactions are reliable, secure, and seamless,” Visa Chief Product and Strategy Officer Jack Forestell said in a statement, according to AP.

The pact means AI agents can complete purchases on a user’s behalf at virtually any merchant that accepts Visa. Details about the financial terms of the agreement, or whether specific transaction fees will apply, were not immediately detailed.

Developers are caught between the joy — or pressure — of using agents to ship 10x faster today and the dread of how they will maintain that code tomorrow. The gap between “vibe” code and code that can be deployed to millions of users is vast and easy to underestimate. Closing the gap requires care, expertise, and effort, with the payoff coming later. Agents are able to complete increasingly complex programming tasks but without the quality we need. What’s missing, and how can we fill the gap?

Why agent-generated code degrades: the bloat problem

Enterprise code has to clear three bars: it must be maintainable, reliable, and secure. Out-of-the-box AI agents can miss all three. Let’s focus on the biggest and most visible maintainability issue, which is bloat: redundant validation, defensive checks that cannot fire, near-duplicate functions, dead code that nothing removes. A None check on a parameter typed as dict. A try/except around a call that never throws. Two functions, identical except for the negation in their return statement.

Bloat varies dramatically by model. Sonar’s LLM Leaderboard runs every frontier model through 4,400+ Java tasks and analyses the code generated. To complete the benchmark, GPT-5.4 High generated 1,159,000 lines of code at an 81.05% pass rate, while Claude Opus 4.7 Thinking generated only 336,000 lines of code to return a better than 82.52% pass rate. Different models generate dramatically different code to achieve similar outcomes.

Bloat is not just messy. Carnegie Mellon researchers studied 807 open-source projects that had adopted Cursor, matched against 1,380 controls, measured by SonarQube. A short-term velocity gain disappeared by month three, while static analysis warnings rose 30% and code complexity rose 41% — both persistent. The harder it became to change the codebase and the more bugs it contained, the more the velocity was dragged down. Any experienced developer will know how this goes: quality problems compound until the code feels impossible to change and the only option is the dreaded rewrite.

Three forces produce bloat once a model is in use:

1. Agents do not feel the maintenance burden. Armin Ronacher, the creator of Flask, made the point on the Pragmatic Engineer podcast in late April. Humans feel the cost of bad code over time, and as Ronacher put it, “if the pain gets too big, you as a human are incentivized to fix the cause of your pain” — so we refactor. Agents do not. They obliviously extend bad structure indefinitely. A senior engineer’s job is to say no to unnecessary abstraction. The agent has no equivalent reflex.
2. Training rewards apparent completeness. Pretraining corpora are full of explanatory material — Stack Overflow answers, tutorials, README snippets — deliberately self-contained and verbose. Post-training compounds the effect: human raters prefer outputs that look thorough, so models learn that “comprehensive” reads as better. When uncertain which edge case matters, the safe move is to handle all of them. Each guard is locally defensible. The aggregate is bloat.
3. Iterative generation has no deletion pressure. Agents add but rarely delete. Removing dead code does not make any test go green, so superseded functions accumulate alongside their replacements. SlopCodeBench, a March 2026 benchmark across 11 coding models, found rising structural complexity in 80% of trajectories and rising verbosity in 89.8%. Agents continue to patch bad code, treating every task as if it’s their last.

AC/DC: the loop that compensates

What closes the gap is a loop around each iteration of agent work. The agent does what it is good at — generating code — and our job is to wrap that with three steps the agent cannot reliably do on its own. At Sonar we call this the Agent Centric Development Cycle, or AC/DC: guide, verify, solve.

Guide

Many teams overcorrect on context. They paste the style guide, three years of architectural decisions, and the entire onboarding doc into the agent’s instructions and expect output to improve. ETH Zurich researchers tested this and found the opposite: large context files often reduced task success against no context at all, and added 20% or more to inference cost.

Keep agent-facing context short — under 200 lines is a useful heuristic — and restrict it to fundamentals that can’t easily be inferred from the code: naming conventions, architectural invariants, what has been tried and failed. However, this will only get you so far, so make sure to provide specific context for each task. If you have architectural expectations, don’t expect the agent to guess them. Software architecture tools can be used to provide additional context in the guide phase.

Task shape matters too. Break the work into steps and agree on a plan; ask the agent to provide three solutions and evaluate the impact on quality of each. There is no perfect software architecture, and you understand the trade-offs in your codebase best, so think critically about the changes before they happen. Without this, the agent will confidently pick an option, seemingly at random, and the further it goes the harder it is to “unpick.” If you want to test this, ask three instances of your preferred agent to complete a task that involves some polymorphism and watch each one confidently suggest a different solution.

Verify

The most expensive verification mistake is doing it last. Reviewing 200-line pull requests (PRs) after the agent is done is the dynamic behind the Faros/DORA figures Addy Osmani highlighted: 98% more PRs merged in high-adoption teams, review times up 91%. Verification inside the loop is different. Unit test runs, static analysis, and security scanners produce output the agent can act on. This is where AI-native tooling belongs: purpose-built for the agent to invoke, not just for humans to consult through a UI.

Human reviewers cannot keep up. When agents merge twice as many PRs per week and each one takes nearly twice as long to review, doubling the review staff still leaves you behind. Automated verification is the only response that scales. Fast feedback has always been a fundamental tenet of good software engineering. Feeding it directly back to the agent protects the developer from simple mistakes and leaves them headroom to work on the harder ones.

Solve

If verification happens within the agentic loop, the agent can fix any issues whilst the code is being generated without expensive remediation steps. Static analysis tools can guide the agent on how to resolve the issue quickly. Some cases need human judgment — a None check at a system boundary may document a real precondition. But most of the work is mechanical. Automate the obvious fixes and let engineers spend their attention on the cases that are not.

The investment that compounds

Better models will keep arriving. They may not change the mechanism of bloat or the dynamics of compounding decay. The loop is what does — bounded tasks, sharp context, in-loop verification, and a deliberate “solve” step to clear bloat before it accumulates.

The same logic governs how autonomy should expand. Reduce human interventions only when the agent’s guide, verify, and solve cycle is making them redundant. Our biases can sting us here: an agent’s ability to write code can lead us to agree with it more than we should. Don’t trust blindly; wait for the evidence.

The teams that will be shipping enterprise-quality code with AI agents in 18 months are not the ones running the “best model.” They are the ones treating workflow as the engineering investment, with the seriousness once given to build systems and CI. The model is the tool, the workflow is the discipline. That is where the durable advantage compounds.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

Hot AI companies can’t stop talking about forward-deployed engineers (FDEs), which are now very much in vogue. 

FDEs, in case you haven’t heard, are hired by companies looking (hoping?) to successfully deploy AI tools and services. It’s one of the hotter professions in a world still trying to understand the impact of AI on careers.

So, what exactly are FDEs — are they techy lone rangers like the ones OpenAI, Google and Microsoft are hiring? Turns out it’s not so much about individual engineers who swoop in to design and roll out AI deployments; it’s more about a team of engineers working together at customer sites.

At least, that’s the view at Amazon Web Services (AWS).

In fact, according to Taimur Rashid, managing director of the AWS Generative AI Innovation Center, the FDE concept pre-dates the current generative AI (genAI) gold rush. The same kinds of engineering teams were needed for the earlier machine-learning and cloud eras to help companies with deployments.

Rashid recently talked about how AWS sees FDEs as a profession in a conversation with Computerworld. And he weighed in on the desired job skills the company seeks in this increasingly AI-centric era.

What is an FDE? “We view it as a team. It’s a cross-functional team that has engineers, scientists, strategists, and folks that can piece technology and business together. In some cases, we do have to have security engineers in there, too. 

“I see them as anesthesiologists. They have to prep so many things, monitor things throughout. We see ourselves as a frontier deployment team helping customers adopt all forms of AI, whether it’s genAI, agentic AI, even emerging trends like physical AI. We’re helping these companies become frontier themselves.”

How does an FDE engagement begin, and how is it structured? “Where we see the forward deployed model is when customers come in — for example, we have our executive briefing center in Seattle and in Arlington, VA. When customers share what they’re trying to do, very quickly a customer’s like, “What’s the quickest way I can go and build something with you?” 

“We’ll forward deploy our people in, we’ll embed them in your business and we’ll go through these 45-day sprints that we typically design. Through those successive sprints, we’re building stuff together, we’re proving value, and then they can expand that to a much broader engagement.

Where do FDEs actually sit? Client side, internally, or in-between? “It’s mixed, and it largely depends on what the customer’s preference is. We’ve seen models where the customer has been very adamant that, ‘We want your teams with us in our business.’  In those cases, we forward deploy the majority of the teams on site. 

“We have models where customers are fine with you being wherever you’re based, as long as you’re still embedded virtually. And then there’s a hybrid. We deploy anywhere from five to seven people. Sometimes, the baseline is actually three.”

Will cost savings be the job of an FDE, or someone else on the team? “I expect these teams to be able to architect systems that have those cost requirements in mind, whether it’s use a different model for a different use case that doesn’t increase the per-token cost…or think about ways where you can use semantic caching. I personally think you may have a high spend in token consumption, but if you’re generating revenue, as long as the economics work out, then you’re at peace.”

What challenges have you gone through deploying FDEs in the real world? “One of the challenges worth highlighting is when customers get really excited about us forward deploying resources, what they end up realizing is [that] they’re not set up to absorb that right away. They realize they have to go through … process-related things, security access — all those operational things. 

“One very good example is the Commonwealth Bank of Australia. They said: “Prioritization’s a big thing, and if you forward deploy and you’re 100% dedicated, how do we ensure that our teams are also equally 100% dedicated?’ When you’re sitting in your office, you’re distracted by your day-to-day. So they said, ‘Why don’t we create a neutral ground in Seattle? You fly your people, we’ll fly our people. We’ll give them three weeks of dedicated time so they have no distractions.’”

Have you gone into projects where they want AI but have no security or governance ready? “I’ve been through this before, certainly. We do see customers that have security processes and capabilities, but it’s not as tight as it should be in the age of AI. Governance is the biggest area where customers right now have the biggest gap. I’m talking governance around agents. In the past two months, almost 100% of the conversation around agents is not about capability. It’s all about governance. 

“That is a big area right now where forward deploy teams are helping with governance education, and building the scaffolding for that.”

How does software engineering fit into the FDE model? “One of the greatest learnings is that as we forward deploy resources and get customers to take AI and integrate it into their systems, the knowledge of software engineering is so important. Today, a customer can use one of the Claude models and scan their code base and look at vulnerabilities. 

“The tough part is not assessing those vulnerabilities, it’s remediating [them]. Remediating [them] requires software engineering experience, because you have got to merge code, test it, deploy it. We largely see that the frontier software development teams are smaller and they’re managing agents that are doing various tasks across the software development lifecycle.”

AWS has many models at your scale, open source, closed — it’s more complex than what other AI vendors offer.  How do you nail down the talent? “It’s massive, and when you look at not only scale, it’s the complexity of the stack. We take an approach where we fundamentally do three important things: No. 1, we want to ensure people understand concepts; they have to understand pre-training, post-training, reinforcement, fine-tuning. 

“Secondly, we make sure that our teams are well versed in their first-party services. The third thing is that by design, AWS has always been about choice. We say, ‘Let’s do 80-20 here. What is 20% of those specialties that we need to have, which can cover 80% of what most customers are trying to do?’”

What skills should software developers learn to move into FDE work — the top three or four things? “We look at three categories. First category is entirely functional. Are they more engineering specific? Are they science specific? Are they security specific? Our litmus test is not only knowledge of the function, but the actual hands-on work that they can do with it. Secondly is around domain. I focus on what is their domain understanding across the whole AI lifecycle? The third thing is cultural. We are looking for folks that are okay at dealing with ambiguity, being good at stakeholder management, and having that startup mindset. 

“This AI transformation’s not for the faint of heart.”

There’s a rush for FDEs from OpenAI, Google, and others. What’s different about what you look for? “I don’t know entirely what the others are looking for, but I’ll tell you what we have been looking for in the past. When we hired solution architects, it was about systems level understanding. But what I see more and more is hands-on experience, cultural mindset, all these things are very important. If I had to pick one thing that is going to be very important in the talent that we either upskill or future talent that we hire, it has to be the application of AI towards software engineering and system integration tasks.

You’re upskilling AWS talent as well? “We do. You will see some publications coming out in the next couple of weeks around how do we do this across our software teams and how does that translate to customer-facing roles.”

We’ve all heard the mantra from the quants in the business community: you can’t manage what you can’t measure. And if that’s true for human intelligence, it should be true for the artificial kind too.

How do we measure agents and large language models (LLMs)? We’re just beginning to come up with statistical metrics. Here are several of the most common metrics that designers and users toss about when they’re evaluating a model.

[ See also: 27 questions to ask before choosing an LLM ]

Time to first token

How long does it take to generate the first token? For real-time applications with time constraints, faster responses can be essential. It’s well-known that people hate waiting even a few milliseconds. The teams that develop user interfaces learned decades ago that it’s important for the software to respond quickly when a human is waiting for an answer. Even a few seconds of delay mean that the human will wander off to another window to check some email or place some bet on a prediction market. Time to first token is a good measure for models that will be working directly with the fickle human intelligences and their latent attention deficit disorder.

Time per output token

Take the total time it takes to respond and divide by the total number of tokens. The time to first token measures how long it takes to start a response and this measures the average speed as the model through all of the tokens. In basic LLMs, this value is generally fairly constant. Once the prefill is done and the LLM enters the decode phase, the output tokens usually appear at a constant stream. When the output is long enough, the startup time to first token is amortized away. In some of the more complicated architectures with loops for planning or gathering data from various tools, the average speed can vary as the model shifts in and out of making agentic decisions.

Tokens per second

This is just the reciprocal of the average time per token. Sometimes it is reported separately for different stages in the pipeline.

Throughput (requests per minute)

If a system supports more than a single user, tracking the number of different requests makes sense. These throughput numbers can be quite useful for measuring the power of some of the newer pipelines that are more efficient when they’re answering multiple prompts at the same time.

Error rate

Not every request gets an answer. The error rate tracks how often rate limits, timeouts, or model “refusals” get in the way. Better accounting tracks each independently because the number of failures in each category can be very different.

Token efficiency

Not all work tokens are visible and not all tokens are part of the final outcome. This measures how much work is done to produce the final result. As models become more complex or agentic and the pipelines become more sophisticated, the efficiency tends to drop. Agentic reasoning and strategic planning typically require more tokens that don’t appear in the final answer. This is generally a measure of how expensive a model might be to run.

Tail latency

It’s all well and good to measure the average time to answer, but in some cases a few very slow responses can really color people’s judgement. Some applications require good performance all of the time. Would you want to ride in an autonomous car that gets steering instructions very quickly “on average” instead of always? What if that’s only 99% of the time? Tail latency uses a mixture of queuing theory and detailed measurements to track the worst moments in the long tail of the latency graph. It’s useful when even occasional delays are problematic. 

Total cost of ownership

Projects that use an API or buy output from providers just look at the cost per 1M tokens. They’re effectively renters. The groups that are buying GPUs and paying for electricity, though, will add up these costs and other indirect costs like depreciation and maintenance to come up with a number that estimates how much the tokens really cost to produce. This value will depend upon demand and utilization rates—that is, on how many users are sending in prompts and how efficiently the model fits in a particular GPU and its RAM.

Parameters

Many models have numbers in their name followed by a B. This is meant to roughly capture the number of parameters, or the number of variables the model uses to generate outputs from inputs. The number “70B” means that there are about 70 billion parameters in the model. This is a good estimate for the complexity of the model and the size of the training set that has been stuffed into it. Generally bigger numbers mean a larger amount of information is hiding inside the model. It often means that it will take a bigger GPU with more RAM to generate an answer with it. It’s not a very precise number, though, because there are many other areas of the architecture that can influence whether the model can generate the answer you want inside your budget. There continue to be advances and it’s not uncommon for someone to claim that a new model with X parameters is better than an old model with 2X or 3X parameters.

Hallucination rate

While everyone wants LLMs to generate accurate output, measuring it can be difficult because deciding what’s accurate is sometimes complicated. One approach is to ask the LLM to summarize a document. Then another model evaluates how well the summary matches the original. While this may not catch all subtle slips, it will capture enough of the worst departures from reality. Some researchers have built complex test sets with curated answers. The LLMs that deliver the expected answers get the highest scores. Some common benchmarks are TruthfulQA, HaluEval, QAFactEval, and Vectara’s Hallucination Evaluation Model (HHEM).

Toxicity and bias scores

If measuring accuracy is difficult, building a metric to detect toxic or biased output is even more challenging because the definitions can be so protean. Still, some teams have built LLMs that key on particular concepts or word choices. They can detect some of the most obvious red flags that could generate political trouble. Some well-known versions include Granica Screen and Perspective API.

PII leakage

One of the biggest fears is that LLMs will somehow absorb information that may be considered personal and private. Some of the simplest measures can be as simple as regular expressions that look for the sixteen digit numbers used for credit card transactions. Many of the model builders work on eliminating personally identifiable information (PII) from the training set before beginning.

Tool-calling accuracy

As models grow more complex and agentic, they often gain access to various tools or Model Context Protocol (MCP) gateways that can help them find the best answers. Not all models take advantage of this help. The tool-calling accuracy scores count how often the models choose the best tool for the job. One particular example of this measurement is BFCL (Berkeley Function Calling Leaderboard).

Prompt sensitivity

The value captures how small changes in the language of the prompt induces the model to produce different results. It’s like a derivative from calculus class, although it’s generally computed experimentally using some collection of test prompts. There are a number of different approaches that depend upon different types of changes. Some test sets are built with small rephrasing of the request that are semantically the same. Others mix together different ways of specifying the problem, some with examples, say, and some without. Some specific examples include PromptSE and ProSA.

Semantic similarity and conciseness

Some metrics evaluate the answers by comparing them to a set of gold standard answers. This often involves feeding them to a vector embedding model and searching a retrieval-augmented generation (RAG) database for similar answers. This can track how concise or fluffy the answers might be as well as looking for how much variability might be introduced through changing parameters like the temperature. One common example is the BERTScore.

Grounding score

Many systems that combine an LLM with a vector search tool for RAG measure the effectiveness of the combination with a benchmark like the grounding score. The LLM is presented with extra data from the vector search and the benchmark measures how closely it follows this extra information. That is, how much of the answer comes from the provided source documents and how much is synthesized using the data in its training set. Some examples include RAGAS, TruLens, ARES (Automated RAG Evaluation System), RGB (Retrieval-Augmented Generation Benchmark), HaluEval, and HalluHard. A similar concept is called “context adherence,” “context precision,” “context recall,” or “faithfulness.”

Model variability

Most LLMs fold in a certain amount of random entropy, and this amount is often controlled by a parameter called the “temperature.” The model variability is a measure of how much the answers will change between runs. Some applications like chatbots require a certain amount of variability because the randomness adds a bit of “life” to the answers. Other applications like those in mission-critical areas like law or medicine will undermine confidence if the answers vary.

Format compliance rate

In some roles, LLMs are asked to produce data in strict formats like JSON or CSV. This is often important if the data will be fed into some pipeline for further processing or storage. The format compliance rate tests a number of common formats and measures how often the LLM returns semantically correct data. Agentic systems that glue together multiple LLMs and other tools rely heavily on LLMs with good scores on this benchmark.

Instruction following

Some prompts include very specific instructions and the adherence can be measured empirically. For example, some prompts will ask the LLM to produce exactly 300 words or a poem in rhyming couplets. These tests use a collection of sample prompts that ask for answers that can be easily measured. Some specific examples include IFEval, FollowBench, and the BFCL (Berkeley Function Calling Leaderboard), a value that is mentioned above in the section on tool usage.   

Subgoal success rate

As agentic models become more common, it’s helpful to track how well the model performs on each of the various parts of the agent’s strategic plan. All of the metrics here can be broken down and tracked for each of the subgoals.

Plan stability

Agentic models start with a plan. Some of them are smart enough to abandon the plan or at least adjust it as the work evolves. Plan stability measures how often the plans are adjusted. A high rate of adjustment could mean that the agent is a bad planner or just flexible or maybe both.

Self-correction score

Some agents are able to dive deeper and recognize their mistakes. The self-correction score measures how often the model will make a mistake and then recognize it, either on its own or after being prompted with the question, “Are you really sure?”

Jailbreak resistance

Some users try to find clever ways to lure the LLM into tossing aside any restrictions on topics or answers. In the past, some LLMs could be fooled by being told the answer was part of a play or a work of fiction. So discussing forbidden subjects wasn’t a problem because it was all pretend. Newer models have more elaborate defenses. Measures of the ability to resist deception include JailbreakBench, AgentHarm, and Tele-AI-Safety. 

Prompt injection vulnerability

Sometimes untrusted data from extra sources or skills may include malicious instructions that can exploit the LLM. Benchmarks such as Skill-Inject and SPIKEE (Simple Prompt Injection Kit for Evaluation and Exploitation) work with known attack vectors and measure how susceptible a model is to targeted prompt injection attacks.

Copyright infringement score

Some LLMs can regurgitate the data in their training corpus in a way that seems like plagiarism or copyright infringement. This can be an issue when the training material wasn’t carefully licensed. The copyright infringement score measures how often the LLM may parrot the training material a bit too closely. Tools for defending against this include CopyrightCatcher and DE-COP.

RULER

How well can a model extract information from the entire context? NIAH (needle-in-a haystack) benchmarks measure how well a model can retrieve small, crucial bits of information from long contexts. RULER takes NIAH tests further with the ability to vary the types and quantities of needles, the size of the haystack, and the complexity of the task.

GSM8K 

The developers of GSM8K (Grade School Math 8K) set out to benchmark an LLM’s ability to tackle multistep mathematical problems, so they gathered 8,500 problems that are common in grade school math classes. While the focus is explicitly on solving math homework problems, the benchmark also measures the ability to construct reasoning chains.

GPQA

The Graduate-Level Google-Proof Q&A is composed of hundreds of hard questions that might normally be answered by humans in graduate school, generally in science. To make the benchmark harder, the researchers focused on questions that non-experts often get wrong. The term “Google-proof” means that the benchmark includes questions that can’t be easily answered by asking a search engine.

MMLU-Pro

The MMLU-Pro benchmark builds on the Massive Multitask Language Understanding dataset to test a model’s understanding of a broad set of scientific knowledge. It includes more than 12,000 questions about general scientific fields like biology, chemistry, economics, and law. 

MBPP

Google created MBPP (Mostly Basic Python Problems) to evaluate how well a model was solving coding questions. Each problem comes with a statement, a gold standard solution, and several similar test cases. The number of accurate answers to these questions is a good measure of how well the model will solve many of the simpler Python coding problems presented by users.

SWE-bench

This collection of several thousand software engineering challenges evaluates how well a model solves programming problems. The developers created it by selecting a number of issues and corresponding pull-requests from a dozen or so Python projects. After some limitations appeared, the creators expanded the set by creating SWE-Bench+, SWE Bench Verified, and SWE-Bench Pro.

LMSYS Chatbot Arena

Instead of creating a fixed set of test prompts, the Large Model Systems Organization’s Chatbot Arena is a dynamic system that feeds the same prompt to different models and then asks humans to pick the best results. These head-to-head contests produce an Elo-like rating that is similar to the one used to score chess players.

Price

The rest of these metrics are useful, but as the real estate agents say, the three most important numbers on a property listing are price, price, and price. The cost is a bit less important for measuring AIs, but only a bit. Price can make a huge difference between a project being profitable and a moneysink. When the cost for each inference is a tad too high, it’s impossible to make it up with volume.

The key caveat is that a cheaper model isn’t a good idea if it generates answers that are filled with hallucinations or worse. The quality of the answers can differ greatly, and saving a few pennies can be a mistake. To make matters more complicated, there’s an explosion in different styles and approaches. Sometimes it makes sense to pay a bit more for a model that delivers answers with the right vibe.

When it comes to AI deployments, IT leaders are often caught in an awkward middle space, trying to reconcile conflicting directives from senior management with constantly changing AI models, capabilities, and costs; data governance and security needs; and the limitations of their own team.

“Very few real benefits can be attained by simply purchasing an AI product and giving it to employees. Vendors have been overselling that fallacy for the past three years,” said Nader Henein, a Gartner VP analyst.

“The reality is that strong AI value and consistent ROI are almost always a result of deep and intentional integration of AI capabilities into existing workflows. For that you need specialized teams, which do not come cheap, and organizations have been recruiting those teams in a variety of ways,” Heinen said.

Among the options available to IT leaders looking for help with AI deployments are traditional IT consultancies, AI-specific consultancies, and independent contractors. Large enterprises with deep pockets can consider acquiring an AI firm and integrating its technology and expert staff. The use of open source to reduce vendor lock-in is a strategy that can sit on top of those others, an approach that Capital One has used. 

But the option that has been getting the most attention recently is bringing in forward-deployed engineers (FDEs), teams of experts from AI vendors that embed with a customer’s in-house engineers to oversee AI rollouts within the enterprise environment. Both OpenAI and Anthropic have recently announced FDE offerings, for example, and Microsoft is partnering with consulting giant EY in a new FDE program for agentic AI deployments.

Engineering teams employed by AI vendors have key strengths, such as understanding their models better than anyone else, having experience integrating those models into different types of enterprise environments, and knowing about upcoming model capabilities before they’re announced. But they also have the obvious drawback of vendor lock-in. Even if future rollouts are not within their contracted deliverables, those vendor employees could subtly influence a client’s future AI efforts. 

Flavio Villanustre, CISO for LexisNexis Risk Solutions, cautions IT executives to move into FDE programs carefully. 

FDEs “are financially incentivized to grow customers’ use of a vendor’s AI products and to create stickiness with that vendor’s services,” he said. “While FDEs may be a reasonable value-added service by the AI vendor, customers should always find other unbiased expert opinions that can evaluate competitive solutions across multiple vendors.”

This is particularly important at a time when “investor-subsidized AI token business models are starting to show cracks,” Villanustre said. “Also, in the current rapid pace of innovation in this field where AI vendors are constantly leapfrogging each other, retaining the agility to move from one vendor to the next could create significant competitive advantages.”

Analysts, consultants, and other industry experts who spoke with Computerworld about FDEs echoed Villanustre’s caution, citing concerns around hidden costs, confidentiality, observability, and vendor lock-in.

Long-term costs and vendor lock-in

A key issue that IT executives need to consider is how long the FDE teams will be needed. The enterprise will likely need an ongoing series of AI deployments synced with the current AI model(s). If help is needed today, why would that change tomorrow?

Enterprises tend to overlook those longer-term costs, said John Sangyeob Kim, an AI engineer at software development vendor Solidroad.

“Deployment is maybe 20% of the total cost. The other 80% is keeping the system running through model upgrades, data drift, and edge cases that only appear after months in production,” Kim said. “Most contracts price the first part and assume the rest. Deployment isn’t the hard part of enterprise AI anymore. The next eighteen months are.”

And whether it’s intentional or not, FDEs will naturally favor their own product portfolio — it’s what they know best.

“FDEs from model labs are good at making their own models work in your environment. They are less suited for multi-model systems, because their incentive is to keep you inside their ecosystem,” Kim said.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said IT leaders should look at the FDE model as a strategy involving ongoing operational power. 

“Whoever shapes the deployment pattern shapes the enterprise’s future muscle memory. Whoever owns the evaluation layer owns the truth layer. Whoever controls the integration logic controls the dependency map,” Gogia said. “This is why the FDE model matters. It is not just another delivery option. It is the frontier AI vendor moving closer to the customer’s workflow, operating model, and decision architecture.”

That proximity cuts both ways, Gogia noted. “FDEs are embedded inside the customer’s [environment], but they are also connected to the vendor’s commercial center of gravity. Their instinct will be to build around the model family, tooling assumptions, deployment patterns, and product roadmap they know best. This is perfectly natural. It is also precisely why CIOs must be cautious,” he said.

Allowing AI vendor employees an outsized say in enterprise deployment decisions could lock in model vendor dependency, which in turn will fuel high prices that can’t be fought effectively.

“FDEs can accelerate deployment and deepen dependency at the same time,” Gogia said. “Frontier AI vendors are no longer content to sell access to models. They increasingly want to shape how enterprises deploy intelligence. That is a larger prize.”

What happens when the FDE team leaves?

FDE post-departure risks are severe and often underappreciated, according to Justin Greis, CEO of consulting firm Acceligence and former head of the North American cybersecurity practice at McKinsey.

For one thing, the FDE team learns a massive number of operational details from the enterprise deployment. Although NDAs and confidentiality contracts protect any data accessed, they often don’t regulate observed processes and procedures. 

“The learnings are absolutely going to be taken from client to client,” Greis said. “Whoever helps deploy AI will learn far more than what appears in the statement of work. They will learn the real workflows, the undocumented exceptions, the data-quality gaps, the approval bottlenecks, the security workarounds, and the places where the business depends on a few people knowing what to do when the process breaks. That knowledge may be as sensitive and precious as the data itself.”

Another critical but often overlooked issue is how much meaningful control will IT have over the project if and when the FDE team leaves.

“The danger is not using outside help. Most companies will need outside help,” Greis said. “The danger is using outside help in a way that leaves the enterprise less capable and more dependent when the engagement is over.”

It is precisely those operational decisions that IT often neglects, said Solidroad’s Kim.

“The best predictor of success is not the vendor. It is whether one internal engineer truly understands the system before the implementer leaves. What matters is who owns the evaluation loop after the demo,” Kim said.

“What happens to our prompts, scorers, and guardrails when the model version changes? If we paused this engagement tomorrow, what would actually stop working, by design or by accident?” Kim asked. “Where do you want the enterprise’s AI learning, control, and dependency to live after the engagement is over?”

Kim argues that observability — the ability to understand and manage all elements of a complex enterprise environment — is a critical function to which IT often gives insufficient attention. Determining whether the project uses the enterprise’s observability stack or the vendor’s observability stack is crucial.

“If the implementer is using their observability stack, that is fine during the build, but you need a plan to migrate it to something you own before they leave; otherwise the visibility walks out of the door with them,” Kim said. “If they are using yours, that is the best case. It means they are working inside the system your team will operate long-term.”

A major problem crops up when they are using neither the enterprise’s nor the vendor’s observability stack. “Neither means they are building the system without any production observability layer at all, and you inherit a system you cannot see into. The first time something breaks in production, you have no traces, no failure history, and no way to tell whether the issue is a model regression, a data problem, or a code bug,” Kim said.

“If observability was not a priority during the build, evals and regression testing usually weren’t either, so you are inheriting a system you cannot measure and cannot safely change. That’s the worst possible handoff position,” he said.

Weighing the alternatives

While the FDE approach is not new, it is just now beginning a surge in popularity, and there are a finite number of such specialists available. That means not all companies even have the option of using FDEs.

This availability disconnect is especially prominent for non-US deployments, where on-site FDEs are rarer, said Gartner’s Henein. “Where is the development happening? There may not be FDEs available in that region,” he said. 

There are plenty of other places enterprises can turn to for AI help. Ishraq Khan, CEO of coding productivity tool vendor Kodezi, encourages IT executives to consider a wide range of options but notes that all approaches have major drawbacks.

“Traditional consultancies are usually stronger at governance, process, compliance, and organizational coordination. They know how large enterprises operate politically and structurally. The downside is that many move slower and often lack deep frontier AI specialization,” Khan said.

Gogia from Greyhound Research put it more colorfully: Traditional IT consulting firms “know how to get legal, risk, security, finance, HR, and business units into the same room without anybody setting fire to the carpet. For regulated enterprises, that matters,” he said.

Specialized AI consultancies have a different set of strengths, Khan said. “AI-native consultancies move much faster and are often more technically current, but many are still immature operationally. Some can build impressive demos without fully understanding long-term maintainability, governance, or production reliability.”

Greis from Acceligence commented on two other options for bringing in outside AI help. Using an independent contractor “can be great for eval design, architecture reviews, red teaming, agent design, or getting a stalled team unstuck,” he said, but it can increase the risk of “key-person dependency,” where a single external person is the only one who understands the system.

As for purchasing an AI firm and onboarding its employees, a practice known as “acquihiring,” Greis said it can work well when the AI capability and expertise being brought in are truly strategic for the acquiring enterprise. But there is a risk that the acquired team will be smothered by the parent company’s bureaucracy: “You buy a speedboat, bolt it to an aircraft carrier, and then wonder why it stopped moving,” he said.

Finally, an open-source strategy can give companies flexibility and reduce vendor dependence, but “many companies underestimate the operational burden that comes with it,” Kodezi’s Khan said. “Open source only helps if the organization has the internal talent and discipline to maintain it properly.”

Bottom line: enterprises need to define their true objectives before deciding on an approach. Khan offered several key questions for CIOs to consider: “Who owns the deployment after implementation? Can we move providers later without rebuilding everything? What happens if the vendor relationship changes or disappears? Are we optimizing for short-term deployment speed or long-term operational resilience?”

In any scenario where outside firms have direct access to enterprise systems, IT needs to be kept fully in the loop. “The worst outcome is when an enterprise successfully deploys AI but no longer fully understands how its own systems operate underneath,” Khan said.

External help for AI deployments: 6 options

	Pros	Cons
AI vendor FDEs	+  Best expertise on the main model being used	–  Vendor lock-in –  Operational detail leaks
Traditional IT consultancies	+  Best understanding of change management, legacy integration, global rollout, governance, and operating-model redesign	–  Can be too slow, too expensive, or too generic
AI consulting firms	+  More practical AI deployment experience than traditional consultants +  Less vendor lock-in than model-provider FDEs	–  May not sufficiently understand enterprise-grade requirements: security, identity, auditability, compliance, incident response, cost controls, and long-term maintainability
Independent contractors	+  Useful for precision tasks: eval design, architecture reviews, red teaming, agent design, or getting a stalled team unstuck	–  Risk of ‘key-person dependency’
‘Acquihiring’ an AI firm	+  Works when the acquired capability is truly strategic	–  Acquired team can be smothered inside existing bureaucracy
Deploying open-source products	+  Reduces dependency on one model vendor +  Attractive for data sovereignty, control over enterprise systems, cost efficiencies, and regulated environments	–  Enterprise takes on full responsibility for security, patching, evaluation, deployment, monitoring, and lifecycle management

Related reading:

• Here’s one career emerging from the AI shift: ‘forward-deployed engineers’
• The forward-deployed engineer: Why talent, not technology, is the true bottleneck for enterprise AI
• The new AI lock-in

Heads-up, my fellow Android-appreciating animals: Google’s in the midst of rolling out a subtle change to its privacy settings that’s well worth your while to notice.

The change includes a new clause that says the company can use images, files, video, and audio from your interactions with Google Lens, Search, and Gemini Live to train and improve its AI models.

By default, that switch will soon be on and active for your account.

But with about 20 seconds of one-time effort, you can opt out and flip it off (both literally and metaphorically, if you’re so inclined) once and for all.

Lemme show ya how.

[Get level-headed knowledge in your inbox with my free Android Intelligence newsletter. Something new and useful every Friday — from my keyboard to your email.] 

Google’s new AI training privacy default

First things first — the nature of the change: According to Google, starting in the next few days, a new “Search Services History” section within the general Google account settings will lead to a significant-seeming policy shift. As per an email the company sent out to users this week:

Your media [will now be] saved when Search Services History is on. Saved media includes your images, files, audio, and video from your interactions with Search services to help improve your experience. … Your saved media is also used to develop and improve Google services and technologies, including AI models and safety measures.

Riiiiiiiiiiight.

Now, to its credit, Google does say the data will never be associated with your account or identity once it’s used for these purposes, and it’ll rely on “filters” to “automatically remove a broad range of identifying info or sensitive personal information.” But still, whether you’re working with important corporate info or simply put off by the idea of your personal media being fed into the AI training machine, this may be news you aren’t exactly thrilled to hear.

If you’re finding AI increasingly creepy or you’re just not so keen on knowing whatever media you submit to search-related services will be used to train and develop AI for the future, now’s the time to proactively speak up and change your Google account settings to shut down this setup before it begins.

That, unfortunately, is where things get slightly complicated — ’cause for most of us, this new Search Services History section doesn’t seem to be present and available just yet.

But that doesn’t mean you’re plum out of luck.

Your 20-second opt-out roadmap

To start on your AI training opt-out adventure, make sure you’re signed into whatever Google account you rely on for work and/or personal purposes, then head to the Google Activity Controls page and see if you see a section there called “Search Services History.”

If you do, this is especially easy for you: Just use the option right then and there to disable the “Save Media” setting within that section, which will stop any media files from being saved and used without eliminating the entire history of things you’ve searched. (If you’d rather eliminate all of your Google Search history from being saved and used even for your own future discovery and recommendations, you can also opt to turn that entire section off. Just be aware that it may have some wide-reaching effects on the personalization you see across a lot of Google services.)

If you don’t see that section — and, again, that appears to be the case for most of us at this point — you’ve got two options for the moment:

1. You can completely disable all of “Web & App Activity.” Google says if you do this, once your account transitions over to the new approach, all of those “Search Services History” settings will stay off as well. Just be aware that doing so will prevent any and all search history from being saved for you from here on out — which, again, means you won’t be able to revisit your search history yourself and won’t see suggestions and personalization based on past searches throughout Google apps in the future.
2. If you want to avoid entirely eliminating all of your search history, you can for now uncheck the boxes only for “Include voice and audio activity” and “Include Visual Search History.” That’ll stop search-related media from being saved to your Google account for the time being — though I’d also suggest setting yourself a reminder to look back at that same page once a week or so until you see “Search Services History” appear and can confirm that “Save Media” is unchecked as a result of that previous preference. Right now, Google isn’t explicitly saying that such a preference will carry over, so I’d put it on yourself to double-check and make sure (and then make the needed adjustment in the new interface, if not).

The choice is ultimately 100% yours — but in this case, it’s up to you to take action and opt yourself out if you aren’t comfortable with the default. It’s an unfortunate position to be put in, but now you at least know what’s happening and how you can make your own decision to take back control.

Find the tips and tools that’ll *actually* help you with my free Android Intelligence newsletter. No hype, no nonsense — just useful new stuff in your inbox every Friday, from one (alleged) human to another. 

Modern AI systems have evolved beyond the simple chatbots that quickly became popular. Now they use semantic tools to manage workflows and link machines to machines, providing a flexible and effective framework for the next generation of business automation. What you used to build in Microsoft’s Power Platform or construct inside Biztalk is now an agent, built around large language models (LLMs) that can parse both your data and the APIs that you want to use your data with, orchestrating workflows with a level of autonomy that traditional tooling can’t match.

That shift has offered new opportunities, much like those that came with business platforms like Microsoft Dynamics and Salesforce. Here, tools built to solve one set of business problems could be turned into applications that could be sold to other companies. What worked for you to solve one of your problems could now be an added revenue stream, sold through platform marketplaces that helped customers manage installations and customizations.

Agents are business applications now

Modern agents are much like those business applications. Often developed to solve a specific need, but quicky adopted by organizations and refactored to apply enterprise standards (using tools like the Agent Governance Toolkit and frameworks like Microsoft’s Agent Framework), they’re rapidly maturing and are ready to be shared more widely. The process of sharing needs to be curated and controlled, and, if possible, tied to a revenue stream.

There’s certainly some urgency here. Until recently, subsidized tokens have kept costs artificially low. Now companies like GitHub and Anthropic are moving to a more sustainable (for them) pricing model, increasing the cost of inferencing and squeezing companies’ AI budgets. As a result, switching AI projects away from a cost to a revenue source is high on CIOs’ agendas. If those tuned and trained agents can be sold on a marketplace, then that token budget can be justified.

Microsoft has always been a company built on partner relationships, starting with individual developers and working all the way up to the largest software companies and consultancies. That reach is key to helping partners extract as much value as possible from their agents, as it allows Microsoft to integrate its partner sales tools into its own products and services, as well as into other platforms.

Extending the Microsoft Marketplace for AI developers

We’re already familiar with many of Microsoft’s marketplaces, built into individual tools like Teams, into platforms like Microsoft 365 or Visual Studio, or into the Windows Store. Now the company is doing the same for AI developers, extending Microsoft Marketplace to software agents. Announced at Build 2026, the updated Microsoft Marketplace provides ways to publish code—apps and agents—developed across all of Microsoft’s development platforms, including Copilot Studio, opening the marketplace up to traditional and non-traditional developers alike.

Perhaps the most important aspect of this new Marketplace is its own intelligence, using context to expose your code to the right audience. If you’ve developed an agent for use with Microsoft 365, it will be exposed inside the Microsoft 365 Copilot Agent Store; for Visual Studio, in the Visual Studio Marketplace; or for Teams, in the Microsoft Marketplace. All of these are different views on the same back end, using AI to ensure that the relevant agents are displayed.

Replacing search with Intelligent Discovery

This is extended by another new service, Intelligent Discovery, which adds natural language support to search, using AI to infer user intent and highlight the most relevant tools. Building on the familiar metaphor of the search bar, the initial smart search model offers a freeform way to explore the Marketplace. While there are suggested prompts, they’re not necessary. The search tooling allows you to generate comparisons between tools using your own criteria, with the Marketplace AI generating views based on your requirements.

Microsoft’s aim here is to shift discovery from keywords to use cases, so that buyers can quickly get the tools they need without having to evaluate different solutions, before completing a purchase. By handing that aspect of the buying process over to Marketplace’s AIs, customers can go straight to trials or even to buying agents and applications.

For a tool like this to be successful it needs to be trustworthy. By building it on top of the same development frameworks as your agents, Microsoft can take advantage of the AI guardrails built into Microsoft Foundry as well as low-level tooling like the Agent Governance Framework. Restricting the intelligent search to the Marketplace catalogue reduces the risk of hallucination, as output is grounded in Marketplace data and metadata.

A developer-friendly marketplace

Microsoft is providing tooling to help developers get their listings right. According to Cyril Belikoff, Microsoft’s vice president of Commercial Cloud and AI, “We actually have a separate AI tool that we give to software companies to optimize their listings, called a listing optimizer, funny enough, and that listing optimizer reviews their listing and then provides them with particular guidance on how to best improve it, so that it can be best discoverable in today’s search world.”

You can expect the listing optimizer to be tuned to work with the new Marketplace tooling, but for now it still focuses on traditional search. As Marketplace is a B2B platform, there’s a lower risk of spam applications, but even so, Microsoft remains aware of the possibility of a new system being gamed, and will be rolling out Intelligent Discovery carefully, monitoring its performance as more customers get access over time.

Having a new discovery method is one thing; getting quality AI applications in the Marketplace is another. Microsoft is validating all code submitted, though the criteria will differ between target platforms. An agent built for Teams will be treated differently than one built on Microsoft 365’s WorkIQ. It’s an approach that allows Microsoft to support new standards as they become available.

Alongside its agent development tooling, Microsoft is rolling out a new set of guidelines and processes to help developers get ready to sell their agents. Hosted on GitHub, these offer code templates as well as a link to the App Advisor guidance tools.

Still gaps to fill

This first release of Intelligent Discovery is promising, but some key features are missing. With agent token costs an increasing problem for businesses, it would be nice to see tools that help predict costs, integrating with finops tooling. We’re living in an age of shadow AI, and putting the AI we used to buy with credit cards in Microsoft Marketplace is one way to shine a light on those shadows — bringing the necessary control and governance to AI purchases, and maybe even providing support for site licensing.

Microsoft Marketplace is becoming a useful resource for AI application developers. It encompasses the entire development life cycle: offering tools that can help you build agents, the models that you need to power your agents, and finally a way to monetize your work. There’s a longer-term opportunity here, for both the Marketplace and Intelligent Discovery to offer Model Context Protocol (MCP) interfaces, ensuring that tooling and tool discovery become part of the developer workflow, and making developers aware of new tools that might help solve a problem or simplify a task.

I’ve been using and writing about Microsoft Copilot since it was publicly released in 2023. I’ve reviewed it, written articles about using it more effectively, explained how to curb hallucinations in it and other similar tools, and detailed how to use it in concert with Microsoft 365. It’s also been my go-to generative AI (genAI) tool for personal projects and advice.

But the time has come for me to leave it behind for my personal use. It’s become abundantly clear that for those tasks, Google Gemini is better. Here’s why.

Copilot is inept at solving a tech problem

Like many people who know something about technology, I’m the IT staff for friends and family. I’ve often used Copilot to help solve issues I can’t fix myself. Sometimes Copilot helps. And other times…, well, the last time I turned to it for troubleshooting advice is when I realized it was time to abandon Copilot.

My wife had bought a new iPhone, and I noticed she was receiving texts sent to her email address but hadn’t received any sent to her phone number. I asked Copilot for help.

I won’t go into the details of the wild goose chase Copilot sent me on — I’ll just offer a few lowlights. It first told me, with absolute authority, that there are “only two real explanations” for the problem and asked me to look at several settings to confirm which explanation would fix the issue.

It turned out that neither of the “two real explanations” were the cause. Undeterred, Copilot assured me, again with complete confidence, that it was going to send me “straight to the switch” that would immediately solve the problem.

I tried it. The switch didn’t work. Neither did the “final fix” it promised me. Nor did any of the other many “solutions” if offered after that so-called final fix. For more than an hour, it flailed with utter confidence and utter futility trying to diagnose and fix the problem. 

And then came the final indignity: After doing some digging, I realized Copilot was trying to solve the problem based on an old version of iOS, not the current one on my wife’s phone. When I confronted Copilot about that, it briefly apologized and promised it knew the solution: I had to call the cellphone carrier.

That was it for me. I’d had enough. I turned to Gemini for help. 

Thirty seconds later, Gemini diagnosed the problem and recommended a simple fix, which didn’t require a call to my phone carrier. It worked like a charm. Gemini had solved a tech problem in 30 seconds that Copilot couldn’t resolve after an hour. 

Copilot whiffs on personal research 

I often used Copilot for personal research projects. A recent one involved Parisian neighborhoods in the 1870s. I was looking for information about the area around the Saint-Lazare train station. When I asked Copilot, it told me the area was dangerous and poverty-ridden back then, with poor housing whose exteriors were heavily stained by coal smoke from arriving and departing trains.

That didn’t sit right with me. I recalled a well-known painting Paris Street; Rainy Day by the Impressionist painter Gustave Caillebotte, which depicted the neighborhood in the 1870s as wealthy and fashionable, filled with elegant Hausmann-style apartment buildings. I asked both Gemini and Claude about the neighborhood in the 1870s. They both told me it was expensive, fashionable and sought after by the well-off. I confirmed that with my own follow-up research.

Once again, Copilot had whiffed.

Copilot gives bad scheduling advice

I swim for exercise three or four times a week at my health club’s indoor pool. The club closed the pool for several months, so I decided to swim at the pool of an elementary school a short walk from my house. I hadn’t exercised there before and wanted to find the times on Monday through Friday when the pool would be least crowded. I asked Copilot for help.

As always, Copilot spoke with a solid air of authority. And once again, it was wrong. It told me that the least crowded time for public swimming on weekdays was between 11:30 a.m. and 12:30 p.m. or between 12:00 p.m. and 1:00 p.m.

On one count it was right: the pool would certainly not be crowded with public swimmers at those times. Because the pool doesn’t open to the public until 3 p.m. 

I turned to Gemini, which told me that 3 p.m., when the pool opened, would be the least-crowded time. Claude was no help. It demurred and said it didn’t know the answer – a rare, refreshing admittance of ignorance from a chatbot.

Gemini was on target again — 3 p.m. did indeed turn out to be the least-crowded time to swim. I often get a lane to myself, and at worst have to split a lane with one other swimmer. I asked several lifeguards if 3 p.m. was the least-crowded time on weekdays; they all confirmed it was.

Bye-bye, Copilot

For all those reasons, when it comes to personal research and advice, I’ve abandoned Copilot. I typically use Gemini now, although on occasion, I ask for a second opinion from Claude.

For my Computerworld work, I’ll keep using Copilot, and continue to write reviews of it, offer advice on how to use it and keep you informed about the latest news about it.

But other than that, for my personal use, Copilot is dead to me.

I’ve been around long enough to remember when deploying an application meant copying a *.exe file from the developer’s machine right into production. I am not making this up. It was that simple, and that fraught with peril. Applications weren’t complex — they were often not anything more than that simple *.exe file — and the process around deployment didn’t need to be anything complex, but it probably should have been.

Proper deployment of an application is something we’ve learned to do over the years. The process of properly building, testing, and deploying an application has grown more complex for two reasons. First, the process must ensure that every deployment succeeds. Deploying complex applications can be convoluted and challenging, and a strict deployment process ensures everything happens properly and runs correctly. Second, the process must thoroughly test the application to make sure that all the moving parts work together to create a properly functioning application. 

Today’s continuous deployment processes were hard-won from many lessons learned. Eventually, these practices became formalized, even to the point where the Sarbanes-Oxley Act was understood to require that IT departments formally document their deployment processes. 

This kind of governance is what separates professional software development organizations from those that, well, don’t know what they are doing. 

Agentic growing pains

Agentic development is headed in this same direction, but it’s all happening a bit more quickly. It was just a few months ago that people began to use AI to write code seriously. At first, most of us were doing it furtively, having Claude Code find and fix bugs, and then quietly checking in the solutions. Maybe we were a bit hesitant to mention that we had done this, but then we felt guilty about doing it and taking credit, and eventually we mentioned it. But it soon became apparent — like “within a week” soon — that Claude Code was up to the task, and we became pretty open about it. 

Very quickly, it not only became accepted but actually encouraged, and we were off to the races. In a month, everyone was tokenmaxxing.

It almost seems a bit out of control. Sure, there is a lot of code being generated and non-trivial amounts of money being spent, but it isn’t quite clear if the results are worth the effort. I’m not at all sure if anyone can say that the money spent is returning the value needed.

At some point, as an industry, we are going to have to get control of all of this. I fear that there will be a rush to impose governance over the whole thing. This, like deploying an .exe directly, is also fraught with peril. Right now, there appears to be little control over what tools are being used where, how much is being spent for what purpose, and what that spending is actually getting us. 

The governance over our deployment processes was successful and useful because it arose organically. The accepted, codified procedures arose from the lessons that practitioners learned by actually building and deploying applications. We all should work to ensure that a similar process happens with agentic coding.

Developers know best

Because agentic coding is happening so quickly and so furiously, the danger is that a governance process will be imposed over the top just as quickly and furiously. I want to encourage us all to take a deep breath, slow down a bit, and take a close look at what we are doing, and more importantly, how we are doing it. 

Agentic coding will be governed in some manner. It’s critical that we practitioners take the lead in providing that governance, or we’ll have governance forced upon us. We are the ones that know what matters and how the tools should be used. We are the ones with skin in the game and the ones keeping pace with the technology. 

Top-down governance of a technology moving this fast will never be able to keep up. Or as Uri Haramati, co-founder and CEO of Torii says, “The person closest to the tool usually understands why it’s being used, and governance works better when it includes those people instead of trying to control them.”

Copying that *.exe file into production is comically reckless. We don’t do that anymore because we know better. It took time, but we learned the right way to deploy software. Right now, we are in the “copying the *.exe” phase of agentic coding, and we need to figure out the right way to do it before someone comes along and does it for us.

MUNICH — Nextcloud has integrated Euro-Office into its workplace application suite, one of several updates to Nextcloud Hub unveiled on Tuesday that include a new compliance app for large organizations and a program to support developers building for its platform.

The announcements came during the company’s Nextcloud Summit 2026 here.

Euro-Office, announced in March, is billed as an open source, sovereign alternative to Microsoft Office for European organizations keen to reduce their reliance on US tech providers. It consists of four browser-based applications: a document editor, spreadsheet program, presentation tool, and a PDF editor — each enabling collaborative editing. Euro-Office documents can also be opened directly from the Nextcloud Files mobile app.

Nextcloud is one of several European companies that support Euro-Office, which is built on the open-source code base of OnlyOffice and distributed under the GNU Affero General Public License v3 (AGPL v3).

The integraton means Nextcloud users can now choose between two options in Nextcloud Office: Euro-Office and the existing Collabora integration. 

“Euro-Office uses a different architectural approach that can result in a better performance in the browser, a different user experience…, so it’s important that this option is available,” Jos Poortvliet, Nextcloud co-founder and vice president of communications, said at the Tuesday event.

Other changes in the Nextcloud Hub 26 Spring release include updates to Nextcloud‘s Talk video and voice meeting app, including AI noise suppression and the ability to start a call from any Nextcloud Hub app – an addition that will make collaborative editing easier, said Poortvliet. 

For Nextcloud Assistant, there are new AI agent capabilities. In addition to existing capabilities such as managing calendars and tasks, AI agents can now create cards in Nextcloud’s Deck task management app and update information in the Forms app.

There are also improvements to the AI assistant’s interface, which can be moved around to avoid blocking other applications and allow users to copy and paste text more easily without opening another tab. To meet EU AI Act requirements, Nextcloud will make it easier to see which  provider supplies the large language model (LLM) the Assistant runs on.

Nextcloud will also integrate the AI assistant directly into its Nextcloud Office suites via a sidebar chat interface, allowing users to address problems such as errors in the spreadsheet app.

There’s also a new Governance app that helps large organizations — particularly governments and highly regulated industries — meet regulatory requirements with compliance tools to manage data held in Nextcloud Hub. It contains several features,  including sensitivity labels to control access rights; data retention and archive capabilities; and a legal hold option that preserves documents for legal purposes such as a court case.

The Governance app includes a Compliance Manager that provides a compliance score based on an organization’s regulatory requirements, and measures progress towards certain targets. Admins can also search and review documents shared by employees and generate audit reports for compliance. The Governance app is available to Nextcloud Enterprise customers.

Nextcloud also launched a program to support independent software providers interested in building apps on its platform. 

With AI making it easier for developers to build software that integrates with its platform, Nextcloud expects a 10-fold increase in the number of available apps — from 600 now to 6,000 over the next 12 months, according to Nextcloud CEO Frank Karlitschek.

Nextcloud promised to promote apps developed by partners in its App Store and sell subscriptions as part of the ISV program, as well as provide documentation and technical help to customers. In return, developers would provide guarantees to customers around security processes and long-term support.

“We can strengthen our ecosystem, the developers also make some money — because obviously we do a revenue share here — and we leverage the dynamics that we expect from AI coming very soon,” said Karlitschek.

Editor’s note: NextCloud paid for Matthew Finnegan’s travel and hotel costs for NextCloud Summit 2026, but had no editorial role in the creation of this story.

WWDC26 felt like a defining platform moment. Apple is no longer simply promising that AI will arrive eventually; it is arguing that Apple Intelligence and Siri AI should become central to the future of its ecosystem. If that works, the company will have turned AI from a perceived weakness into a new reason to stay inside Apple’s world.

Still, the bigger question is execution. Apple did not present AI as a lab experiment; it presented a polished, consumer-ready experience. That raises expectations. 

Apple must deliver this time

Users will not judge Apple Intelligence by model architecture or parameter counts. They will judge it by whether Siri understands them, whether actions work reliably, whether personal context feels useful rather than intrusive, and whether the experience is consistent across devices.

Since Monday’s announcements, we’ve learned that some features will not work on all devices — and there’s speculation Siri AI may not fully escape beta until 2027. “Until Apple puts a stake in the ground and says when the new Siri features will be available, the debate remains: Does Apple actually have the chops in personalized AI? The demo suggests yes. The lack of timing suggests maybe,” wrote analyst Gene Munster.

Optimists argue that Apple has regained momentum by presenting a coherent AI story, one built around privacy, integration and everyday utility rather than spectacle. Skeptics counter that many of the features resemble capabilities already available elsewhere, and say the company still needs to prove it can ship them at scale and make them a meaningful reason for consumers to upgrade.

What the analysts say

That balance is visible in analyst reaction. In a client note seen by Computerworld, Erik Woodring of Morgan Stanley described the keynote as clear progress on Apple’s AI roadmap and said it suggested monetization opportunities could arrive earlier than expected — even if the overall journey will be “a marathon, not a sprint.”

UBS, in contrast, said the privacy-focused AI additions are useful but unlikely to be a material driver of iPhone demand in the near term, while Barclays called the changes interesting but incremental, and not enough to drive an upgrade cycle.

Ben Wood, chief analyst at CCS Insight, argued that Apple had to answer concerns about its AI shortcomings and now has to prove that its privacy-led, integration-first approach translates into a meaningfully better everyday experience. “Consumers will not judge Apple Intelligence by model sizes, partnerships or technical architecture,” Wood told me. “They will judge it by whether Siri understands them, whether actions work, whether personal context feels useful rather than intrusive, and whether the experience is consistent across devices.”

Dipanjan Chatterjee, vice president principal analyst at Forrester, said Apple’s strength lies in shifting the focus from the underlying technology to outcomes such as usefulness, simplicity and trust, while warning that the company still has skeptics to win over after its stop-start AI rollout. “The lesson for brands is clear: market the value, not the ingredients,” said Chatterjee. “After stumbling with the Apple Intelligence roll-out, Apple’s success will hinge on delivering the new Siri experience quickly, and ensuring it works as promised for iPhone users at scale.”

What about Apple developers?

While many are infuriated about Europe’s inability to build compromise, Apple’s developer army otherwise seems positive about what the company has accomplished. 

“On the AI front, it never made sense to me for Apple to develop their own LLM, so focusing on powerful, fast and private for implementation of Apple Intelligence seems to be an effort that is progressing rather nicely.  In all, a pretty good Keynote, I’d say,” Rich Siegel, founder and CEO of Bare Bones Software, said in an interview.

“It’s great to see Apple continue to pursue a vision of AI that leverages local systems, preserves privacy, and integrates with third party tools,” said Ken Case, CEO of the Omni Group. “A lot of our work around the Apple Foundation Models and automation, App Intents, and adopting Swift look to be fruitful investments, but it’s clear there’s more to do starting this summer. It’s also welcome to see them refine Liquid Glass, giving customers more control and listening to feedback they’ve heard over the past year.”

“I expected that this year’s Siri revamp would be the biggest personal assistant update Apple has ever done, and that’s exactly what we got,” said Sergii Kryvoblotskyi, director of AI and research at MacPaw. “Since Apple acquired Siri back in 2010, it has lacked one thing: real intelligence. Behind the great speech recognition service it provided, the tech was not ready to provide real value to users.”

“Most notable for me was Siri AI and the push towards on-device and more capable models that can do more with user context,” Matt Vlasach, Jamf senior vice president, enterprise products and solutions engineering,” said in an interview. “While obvious for consumer use cases, as illustrated in the keynote, the opportunity to evolve this to the work context using a more advanced Apple Intelligence framework is an exciting evolution.”

“OS 27 feels like a deliberate reset, less about new features and more about polish and quality-of-life improvements, which most users will welcome,” said John Richards, general manager, IT products, at Iru. “The new capabilities are focused entirely on Apple Intelligence and Siri AI, and what’s encouraging is how much Apple leaned into privacy with the Gemini partnership. That combination of capability and privacy-first design is the right instinct.”

“The single biggest request I made at Apple’s Foundation Models workshop in Madrid was opening Private Cloud Compute to third-party developers,” said Serhii Popov, senior software Eengineer at CleanMyMac. “It’s here and free for apps under 2 million users. That’s a real breakthrough and a huge opportunity for a lot of great apps.”

How will integrated AI change things?

Joel Rennich, senior vice president for product management at JumpCloud, looked ay how on-device AI will transform other paradigms. For starters, it shifts identity from simple authentication to governing what actions an AI agent is allowed to take.v“Enterprises will need identity frameworks that govern both human and non-human actors consistently,” he said.

“iOS 27 and Apple Intelligence point toward an operating system that does not just launch apps, it executes intent,” Rennich said. “Instead of users navigating between tools, the OS increasingly mediates outcomes directly through AI. This changes how work is initiated and completed on devices.

“With Apple Intelligence integrated across core experiences like Siri, Safari, and system services, AI is no longer an overlay but infrastructure. The separation between where data lives and where it is used becomes increasingly invisible to the user. Intent becomes the primary input, not app selection.”

I also spoke with Hexnode CEO Apu Pavithran, who pointed to some of the concerns enterprise users might have following WWDC: “The keynote didn’t speak much to admins,” he said. “The features that matter most at the management layer, such as how Apple exposes Siri AI through MDM APIs, whether IT gets granular per-app controls for Apple Intelligence, how shared device deployments handle the new assistant — these will be answered in the developer documentation. This week, that’s where IT teams should be looking.”

“Admins should dig in immediately and see what’s changed. Watch the developer docs, audit how Apple Intelligence interacts with existing device policies, and remember that the keynote is only a part of the story for enterprises,” he said.

Making AI great again

“Rebuilt from the ground up, Apple is trying to make AI feel native, useful and invisible across the devices people already use every day,” Francisco Jeronimo, vice president for client devices at IDC, said in an interview. “This matters, because the winning AI experience for consumers will not be the loudest or most technically complex. It will be the one that understands context, respects privacy, works reliably across apps, and reduces friction without forcing users to change behavior.”

“[Apple] is also clearly seeking to differentiate through its privacy promises,” said CCS Insight’s Wood. “This looks like a step in the right direction, but there is no room for complacency, and Apple still has a long AI journey ahead.”

Pavithran reflected on something more. “Overall, it’s hard not to think of this year as a deliberately measured keynote, one that’s intentionally playing it safe and seeking to rewrite the AI narrative,” he said. “I won’t be surprised if this ends up setting the stage for a much bigger installment next year with incoming CEO John Ternus hitting the ground running with some ‘wow’ features like new hardware or agentic AI at scale.”

That we can now seriously consider that possibility shows the extent to which Apple has regained momentum in AI on its platforms.

You can follow me on social media! Join me on BlueSky,  LinkedIn, Mastodon and The Core.