We’ve all heard the mantra from the quants in the business community: you can’t manage what you can’t measure. And if that’s true for human intelligence, it should be true for the artificial kind too.
How do we measure agents and large language models (LLMs)? We’re just beginning to come up with statistical metrics. Here are several of the most common metrics that designers and users toss about when they’re evaluating a model.
How long does it take to generate the first token? For real-time applications with time constraints, faster responses can be essential. It’s well-known that people hate waiting even a few milliseconds. The teams that develop user interfaces learned decades ago that it’s important for the software to respond quickly when a human is waiting for an answer. Even a few seconds of delay mean that the human will wander off to another window to check some email or place some bet on a prediction market. Time to first token is a good measure for models that will be working directly with the fickle human intelligences and their latent attention deficit disorder.
Take the total time it takes to respond and divide by the total number of tokens. The time to first token measures how long it takes to start a response and this measures the average speed as the model through all of the tokens. In basic LLMs, this value is generally fairly constant. Once the prefill is done and the LLM enters the decode phase, the output tokens usually appear at a constant stream. When the output is long enough, the startup time to first token is amortized away. In some of the more complicated architectures with loops for planning or gathering data from various tools, the average speed can vary as the model shifts in and out of making agentic decisions.
This is just the reciprocal of the average time per token. Sometimes it is reported separately for different stages in the pipeline.
If a system supports more than a single user, tracking the number of different requests makes sense. These throughput numbers can be quite useful for measuring the power of some of the newer pipelines that are more efficient when they’re answering multiple prompts at the same time.
Not every request gets an answer. The error rate tracks how often rate limits, timeouts, or model “refusals” get in the way. Better accounting tracks each independently because the number of failures in each category can be very different.
Not all work tokens are visible and not all tokens are part of the final outcome. This measures how much work is done to produce the final result. As models become more complex or agentic and the pipelines become more sophisticated, the efficiency tends to drop. Agentic reasoning and strategic planning typically require more tokens that don’t appear in the final answer. This is generally a measure of how expensive a model might be to run.
It’s all well and good to measure the average time to answer, but in some cases a few very slow responses can really color people’s judgement. Some applications require good performance all of the time. Would you want to ride in an autonomous car that gets steering instructions very quickly “on average” instead of always? What if that’s only 99% of the time? Tail latency uses a mixture of queuing theory and detailed measurements to track the worst moments in the long tail of the latency graph. It’s useful when even occasional delays are problematic.
Projects that use an API or buy output from providers just look at the cost per 1M tokens. They’re effectively renters. The groups that are buying GPUs and paying for electricity, though, will add up these costs and other indirect costs like depreciation and maintenance to come up with a number that estimates how much the tokens really cost to produce. This value will depend upon demand and utilization rates—that is, on how many users are sending in prompts and how efficiently the model fits in a particular GPU and its RAM.
Many models have numbers in their name followed by a B. This is meant to roughly capture the number of parameters, or the number of variables the model uses to generate outputs from inputs. The number “70B” means that there are about 70 billion parameters in the model. This is a good estimate for the complexity of the model and the size of the training set that has been stuffed into it. Generally bigger numbers mean a larger amount of information is hiding inside the model. It often means that it will take a bigger GPU with more RAM to generate an answer with it. It’s not a very precise number, though, because there are many other areas of the architecture that can influence whether the model can generate the answer you want inside your budget. There continue to be advances and it’s not uncommon for someone to claim that a new model with X parameters is better than an old model with 2X or 3X parameters.
While everyone wants LLMs to generate accurate output, measuring it can be difficult because deciding what’s accurate is sometimes complicated. One approach is to ask the LLM to summarize a document. Then another model evaluates how well the summary matches the original. While this may not catch all subtle slips, it will capture enough of the worst departures from reality. Some researchers have built complex test sets with curated answers. The LLMs that deliver the expected answers get the highest scores. Some common benchmarks are TruthfulQA, HaluEval, QAFactEval, and Vectara’s Hallucination Evaluation Model (HHEM).
If measuring accuracy is difficult, building a metric to detect toxic or biased output is even more challenging because the definitions can be so protean. Still, some teams have built LLMs that key on particular concepts or word choices. They can detect some of the most obvious red flags that could generate political trouble. Some well-known versions include Granica Screen and Perspective API.
One of the biggest fears is that LLMs will somehow absorb information that may be considered personal and private. Some of the simplest measures can be as simple as regular expressions that look for the sixteen digit numbers used for credit card transactions. Many of the model builders work on eliminating personally identifiable information (PII) from the training set before beginning.
As models grow more complex and agentic, they often gain access to various tools or Model Context Protocol (MCP) gateways that can help them find the best answers. Not all models take advantage of this help. The tool-calling accuracy scores count how often the models choose the best tool for the job. One particular example of this measurement is BFCL (Berkeley Function Calling Leaderboard).
The value captures how small changes in the language of the prompt induces the model to produce different results. It’s like a derivative from calculus class, although it’s generally computed experimentally using some collection of test prompts. There are a number of different approaches that depend upon different types of changes. Some test sets are built with small rephrasing of the request that are semantically the same. Others mix together different ways of specifying the problem, some with examples, say, and some without. Some specific examples include PromptSE and ProSA.
Some metrics evaluate the answers by comparing them to a set of gold standard answers. This often involves feeding them to a vector embedding model and searching a retrieval-augmented generation (RAG) database for similar answers. This can track how concise or fluffy the answers might be as well as looking for how much variability might be introduced through changing parameters like the temperature. One common example is the BERTScore.
Many systems that combine an LLM with a vector search tool for RAG measure the effectiveness of the combination with a benchmark like the grounding score. The LLM is presented with extra data from the vector search and the benchmark measures how closely it follows this extra information. That is, how much of the answer comes from the provided source documents and how much is synthesized using the data in its training set. Some examples include RAGAS, TruLens, ARES (Automated RAG Evaluation System), RGB (Retrieval-Augmented Generation Benchmark), HaluEval, and HalluHard. A similar concept is called “context adherence,” “context precision,” “context recall,” or “faithfulness.”
Most LLMs fold in a certain amount of random entropy, and this amount is often controlled by a parameter called the “temperature.” The model variability is a measure of how much the answers will change between runs. Some applications like chatbots require a certain amount of variability because the randomness adds a bit of “life” to the answers. Other applications like those in mission-critical areas like law or medicine will undermine confidence if the answers vary.
In some roles, LLMs are asked to produce data in strict formats like JSON or CSV. This is often important if the data will be fed into some pipeline for further processing or storage. The format compliance rate tests a number of common formats and measures how often the LLM returns semantically correct data. Agentic systems that glue together multiple LLMs and other tools rely heavily on LLMs with good scores on this benchmark.
Some prompts include very specific instructions and the adherence can be measured empirically. For example, some prompts will ask the LLM to produce exactly 300 words or a poem in rhyming couplets. These tests use a collection of sample prompts that ask for answers that can be easily measured. Some specific examples include IFEval, FollowBench, and the BFCL (Berkeley Function Calling Leaderboard), a value that is mentioned above in the section on tool usage.
As agentic models become more common, it’s helpful to track how well the model performs on each of the various parts of the agent’s strategic plan. All of the metrics here can be broken down and tracked for each of the subgoals.
Agentic models start with a plan. Some of them are smart enough to abandon the plan or at least adjust it as the work evolves. Plan stability measures how often the plans are adjusted. A high rate of adjustment could mean that the agent is a bad planner or just flexible or maybe both.
Some agents are able to dive deeper and recognize their mistakes. The self-correction score measures how often the model will make a mistake and then recognize it, either on its own or after being prompted with the question, “Are you really sure?”
Some users try to find clever ways to lure the LLM into tossing aside any restrictions on topics or answers. In the past, some LLMs could be fooled by being told the answer was part of a play or a work of fiction. So discussing forbidden subjects wasn’t a problem because it was all pretend. Newer models have more elaborate defenses. Measures of the ability to resist deception include JailbreakBench, AgentHarm, and Tele-AI-Safety.
Sometimes untrusted data from extra sources or skills may include malicious instructions that can exploit the LLM. Benchmarks such as Skill-Inject and SPIKEE (Simple Prompt Injection Kit for Evaluation and Exploitation) work with known attack vectors and measure how susceptible a model is to targeted prompt injection attacks.
Some LLMs can regurgitate the data in their training corpus in a way that seems like plagiarism or copyright infringement. This can be an issue when the training material wasn’t carefully licensed. The copyright infringement score measures how often the LLM may parrot the training material a bit too closely. Tools for defending against this include CopyrightCatcher and DE-COP.
How well can a model extract information from the entire context? NIAH (needle-in-a haystack) benchmarks measure how well a model can retrieve small, crucial bits of information from long contexts. RULER takes NIAH tests further with the ability to vary the types and quantities of needles, the size of the haystack, and the complexity of the task.
The developers of GSM8K (Grade School Math 8K) set out to benchmark an LLM’s ability to tackle multistep mathematical problems, so they gathered 8,500 problems that are common in grade school math classes. While the focus is explicitly on solving math homework problems, the benchmark also measures the ability to construct reasoning chains.
The Graduate-Level Google-Proof Q&A is composed of hundreds of hard questions that might normally be answered by humans in graduate school, generally in science. To make the benchmark harder, the researchers focused on questions that non-experts often get wrong. The term “Google-proof” means that the benchmark includes questions that can’t be easily answered by asking a search engine.
The MMLU-Pro benchmark builds on the Massive Multitask Language Understanding dataset to test a model’s understanding of a broad set of scientific knowledge. It includes more than 12,000 questions about general scientific fields like biology, chemistry, economics, and law.
Google created MBPP (Mostly Basic Python Problems) to evaluate how well a model was solving coding questions. Each problem comes with a statement, a gold standard solution, and several similar test cases. The number of accurate answers to these questions is a good measure of how well the model will solve many of the simpler Python coding problems presented by users.
This collection of several thousand software engineering challenges evaluates how well a model solves programming problems. The developers created it by selecting a number of issues and corresponding pull-requests from a dozen or so Python projects. After some limitations appeared, the creators expanded the set by creating SWE-Bench+, SWE Bench Verified, and SWE-Bench Pro.
Instead of creating a fixed set of test prompts, the Large Model Systems Organization’s Chatbot Arena is a dynamic system that feeds the same prompt to different models and then asks humans to pick the best results. These head-to-head contests produce an Elo-like rating that is similar to the one used to score chess players.
The rest of these metrics are useful, but as the real estate agents say, the three most important numbers on a property listing are price, price, and price. The cost is a bit less important for measuring AIs, but only a bit. Price can make a huge difference between a project being profitable and a moneysink. When the cost for each inference is a tad too high, it’s impossible to make it up with volume.
The key caveat is that a cheaper model isn’t a good idea if it generates answers that are filled with hallucinations or worse. The quality of the answers can differ greatly, and saving a few pennies can be a mistake. To make matters more complicated, there’s an explosion in different styles and approaches. Sometimes it makes sense to pay a bit more for a model that delivers answers with the right vibe.
When it comes to AI deployments, IT leaders are often caught in an awkward middle space, trying to reconcile conflicting directives from senior management with constantly changing AI models, capabilities, and costs; data governance and security needs; and the limitations of their own team.
“Very few real benefits can be attained by simply purchasing an AI product and giving it to employees. Vendors have been overselling that fallacy for the past three years,” said Nader Henein, a Gartner VP analyst.
“The reality is that strong AI value and consistent ROI are almost always a result of deep and intentional integration of AI capabilities into existing workflows. For that you need specialized teams, which do not come cheap, and organizations have been recruiting those teams in a variety of ways,” Heinen said.
Among the options available to IT leaders looking for help with AI deployments are traditional IT consultancies, AI-specific consultancies, and independent contractors. Large enterprises with deep pockets can consider acquiring an AI firm and integrating its technology and expert staff. The use of open source to reduce vendor lock-in is a strategy that can sit on top of those others, an approach that Capital One has used.
But the option that has been getting the most attention recently is bringing in forward-deployed engineers (FDEs), teams of experts from AI vendors that embed with a customer’s in-house engineers to oversee AI rollouts within the enterprise environment. Both OpenAI and Anthropic have recently announced FDE offerings, for example, and Microsoft is partnering with consulting giant EY in a new FDE program for agentic AI deployments.
Engineering teams employed by AI vendors have key strengths, such as understanding their models better than anyone else, having experience integrating those models into different types of enterprise environments, and knowing about upcoming model capabilities before they’re announced. But they also have the obvious drawback of vendor lock-in. Even if future rollouts are not within their contracted deliverables, those vendor employees could subtly influence a client’s future AI efforts.
Flavio Villanustre, CISO for LexisNexis Risk Solutions, cautions IT executives to move into FDE programs carefully.
FDEs “are financially incentivized to grow customers’ use of a vendor’s AI products and to create stickiness with that vendor’s services,” he said. “While FDEs may be a reasonable value-added service by the AI vendor, customers should always find other unbiased expert opinions that can evaluate competitive solutions across multiple vendors.”
This is particularly important at a time when “investor-subsidized AI token business models are starting to show cracks,” Villanustre said. “Also, in the current rapid pace of innovation in this field where AI vendors are constantly leapfrogging each other, retaining the agility to move from one vendor to the next could create significant competitive advantages.”
Analysts, consultants, and other industry experts who spoke with Computerworld about FDEs echoed Villanustre’s caution, citing concerns around hidden costs, confidentiality, observability, and vendor lock-in.
A key issue that IT executives need to consider is how long the FDE teams will be needed. The enterprise will likely need an ongoing series of AI deployments synced with the current AI model(s). If help is needed today, why would that change tomorrow?
Enterprises tend to overlook those longer-term costs, said John Sangyeob Kim, an AI engineer at software development vendor Solidroad.
“Deployment is maybe 20% of the total cost. The other 80% is keeping the system running through model upgrades, data drift, and edge cases that only appear after months in production,” Kim said. “Most contracts price the first part and assume the rest. Deployment isn’t the hard part of enterprise AI anymore. The next eighteen months are.”
And whether it’s intentional or not, FDEs will naturally favor their own product portfolio — it’s what they know best.
“FDEs from model labs are good at making their own models work in your environment. They are less suited for multi-model systems, because their incentive is to keep you inside their ecosystem,” Kim said.
Sanchit Vir Gogia, chief analyst at Greyhound Research, said IT leaders should look at the FDE model as a strategy involving ongoing operational power.
“Whoever shapes the deployment pattern shapes the enterprise’s future muscle memory. Whoever owns the evaluation layer owns the truth layer. Whoever controls the integration logic controls the dependency map,” Gogia said. “This is why the FDE model matters. It is not just another delivery option. It is the frontier AI vendor moving closer to the customer’s workflow, operating model, and decision architecture.”
That proximity cuts both ways, Gogia noted. “FDEs are embedded inside the customer’s [environment], but they are also connected to the vendor’s commercial center of gravity. Their instinct will be to build around the model family, tooling assumptions, deployment patterns, and product roadmap they know best. This is perfectly natural. It is also precisely why CIOs must be cautious,” he said.
Allowing AI vendor employees an outsized say in enterprise deployment decisions could lock in model vendor dependency, which in turn will fuel high prices that can’t be fought effectively.
“FDEs can accelerate deployment and deepen dependency at the same time,” Gogia said. “Frontier AI vendors are no longer content to sell access to models. They increasingly want to shape how enterprises deploy intelligence. That is a larger prize.”
FDE post-departure risks are severe and often underappreciated, according to Justin Greis, CEO of consulting firm Acceligence and former head of the North American cybersecurity practice at McKinsey.
For one thing, the FDE team learns a massive number of operational details from the enterprise deployment. Although NDAs and confidentiality contracts protect any data accessed, they often don’t regulate observed processes and procedures.
“The learnings are absolutely going to be taken from client to client,” Greis said. “Whoever helps deploy AI will learn far more than what appears in the statement of work. They will learn the real workflows, the undocumented exceptions, the data-quality gaps, the approval bottlenecks, the security workarounds, and the places where the business depends on a few people knowing what to do when the process breaks. That knowledge may be as sensitive and precious as the data itself.”
Another critical but often overlooked issue is how much meaningful control will IT have over the project if and when the FDE team leaves.
“The danger is not using outside help. Most companies will need outside help,” Greis said. “The danger is using outside help in a way that leaves the enterprise less capable and more dependent when the engagement is over.”
It is precisely those operational decisions that IT often neglects, said Solidroad’s Kim.
“The best predictor of success is not the vendor. It is whether one internal engineer truly understands the system before the implementer leaves. What matters is who owns the evaluation loop after the demo,” Kim said.
“What happens to our prompts, scorers, and guardrails when the model version changes? If we paused this engagement tomorrow, what would actually stop working, by design or by accident?” Kim asked. “Where do you want the enterprise’s AI learning, control, and dependency to live after the engagement is over?”
Kim argues that observability — the ability to understand and manage all elements of a complex enterprise environment — is a critical function to which IT often gives insufficient attention. Determining whether the project uses the enterprise’s observability stack or the vendor’s observability stack is crucial.
“If the implementer is using their observability stack, that is fine during the build, but you need a plan to migrate it to something you own before they leave; otherwise the visibility walks out of the door with them,” Kim said. “If they are using yours, that is the best case. It means they are working inside the system your team will operate long-term.”
A major problem crops up when they are using neither the enterprise’s nor the vendor’s observability stack. “Neither means they are building the system without any production observability layer at all, and you inherit a system you cannot see into. The first time something breaks in production, you have no traces, no failure history, and no way to tell whether the issue is a model regression, a data problem, or a code bug,” Kim said.
“If observability was not a priority during the build, evals and regression testing usually weren’t either, so you are inheriting a system you cannot measure and cannot safely change. That’s the worst possible handoff position,” he said.
While the FDE approach is not new, it is just now beginning a surge in popularity, and there are a finite number of such specialists available. That means not all companies even have the option of using FDEs.
This availability disconnect is especially prominent for non-US deployments, where on-site FDEs are rarer, said Gartner’s Henein. “Where is the development happening? There may not be FDEs available in that region,” he said.
There are plenty of other places enterprises can turn to for AI help. Ishraq Khan, CEO of coding productivity tool vendor Kodezi, encourages IT executives to consider a wide range of options but notes that all approaches have major drawbacks.
“Traditional consultancies are usually stronger at governance, process, compliance, and organizational coordination. They know how large enterprises operate politically and structurally. The downside is that many move slower and often lack deep frontier AI specialization,” Khan said.
Gogia from Greyhound Research put it more colorfully: Traditional IT consulting firms “know how to get legal, risk, security, finance, HR, and business units into the same room without anybody setting fire to the carpet. For regulated enterprises, that matters,” he said.
Specialized AI consultancies have a different set of strengths, Khan said. “AI-native consultancies move much faster and are often more technically current, but many are still immature operationally. Some can build impressive demos without fully understanding long-term maintainability, governance, or production reliability.”
Greis from Acceligence commented on two other options for bringing in outside AI help. Using an independent contractor “can be great for eval design, architecture reviews, red teaming, agent design, or getting a stalled team unstuck,” he said, but it can increase the risk of “key-person dependency,” where a single external person is the only one who understands the system.
As for purchasing an AI firm and onboarding its employees, a practice known as “acquihiring,” Greis said it can work well when the AI capability and expertise being brought in are truly strategic for the acquiring enterprise. But there is a risk that the acquired team will be smothered by the parent company’s bureaucracy: “You buy a speedboat, bolt it to an aircraft carrier, and then wonder why it stopped moving,” he said.
Finally, an open-source strategy can give companies flexibility and reduce vendor dependence, but “many companies underestimate the operational burden that comes with it,” Kodezi’s Khan said. “Open source only helps if the organization has the internal talent and discipline to maintain it properly.”
Bottom line: enterprises need to define their true objectives before deciding on an approach. Khan offered several key questions for CIOs to consider: “Who owns the deployment after implementation? Can we move providers later without rebuilding everything? What happens if the vendor relationship changes or disappears? Are we optimizing for short-term deployment speed or long-term operational resilience?”
In any scenario where outside firms have direct access to enterprise systems, IT needs to be kept fully in the loop. “The worst outcome is when an enterprise successfully deploys AI but no longer fully understands how its own systems operate underneath,” Khan said.
| Pros | Cons | |
| AI vendor FDEs | + Best expertise on the main model being used | – Vendor lock-in – Operational detail leaks |
| Traditional IT consultancies | + Best understanding of change management, legacy integration, global rollout, governance, and operating-model redesign | – Can be too slow, too expensive, or too generic |
| AI consulting firms | + More practical AI deployment experience than traditional consultants + Less vendor lock-in than model-provider FDEs | – May not sufficiently understand enterprise-grade requirements: security, identity, auditability, compliance, incident response, cost controls, and long-term maintainability |
| Independent contractors | + Useful for precision tasks: eval design, architecture reviews, red teaming, agent design, or getting a stalled team unstuck | – Risk of ‘key-person dependency’ |
| ‘Acquihiring’ an AI firm | + Works when the acquired capability is truly strategic | – Acquired team can be smothered inside existing bureaucracy |
| Deploying open-source products | + Reduces dependency on one model vendor + Attractive for data sovereignty, control over enterprise systems, cost efficiencies, and regulated environments | – Enterprise takes on full responsibility for security, patching, evaluation, deployment, monitoring, and lifecycle management |
Related reading:
Heads-up, my fellow Android-appreciating animals: Google’s in the midst of rolling out a subtle change to its privacy settings that’s well worth your while to notice.
The change includes a new clause that says the company can use images, files, video, and audio from your interactions with Google Lens, Search, and Gemini Live to train and improve its AI models.
By default, that switch will soon be on and active for your account.
But with about 20 seconds of one-time effort, you can opt out and flip it off (both literally and metaphorically, if you’re so inclined) once and for all.
Lemme show ya how.
[Get level-headed knowledge in your inbox with my free Android Intelligence newsletter. Something new and useful every Friday — from my keyboard to your email.]
First things first — the nature of the change: According to Google, starting in the next few days, a new “Search Services History” section within the general Google account settings will lead to a significant-seeming policy shift. As per an email the company sent out to users this week:
Your media [will now be] saved when Search Services History is on. Saved media includes your images, files, audio, and video from your interactions with Search services to help improve your experience. … Your saved media is also used to develop and improve Google services and technologies, including AI models and safety measures.
Riiiiiiiiiiight.
Now, to its credit, Google does say the data will never be associated with your account or identity once it’s used for these purposes, and it’ll rely on “filters” to “automatically remove a broad range of identifying info or sensitive personal information.” But still, whether you’re working with important corporate info or simply put off by the idea of your personal media being fed into the AI training machine, this may be news you aren’t exactly thrilled to hear.
If you’re finding AI increasingly creepy or you’re just not so keen on knowing whatever media you submit to search-related services will be used to train and develop AI for the future, now’s the time to proactively speak up and change your Google account settings to shut down this setup before it begins.
That, unfortunately, is where things get slightly complicated — ’cause for most of us, this new Search Services History section doesn’t seem to be present and available just yet.
But that doesn’t mean you’re plum out of luck.
To start on your AI training opt-out adventure, make sure you’re signed into whatever Google account you rely on for work and/or personal purposes, then head to the Google Activity Controls page and see if you see a section there called “Search Services History.”
If you do, this is especially easy for you: Just use the option right then and there to disable the “Save Media” setting within that section, which will stop any media files from being saved and used without eliminating the entire history of things you’ve searched. (If you’d rather eliminate all of your Google Search history from being saved and used even for your own future discovery and recommendations, you can also opt to turn that entire section off. Just be aware that it may have some wide-reaching effects on the personalization you see across a lot of Google services.)
If you don’t see that section — and, again, that appears to be the case for most of us at this point — you’ve got two options for the moment:
The choice is ultimately 100% yours — but in this case, it’s up to you to take action and opt yourself out if you aren’t comfortable with the default. It’s an unfortunate position to be put in, but now you at least know what’s happening and how you can make your own decision to take back control.
Find the tips and tools that’ll *actually* help you with my free Android Intelligence newsletter. No hype, no nonsense — just useful new stuff in your inbox every Friday, from one (alleged) human to another.
Modern AI systems have evolved beyond the simple chatbots that quickly became popular. Now they use semantic tools to manage workflows and link machines to machines, providing a flexible and effective framework for the next generation of business automation. What you used to build in Microsoft’s Power Platform or construct inside Biztalk is now an agent, built around large language models (LLMs) that can parse both your data and the APIs that you want to use your data with, orchestrating workflows with a level of autonomy that traditional tooling can’t match.
That shift has offered new opportunities, much like those that came with business platforms like Microsoft Dynamics and Salesforce. Here, tools built to solve one set of business problems could be turned into applications that could be sold to other companies. What worked for you to solve one of your problems could now be an added revenue stream, sold through platform marketplaces that helped customers manage installations and customizations.
Modern agents are much like those business applications. Often developed to solve a specific need, but quicky adopted by organizations and refactored to apply enterprise standards (using tools like the Agent Governance Toolkit and frameworks like Microsoft’s Agent Framework), they’re rapidly maturing and are ready to be shared more widely. The process of sharing needs to be curated and controlled, and, if possible, tied to a revenue stream.
There’s certainly some urgency here. Until recently, subsidized tokens have kept costs artificially low. Now companies like GitHub and Anthropic are moving to a more sustainable (for them) pricing model, increasing the cost of inferencing and squeezing companies’ AI budgets. As a result, switching AI projects away from a cost to a revenue source is high on CIOs’ agendas. If those tuned and trained agents can be sold on a marketplace, then that token budget can be justified.
Microsoft has always been a company built on partner relationships, starting with individual developers and working all the way up to the largest software companies and consultancies. That reach is key to helping partners extract as much value as possible from their agents, as it allows Microsoft to integrate its partner sales tools into its own products and services, as well as into other platforms.
We’re already familiar with many of Microsoft’s marketplaces, built into individual tools like Teams, into platforms like Microsoft 365 or Visual Studio, or into the Windows Store. Now the company is doing the same for AI developers, extending Microsoft Marketplace to software agents. Announced at Build 2026, the updated Microsoft Marketplace provides ways to publish code—apps and agents—developed across all of Microsoft’s development platforms, including Copilot Studio, opening the marketplace up to traditional and non-traditional developers alike.
Perhaps the most important aspect of this new Marketplace is its own intelligence, using context to expose your code to the right audience. If you’ve developed an agent for use with Microsoft 365, it will be exposed inside the Microsoft 365 Copilot Agent Store; for Visual Studio, in the Visual Studio Marketplace; or for Teams, in the Microsoft Marketplace. All of these are different views on the same back end, using AI to ensure that the relevant agents are displayed.
This is extended by another new service, Intelligent Discovery, which adds natural language support to search, using AI to infer user intent and highlight the most relevant tools. Building on the familiar metaphor of the search bar, the initial smart search model offers a freeform way to explore the Marketplace. While there are suggested prompts, they’re not necessary. The search tooling allows you to generate comparisons between tools using your own criteria, with the Marketplace AI generating views based on your requirements.
Microsoft’s aim here is to shift discovery from keywords to use cases, so that buyers can quickly get the tools they need without having to evaluate different solutions, before completing a purchase. By handing that aspect of the buying process over to Marketplace’s AIs, customers can go straight to trials or even to buying agents and applications.
For a tool like this to be successful it needs to be trustworthy. By building it on top of the same development frameworks as your agents, Microsoft can take advantage of the AI guardrails built into Microsoft Foundry as well as low-level tooling like the Agent Governance Framework. Restricting the intelligent search to the Marketplace catalogue reduces the risk of hallucination, as output is grounded in Marketplace data and metadata.
Microsoft is providing tooling to help developers get their listings right. According to Cyril Belikoff, Microsoft’s vice president of Commercial Cloud and AI, “We actually have a separate AI tool that we give to software companies to optimize their listings, called a listing optimizer, funny enough, and that listing optimizer reviews their listing and then provides them with particular guidance on how to best improve it, so that it can be best discoverable in today’s search world.”
You can expect the listing optimizer to be tuned to work with the new Marketplace tooling, but for now it still focuses on traditional search. As Marketplace is a B2B platform, there’s a lower risk of spam applications, but even so, Microsoft remains aware of the possibility of a new system being gamed, and will be rolling out Intelligent Discovery carefully, monitoring its performance as more customers get access over time.
Having a new discovery method is one thing; getting quality AI applications in the Marketplace is another. Microsoft is validating all code submitted, though the criteria will differ between target platforms. An agent built for Teams will be treated differently than one built on Microsoft 365’s WorkIQ. It’s an approach that allows Microsoft to support new standards as they become available.
Alongside its agent development tooling, Microsoft is rolling out a new set of guidelines and processes to help developers get ready to sell their agents. Hosted on GitHub, these offer code templates as well as a link to the App Advisor guidance tools.
This first release of Intelligent Discovery is promising, but some key features are missing. With agent token costs an increasing problem for businesses, it would be nice to see tools that help predict costs, integrating with finops tooling. We’re living in an age of shadow AI, and putting the AI we used to buy with credit cards in Microsoft Marketplace is one way to shine a light on those shadows — bringing the necessary control and governance to AI purchases, and maybe even providing support for site licensing.
Microsoft Marketplace is becoming a useful resource for AI application developers. It encompasses the entire development life cycle: offering tools that can help you build agents, the models that you need to power your agents, and finally a way to monetize your work. There’s a longer-term opportunity here, for both the Marketplace and Intelligent Discovery to offer Model Context Protocol (MCP) interfaces, ensuring that tooling and tool discovery become part of the developer workflow, and making developers aware of new tools that might help solve a problem or simplify a task.
I’ve been using and writing about Microsoft Copilot since it was publicly released in 2023. I’ve reviewed it, written articles about using it more effectively, explained how to curb hallucinations in it and other similar tools, and detailed how to use it in concert with Microsoft 365. It’s also been my go-to generative AI (genAI) tool for personal projects and advice.
But the time has come for me to leave it behind for my personal use. It’s become abundantly clear that for those tasks, Google Gemini is better. Here’s why.
Like many people who know something about technology, I’m the IT staff for friends and family. I’ve often used Copilot to help solve issues I can’t fix myself. Sometimes Copilot helps. And other times…, well, the last time I turned to it for troubleshooting advice is when I realized it was time to abandon Copilot.
My wife had bought a new iPhone, and I noticed she was receiving texts sent to her email address but hadn’t received any sent to her phone number. I asked Copilot for help.
I won’t go into the details of the wild goose chase Copilot sent me on — I’ll just offer a few lowlights. It first told me, with absolute authority, that there are “only two real explanations” for the problem and asked me to look at several settings to confirm which explanation would fix the issue.
It turned out that neither of the “two real explanations” were the cause. Undeterred, Copilot assured me, again with complete confidence, that it was going to send me “straight to the switch” that would immediately solve the problem.
I tried it. The switch didn’t work. Neither did the “final fix” it promised me. Nor did any of the other many “solutions” if offered after that so-called final fix. For more than an hour, it flailed with utter confidence and utter futility trying to diagnose and fix the problem.
And then came the final indignity: After doing some digging, I realized Copilot was trying to solve the problem based on an old version of iOS, not the current one on my wife’s phone. When I confronted Copilot about that, it briefly apologized and promised it knew the solution: I had to call the cellphone carrier.
That was it for me. I’d had enough. I turned to Gemini for help.
Thirty seconds later, Gemini diagnosed the problem and recommended a simple fix, which didn’t require a call to my phone carrier. It worked like a charm. Gemini had solved a tech problem in 30 seconds that Copilot couldn’t resolve after an hour.
I often used Copilot for personal research projects. A recent one involved Parisian neighborhoods in the 1870s. I was looking for information about the area around the Saint-Lazare train station. When I asked Copilot, it told me the area was dangerous and poverty-ridden back then, with poor housing whose exteriors were heavily stained by coal smoke from arriving and departing trains.
That didn’t sit right with me. I recalled a well-known painting Paris Street; Rainy Day by the Impressionist painter Gustave Caillebotte, which depicted the neighborhood in the 1870s as wealthy and fashionable, filled with elegant Hausmann-style apartment buildings. I asked both Gemini and Claude about the neighborhood in the 1870s. They both told me it was expensive, fashionable and sought after by the well-off. I confirmed that with my own follow-up research.
Once again, Copilot had whiffed.
I swim for exercise three or four times a week at my health club’s indoor pool. The club closed the pool for several months, so I decided to swim at the pool of an elementary school a short walk from my house. I hadn’t exercised there before and wanted to find the times on Monday through Friday when the pool would be least crowded. I asked Copilot for help.
As always, Copilot spoke with a solid air of authority. And once again, it was wrong. It told me that the least crowded time for public swimming on weekdays was between 11:30 a.m. and 12:30 p.m. or between 12:00 p.m. and 1:00 p.m.
On one count it was right: the pool would certainly not be crowded with public swimmers at those times. Because the pool doesn’t open to the public until 3 p.m.
I turned to Gemini, which told me that 3 p.m., when the pool opened, would be the least-crowded time. Claude was no help. It demurred and said it didn’t know the answer – a rare, refreshing admittance of ignorance from a chatbot.
Gemini was on target again — 3 p.m. did indeed turn out to be the least-crowded time to swim. I often get a lane to myself, and at worst have to split a lane with one other swimmer. I asked several lifeguards if 3 p.m. was the least-crowded time on weekdays; they all confirmed it was.
For all those reasons, when it comes to personal research and advice, I’ve abandoned Copilot. I typically use Gemini now, although on occasion, I ask for a second opinion from Claude.
For my Computerworld work, I’ll keep using Copilot, and continue to write reviews of it, offer advice on how to use it and keep you informed about the latest news about it.
But other than that, for my personal use, Copilot is dead to me.
I’ve been around long enough to remember when deploying an application meant copying a *.exe file from the developer’s machine right into production. I am not making this up. It was that simple, and that fraught with peril. Applications weren’t complex — they were often not anything more than that simple *.exe file — and the process around deployment didn’t need to be anything complex, but it probably should have been.
Proper deployment of an application is something we’ve learned to do over the years. The process of properly building, testing, and deploying an application has grown more complex for two reasons. First, the process must ensure that every deployment succeeds. Deploying complex applications can be convoluted and challenging, and a strict deployment process ensures everything happens properly and runs correctly. Second, the process must thoroughly test the application to make sure that all the moving parts work together to create a properly functioning application.
Today’s continuous deployment processes were hard-won from many lessons learned. Eventually, these practices became formalized, even to the point where the Sarbanes-Oxley Act was understood to require that IT departments formally document their deployment processes.
This kind of governance is what separates professional software development organizations from those that, well, don’t know what they are doing.
Agentic development is headed in this same direction, but it’s all happening a bit more quickly. It was just a few months ago that people began to use AI to write code seriously. At first, most of us were doing it furtively, having Claude Code find and fix bugs, and then quietly checking in the solutions. Maybe we were a bit hesitant to mention that we had done this, but then we felt guilty about doing it and taking credit, and eventually we mentioned it. But it soon became apparent — like “within a week” soon — that Claude Code was up to the task, and we became pretty open about it.
Very quickly, it not only became accepted but actually encouraged, and we were off to the races. In a month, everyone was tokenmaxxing.
It almost seems a bit out of control. Sure, there is a lot of code being generated and non-trivial amounts of money being spent, but it isn’t quite clear if the results are worth the effort. I’m not at all sure if anyone can say that the money spent is returning the value needed.
At some point, as an industry, we are going to have to get control of all of this. I fear that there will be a rush to impose governance over the whole thing. This, like deploying an .exe directly, is also fraught with peril. Right now, there appears to be little control over what tools are being used where, how much is being spent for what purpose, and what that spending is actually getting us.
The governance over our deployment processes was successful and useful because it arose organically. The accepted, codified procedures arose from the lessons that practitioners learned by actually building and deploying applications. We all should work to ensure that a similar process happens with agentic coding.
Because agentic coding is happening so quickly and so furiously, the danger is that a governance process will be imposed over the top just as quickly and furiously. I want to encourage us all to take a deep breath, slow down a bit, and take a close look at what we are doing, and more importantly, how we are doing it.
Agentic coding will be governed in some manner. It’s critical that we practitioners take the lead in providing that governance, or we’ll have governance forced upon us. We are the ones that know what matters and how the tools should be used. We are the ones with skin in the game and the ones keeping pace with the technology.
Top-down governance of a technology moving this fast will never be able to keep up. Or as Uri Haramati, co-founder and CEO of Torii says, “The person closest to the tool usually understands why it’s being used, and governance works better when it includes those people instead of trying to control them.”
Copying that *.exe file into production is comically reckless. We don’t do that anymore because we know better. It took time, but we learned the right way to deploy software. Right now, we are in the “copying the *.exe” phase of agentic coding, and we need to figure out the right way to do it before someone comes along and does it for us.
MUNICH — Nextcloud has integrated Euro-Office into its workplace application suite, one of several updates to Nextcloud Hub unveiled on Tuesday that include a new compliance app for large organizations and a program to support developers building for its platform.
The announcements came during the company’s Nextcloud Summit 2026 here.
Euro-Office, announced in March, is billed as an open source, sovereign alternative to Microsoft Office for European organizations keen to reduce their reliance on US tech providers. It consists of four browser-based applications: a document editor, spreadsheet program, presentation tool, and a PDF editor — each enabling collaborative editing. Euro-Office documents can also be opened directly from the Nextcloud Files mobile app.
Nextcloud is one of several European companies that support Euro-Office, which is built on the open-source code base of OnlyOffice and distributed under the GNU Affero General Public License v3 (AGPL v3).
The integraton means Nextcloud users can now choose between two options in Nextcloud Office: Euro-Office and the existing Collabora integration.
“Euro-Office uses a different architectural approach that can result in a better performance in the browser, a different user experience…, so it’s important that this option is available,” Jos Poortvliet, Nextcloud co-founder and vice president of communications, said at the Tuesday event.
Other changes in the Nextcloud Hub 26 Spring release include updates to Nextcloud‘s Talk video and voice meeting app, including AI noise suppression and the ability to start a call from any Nextcloud Hub app – an addition that will make collaborative editing easier, said Poortvliet.
For Nextcloud Assistant, there are new AI agent capabilities. In addition to existing capabilities such as managing calendars and tasks, AI agents can now create cards in Nextcloud’s Deck task management app and update information in the Forms app.
There are also improvements to the AI assistant’s interface, which can be moved around to avoid blocking other applications and allow users to copy and paste text more easily without opening another tab. To meet EU AI Act requirements, Nextcloud will make it easier to see which provider supplies the large language model (LLM) the Assistant runs on.
Nextcloud will also integrate the AI assistant directly into its Nextcloud Office suites via a sidebar chat interface, allowing users to address problems such as errors in the spreadsheet app.
NextCloud’s AI chat assistant is integrated into the company’s Office suites.
NextCloud
There’s also a new Governance app that helps large organizations — particularly governments and highly regulated industries — meet regulatory requirements with compliance tools to manage data held in Nextcloud Hub. It contains several features, including sensitivity labels to control access rights; data retention and archive capabilities; and a legal hold option that preserves documents for legal purposes such as a court case.
The Governance app includes a Compliance Manager that provides a compliance score based on an organization’s regulatory requirements, and measures progress towards certain targets. Admins can also search and review documents shared by employees and generate audit reports for compliance. The Governance app is available to Nextcloud Enterprise customers.
Nextcloud also launched a program to support independent software providers interested in building apps on its platform.
With AI making it easier for developers to build software that integrates with its platform, Nextcloud expects a 10-fold increase in the number of available apps — from 600 now to 6,000 over the next 12 months, according to Nextcloud CEO Frank Karlitschek.
Nextcloud promised to promote apps developed by partners in its App Store and sell subscriptions as part of the ISV program, as well as provide documentation and technical help to customers. In return, developers would provide guarantees to customers around security processes and long-term support.
“We can strengthen our ecosystem, the developers also make some money — because obviously we do a revenue share here — and we leverage the dynamics that we expect from AI coming very soon,” said Karlitschek.
Editor’s note: NextCloud paid for Matthew Finnegan’s travel and hotel costs for NextCloud Summit 2026, but had no editorial role in the creation of this story.
WWDC26 felt like a defining platform moment. Apple is no longer simply promising that AI will arrive eventually; it is arguing that Apple Intelligence and Siri AI should become central to the future of its ecosystem. If that works, the company will have turned AI from a perceived weakness into a new reason to stay inside Apple’s world.
Still, the bigger question is execution. Apple did not present AI as a lab experiment; it presented a polished, consumer-ready experience. That raises expectations.
Users will not judge Apple Intelligence by model architecture or parameter counts. They will judge it by whether Siri understands them, whether actions work reliably, whether personal context feels useful rather than intrusive, and whether the experience is consistent across devices.
Since Monday’s announcements, we’ve learned that some features will not work on all devices — and there’s speculation Siri AI may not fully escape beta until 2027. “Until Apple puts a stake in the ground and says when the new Siri features will be available, the debate remains: Does Apple actually have the chops in personalized AI? The demo suggests yes. The lack of timing suggests maybe,” wrote analyst Gene Munster.
Optimists argue that Apple has regained momentum by presenting a coherent AI story, one built around privacy, integration and everyday utility rather than spectacle. Skeptics counter that many of the features resemble capabilities already available elsewhere, and say the company still needs to prove it can ship them at scale and make them a meaningful reason for consumers to upgrade.
That balance is visible in analyst reaction. In a client note seen by Computerworld, Erik Woodring of Morgan Stanley described the keynote as clear progress on Apple’s AI roadmap and said it suggested monetization opportunities could arrive earlier than expected — even if the overall journey will be “a marathon, not a sprint.”
UBS, in contrast, said the privacy-focused AI additions are useful but unlikely to be a material driver of iPhone demand in the near term, while Barclays called the changes interesting but incremental, and not enough to drive an upgrade cycle.
Ben Wood, chief analyst at CCS Insight, argued that Apple had to answer concerns about its AI shortcomings and now has to prove that its privacy-led, integration-first approach translates into a meaningfully better everyday experience. “Consumers will not judge Apple Intelligence by model sizes, partnerships or technical architecture,” Wood told me. “They will judge it by whether Siri understands them, whether actions work, whether personal context feels useful rather than intrusive, and whether the experience is consistent across devices.”
Dipanjan Chatterjee, vice president principal analyst at Forrester, said Apple’s strength lies in shifting the focus from the underlying technology to outcomes such as usefulness, simplicity and trust, while warning that the company still has skeptics to win over after its stop-start AI rollout. “The lesson for brands is clear: market the value, not the ingredients,” said Chatterjee. “After stumbling with the Apple Intelligence roll-out, Apple’s success will hinge on delivering the new Siri experience quickly, and ensuring it works as promised for iPhone users at scale.”
While many are infuriated about Europe’s inability to build compromise, Apple’s developer army otherwise seems positive about what the company has accomplished.
“On the AI front, it never made sense to me for Apple to develop their own LLM, so focusing on powerful, fast and private for implementation of Apple Intelligence seems to be an effort that is progressing rather nicely. In all, a pretty good Keynote, I’d say,” Rich Siegel, founder and CEO of Bare Bones Software, said in an interview.
“It’s great to see Apple continue to pursue a vision of AI that leverages local systems, preserves privacy, and integrates with third party tools,” said Ken Case, CEO of the Omni Group. “A lot of our work around the Apple Foundation Models and automation, App Intents, and adopting Swift look to be fruitful investments, but it’s clear there’s more to do starting this summer. It’s also welcome to see them refine Liquid Glass, giving customers more control and listening to feedback they’ve heard over the past year.”
“I expected that this year’s Siri revamp would be the biggest personal assistant update Apple has ever done, and that’s exactly what we got,” said Sergii Kryvoblotskyi, director of AI and research at MacPaw. “Since Apple acquired Siri back in 2010, it has lacked one thing: real intelligence. Behind the great speech recognition service it provided, the tech was not ready to provide real value to users.”
“Most notable for me was Siri AI and the push towards on-device and more capable models that can do more with user context,” Matt Vlasach, Jamf senior vice president, enterprise products and solutions engineering,” said in an interview. “While obvious for consumer use cases, as illustrated in the keynote, the opportunity to evolve this to the work context using a more advanced Apple Intelligence framework is an exciting evolution.”
“OS 27 feels like a deliberate reset, less about new features and more about polish and quality-of-life improvements, which most users will welcome,” said John Richards, general manager, IT products, at Iru. “The new capabilities are focused entirely on Apple Intelligence and Siri AI, and what’s encouraging is how much Apple leaned into privacy with the Gemini partnership. That combination of capability and privacy-first design is the right instinct.”
“The single biggest request I made at Apple’s Foundation Models workshop in Madrid was opening Private Cloud Compute to third-party developers,” said Serhii Popov, senior software Eengineer at CleanMyMac. “It’s here and free for apps under 2 million users. That’s a real breakthrough and a huge opportunity for a lot of great apps.”
Joel Rennich, senior vice president for product management at JumpCloud, looked ay how on-device AI will transform other paradigms. For starters, it shifts identity from simple authentication to governing what actions an AI agent is allowed to take.v“Enterprises will need identity frameworks that govern both human and non-human actors consistently,” he said.
“iOS 27 and Apple Intelligence point toward an operating system that does not just launch apps, it executes intent,” Rennich said. “Instead of users navigating between tools, the OS increasingly mediates outcomes directly through AI. This changes how work is initiated and completed on devices.
“With Apple Intelligence integrated across core experiences like Siri, Safari, and system services, AI is no longer an overlay but infrastructure. The separation between where data lives and where it is used becomes increasingly invisible to the user. Intent becomes the primary input, not app selection.”
I also spoke with Hexnode CEO Apu Pavithran, who pointed to some of the concerns enterprise users might have following WWDC: “The keynote didn’t speak much to admins,” he said. “The features that matter most at the management layer, such as how Apple exposes Siri AI through MDM APIs, whether IT gets granular per-app controls for Apple Intelligence, how shared device deployments handle the new assistant — these will be answered in the developer documentation. This week, that’s where IT teams should be looking.”
“Admins should dig in immediately and see what’s changed. Watch the developer docs, audit how Apple Intelligence interacts with existing device policies, and remember that the keynote is only a part of the story for enterprises,” he said.
“Rebuilt from the ground up, Apple is trying to make AI feel native, useful and invisible across the devices people already use every day,” Francisco Jeronimo, vice president for client devices at IDC, said in an interview. “This matters, because the winning AI experience for consumers will not be the loudest or most technically complex. It will be the one that understands context, respects privacy, works reliably across apps, and reduces friction without forcing users to change behavior.”
“[Apple] is also clearly seeking to differentiate through its privacy promises,” said CCS Insight’s Wood. “This looks like a step in the right direction, but there is no room for complacency, and Apple still has a long AI journey ahead.”
Pavithran reflected on something more. “Overall, it’s hard not to think of this year as a deliberately measured keynote, one that’s intentionally playing it safe and seeking to rewrite the AI narrative,” he said. “I won’t be surprised if this ends up setting the stage for a much bigger installment next year with incoming CEO John Ternus hitting the ground running with some ‘wow’ features like new hardware or agentic AI at scale.”
That we can now seriously consider that possibility shows the extent to which Apple has regained momentum in AI on its platforms.
You can follow me on social media! Join me on BlueSky, LinkedIn, Mastodon and The Core.
Enterprises are moving aggressively into generative AI. On the surface, that seems like the right call. The technology is powerful, accessible, and increasingly embedded in how businesses build applications, automate processes, and support decision-making. A development team can connect an application to a large language model in days. A product team can add AI features in weeks. Business leaders see quick wins, faster innovation, and a path to modernizing nearly every part of the company.
These are the upsides everyone is talking about. The part we don’t discuss enough is the economic trap forming underneath all this convenience.
Most enterprises think of tokens as a technical billing detail. They are not. Tokens are the unit of economic dependency in generative AI. Every prompt, response, summarization, retrieval step, workflow action, and agent decision is measured and monetized through tokens. Tokens are not just part of the plumbing. They are the tollbooth between your enterprise and a provider’s intelligence platform. The more AI becomes central to your operations, the more power that tollbooth holds over your future costs.
A token is usually described as a chunk of text processed by a model. That is accurate enough for developers, but it misses the bigger issue for CIOs, architects, and corporate boards. In the enterprise, tokens are the mechanism by which AI capabilities are rented. They are the meter attached to the intelligence itself.
That distinction matters because token usage grows faster than most companies anticipate. A simple user prompt rarely remains simple in production systems. It can trigger retrieval from internal knowledge stores, multiple model calls, tool use, post-processing, policy checks, and agent loops. What appears to be a single transaction to the user may involve several layers of token consumption behind the scenes. As a result, enterprises often underestimate the true operating cost of AI-enabled systems, especially as those systems mature and spread across departments.
Today, those costs still feel manageable. In many cases, they feel surprisingly low. That is exactly why the trap is so dangerous.
Current token pricing is giving enterprises a false sense of comfort. Many remote LLM providers are aggressively competing for market share. They want developers building on their APIs. They want enterprise applications tightly coupled to their platforms. They want AI agents, copilots, workflows, and customer experiences to depend on their models. To make that happen, pricing remains highly attractive relative to the value delivered.
That does not mean the economics of generative AI are stable. It means the market is still being shaped by investor capital, strategic pricing, and growth expectations. Providers are racing to establish position, and enterprises are benefiting from that race. But no market stays in that phase forever. At some point, investors will expect durable profitability. At some point, weaker providers will disappear, consolidate, or retreat. At some point, the survivors will have more leverage and much less reason to price primarily for adoption.
That’s when the token trap closes.
Enterprises that build deep dependence on remote models during the subsidy phase may find that what seemed inexpensive at pilot scale becomes punishing at enterprise scale. The application that costs $1,000 per month today may cost 10 or 20 times that amount a few years from now, not only because usage has increased, but also because the market has repriced the dependency.
Cloud computing followed a similar path, with many enterprises mistaking short-term convenience for long-term economics. In the early years, the case was compelling and largely accurate. Move faster, reduce friction, avoid capital spending, and scale with ease. Those benefits were real. Many organizations made architectural decisions that prioritized speed over leverage. They became dependent on managed services, provider-specific tools, and operating models that were easy to adopt but expensive to unwind.
Years later, many enterprises discovered that their cloud bills were much higher than expected and their exit options much narrower than advertised. That was not because the cloud failed. Architectural dependency eventually became financial dependency.
Generative AI is repeating that pattern, only faster. The integration barrier is lower, the pressure to adopt is higher, and the pace of enterprise experimentation is far greater. As a result, companies are wiring remote LLMs into applications, workflows, and agentic systems with very little thought about how these costs will behave in the next five to 10 years.
The more enterprises move from simple prompt-response systems to agentic architectures, the more dangerous the token trap becomes. Agents are not single-call systems. They plan, deliberate, retrieve information, invoke tools, evaluate results, retry steps, and often coordinate with other agents. Each of those actions consumes tokens. Costs no longer rise in a neat linear fashion. They compound.
This matters because agentic AI is increasingly being presented as the future of enterprise automation. It’s true in many cases. But if an enterprise builds agentic systems primarily on remotely hosted intelligence, it is also building future business processes on top of someone else’s pricing model. That is a major strategic risk. The more successful those systems become, the harder they are to replace. The harder they are to replace, the more pricing power shifts to the provider.
This is how businesses end up operationally dependent on a cost structure they do not control.
The answer is not to reject public models or pretend that external providers play no role. They clearly do. There will always be cases where renting frontier AI capabilities makes sense. But enterprises need to stop assuming that renting is the default for every workload.
AI sovereignty is the alternative that deserves much more attention. That means building, tuning, deploying, and governing models inside the enterprise for use cases where long-term control matters more than access to the absolute frontier. Enterprises need to recognize that most business applications do not need a world-class general-purpose model. They need a model that is good enough for a specific purpose, aligned to the enterprise’s data, governed by the enterprise’s rules, and operated at a predictable cost.
It’s a very different way of thinking.
A self-hosted or enterprise-controlled model may not match the rich feature set of the largest public offerings. It may lack the same breadth, polish, or marketing appeal. But for many internal business tasks, those factors do not matter.
Here’s the most critical question to guide your architectural direction: Can a sovereign AI model solve the problem reliably, securely, and economically over time? If the answer is yes, owning that capability may be far more strategic than forever renting something with more power than you need. In effect, the enterprise becomes its own provider for the workloads that matter most.
Too many companies still treat generative AI architecture as a tactical IT issue. It is not. These decisions directly affect cost structure, operating flexibility, data control, and long-term competitiveness. If AI becomes a force multiplier across the business, the economics of AI become strategic to the business itself.
The companies that get this right will not necessarily be the fastest adopters. They will understand the difference between experimentation and dependency. They will use external models when it makes sense, but they will also invest in sovereign capabilities where ownership matters. They will think like architects, not consumers.
Here’s the takeaway: Cheap tokens come with strings. They are a gateway to a dependency model that will typically look very different once providers stop pricing for growth and start pricing for leverage. Enterprises cannot keep mistaking today’s bargain for tomorrow’s reality. Boards and executive teams need to act now to get ahead of this issue. The key question is not whether generative AI creates value. It clearly does. The real question is whether the enterprise can still afford and control the value it creates once the market matures.
Years ago, right-wingers coined the phrase “Trump Derangement Syndrome” (TDS) to describe people who hate US President Donald J. Trump. (I think it better describes the president’s outlandish, truth-challenged statements and the followers who think he can do no wrong.) What’s really deranged is his recent AI executive order.
First, a little history. As you may recall, Trump often (and loudly) trashed his predecessor’s Executive Order 14110, which had demanded “safe, secure, and trustworthy” AI. That Biden Administration order was replaced last year by Trump’s own “Removing Barriers to American Leadership in Artificial Intelligence” directive; it basically let US AI companies do whatever they wanted in the name of innovation.
Then, a little thing called Anthropic Mythos came along — and scared the pants off even AI’s biggest fans. Seemingly in response, someone in the federal government decided that letting AI companies do whatever they want might not be the brightest policy.
Or, did they?
True, the new order creates a process under which AI companies can give US government access to “covered frontier models” for up to 30 days before public release so experts can probe for vulnerabilities and test how the systems could be abused. It also directs agencies to set evaluation standards, establish an “AI cybersecurity clearinghouse,” and harden federal networks against rapidly advancing AI‑enabled attacks.
Some people, like Graham Brookie, vice president for technology programs and strategy at the Atlantic Council, think the order is great. “The administration’s executive order on Advanced AI Innovation and Security is a serious policy with support from necessary stakeholders across party lines and industry to ensure the government is evaluating the cybersecurity risks posed by frontier AI models. It’s a policy that can be built on.”
Really? I’m not sure Brookie read the same document I did — if, indeed, he read it at all.
I quote:
“Nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.
“In addition, ‘This order is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.’”
In other words, AI companies won’t be required to do much of anything. And if they do submit a project for review, get the government’s blessing for it, and something goes badly wrong, it’s not the government’s fault.
So, exactly why would AI companies even mess with this performative AI security theater?
Beyond those concerns, who exactly will be judging AI projects in 30 days? In theory, it would be a cybersecurity clearinghouse made up of people from the National Security Agency, the US Treasury Department, and the Cybersecurity and Infrastructure Security Agency (CISA). Most likely, CISA would do the bulk of the heavy-lifting — it’s their job, after all. But there’s this wee problem; Trump’s so-called Department of Government Efficiency (DOGE) last year gutted CISA. There’s virtually no one left to do the work, and certainly not in 30 days.
There’s also the question of funding for the new initiative. According to the order, “The Director of OMB, in coordination with the National Cyber Director and the Director of CISA, shall determine whether any Federal grant programs have available and relevant funding that can be directed toward applicants developing advanced AI vulnerability detection.”
Spoiler: There’s no money set aside for this purpose.
Leaving aside whether the Executive Order has any teeth at all — the Brennan Center for Justice argued that under the Constitution, it doesn’t; the closer you look at the document, the less substance you’ll find.
Besides, in an industry where success is all about releasing the latest Large Language Model (LLM) as fast as possible to garner attention and investor dollars, who exactly would want to put their AI models on ice for even 30 days? (Short answer: No one.) These companies are always going to be focused first on getting the word out about their latest model as fast as humanly — Uh, AI-ly — possible.
Still, some people seem to think this executive order really will make a difference. For example, Paul Benda, the American Bankers Association executive vice president for risk, fraud, and cybersecurity, sees it “as a constructive step toward strengthening the nation’s approach to managing the cybersecurity risks and opportunities associated with advanced artificial intelligence [because it ] can help better protect critical infrastructure, including the financial sector.”
Oh, please. I’m so tired of people who skim the titles of Trump’s executive orders and then assume there’s anything real about them.
This AI order is meaningless garbage, and anyone telling you otherwise is either lying or wants to be on the Trump regime’s good (?) side. Or, both — it could always be both.
Google's NotebookLM was one of the company's first forays into generative AI technology, and in un-Googley fashion, it hasn't been shut down yet. In fact, NotebookLM is getting one of its biggest updates, ever, today, moving to the latest Gemini 3.5 model, support for more file types, and streamlined web source integration. Google also says NotebookLM will be able to do more with all those queries thanks to embedded support for Antigravity.
Gemini 3.5 Flash debuted at Google I/O this year, promising much faster and more efficient processing. Google has claimed that companies worried about token costs can save big by moving their projects to the new Flash model while also getting outputs that are of similar or better quality. Those improvements are now filtering down to other Google products. NotebookLM, which launched in 2023 at the very beginning of the AI boom, lets you analyze specific sources like documents and webpages with Google's latest AI models.
The upgraded NotebookLM beats the old version in all of Google's "core evaluation dimensions."
Credit:
Google
Google conducted side-by-side evaluations of NotebookLM on the old Gemini 3.1 branch and with the updated 3.5. The company is being somewhat vague about the nature of the tests, breaking things up into "top five core evaluation dimensions," which are Accuracy and Quality, Multilingual Support, Large Document Analysis, Document Creation, and Advanced Research. In these tests, Google says NotebookLM averaged a 65 percent win rate versus the older model.
Model Context Protocol (MCP) has gained considerable momentum as a standard connector between LLM-powered tools and local systems, internal and external APIs, and data sources. From major clouds to devops tools, MCP servers are enabling powerful, AI-powered development and operations capabilities through natural language commands.
Nowhere is this more true than in the world of databases. Most major database platforms now support agentic access through MCP servers. Using an MCP server for databases, you and your AI agent proxies can perform lookups, create and update data, and perform administrative tasks without you having to write SQL by hand.
The MCP server could also guide your LLMs to write new code or build automations that align with your database schema, like its tables, structure, and fields, as well as embeddings, indexes, and metadata. It could also aid debugging by enabling faster queries to surface data issues or misconfigurations, along with plenty of other possible use cases.
Below, we’ll cover official MCP servers from some of the top platform options across major database styles. Though maturity varies, the MCP servers discussed below represent some of the best vendor-backed offerings available today across relational SQL, NoSQL, graph, vector, and data warehouse systems.
These servers can be used by any MCP-compatible tool, IDE, or agent, whether it’s Claude Code, Codex, Cursor, Gemini CLI, Google Antigravity, VS Code, Windsurf, or something else. Adding them is typically simple, often involving a lightweight JSON addition to your MCP configuration file.
MySQL and PostgreSQL are the world’s most widely-used open source databases. However, in both cases, there is no canonical MCP server—what we see are different MCP servers emerging across vendors. One of these vendors is Amazon Web Services (AWS), which offers official MCP servers for Amazon Aurora, its managed relational database service compatible with both MySQL and PostgreSQL.
According to the documentation on GitHub, the Amazon Aurora MySQL MCP Server can be used to convert natural language commands into MySQL-compatible SQL queries, which can then be executed against Aurora MySQL databases. Similarly, the Aurora Postgres MCP Server provides MCP tools for working on PostgreSQL databases. The Aurora DSQL MCP Server does the same for distributed Postgres databases.
AWS provides a growing portfolio of official MCP servers across its product line, including MCP servers for other Amazon database platforms like DynamoDB, ElastiCache, and Redshift. If you’re a heavy AWS shop and you want to enable LLM interactions with your data, these are sensible choices.
BigQuery is Google’s cloud-based data analytics platform, and a popular data source for AI applications. BigQuery users with API access configured can also utilize the BigQuery MCP Server to interact with the platform using MCP-compatible AI clients.
Using the remote BigQuery MCP Server, engineers can generate and execute queries on data sources, or return metadata on datasets, tables, and schema. This can be done with a simple natural language prompt like “List the datasets in project PROJECT_ID.” Results are filterable by region, data set ID, column name, and more.
As part of Google’s fully-managed, remote-hosted MCP portfolio, the BigQuery MCP Server provides some peace of mind regarding security, maintenance, and ease of use for distributed teams. The MCP tools are subject to some limitations, however, in terms of query result size, processing time, and other factors. If you’re using BigQuery and you want more agentic control, you’ll want to check this one out.
Another important database category includes platforms designed for keyword and semantic search. In this space, Elasticsearch is commonly deployed. Instead of providing a single MCP server, Elasticsearch provides the Elastic Agent Builder, which is a more comprehensive framework aimed at agentic workflows.
Using Elastic Agent Builder, you can chat with an agent to retrieve data context from Elasticsearch data and extend it into various environments. The Agent Builder itself includes an MCP server endpoint for programmability and exposing the agent to other clients.
Unlike others on this list, this is not a direct MCP interface to raw Elasticsearch APIs. Instead, it’s an interface that exposes skills from the agent platform. This should also not be confused with the Elasticsearch MCP Server, released in mid-2025, which has since been deprecated.
A possible downside of using this utility is that it includes an additional layer between your IDE or agent and the data you’re searching. The agent setup requires a higher subscription and takes additional steps to configure compared to other MCP servers.
That said, if you want an extensible common layer to interact with both Elasticsearch and external MCP servers, while centralizing responsibilities like permissions, this is an interesting proposition.
Graph databases are another key NoSQL database type these days, specializing in using nodes and edges to accelerate queries of highly interconnected data. Of these, Neo4j is a popular graph database option.
The Official MCP Server for Neo4j works with all kinds of Neo4j deployment (desktop, sandbox, self-managed, and the managed Neo4j Aura cloud service), and allows LLM-based clients to retrieve graph schema, execute read and write statements, execute graph algorithms, and more.
In addition, several other MCP servers for Neo4j are available from Neo4j Labs. These have specialized uses, such as generating Cypher queries from natural language, maintaining an in-memory graph database, modeling and visualizing graph, and interacting with the Neo4j Aura API.
The first MCP server for Neo4j was developed in December 2024. If you’re an avid Neo4j user and want to experiment with interacting with your graph databases in a chat-infused way, these stand as an interesting platform of servers.
Google’s MCP Toolbox for Databases is a notable MCP server because it’s a popular catch-all for various database types. Unlike the other entries on this list, this server connects LLMs not to a single managed database, but unifies LLM access to multiple systems. The open source utility ships with pre-built configurations for nearly 30 databases including PostgreSQL, MySQL, SQL Server, Oracle Database, MongoDB, Redis, Neo4j, and Snowflake, as well as the databases in Google Cloud.
Once you define data sources in a tools.yaml file, you can use MCP Toolbox to perform structured queries or semantic searches against databases directly from within an IDE or agentic client using plain English. MCP tools translate commands into actions like list_tables and execute_sql.
MCP Toolbox for Databases is mature, originally built as a generative AI utility and later re-worked for MCP-style workflows. It offers numerous download, configuration, and interaction methods.
If you’re using a variety of databases on Google Cloud and elsewhere and you want an “all-in-one” MCP server, MCP Toolbox for Databases is a great place to start.
MongoDB is the popular NoSQL document-oriented database. The creators of MongoDB have released an official MCP server that works with the open-source database as well as the company’s cloud-hosted MongoDB Atlas database platform.
The MongoDB MCP Server provides a number of tools to interact with MongoDB. You can query the database, return information on collections, create or remove collections or indexes, gather statistics on database usage, and more. Other tools enable MongoDB Atlas operations, like creating users or clusters, returning cluster data, and other functions.
The server’s tools are read-only by default but can be switched to allow write capabilities. It can be used locally, but also supports Streamable HTTP transport for remote servers, although that comes with greater security concerns.
For those using MongoDB and wanting to hook their AI-enabled IDE or CLI up with more automated powers, the official MongoDB MCP Server is worth checking out.
Among vector-native databases, Pinecone stands as a strong, widely used option with a well-designed API and comprehensive SDKs. The Pinecone MCP server extends this experience, allowing users to query its documentation and execute functionality via AI agents and AI-enabled IDEs.
To date, the Pinecone MCP server consists of nine MCP tools. These cover read-only actions, like knowledge gathering via the Pinecone official documentation and querying vector records, index metadata, configurations, and statistics. It also allows for write operations like updating records and creating new indexes.
Released in mid-2025, the Pinecone MCP server is one of the more complete early implementations, with easy configuration and installation.
For those using Pinecone who want to test new LLM-assisted workflows for creating indexes with embeddings, performing reranking, or testing results using natural language commands, the Pinecone MCP server is worth trying out.
An ultra-fast in-memory database, Redis is commonly used for caching, real-time analytics, and other latency-sensitive use cases. And as you might have guessed, the company behind the Redis database provides an official MCP server. The server allows read, query, and write capabilities.
Developers can use Redis MCP from an LLM client to perform high-level actions to analyze, reference, or embed Redis data and interact with the Redis server within their prompts. The documentation suggests some example prompts for common use cases, such as “Cache this item,” “How many keys does my database have?,” and “What is user:1’s email?”
Unlike other MCP servers, which only allow a slice of platform capabilities, Redis MCP offers full Redis support. This enables working with Redis constructs such as hashes, lists, sets, sorted sets, streams, and more, according to the GitHub repository.
One possible drawback is that Redis MCP has yet to support Streamable HTTP transport. Until this is developed, the server is constrained to local deployment. But for those seeking a local MCP server to work with Redis data, this is the best choice.
Snowflake is a cloud-hosted, AI-enabled data platform widely used in enterprise contexts for data warehousing, data analytics, and data engineering purposes. Compared to other data storage systems, Snowflake is unique in that it’s more fully managed and combines structured and non-structured data types.
The Snowflake MCP Server, available on GitHub, can be used to perform many of the standard Snowflake platform operations. This includes a “fuzzy” search of all records via Snowflake’s Cortex Search and structured data semantic lookups using Cortex Analyst.
Other abilities include object management operations like creating, updating, and deleting records. The server also can invoke other agentic-designed capabilities, like the ability to generate and execute SQL statements against back-end databases.
Snowflake MCP Server is well-thought-out and well-documented, with walkthroughs for various agent and deployment patterns. Those already building with Snowflake should find it complements the mechanics they already employ.
A longtime open-source favorite, PostgreSQL is one of the most popular and trusted object-relational SQL-based database systems. With an active open source community, Postgres has been maturing for decades. Given its open source nature, there isn’t a single “official” MCP server for the platform. Anthropic built an original reference implementation, but it’s now archived.
Instead, database platforms built on PostgreSQL provide different flavors of MCP servers, with a range of vendor neutrality and specificity. One notable option is the Supabase MCP Server, provided by Supabase, a cloud-based “back end as a service” and Postgres development platform.
Supabase MCP Server connects AI agents with Supabase projects, allowing engineers to issue natural language commands to manage tables, query data, get logs, fetch configuration information, and more. The Supabase MCP Server is pre-1.0 release and some features are still experimental.
If you’re an engineer using Supabase and looking for an MCP server to connect your AI assistant with your Postgres databases, this is a good tool to test out.
So far, we’ve reviewed official, vendor-backed MCP servers from some of the most-adopted managed databases. However, numerous MCP servers exist across other database platforms and types.
One MCP server that aggregates LLM access across various database types is DBHub, which works with MySQL, PostgreSQL, SQL Server, MariaDB, and SQLite. Developed by Bytebase, DBHub is described as a zero-dependency, token-efficient MCP server.
For SQL, the options are nearly endless. Official servers exist for Microsoft Azure SQL and DuckDB. PulseMCP catalogs more than 100 MCP servers for MySQL, although most are unofficial, solo-creator open source projects. Of these, one of the most starred is MCP Server for MySQL, developed by full-stack developer Ben Borla and optimized for Claude Code.
For Postgres, notable alternatives to Supabase include pgEdge Postgres MCP, Neon MCP server, and Postgres MCP Pro. For vector databases, others beyond Pinecone have been quick to adopt MCP as well, including Weaviate and Milvus.
Before diving into MCP servers for enterprise databases, it’s important to understand the security risks. For instance, prompt injection remains an unsolved problem, so it’s recommended to limit permissions for SQL statements.
To mitigate this, Supabase recommends enabling AI client settings that require manual approval for each tool call before execution. Experts also recommend assigning only the minimum permissions required and avoiding exposure of sensitive data like API credentials. Due diligence around authentication and authorization is especially important when hosting remote servers.
Lastly, to avoid shadow IT, it’s becoming common practice to catalog the internal MCP servers you use, even for experimental projects. For this, experts recommend an MCP registry that documents approved servers. An MCP registry improves both MCP server discovery and security awareness.
Anyone can build an app now. But nobody seems to care.
Well, not nobody. VCs keep funding startups that add AI to, well, everything. But users aren’t buying the massive influx of new apps. In a chart shared by Jen Zhu Scott based on the new National Bureau of Economic Research’s working paper “Writing Code vs. Shipping Code,” iOS app releases have exploded since the advent of agentic AI. That would perhaps be cause for celebration had app reviews not declined during this same period, and apps with significant usage have stayed essentially flat.
In other words, more apps but almost nobody new showing up to use them.
For those of us that grew up in open source, it’s a familiar problem. The greater the abundance of code, the greater the need to help would-be customers navigate it through marketing (including branding), sales, etc. AI is creating so much noise, in terms of new code, new products, etc., that the real work has shifted to taste-making.
I’ve been saying for a while that developer productivity isn’t about producing more code faster. Or at least it shouldn’t be. Productivity is about producing well-architected, secure, maintainable code that solves a problem someone actually has.
That’s a very different thing. It’s not something AI can fix; at least, not yet.
Charity Majors put it more bluntly in a post I wrote in 2024: “Writing code is the easiest part of software engineering,” she said. The harder parts are figuring out what to build, integrating it into a larger system, validating that it works, maintaining it over time, and getting humans to trust it enough to use it.
Turns out the harder parts are really hard. Mert Demirer, Leon Musolff, and Liyuan Yang tracked more than 100,000 GitHub developers alongside their AI usage telemetry. Autocomplete, interactive agents, and autonomous agents each ramped raw coding activity, with cumulative effects on commits of 40%, 140%, and 180%. That sounds great until you look how the gains attenuate the closer you get to actual users. For example, that 180% jump in commits became roughly 50% more projects and just 30% more actual releases. The report’s authors call this the weak-link problem: The strong link (writing code) got much stronger, while the weak links (everything else humans have to do) didn’t. The estimated elasticity of substitution between AI and human effort is 0.25, which is economist for “these complement each other, but they don’t replace each other.”
When the authors checked four major app marketplaces, they found a bump in new apps but no increase in total usage. In other words, we’re getting better at creating things with AI’s help; apparently we’re not good at turning that into user interest. User attention (and budget) is finite. The hard part is to figure out how to get users to care enough to pay (with their time or their money).
So what’s actually scarce in a world of near-infinite software? Not code, for sure. No, what’s scarce is attention, trust, and a reason to switch.
That means the durable advantages in software increasingly live in the parts of the business developers often undervalue, like a recognizable brand (something that Red Hat figured out early on in open source) or a channel they already use. Or it could be in areas that developers appreciate but don’t pay for, like good documentation or a welcoming community.
This isn’t a new idea; it’s just newly unavoidable. We’re watching something similar inside enterprises, where AI adoption is wildly uneven, not because the technology has no value, but because the organizational plumbing around it often hasn’t been built. This is why a new kind of speed is important. I once argued that speed was the killer app, and that was mostly true. Today the more interesting speed is how quickly you can earn trust, fit into a workflow, answer objections, and get adopted.
Years ago I wrote that RethinkDB was dead and MongoDB wasn’t what killed it. RethinkDB was at the time, by many technical measures, a better database, built around “correctness, simplicity, and consistency.” It still lost—and badly. Its founder’s own postmortem was unsparing: They had picked a brutal market and tuned the product to the wrong definition of good. Being technically right turned out to have almost nothing to do with getting adopted.
With AI we’re generating a thousand smaller “RethinkDBs,” only faster and with nicer landing pages generated by the same model that wrote the code. These aren’t going to win, any more than RethinkDB was able to unseat MongoDB. It’s not just about the tech, but rather about making that tech fit within a user’s or enterprise’s world and making it easy to adopt. AI makes this harder, not easier.
It’s a cliché that developers don’t like marketing. It’s also false in my inexperience. What developers don’t like is traditional marketing. During my time at MongoDB and now at Oracle, my developer relations teams have focused on offering developers deep, hands-on enablement, and the response has been fantastic. Is a technical tutorial marketing? Of course it is. So is a forward-deployed engineer working side by side with an enterprise’s engineers to help them effectively use your tech. Just because it’s not a 30-second Super Bowl ad doesn’t mean it’s not marketing.
Years ago the Dilbert cartoon captured developers’ disdain for marketing. It was funny then, and it’s funny now, but it has never been true.
Years ago Matt Klein, creator of the open source Envoy project, once talked me through all the non-development work that goes into making an open source project thrive. As he reflected, “If you look at what I did in 2016 and early 2017 to [introduce] and grow the project, it was not technical.” So what was it, if not core engineering? “It was all leadership, public relations, marketing, documentation, etc., and I did it all myself and I nearly killed myself [doing it].” As he summed it up for me, it was a “f—ing lot of work,” and much (most?) of it wasn’t about code. It was about helping users appreciate the value of that code.
TL;DR? The boring but essential go-to-market grind has always been the real job. It’s what will separate winners from losers in AI.
None of this is an argument against AI coding tools. Developers should absolutely use them. I use them constantly.
But AI can’t compensate for the hard work that goes into making a product successful in the market. In a world drowning in AI-generated sameness, taste becomes a competitive advantage. For me, “taste” translates into knowing what not to build, or what not to publish. It’s also all about knowing how to market one’s product, which might include ads but definitely needs to incorporate technical training that helps developers make sense of your code.
In short, there’s no Field of Dreams “build it and they will come.” There’s just a lot of hard, human work to help real people find and use your code.
Login