A slew of recent studies indicate that AI isn’t dramatically displacing workers. At the same time, there were no indicators that the fast-moving technology is creating a lot of jobs, even as AI skills remain in high demand.

Stanford University researchers in an August study found that any AI-related job declines were concentrated among young workers aged 22–25. But for older and more experienced workers that wasn’t the case.

“Claims of economy-wide AI-driven job losses are not visible in payroll data through June 2026,” the researchers said, citing ADP payroll information. Stanford partnered with ADP to conduct the research.

The decline for young workers is more often related to the automation of some work tasks by AI, as opposed to its use to complement work. In contrast, AI is more often used by experienced workers to augment what they do. 

According to the Stanford report, workers between 22 and 25 and employed in jobs highly exposed to disruption by AI are falling behind their older counterparts. Their employment levels in June were about 19% below the same age group in less-exposed occupations. That’s compared to where they would be if both groups had kept pace with each other.

“Experienced workers show no comparable gap,” the researchers said, adding that “this is consistent with recent reports of a worsening job market for entry-level workers.”

Employment of younger workers in AI-exposed jobs fell about 11% from late 2022, while the same age group in less-exposed jobs actually grew about 10%. “For older age groups, we find much less marked differences in employment growth across AI exposure quintiles,” the researchers said.

Stanford, which created a lab last year to study AI’s impact amid heightened fears of the technology’s economic effects, said its numbers were based on available indicators.

AI’s adverse impact on young workers has been well documented, with many of them laid off in the initial wave of AI automation. AI has also suppressed wages for entry-level workers.

AI was cited for 116,175 job cuts in 2026, but that number is declining. The technology  was blamed for 3,462 cuts in August, according to data from Challenger, Gray and Christmas.

It’s the “lowest monthly total since December 2025 when 142 cuts were attributed to AI,” the company wrote in a blog entry. The research firm also noted aggressive hiring plans for companies in 2027.

AI is adding value to companies in different ways, and humans are an important part of that process, said Michael Chui, a senior fellow at QuantumBlack, AI at McKinsey. “The night shift is real now,” he said. “People are sending off their agents to write code and checking in the morning.”

MIT late last year established an AI labor index to determine how agents are affecting the workforce. The school’s Project Iceberg looks at how the coordination and interaction with agentic AI changes how work is done.

Despite the disruptions, demand for AI-related talent continues to rise, tech industry consortium CompTIA said last week. About 320,000 job listings in August required AI-related capabilities, a 4.5% increase from July.

AI-related hiring is outperforming the broader tech market, with demand for AI skills up 96% year-over-year, said Kye Mitchell, head of Experis North America, which is part of ManpowerGroup.

“We’re seeing real growth…in marketing management and project management roles, suggesting employers are looking beyond the people building AI to the people applying it,” Mitchell said.

More broadly, the US economy added 162,000 jobs in August, according to US Department of Labor figures released last week. CompTIA noted that tech employment across all sectors rose by 86,000 workers in August.

But within the tech sector, 14,700 positions were cut, CompTIA said.

Databricks on Wednesday introduced Adaptive Instructed-Retriever, a new retrieval model designed to improve enterprise AI search by taking additional steps for complex queries while stopping early on simpler ones in order to help its customers balance answer quality, latency, and cost.

The new model builds on research behind Databricks’ earlier Instructed-Retriever-1 but takes a different approach to handling complex queries.

While Instructed-Retriever-1 uses parallel, single-step search and incorporates enterprise data schemas and custom instructions to improve retrieval, Adaptive Instructed-Retriever can combine parallel retrieval with sequential, multi-step search when additional evidence gathering is needed.

That distinction matters because applying multi-step search to every query can increase latency and the computational resources required for retrieval, while limiting every query to a single search step can hurt results for complex, multi-hop questions.

Adaptive Instructed-Retriever, the company said, is designed to take additional steps only when they are likely to improve retrieval quality, allowing it to return earlier on simpler requests.

To achieve that capability, Databricks trained the model using synthetic enterprise retrieval environments and an agentic data synthesis process, reusing training data from Instructed-Retriever-1 while adding synthetic multi-hop questions designed to benefit from multiple search steps.

It then used online reinforcement learning to train the model to weigh the potential improvement in retrieval quality against the cost of additional search steps, rewarding high-performing search trajectories while penalizing steps that did not produce corresponding gains.

That training approach, the company said, can produce multiple versions, or checkpoints, of the model with different quality-latency trade-offs, allowing one trained with a lower penalty for additional search steps to spend more time improving results and another with a higher penalty to limit search steps and reduce latency.

For customer enterprises deploying the model in production, these versions or checkpoints would allow retrieval performance to be matched more closely to the requirements of different applications rather than applying the same search strategy across all workloads, it added.

Adaptive retrieval could help control AI search costs

That flexibility could become increasingly valuable as enterprises move AI agents and other applications into production, where improving answer quality often requires additional inference, compute and retrieval resources that can raise the cost of running them at scale, analysts say.

“Repeated searches can improve answers to complex questions but add latency and compute costs, while a single search can be faster and cheaper but fall short on harder queries. The ability to determine when additional search is worthwhile could therefore help enterprises avoid spending resources without improving answer quality,” said Ashish Chaturvedi, executive research leader at HFS Research.

For CIOs, that ability to limit unnecessary search could eventually offer greater predictability over compute consumption and costs.

“The problem with agentic AI at scale is that consumption is hard to forecast, agents searching and re-searching create compounding, unpredictable cost and latency, and finance teams hate these variable bills. Knowing your agents will search within a defined ceiling, and that you can set that ceiling per workload, is what makes agentic search safe to run at scale rather than a runaway meter,” Chaturvedi noted.

The economics could become even more compelling, Chaturvedi added, if the specialized model can deliver the claimed retrieval quality of larger general-purpose models with lower latency.

Adaptive Instructed-Retriever, according to Databricks, matched or exceeded the retrieval quality of Claude Sonnet 5, GPT-5.6 Luna and DeepSeek-V4-Flash in its internal evaluations while completing requests in 5.8 seconds, or more than twice as fast as those models.

That difference could matter economically because retrieval is a high-frequency operation for data agents, Chaturvedi said, potentially allowing enterprises to avoid applying the cost and latency of larger models to every retrieval request.

Developers could shed some retrieval orchestration

There are advantages for developers too, especially in terms of reducing orchestration-level work.

Traditionally, developers handling complex, multi-hop questions often have to design the control flow for multiple search rounds themselves, including deciding when to refine a query, search again, or stop, but Adaptive Instructed-Retriever’s learned search policy shifts those decisions into the model, potentially reducing the amount of bespoke retrieval orchestration developers need to build and maintain, Chaturvedi said.

That, in turn, could make sophisticated retrieval capabilities more accessible to teams that lack the resources, expertise, and time to build a search agent from scratch, echoed Stephanie Walter, practice leader of AI stack at HyperFRAME Research.

More flexibility brings new trade-offs

However, there are trade-offs that developers should consider, analysts pointed out.

While the model’s checkpoints can help prevent individual queries from consuming an unpredictable amount of compute, teams will still need to test and monitor different settings to determine which one of these delivers the best balance of cost, latency and retrieval quality for each application or use case, Walter said.

That, in turn, adds another layer of performance and cost optimizations developers need to manage, Chaturvedi echoed.

More so because being able to choose the right model checkpoint means working through real business questions and documenting where answers are coming from in your database or other knowledge bases, said Mike Leone, principal analyst at Moor Strategy and Insights.

“Very few enterprise development teams have built such a curated list. In practice, that means most teams are likely to simply select a setting once and leave it unchanged,” Leone added.

Even with the right checkpoint in place, however, Adaptive Instructed-Retriever does not address every challenge that can affect retrieval quality. Teams would still need to prepare and organize their data, enforce permissions, evaluate retrieval quality, and verify final answers, Walter said.

Echoing Walter, Leone added that a smarter search policy cannot compensate for poorly structured underlying data or a lack of agreement over what key terms mean, meaning it could still retrieve the wrong answer even if it does so more efficiently.

There are also caveats that CIOs should take note of.

Adaptive Instructed-Retriever’s performance and quality claims, Chaturvedi said, are based on Databricks’ own benchmarks against competitors selected by itself, meaning that these claims will need to be tested against an enterprise’s actual workloads rather than accepted at face value.

Even if those results hold up, however, CIOs will still need to decide whether the potential gains in retrieval performance, latency, and cost justify adding a specialized model to their AI stack.

Rather, using a larger general-purpose model for both retrieval and reasoning could offer a simpler alternative by reducing the number of models enterprises need to deploy and manage, Chaturvedi said. The economic case for Adaptive Instructed-Retriever for CIOs, therefore, will depend on whether its claimed performance holds up on real enterprise workloads and whether the potential savings outweigh the additional complexity of using a specialized model, Chaturvedi added.

AI note-taking applications are increasingly used by workers to record meetings, generate conversation summaries and suggest post-meeting action items. 

Along with the promised productivity benefits — enabling users to focus on meetings rather than actively taking notes — the emergence of these AI tools has raised privacy questions, particularly around obtaining prior consent by meeting participants for their use. Those concerns, in turn, have prompted a spate of lawsuits against software vendors that sell AI notetaking tools.

In some ways, the underlying questions are not new: It’s long been possible to record a phone call with a dictaphone, and laws around surreptitious recording have been around for decades. Yet widespread access to AI notetakers via desktop or smartphone apps means that it’s easier than ever to record a conversation for future reference. 

Among the thorny questions arising from the technology’s use: what happens to conversation data sent to a software vendor’s servers for processing? Are the recording and transcript used to train those vendors’ AI models, for example, or create biometric voiceprints? Those practices are among the issues being considered in US courts. 

Otter, which claims to have 35 million users, was subject to a class-action complaint in a federal court in California last year. That complaint accused Otter of recording individuals without consent and using their voices to train its speech recognition AI tools. Last month, a judge rejected Otter’s attempt to dismiss the main claims, though the scope of the case was narrowed.

A lawsuit filed against another vendor, Fireflies, in an Illinois court late last year, alleges the company collects and stores biometric voiceprints without user consent, in violation of the Illinois Biometric Information Privacy Act (BIPA). Fireflies claims to have more than 20 million individuals and 1 million organizations as customers.

Earlier this year, a class-action complaint in a Washington court claimed that a live transcription feature in Microsoft’s Teams collaboration application also violates BIPA by collecting biometric data without consent. 

And most recently, a complaint alleged that Granola, a well-funded startup, designed its product to be used without the knowledge of all meeting participants, in violation of the Electronic Communications Privacy Act (ECPA). That complaint also claims Granola trains its AI models on conversation data without consent. 

None of these cases has yet been resolved, and it’s unclear whether any of the vendors broke the law in the design and delivery of their products and services. But the lawsuits highlight considerations for businesses that allow the use of AI notetakers, both in terms of prior consent for recordings and understanding how conversation data is handled.

More broadly, the lawsuits raise questions about how existing privacy and consent rules apply to new technologies that make it easier to record others, whether through AI note-taking apps, smartglasses, or other recording devices.

Computerworld spoke with Brian McGinnis, partner at law firm Barnes & Thornburg and a founding member and co-chair of the firm’s Data Security and Privacy Law practice group, about the focus of the lawsuits, potential outcomes, and how businesses can deploy AI notetaking apps safely.

Several cases have already been brought against popular AI note-taking apps. What are some of the main commonalities between these? Which laws are the vendors accused of breaching? “The common allegation is that these companies capture communications of people who did not agree to the recording or receive adequate notice. Some of the lawsuits also allege that meeting data is used to train AI models and that consent cannot meaningfully be withdrawn once the data has been processed.

“There are various federal and state claims. You’ve got the Electronic Communications Privacy Act, a federal wiretapping statute. As a general matter, the Electronic Communications Privacy Act permits an interception when one party consents, subject to statutory exceptions and questions about whether the technology constitutes an unlawful interception or third-party eavesdropping. In other words, if you, as the user, provide consent, you can be on a meeting with 20 other people and it’s deemed to be sufficient; you don’t necessarily need to get the consent of other people. 

“But California and a minority of other states are what we call ‘two-party consent’ states, meaning each individual on the call has to give consent; it’s not sufficient for you as the person who turns the notetaker on to provide the consent — you also have to get the consent of others that are being recorded. There are state laws around that.

“There’s a law called CIPA, the California Invasion of Privacy Act, that’s being utilized in this context.  We see a lot of suits being brought under that law — it’s a wiretapping statute designed for telephone wiretapping that’s now being applied to the internet.”

What about the use of biometric data? “A growing number of states regulate biometric data, with Illinois’ BIPA being particularly prominent because it provides a private right of action. This means an individual can sue a company for violation of the law.  

“That law covers biometric information. With an audio recording or recording of ‘dumb’ video that isn’t running any algorithms, you’re not necessarily collecting any biometrics. But when you start identifying people, you’re recording things like faceprints or voiceprints, which are in the definition of biometric information within the statute. Now you’re not only collecting personal information, but also biometric information, which is considered very sensitive and much more highly regulated. 

“Then you’ve got the private right of action that goes against it. Part of the argument here is that recordings taken by the AI notetakers can be used to produce some form of biometric information, such as a voiceprint.  

“We’ve seen a lot of cases, and a lot of changes in industry as a result of this law. Shutterfly had a famous case about scanning for people’s faces and things like that in Illinois that changed the photo storage and processing industry a little bit. 

“A lot of companies stay out of Illinois to avoid this law specifically. But a customer organization might not know exactly who’s in a meeting and where a person is located at the time of the meeting. So, you need to either follow that law and get individual consent, or stay out of states with biometric laws if you want to use some of these tools. 

“It’s just a further challenge for these applications if the goal is to be used as much as possible with as little detection as possible.”

What are some of the potential outcomes for these cases? “I think it’d be unlikely to get an outright ban, absent passing some kind of new law that says these tools are per se illegal for use. It’s much more likely the outcome will require some changes and controls over the way that they get used. The clearest case would be some kind of a pop-up notice: ‘Hey, this meeting’s being recorded, here’s who it is, here’s their privacy policy, here’s their terms of service – do you consent to it?’ And getting opt-in consent from anybody who wants to be recorded. 

“The notion that only one person in the meeting has to say it’s okay and you can just automatically record everybody else, I think that’s probably at risk, and could potentially be replaced with a standard that requires everybody to consent before it’s considered legal.

“But the newer —and more interesting — wave of these is the Granola case, where part of its marketing is that people aren’t aware that it’s there. The Granola complaint alleges that the product was designed to operate without alerting other participants. It’s possible that kind of activity could result in a decision that would ultimately ban it outright. In other words, that it’s illegal to utilize these tools if you aren’t providing notice to everybody and/or aren’t getting consent from everyone on the call. That’s a possible outcome that we could see.

“To me, that’s interesting when you think beyond AI note-taking and into the broader world, with conversations around, like, Meta Glasses, or any of these kinds of AI wearables. Granola in particular has an Apple Watch app, and there are other wearable or physical devices — there’s one [Plaud Note] that sticks on the back of your phone and is essentially an always-on recording device. 

“With these devices, you’re going from an online meeting where you can provide notice and there’s a structure to obtain consent, to walking down the sidewalk and recording people and casual conversations. Maybe it’s somebody you’re having a conversation with, maybe it’s somebody at the table next to you in the coffee shop that you have no relationship with whatsoever — what are the laws around consent and notice in those cases, where there’s no digital interface to put that up in front of people? Do you have to go around with a pad of paper and a pen and get people to sign consent to use these things? Do you have to physically tell them?

“Those are the more interesting conversations that this line of cases is just at the beginning of helping us get some answers on. In other words; how do you get consent? How do you provide notice in a world where we don’t have documents or screens in front of us to easily handle that? Those are interesting questions that the law will have to figure out here.”

These tools are increasingly used in the workplace. How can businesses be sure they deploy the technologies safely? “We’re getting a lot of questions from our clients about this topic, because they’re really unsure and uncertain of how to handle these tools.

“Obviously, there’s a push for broad use of AI within their companies, for productivity increases within their company. People like the tools; they want to be able to use them. But then we also hear a lot of stories about ‘I jumped on a meeting; I didn’t even know it was being recorded, then I got an email afterward with a transcript of it,’ and ‘half of what it recorded wasn’t actually what I said, or it was interpreted incorrectly’ — things like that. So there’s a lot of consternation amongst our clients about the people within their organizations using it. 

“I can’t tell clients definitively that they’re legal as they are; there are ways to use this legally and safely that aren’t going to get your organization sued, but you probably have to do some things that are beyond what’s provided ‘out of the box’ by the software providers. 

“With Granola, for example, my understanding is that certain notice features may not be enabled by default. You can turn on video or audio watermarking, and you can turn on the notice that pops up in these things, but I think it ships without those features enabled. That puts the responsibility on the individual user to determine and then implement their own legal privacy and legal compliance mechanisms using their settings.

“You really have to play to the most stringent state’s law. I would advise a company that, to decrease your chances of getting in trouble for use of these tools, you need to obtain consent of all parties on the call. You can do that verbally, as well; written is even better. 

“Then it’s about having an internal AI note-taking policy. Think of a BYOD policy, which all these companies have, or an AI usage policy — this could be part of that policy, or a standalone policy: ‘Here’s how our organization thinks about these tools:  you can only use these approved tools and, if you’re going to use them, you have to turn on these features. You have to get consent from everyone. Here are limits on what you can do with the output of those transcripts or recordings.’ 

“If you set all that up and do the compliance and governance work, I think you can use these tools and most likely stay on the right side of the law. Certainly, the law doesn’t prevent consenting adults from consenting to the use of these kinds of tools.  

“But the further you get away from that written consent standard, the more problematic it becomes. If you just want to go to a notice standard and not obtain actual opt-in consent, or, even worse, if you want to try and do this without anybody knowing, that potentially could get challenged.”

By now, you probably know my stance on Gemini and other generative AI gobbledegook — right?

In short: It’s an insanely powerful form of technology (obviously). But it’s also wildly inaccurate and ineffective as an all-purpose answer engine, and by cramming it into every last nook and cranny, Google is doing a serious disservice to its users — and to itself.

That being said, generative AI can be incredibly useful. The key is simply to frame it as a purpose-specific tool for the right type of limited task as opposed to treating it as the end-all answer for everything, as most tech players seem overly eager to do these days.

Here in the land of Android, I’m always looking for purpose-specific ways Gemini can make our lives easier. Over time, I’ve found those ideas often aren’t the big headline-making features Google and other tech companies market but rather random little off-the-beaten-path possibilities that rarely get emphasized.

Today, I’ve got a new Gemini Android advantage for you to explore — another one that I’ve never seen advertised anywhere but that works brilliantly well and solves a long-standing limitation of our modern mobile gizmos.

It’ll take you all of a minute to master and cost you precisely $0 to pull off. Ready?

[Get next-level knowledge in your inbox with my free Android Intelligence newsletter — one useful new thing to try every Friday, straight from me to ye.]

Gemini’s custom sound advantage

This Gemini-provided power-up very much follows the path of thinking about AI as a limited, purpose-specific tool for a narrow task — the exact sort of area where AI excels.

I won’t keep you waiting: Gemini, as I discovered whilst poking around within it recently, is really good at generating unique, custom ringtones and notification sounds based on your personal preferences and specifications.

It almost seems obvious, once you think about it. But I sure hadn’t thought about it up until now — and I suspect most other folks haven’t, either.

The advantage here goes beyond just superficial silliness, too: By creating your own custom ringtones and notification noises, you can know exactly what event your phone is alerting you to within a split second of hearing the associated sound — without having to rely on the same limited and often underwhelming pool of default options that everyone else uses (or, worse yet, having to lean on shady, ad-ridden “ringtone apps” to find mediocre alternatives). That means you’ll know within a heartbeat when your boss or an especially important client is calling or if an incoming call is a random unknown number.

On the notification front, you can create a specific, memorable sound to accompany a new message in an important work-related Slack channel and a different distinctive sound for a high-priority email. They’ll all be sounds that you make and identity with and that no one else in the world uses. That’s pretty powerful.

And creating these custom sounds couldn’t be much easier, either, once you realize it’s possible:

First, just fire up Gemini on your phone (or on any device you’re using) and tell it what you want. So, for instance…

• Create a soft, subtle ringtone that’s reminiscent of an old office phone sound.
• Create a ringtone that sounds like an 8-bit version of The Final Countdown.
• Create a one-second high-pitched notification sound that brings to mind the Super Mario Bros theme song.
• Create a short notification sound that mimics the Back to the Future time circuits noise.
• Create a notification sound that sounds like a robot saying “ALERT, ALERT!”

The only limit is your imagination. And if you’re not feeling especially imaginative, you can even ask Gemini to give you a list of ideas for distinctive, memorable ringtones or notification sounds based on geeky nostalgia, 80s pop-metal, or whatever it is that tickles your fancy.

Once you’ve sent a request, Gemini will — somewhat confusingly — spit back a big honkin’ block of HTML code. This is because, for reasons unknown, the system can’t or won’t just provide a sound file directly. Go figure.

But from there, it’s just one more quick step to get the file you need: Click or tap the copy icon in the upper-right corner of that code block — the icon that looks kinda like two stacked, rounded rectangles — to copy the entire chunk of code onto your system clipboard.

Then open up your browser and go to the website JSFiddle.net. It’s a well-known, long-standing tool for running code like HTML right in your browser and seeing the result, and it works entirely in the browser and without any downloads or special permissions required.

Tap or click into the “HTML” field, then paste in the code from your clipboard (by long-pressing any open space, on Android, and selecting “Paste” from the menu that appears). Click or tap the “Run” button, and — ta-da: You’ll see a player to listen to your custom Gemini-generated sound and a button to download it.

Provided you like what you hear, hit the button to download the file, then head into the Sound section of your Android system settings to save it and make it available for applying as any kind of alert or ringtone. The exact steps for doing that will vary from one type of Android device to another, but on a Pixel or another phone that follows Google’s core Android interface, you’ll tap on either “Ringtone” or “Notification,” then tap “My Sounds” and tap the plus icon to import your own custom file from your phone’s local storage.

On a Samsung device, you’ll start the same way but look for the plus icon within the “Ringtone” section of the settings, then tap the “Folders” tab to find your locally downloaded files. Samsung doesn’t make it easy to add in your own custom notification sounds, annoyingly, so what you’ll have to do is open up the Google Files app, find the file you downloaded in your “Downloads” folder, then tap the three-dot icon alongside it to copy it to the folder called “Notifications.” Once the file is in that folder, it should show up as a notification noise option within the Samsung Android settings.

And remember: You can use Android’s notification channels to set specific sounds to be used for different types of alerts, even within a single app — and you can use the Google Contacts app to set custom ringtones for different people, too.

With Gemini as your personal composer, you’ll never be stuck with generic overused sounds again — and you’ll always know exactly what’s happening from the first note of whatever noise you hear.

Keep the experience-enhancing wisdom coming with my free Android Intelligence newsletter. One new useful trick in your inbox every Friday!

We spent the last decade building entire finops departments just to decipher what the cloud bill was saying. Just when we figured out how to stop leaving idle compute instances running, generative AI introduced an infinitely more opaque, faster-moving layer of spend.

AI bill shock has spread like a slow moving hurricane across the industry. Aside from the spike in large language model (LLM) costs, the deeper architectural problem is attribution: where exactly is the money going, and exactly what value is it delivering to the business?

When API calls are wrapped in layers of automated agents and prompt templates, your application becomes a black box that consumes capital to spawn tokens. If a features engine is burning thousands of dollars a month just to have an LLM output pleasantries or process low-value data, that isn’t innovation, it’s an uncovered manhole, a gaping liability. Architectural maturity means treating tokens like any other constrained resource.

Fortunately, there are a few good levers we can pull to control AI spend. Here are the five key tools for taming the beast.

Model routing

Don’t use a nail gun if a thumbtack will suffice.

The most powerful models like the latest Claude Opus are some of the most sophisticated software systems ever built. They are capable of handling extremely complex and subtle use cases. They are also voracious beasts when it comes to compute (and therefore dollar) consumption. They are often overkill.

We naturally start with the most powerful model we have when prototyping and sketching out APIs and requirements. That’s because we are in the “get it working” mode and we don’t want to be tangling with model limitations. But for production, it is absolutely essential to dial in on what is the least powerful model we can use.

Sometimes we can address this by tuning the model choice and discovering what will work manually. But in larger enterprise architectures, we can move to a conditional routing layer. Frameworks like RouteLLM or Semantic Router can dynamically direct simple classification, basic text parsing, and user intent detection to fast and cheap utility models like GPT-4o mini or Claude 3 Haiku.

You can also use advanced AI gateways for routing and spend allocation. At the infrastructure level, wrapping this logic in a dedicated AI API gateway (such as Kong, Cloudflare AI Gateway, or Portkey) gives you the ultimate control. A mature gateway allows you to implement “cascade routing” to automatically fall back to a cheaper or open-source model if your primary LLM hits rate limits or latency spikes. More importantly, an AI API gateway centralizes your telemetry, stripping away the black box so you can enforce hard token budgets per tenant or per microservice. You are essentially building a load balancer for cognitive tasks.

Of course, AI gateways also introduce complexity and cost themselves, so these factors must be balanced into the equation. But the bottom line is, there is a right-sized model for your project. You can save a bundle by routing requests to a good-enough, Goldilocks model.

A side note: Another trend in model routing is “repatriation of compute.” That entails running models (often open-source models) on local hardware, instead of renting generative AI endpoints on the cloud.  This implies all the trade-offs and benefits of other on-prem options.

Semantic caching

In conventional web apps, caching is a solved problem. You put Redis in front of your database, map an exact query to a cached payload, and you’re done. But LLMs break traditional caching because human language is infinitely variable. “How do I reset my password?” and “I forgot my login info” are entirely different strings, but communicate identical intents.

If you rely on overly precise matching, your hit rate will be effectively zero, and you will pay the full token price for every phrasing permutation.

Enter semantic caching. It’s like regex for meaning. Instead of hashing the raw text string, you run the incoming prompt through a fast embedding model and do a similarity search against previously answered prompts. You define a confidence threshold (say, 0.92), and if it is met, you serve the cached response (bypassing the LLM and its cost entirely).

The architectural effect is twofold. First, a semantic cache hit reduces your inference cost for that query to exactly zero. Second, it drops response latency from seconds to milliseconds. You can implement this using purpose-built frameworks like GPTCache, or, if you prefer a leaner infrastructure, by using pgvector inside your existing PostgreSQL database to store and match the embeddings.

However, just like dynamic routing, semantic caching demands a rigorous architectural calculus. You are trading generation costs for embeddings and lookup costs. To check the cache, you still have to tokenize the prompt, call a cheap model (like text-embedding-3-small), and execute a vector search.

You also introduce the very real danger of semantic flattening. Tuning the similarity threshold is a delicate art. Set it too low, and your application starts serving general, recycled answers to nuanced user questions. For open-ended, creative, generative AI applications, semantic caching is practically useless. But for retrieval-augmented generation (RAG) implementations, customer support bots, and internal knowledge bases where users ask the same 20 questions a thousand different ways, it is the most effective cost-reduction lever you can pull.

Prompt caching

Whereas semantic caching stores the response to a given intent, prompt caching stores the data needed to contextualize the question. When the user enters a prompt, it is sent to the generative AI endpoint. But instead of your application having to repeatedly gather and send the massive contextual info needed to frame the prompt—from a RAG pipeline, a database, or other source—that information is already pre-loaded in the context cache. The model simply applies the new question to the cached data, slashing both your latency and your input costs.

The core idea behind prompt caching is that the AI engine itself retains the relevant contextual data for incoming prompts. Tokens cached this way get a massive discount (50% to 90%) compared to raw input tokens. But the details of how it is handled vary between providers.

For OpenAI, prompt caching is automatic, but requires attention to how your app uses the endpoints in order to leverage them. OpenAI enforces a 1,024-token minimum threshold. More importantly, it requires an exact, byte-for-byte prefix match starting from token zero. If you inject a dynamic variable (like a timestamp or a user ID) at the very top of your system instructions, you instantly invalidate the cache and pay full price for every token that follows.

Some other providers, like Anthropic Claude and Google Gemini, use explicit caching. You must deliberately engineer the cache into your architecture, either by hitting a dedicated API endpoint to upload the static document (the context data) with a defined time to live, or by injecting strict cache control breakpoints into your JSON payload. Explicit caching often requires paying a “write premium” up front, but it gives you the control to guarantee your heavy RAG data stays alive in memory for hours rather than minutes.

Prompt discipline

Just because a model can consume gargantuan amounts of tokens, doesn’t mean it should.

With models now capable of ingesting one million to two million tokens in a single prompt, there is a temptation to brute-force use cases into submission by dumping the entire PDF, the entire codebase, or the last three years of logs into the payload and let the LLM sort it out.

This is a finops blunder. You are paying for every single token you send. Furthermore, from an engineering standpoint, stuffing the context window runs into the very real “muddy middle” problem (aka “context rot”), where the model loses resolution into the middle of the content. As the payload grows, performance and accuracy suffer.

The cure is a strict RAG diet. Naive RAG implementations might pull the top 20 results from a vector database and blindly concatenate them into the prompt. Architectural maturity requires a more ruthless filtering process before those tokens ever reach the expensive LLM.

This is where reranking becomes key. Reranking takes the compiled prompt context and makes sure it is efficiently structured before it hits the LLM. Instead of sending raw search results directly to your primary model, you introduce a tiny, highly-efficient cross-encoder (like Cohere Rerank or an open-source BGE model). You cast a wide net in your cheap vector database to fetch 50 potential matches, then pass them to the reranker to score their actual relevance against the user’s specific prompt. The reranker ruthlessly trims the fat, allowing you to pass only the top three or four hyper-relevant chunks to the expensive frontier model.

You are trading a massive, expensive LLM context payload for a tiny, cheap reranking step. The architectural math here is almost always a resounding win. While a cross-encoder might add 100 milliseconds of latency to the pipeline, it routinely slashes prompt token counts by 80% or more, while simultaneously increasing the accuracy of the final generation. In the generative AI spend playbook, giving your LLM less to read is one of the highest-leverage moves you can make.

Response constraints

By default, LLMs are polite, conversational, and verbose. If you ask an API for data, it desperately wants to start with “Certainly! Here is the JSON you requested,” and end with “Please let me know if you need anything else!”

In a web interface, this is charming. In an application’s API architecture, it is a leak in the finops hull. Generation tokens (output) are almost universally priced 3x to 5x higher than prompt tokens (input). Every pleasantry is burning your most expensive commodity.

Taming LLM responses requires strict API-level enforcement. First, max_tokens must be treated as a hard circuit breaker to prevent runaway generation loops, not as a formatting tool. (Relying on max_tokens to shorten answers often just results in truncated, unparseable JSON.) Second, use stop sequences (stop_words). If your model only needs to output a SQL query, setting a stop sequence for the markdown closing tag (“`) or the word “Explanation:” forces the model to sever the connection the moment the computational work is done.

Modern APIs also offer structured outputs (or strict JSON mode), which forces the model to adhere to a rigid schema you define. By combining this with zero-tolerance system prompts (“Output ONLY valid JSON. Omit all conversational text and pleasantries. Failure to do so will break the system”) you force the LLM to behave like a traditional API endpoint rather than a chatbot.

The architectural balance here is to avoid degrading the model’s reasoning ability by strictly limiting its input. LLMs do not have internal working memory, they “think” by iterating on token output. If you force a model to solve a complex logic problem and output a single boolean true or false without giving it space to reason first, its accuracy will plummet. The finops goal isn’t really to force minimal words; it is to ensure that every expensive output token is performing actual computational work (like chain of thought reasoning).

For more on how to manage AI back ends, take a look at this article.

How to effectively consume generative AI to satisfy user needs is the first battle. The second is keeping the finances from undermining that hard work. The five keys discussed here should give you all the power you need to make sure the engineering and the cost controls are working together.

I guess I’m a Go developer now.

My first programming language was BASIC. My seventh-grade math teacher, Mr. Schulte, taught BASIC to us on Fridays. He gave us programming assignments for homework during the week. Some of us loved it so much we stayed after school to play games and create extra programs, including our own crude, text-based adventure games. I remember hearing a story about a crazy new language called Pascal that didn’t have line numbers. How in the world would that work?

Years later I became a Pascal programmer — an ardent one. So ardent, in fact, that I would argue for hours with Visual Basic developers on CompuServe, AOL, and Usenet, telling them how Turbo Pascal, and then Delphi, was clearly a superior language. I was completely self-taught — if you count all the help that others gave me as teaching myself. I wrote shareware, became pretty competent, and eventually started helping others. 

People would tell me I should branch out and learn other languages — C++ or C#. But I wanted to be a master of one instead of a jack of all trades. Besides, we were all Windows developers and there was no better tool for writing Windows applications than Delphi. It became a point of pride for me, and I would stick up for Delphi and its company Borland no matter what. It got so bad that my friends would tease me, calling me “Baghdad Nick” after the famously optimistic Iraqi foreign minister during the Iraq War. 

Delphi by day

Looking back, I was foolish. I did become a pretty good Object Pascal programmer, but I would have benefited from learning JavaScript or a functional language. I did eventually learn TypeScript and a bit of C#. (I guess I was following Anders Hejlsberg around…) 

But if you were to ask me to this day what kind of code I write, I’d say I’m a Delphi developer. Indeed, that is my day job. I understand Delphi deeply, knowing all the nuances and insider tricks to making the language my own.

But over the course of a day or two, I’m now a Go developer, too. 

If coding all day isn’t enough, I have taken on a side project for a friend. I’ve been writing Delphi code for him for a while, but now he wants to shift gears. He has asked me to build a cross-platform application for his company written in Go and Wails Version 3. Now, I’m no Go expert. In fact, I only have a passing familiarity with the language. I can decipher what is going on, but I’m far from understanding the intricacies of what makes Go tick.

But that doesn’t really matter anymore, does it?

Claude bootcamp

I asked Claude to give me a tutorial on Go and Wails. It did a great job, quickly bringing me up to speed on the basics. (How cool is it that you can so easily turn a struct into a JSON object?) I described what I wanted, and asked for recommendations for libraries. The project uses Go for the “back end” and React on the “front end.” (I use the quotes because those “ends” are actually all built into a single binary.) 

In about an hour I came to understand the architecture and got a basic application up and running. I told Claude what I wanted, and it obliged. I described the basic data structures and objects needed, and Claude wrote them quite elegantly. I didn’t actually write a line of code, but there the project was, in all its glory. Go, TypeScript, and HTML. 

Claude managed the Git repository and even created a nice little CI/CD pipeline for me, building binaries for Mac, Linux, and Windows, all in one step.   

I went from being basically ignorant of the Go language to having a moderately sophisticated application in a day.

The thing is, I’m still not that good at Go. Not that good at all. But I’m a Go programmer nonetheless. Maybe next weekend I’ll become a Python programmer.

OpenAI’s rollout of its GPT-6 Astra model ran into early access issues after paying ChatGPT users were unable to use the system shortly after launch, prompting CEO Sam Altman to apologize and say the release had been “messy.”

“First, sorry for the messy rollout,” OpenAI CEO Sam Altman acknowledged the issue in a post on X. “Second, when we screw up, we try to make it right.”

OpenAI had said GPT-6 Astra would be rolled out across ChatGPT tiers and APIs, positioning it as its most advanced model to date. However, the initial rollout did not translate into immediate access for all users, highlighting the gap between model launch and availability across subscription tiers.

Only the organizations enrolled in its Daybreak cybersecurity program were able to access the model, whereas Plus, Pro, Business, and Enterprise ChatGPT subscribers, along with developers using the OpenAI API, were left out.

“Third, we should be able to begin broad rollout to API customers and ChatGPT subscribers in the near future. As usual, we will start with pro subscribers,” Altman continued in the post.

Altman wrote in a follow-up X post on Friday that OpenAI had extended Astra to Pro, Enterprise, and Business Premium users in ChatGPT’s Work and Codex products and had opened it up through the API.

“It might take a few days to roll out to our Plus and Business users,” OpenAI’s official X account posted on September 5.

The company did not immediately respond to a request for comment.

Phased rollout continues without firm timelines

OpenAI introduced GPT-6 Astra on September 4, stating that availability would expand over time rather than being enabled simultaneously for all users.

Altman’s post followed complaints about access during the initial rollout window, although OpenAI has not disclosed how many users were affected or how access varied across tiers.

In a subsequent post on X, Altman said: “We are working towards getting Astra in everyone’s hands as quickly as we can; I know it is frustrating,” indicating that access was being expanded incrementally.

OpenAI technical staff member Thibault Sottiaux confirmed in a separate X post that Plus and Business users had gained access too, crediting the company’s infrastructure: “more scalable than we anticipated.”

The rollout approach is consistent with OpenAI’s initial communication that Astra would be made available over several days, rather than at once. Gartner also noted that the model was first released to a limited set of organizations before broader expansion.

Rollout highlights the gap between launch and access

Analysts said the sequence reflects a distinction between model announcement and actual availability.

Greyhound Research said the Astra rollout should be treated as an operational signal rather than a confirmation of readiness.

“Announced, available, entitled, and production-ready are four separate states,” said Sanchit Vir Gogia, chief analyst at Greyhound Research. “This must be treated as operational evidence, neither dismissed as theatre nor inflated into proof that Astra has failed.”

He added that staged availability reinforces the need for enterprises to verify what level of access they actually receive, rather than assume uniform rollout across users or environments.

Gartner said enterprises will need to strengthen governance as they evaluate Astra’s capabilities.

“CIOs must balance Astra’s advanced automation with stronger cybersecurity, governance, and cost controls before adoption,” Gartner analysts said in an initial note on the launch shared with Computerworld

The firm said Astra’s ability to execute more autonomous workflows will require tighter evaluation controls and observability.

It also pointed to challenges around identity, security posture, and accountability as AI agents take on more complex roles.

Contracts and control models under scrutiny

Greyhound Research said the rollout raises questions about how enterprises define access and operational control.

“A conventional uptime SLA is too narrow for Astra,” said Gogia. “Critical describes the engine. It does not tell the buyer how much of that engine reaches the road.”

He said enterprises need to account for how such systems behave in production, particularly when access, interruption, or task continuity may vary.

“A stop leaves a state the enterprise did not choose, and that state needs a record it can defend,” Gogia said.

The rollout also highlights changes in how governance responsibilities are distributed.

Gogia said administrative controls alone do not address enterprise requirements.

“Admin opt-in is not a safety certificate,” he said. “It is the point at which accountability crosses from vendor release policy into an enterprise governance decision.”

He added that such controls do not extend automatically to API-based deployments, where enforcement depends on enterprise-level systems.

Capability gains introduce trade-offs

OpenAI has positioned GPT-6 Astra as an advance in reasoning, coding, and automation capabilities.

Gartner said these improvements introduce trade-offs that enterprises will need to evaluate in production settings.

While Astra may reduce token usage for some tasks, organizations must consider overall task costs, including validation and oversight, the firm noted.

Gartner also cautioned against over-indexing on early capability claims. “Without more evidence, CIOs should ignore the AGI hype for now and instead focus on use-case-specific evaluations, demonstrated business outcomes and reliable autonomy,” the firm said.

The story originally appeared on CSO.

A year and a half ago I sat in a conference in Silicon Valley and watched an AI agent write production-quality code in real time. I knew our engineering model was going to change, and within a few weeks my team was past the question of whether to adopt AI. The real debate was how far we were willing to go in redesigning how we build legal software around it.

Since then, output per engineer across our R&D organization has roughly tripled. We ship nearly twice as many releases per quarter, deployments have grown from 82 to more than 155, and customer reported defects are down 65% per million lines of code across the last 18 months. We track all of this with DORA metrics, cycle time, pull requests merged per developer, and lines changed per developer against a fixed baseline, and the improvement holds up on every measure.

Getting there had less to do with the tools than with how the organization works. Every CTO I talk to is running some kind of AI coding pilot and seeing decent results: faster prototyping, better test coverage, quicker first drafts. Almost none of them have changed the development life cycle itself, and that is where the real gains were for us.

We spent the past 18 months rebuilding our product development life cycle around AI. Requirements, development, testing, security, deployment, and governance all changed, some beyond recognition. We got plenty wrong along the way, and a few of the lessons would have saved us months if someone had handed them to me at the start.

The biggest gains came from removing handoffs

We assumed most of the productivity gain would come from AI writing code faster. It didn’t. The biggest gains came from getting rid of the handoffs between stages.

In the old model, a feature passed through at least four handoffs on its way out the door: product to development, development to QA, QA to security, and security to deployment operations. Each of those queues cost us time and context. A feature could be code complete in a day and then sit for two weeks waiting on teams that each had their own backlog and their own priorities.

So, we restructured. A team now carries a feature end to end, and AI agents handle the rote work at each stage. A feature that used to take 15 days to go from spec to a working pull request now takes about four hours.

If you take one thing from this article, measure the total time from idea to deployment rather than lines of code generated, and go find where the work sits idle. That is where the gains are hiding.

Governance accelerated adoption

Most engineering organizations add governance after the innovation has proven itself. We flipped that order. We build software for lawyers, where a wrong answer on high-stakes work can cost a client dearly, so moving fast and cleaning up later was never on the table.

Before we scaled past the first pilot teams, we set standards for code review, security scanning, and quality on anything AI produced. Confidence scoring lets routine approvals pass through automatically while anything below threshold goes to a human. Security and quality gates run inside the pipeline itself rather than as a checkpoint at the end.

What we didn’t expect was how much this sped up adoption. Engineers trusted the system, so nobody had to wonder whether code from an agent would pass review, introduce a vulnerability, or get them blamed when the model got something wrong. Eighteen months ago, about 3% of our pull requests had any AI assistance. Today it’s 68%, every engineer in the organization uses AI, and vulnerability density has fallen 76% over the same period. Governance came first and adoption followed, which is the opposite of the sequence most people assume.

Pick your tools and get good at them

A better model or agent ships almost every week, and early on we burned real energy evaluating them. At some point we stopped and standardized on a small set of coding agents.

Going deep paid off more than staying current. We learned the strengths and failure modes of our coding agents and tuned our workflows around how they actually behave, which matters more than whatever the next release promises. Teams that chase every new model spend all their time evaluating, and every switch resets the muscle memory and the trust they built with the last tool. They end up with a lot of pilots and not much change.

Automate the entire life cycle

The obvious place to point AI coding tools is at the coding. Some of our biggest wins came from the edges of the life cycle.

We built or deployed dedicated agents for every stage: requirements, sprint planning, code generation, test authorship, security scanning, deployment, and site reliability. Each one owns a single job and gets measured on a single outcome. What makes it work is a centralized knowledge repository that product and engineering maintain together, a curated and structured knowledge base with retrieval-augmented generation on top. Every agent pulls product context, domain rules, and prior decisions from that same source of truth, so what comes out of one stage is clean input for the next. Without it we would have a set of fast agents that lose the thread at every handoff.

Requirements definition was the most dramatic early win. It used to take weeks. We built an agent that grills a raw product idea the way a tough stakeholder would, pushing on gaps until the spec is ready for development, and that phase now finishes in an afternoon.

Testing moved even faster as quality engineers shifted their focus to optimizing and improving the agents that build tests. AI now generates 99% of our new tests, up from 47% a year ago, with more than 39,000 AI-developed tests in the suite, a number we could never have staffed our way to. Each test is generated directly against the code change it covers and checked against the functional requirements defined in the product requirements document, so growth in volume doesn’t come at the cost of traceability. Engineers spend their time reviewing, refining, and hardening what the system produces rather than writing first drafts, and that shift, from authoring tests to curating and strengthening them, is a big part of why quality improved while we sped up.

We started with one area of the product development life cycle, got it right, then expanded. I would recommend that sequence to anyone. The key is to develop a model that can be expanded to the rest of the phases after experimenting on what works rather than trying to roll it out to every area and having to rebuild the framework later.  

Design for the velocity you’re about to create

We set a public and very specific goal: double R&D productivity in 12 months. It was uncomfortable, and there were stretches where I wasn’t sure we would make it. But a number that aggressive forced everyone to accept that incremental improvement wasn’t going to get us there, and we had to take apart old ways of working while the technology underneath was still maturing. We hit the goal inside the year, and output has kept climbing ever since.

What I underestimated was how fast the bottleneck would move downstream. Once engineering sped up, the constraint moved to go to market. Every release still needs documentation, enablement content, briefings for customer success, and customers who are ready for what’s coming, and for a while we were shipping faster than any of that could keep up. We now treat go to market as its own automated phase of the life cycle, with the same agent-driven approach we applied to code and testing, and a simple rule that nothing ships until the rest of the organization can support it. If I were starting over, I would build that capacity from day one instead of finding out about it the hard way.

The real transformation is organizational

The tools matter, but everyone can buy the same tools. What we actually did over these 18 months was redesign the system around AI: the way work flows between teams, the governance that lets people trust the output, the small set of tools we committed to, and a target aggressive enough to force all of it. Any one of those alone would have produced another interesting pilot. Together they produced a different engineering organization.

None of this is finished. The models keep improving and the life cycle will keep changing with them. The organizational groundwork is what lets us absorb each improvement as it arrives, and for any technology leader still stuck at the pilot stage, that groundwork is where I would start.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

For a growing number of organizations, AI has become not only a key component of the software development process, but the centerpiece.

These enterprises have shifted toward an AI-first development strategy, where artificial intelligence is integrated into all phases of the software development life cycle, rather than being a feature to use selectively during the process.

In most cases, this means a move toward agentic workflows, with developers acting as architects responsible for overseeing AI agents. But in a larger sense, AI-first development is a transformation of how developers explore and understand code, with a focus on designing and producing more and more applications with AI in mind.

Why AI-first development?

For what is surely a rising number of software development teams and organizations, the AI-first approach is becoming more necessary than optional.

“AI-first development matters because the alternative is increasingly untenable,” says Mona Rajhans, senior software engineering manager at cybersecurity products provider Palo Alto Networks, who leads a 14-person AI engineering team. Simply put, development teams that use AI to build applications can accomplish far more in less time.

By the same token, it is increasingly important to design applications with AI components and agentic users in mind. “Bolting AI onto an existing architecture after the fact is like adding a second floor to a house with no load-bearing walls,” Rajhans says. “It works until it doesn’t, and when it breaks it breaks badly.”

One of the biggest advantages of AI-first development is the speed with which teams can complete projects and get products into development. This includes identifying and addressing issues much earlier than would be otherwise possible.

“Work that used to stall in the backlog for quarters, like platform migrations and updating aging libraries, now clears in days,” says Collin Hogue-Spears, senior director, DTE, at application security software company Black Duck Software. “Defects surface at design time rather than in production, where a catch costs a code review instead of an incident.”

When you apply AI from the onset of your software project, “you’ll typically learn about issues with your concept in hours rather than weeks,” says Bob Hutchins, CEO at consulting and advisory firm Human Voice Media. “I developed full production applications where the first working prototype was completed prior to the kickoff meeting for a traditional shop.”

AI-first development also helps to keep unnecessary costs down. “Retro-fitting AI into projects is often costly and difficult,” Hutchins says. “Project designs that include AI from day one result in cleaner data flow patterns, much clearer permission schemes, and user interface elements that understand how agents will interact with these systems.”

Implementing a chatbot to an existing application produces a chatbot, Hutchins says. “Designing for AI provides an entirely new type of product,” he says.

Perhaps most important, using AI-first development causes teams to improve their documentation and overall clarity in their thought processes, Hutchins says. “Since an AI cannot read minds, teams which follow this approach will provide better documentation specifications, because the specification itself will serve as the actual instruction set.”

How to succeed at AI-first development

Not all development teams are ready for this sea change in the development process. Many struggle to adopt AI-first development for any number of reasons. Maybe they have underlying engineering systems that rely on undocumented knowledge, implicit rules, or manual oversight. Or they don’t have the right quality assurance, security, and testing processes and skills in place.

AI requires precise context in order to function correctly, and teams that rely on undocumented legacy codebases and obscure architectures might find that AI generates inaccurate information that ends up requiring significant rework by humans.

By following a few good practices, developers and teams can prepare for an AI-first environment and excel at this new paradigm for development.

Add new skills and roles

One of the most important steps is to add new skill sets and roles that fit into the AI-first strategy, even if that means uncomfortable changes for some. Without the needed skills in place, nothing else in the AI-first approach will work well.

“The scarce ability is no longer writing code, but reading it quickly and assessing it accurately,” Hogue-Spears says. “The strongest developers will work as architects: they frame the problem, direct the agent, and verify the result. That is how one engineer safely oversees the work of several.”

Organizations need to focus staffing on the senior architecture roles and cut or redesign the entry-level junior coding roles, says William Flaiz, founder of CleanSmartLabs, a provider of data cleansing products.

“Tools like Claude Code can write good clean code fast so that is no longer the bottleneck,” Flaiz says. “The judgment and experience of what to build, how to architect it, and what is going to break comes from the judgment of senior architects. Skip this and you will ship product fast, but it will break under user load.”

Consider training programs to ease the transition

Much of AI-first is relatively new for a lot of organizations, and many developers and managers will need to learn what it means to be architects in this new world.

The architect role “is changing in a way most organizations have not caught up to yet,” Rajhans says. “The job is less about writing code and more about designing the boundaries, where the agent acts, where the human decides, and what happens when the agent is wrong. That last question is the one most teams skip until it is too late.”

Development teams also need to shift the developer role to an orchestrator role for agentic development, Flaiz says. Training will be needed in a lot of cases.

“They have enough product context and user understanding to where agents hand off, conflict, and which agent wins when in conflict,” Flaiz says. “This is not a programmer role in the traditional sense, but a new role that is a good transition for junior and entry-level developers.”

In addition, Flaiz recommends integrating user experience expertise into the development process, “not as a starting point only or as a gate check at the end,” he says. “AI can write requirements, specs, and assist with architecture, but it doesn’t understand your users. It doesn’t know the habits of how real users interact with software. That comes from watching [user experience] tests, analyzing behavior, and being human.”

Shift toward agentic workflows

A key part of AI-first development is shifting toward agentic workflows, where autonomous agents deploy reasoning, planning, and external tools to achieve complex goals. This is the scenario in which developers will act as architects who oversee AI agents as they work to solve problems and build applications.

“The fundamental transition here is from typing—writing code—to specifying what you want done,” Hutchins says. “People who thrive within agentic workflows are individuals who can clearly define systems, think through potential [uncommon scenarios], and assess others’ work objectively.”

While keying in code might seem to matter at times, the skills required for agentic workflows “represent true architecture and editorial judgment,” Hutchins says. “As such, senior engineers’ values increase rather than decrease.”

With the new workflows, “developers will spend less time writing every line of code manually and more time defining goals, considering product constraints, iterating through architecture, designing intuitive user interfaces, envisioning data flows, and calculating expected outcomes,” says Bob Brauer, founder and CEO of Interzoid, a consultancy and provider of data enrichment and structured intelligence services.

“AI tools and agents can then help generate code, test plans, documentation, deployment scripts, and monitor ongoing maintenance requirements,” Brauer says.

Create new testing and review processes

Another good practice is to create a new review process prior to building agent pipelines. “AI will generate ‘plausible’ garbage without hesitation,” Hutchins says. “Organizations that are currently suffering damage due to [poor] generation are the same ones that grew their generation capabilities faster than their ability to verify generated output. We have already seen consultants return client monies paid for fake citations. This represents a process failure, not a technology failure.”

Reviews need to be ongoing. “Design for the human checkpoint,” Hutchins says. “For every workflow the questions should be ‘where does a person with judgment need to review before we send this?’ If you cannot answer this question for your current workflow, then you are not implementing AI-first development. You are simply allowing unsupervised delegation.”

As for testing, “developers have to get comfortable with non-determinism,” Rajhans says. “AI-first systems do not behave the same way twice. Testing, observability, and failure handling all have to be redesigned from scratch around that reality. Teams that treat AI outputs like function returns get burned fast.”

Start small and proceed with caution

Despite the urgency among many business leaders to move aggressively toward AI, a somewhat cautious approach might be best for both near-term and long-term goals.

“Begin small and track everything,” Hutchins says. “Allow developers access to approved tools and clearly defined guidelines, then determine where [and] when AI reduces developer time verses when AI generates additional rework.”

AI-first development will become increasingly critical as competition intensifies and software teams face growing pressure to deliver faster, more sophisticated, and easier-to-use applications, Brauer says.

“By leveraging AI across requirements gathering, architecture/design, coding, testing, deployment, and ongoing maintenance, development teams can significantly increase their effectiveness and create greater organizational value,” Brauer says. “Organizations that embrace AI-first development will be better positioned to deliver higher-quality software faster and gain a meaningful competitive advantage.”

IT industry experts and analysts are still trying to piece together Nvidia’s surprise plan to pay $12.9 billion for open-source AI company Hugging Face.

Nvidia dominates AI with its GPUs, and the company generates billions of dollars in revenue through a proprietary approach to the fast-moving technology. Hugging Face, on the other hand, hosts open models and has been a neutral player between chip vendors and model labs.

“This is about Nvidia having more say in how the stack gets built,” said Stephanie Walter, analyst at Hyperframe Research.

Hugging Face is wildly popular with developers, and Nvidia is buying early influence with that crowd. “You have a better chance of being part of the production environment later,” Walter said, adding that she wasn’t sure how Nvidia reached a nearly $13 billion price tag for the acquisition.

“Hugging Face has near-uncontested market primacy over where developers go for open-weight model releases. Now Nvidia owns that,” said Mark Petty, senior director analyst at Gartner.

Nvidia’s chase for developers should force IT decision-makers to review how much of the AI stack they control, said Hector Liu, director of Institute of Foundation Models’ Silicon Valley Lab. IFM is part of the Abu Dhabi-based Mohamed bin Zayed University of Artificial Intelligence.

Liu said CIOs should ask themselves three questions: “Can you run the model on hardware you already have, without a dependency you didn’t choose? Can you see how it was built?” And, “is the license one you can build a business on?

“A model that passes all three is durable, no matter who buys whom next year,” Liu said.

IFM’s latest K2 Horizon model, which was introduced on the same day Nvidia’s deal was announced, is hosted on Hugging Face. The open model was built to answer all of those questions.

K2 Horizon runs on AI hardware from Nvidia, AMD, Cerebras, and major cloud providers, said Liu, who doesn’t expect that to change. “Nvidia has said Hugging Face stays an open platform for every builder and every accelerator, and we’ll take that at face value,” Liu said.

Nvidia pledged to maintain Hugging Face’s hardware and model independence, and said its compute won’t be required. 

Even so, Nvidia isn’t paying nearly $13 billion for a model repository, said Jake Newfield, CEO at Hermetiq — it’s buying the front door to open AI.

(Hermetiq develops AI build and code observability tools.)

AI-generated output is becoming abundant and the infrastructure that makes it testable, reproducible and deployable is becoming strategically valuable, Newfield said. “Nvidia can accelerate it with capital and compute, but the real test is operational neutrality,” he said. 

That involves assessing whether competing hardware remains equally supported across the tooling, benchmarks and deployment paths developers actually use, Newfield said.

Beyond distribution, Nvidia is also buying Hugging Face for data, Petty said. That data showed that agents overtook humans as its largest traffic source in July, the kind of insight that could prove valuable down the road.

“Every model pulled tells Nvidia what the market wants next,” Petty said.

Nvidia has every reason to grow demand for open models and to keep them cheap, Petty said. “That’s good for enterprises, as it prevents market consolidation around a small number of proprietary model owners,” Petty said.

Closed and open models will coexist and lead to a world “where you’re going to continue to train these models and run these models at scale,” Justin Boitano, Nvidia’s vice president for Enterprise AI, said in a press conference after the deal’s announcement.

Nvidia will benefit “through the training that’s done and the inference that’s done on our hardware as models get diffused into the ecosystem at scale,” Boitano said.

That’s one motivation to keep the ecosystem open and neutral, so “developers can work wherever they want to work,” Boitano said.

Open source makes AI more accessible by lowering the barriers to experimentation and adoption, said Jon Carvill, senior vice president of marketing at AI chip maker Nuvacore. “Bringing Nvidia and Hugging Face closer together should help accelerate that choice, access and innovation,” Carvill said.

But there are still unanswered questions around whether Hugging Face will remain open and how much proprietary control Nvidia might exert, said Jack Gold, principal analyst at J. Gold Associates.

Microsoft traveled a similar path with its acquisition of open-source repository GitHub, which “did not really pan out as well as Microsoft hoped,” Gold said. “With Nvidia’s acquisition, will it still be as open to competitive hardware-software access, or will there be some barriers employed?”

OpenAI launched GPT-6 Astra on Thursday, disclosing that the new flagship model has crossed the “Critical” threshold for cybersecurity risk under its Preparedness Framework, a classification the company said triggers additional deployment restrictions.

“GPT‑6 Astra is rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and AWS,” OpenAI said in a statement.

Enterprise administrators must manually enable Astra for their workspace, since access is off by default at launch, according to the company.

Developers can access Astra in the API as gpt-6-astra or through Amazon Bedrock, OpenAI said, priced at $10 per million input tokens and $50 per million output tokens. Pro, Business, and Enterprise users also get a variant called Astra Pro, and the company said Astra supports Zero Data Retention for eligible API customers.

Company claims perfect score on exploit benchmark

OpenAI said it tested Astra without production safeguards on ExploitBench, and that the model scored 100%, up from 78.5% for predecessor GPT-5.6 Sol. On ExploitGym, a broader exploit-development benchmark, the company said Astra reached a 42.4% success rate against 30.3% for Sol, while using fewer output tokens.

“Its ability to identify and develop zero-day exploits can help defenders find and patch weaknesses, but it also creates a need for stronger safeguards,” OpenAI said in the blog post.

OpenAI also tested Astra on vulnerabilities disclosed in the three months before launch, to check whether it could find flaws on its own rather than recalling old exploits from training data. The model found two new zero-day vulnerabilities during that test, OpenAI said, and it is now disclosing both to the software makers involved.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said the Critical label is a disclosure event rather than a capability event.

“Astra’s capability did not change between 10 August, when OpenAI said Critical capability could not be ruled out, and September 1, when it said the threshold was met,” Gogia noted. “The testing changed. The model did not.”

That inverts the obvious enterprise response, he said.

“Astra is now the only frontier model whose cyber capability an enterprise actually knows, because it is the only one measured against a published threshold, while every unlabelled model already sitting behind enterprise credentials has never been measured that way and will not be until its vendor chooses to measure it,” Gogia pointed out. “Those models are not safer.”

OpenAI said the public version of Astra will refuse advanced offensive tasks such as generating proof-of-concept exploits, though it plans to loosen those restrictions for vetted defenders through a program called OpenAI Daybreak in the coming weeks.

The launch follows OpenAI’s rollout of GPT-5.6 Sol, which the company said scored 73.5% on ExploitBench at launch, and comes months after Anthropic’s Fable and Mythos models were briefly pulled from export markets over similar concerns.

Governance shifts from the model to the harness around it

Gogia said the bigger shift is that reasoning now translates into state change, since a wrong chatbot answer is an information problem while a wrong agent action inside a customer-record system is an operating event.

“The governance unit therefore moves off the model,” he said, arguing the relevant question is no longer which model is approved, but how much damage a given identity can do before a control intervenes.

Amit Kumar Jena, head of AI development at Kanerika said the visibility problem is concrete: when an agent acts through a user interface, systems of record log the action as a person, so an agent that updates 400 ERP rows shows up as a service account making 400 updates, with no record of which instruction or model version produced them.

“You lose granularity inside the exact system a regulator or auditor will ask to see,” Jena added.

OpenAI said it built a new evaluation, informed by an incident involving Hugging Face, to test whether a model given an impossible task would exceed its authorized scope.

“Compared to GPT‑5.6 Sol, which without production safeguards went beyond the authorized target 48% of the time, GPT‑6 Astra did this in 0% of cases,” the statement added.

Gogia said the more uncomfortable finding is that Astra behaves better and watches worse: OpenAI reports decreased chain-of-thought monitorability against Sol, with Astra less likely to reveal incriminating reasoning, and its monitoring covers OpenAI’s own external deployment but nothing published extends that telemetry to customers. “OpenAI being able to monitor Astra does not mean an enterprise can audit Astra,” Gogia said.

The article originally appeared on CSO.

OpenAI launched GPT-6 Astra on Thursday, disclosing that the new flagship model has crossed the “Critical” threshold for cybersecurity risk under its Preparedness Framework, a classification the company said triggers additional deployment restrictions.

“GPT‑6 Astra is rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and AWS,” OpenAI said in a statement.

Enterprise administrators must manually enable Astra for their workspace, since access is off by default at launch, according to the company.

Developers can access Astra in the API as gpt-6-astra or through Amazon Bedrock, OpenAI said, priced at $10 per million input tokens and $50 per million output tokens. Pro, Business, and Enterprise users also get a variant called Astra Pro, and the company said Astra supports Zero Data Retention for eligible API customers.

Company claims perfect score on exploit benchmark

OpenAI said it tested Astra without production safeguards on ExploitBench, and that the model scored 100%, up from 78.5% for predecessor GPT-5.6 Sol. On ExploitGym, a broader exploit-development benchmark, the company said Astra reached a 42.4% success rate against 30.3% for Sol, while using fewer output tokens.

“Its ability to identify and develop zero-day exploits can help defenders find and patch weaknesses, but it also creates a need for stronger safeguards,” OpenAI said in the blog post.

Recommended for you

OpenAI also tested Astra on vulnerabilities disclosed in the three months before launch, to check whether it could find flaws on its own rather than recalling old exploits from training data. The model found two new zero-day vulnerabilities during that test, OpenAI said, and it is now disclosing both to the software makers involved.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said the Critical label is a disclosure event rather than a capability event.

“Astra’s capability did not change between 10 August, when OpenAI said Critical capability could not be ruled out, and September 1, when it said the threshold was met,” Gogia noted. “The testing changed. The model did not.”

That inverts the obvious enterprise response, he said.

“Astra is now the only frontier model whose cyber capability an enterprise actually knows, because it is the only one measured against a published threshold, while every unlabelled model already sitting behind enterprise credentials has never been measured that way and will not be until its vendor chooses to measure it,” Gogia pointed out. “Those models are not safer.”

OpenAI said the public version of Astra will refuse advanced offensive tasks such as generating proof-of-concept exploits, though it plans to loosen those restrictions for vetted defenders through a program called OpenAI Daybreak in the coming weeks.

The launch follows OpenAI’s rollout of GPT-5.6 Sol, which the company said scored 73.5% on ExploitBench at launch, and comes months after Anthropic’s Fable and Mythos models were briefly pulled from export markets over similar concerns.

Governance shifts from the model to the harness around it

Gogia said the bigger shift is that reasoning now translates into state change, since a wrong chatbot answer is an information problem while a wrong agent action inside a customer-record system is an operating event.

“The governance unit therefore moves off the model,” he said, arguing the relevant question is no longer which model is approved, but how much damage a given identity can do before a control intervenes.

Amit Kumar Jena, head of AI development at Kanerika said the visibility problem is concrete: when an agent acts through a user interface, systems of record log the action as a person, so an agent that updates 400 ERP rows shows up as a service account making 400 updates, with no record of which instruction or model version produced them.

“You lose granularity inside the exact system a regulator or auditor will ask to see,” Jena added.

OpenAI said it built a new evaluation, informed by an incident involving Hugging Face, to test whether a model given an impossible task would exceed its authorized scope.

“Compared to GPT‑5.6 Sol, which without production safeguards went beyond the authorized target 48% of the time, GPT‑6 Astra did this in 0% of cases,” the statement added.

Gogia said the more uncomfortable finding is that Astra behaves better and watches worse: OpenAI reports decreased chain-of-thought monitorability against Sol, with Astra less likely to reveal incriminating reasoning, and its monitoring covers OpenAI’s own external deployment but nothing published extends that telemetry to customers. “OpenAI being able to monitor Astra does not mean an enterprise can audit Astra,” Gogia said.

The article originally appeared on CSO.