Microsoft has had a narrow escape from total embarrassment: A security company uncovered a critical vulnerability that could have compromised all Azure Cosmos DB databases — both those of customers and Microsoft’s own.

Google subsidiary Wiz found a flaw in the database’s Gremlin API, usually used for storing and managing property graph data.

If bad actors had discovered it first, they could have exploited it to acquire what Wiz called the Cosmos Master Key, which would have enabled them to use the primary key of any Cosmos database, resulting in read and write access to any account. They would also have had access to a list of every database on the service, with identifiers such as subscription and tenant IDs.

Azure Cosmos DB is a NoSQL database that underpins Microsoft’s cloud services. It can be accessed through SDKs for framework such as Python, Node.js, Java, and .NET.

Wiz described how it discovered the vulnerability in a blog post. It disclosed details of the flaw to Microsoft in November 2025.

Microsoft deployed a hot fix within two days, but it took another eight months to re-engineer the infrastructure, removing the Cosmos Master Key and introducing new guardrails to Cosmos DB to prevent similar attacks.

It is not the first time Cosmos DB customers’ primary keys have been under threat: In 2021, Wiz found a flaw in data exploration tool Jupyter Notebook that could be exploited to access the database keys and other secrets.

This article first appeared on CSO.

A client called me last week with a problem I have been hearing about more often lately. They had made significant reserved instance commitments with a major cloud provider, overbuying for what they thought would be heavy AI training workloads. Now they were sitting on thousands of dollars in idle capacity every month. Their plan was simple: resell it to someone else. Except they couldn’t.

I have been doing cloud consulting for a long time, and this situation once had a straightforward solution. You went to the marketplace, listed your unused reservations, and found a buyer. The process was a bit clunky, but it worked. These days, the answer is far more complicated, and my client learned this the hard way.

AI has made this problem increasingly common. Companies initially committed to compute capacity based on ambitious training plans. Prototype projects were expected to scale, and inference workloads were projected to grow substantially. Then reality hit. Some projects did not materialize. Some models trained faster than expected. Some inference patterns were lighter than anticipated.

Many organizations now hold reserved capacity they can’t use, discard, or share without a complex, increasingly restricted process. This reality is something every company with significant cloud spend needs to clearly understand.

The history of cloud resale

There was once a functioning resale market for cloud reserved instances. AWS, for example, maintained a Reserved Instances Marketplace where companies that had purchased reserved capacity could sell those reservations to other AWS customers. This was a legitimate, AWS-sanctioned process. Companies would register as sellers, list their unused reservations with pricing and terms, and if a buyer appeared, the marketplace would facilitate the transaction.

The resale market was useful for companies that had overestimated their needs or whose business changes reduced their cloud consumption. Instead of simply absorbing the cost of unused commitments, they could recoup some of that investment by selling to other organizations with unmet demand. It created a secondary market that added liquidity to what was otherwise a rigid financial arrangement.

My client had some experience with this resale market a few years ago and assumed they could use it again. They were unpleasantly surprised to learn that the rules had changed.

AWS changes the rules

In January 2024, AWS implemented a significant policy change that effectively shut down the resale of EC2 Reserved Instances on its platform. AWS stopped allowing companies to resell their unused reserved capacity through the Reserved Instance Marketplace or any other official channel. If you have a reserved instance commitment with AWS, you are essentially stuck with it unless you can use it yourself or modify your reservation.

This change had a real impact on companies that had relied on resale as part of their cloud financial management strategy. It reduced flexibility and increased the risk of long-term reserved commitments. When I explained this AWS policy change to my client’s representatives, I could hear the frustration in their voices. They had made their commitment in good faith, carefully modeled their expected AI workloads, and now faced the reality that there was no easy exit.

The reasoning behind this change is not entirely clear, but AWS likely viewed capacity resales as something that complicated their billing and commitment models without providing enough benefit to the overall ecosystem. Regardless of the company’s reasons, the primary resale path for the largest cloud provider has been effectively closed.

What options still exist?

What can companies do now when they find themselves with reserved capacity they no longer need? The first possibility is to work directly with the cloud provider to modify or exchange the reservation if it is convertible. Some reservation types allow modifications, such as changing the instance type, region, or tenancy. This will not eliminate the commitment, but it may help companies better align their reservations with actual workload needs.

The second option is to use third-party brokers and marketplaces that operate independently of the cloud providers. Although AWS has shut down its official resale channel, brokers and marketplaces still facilitate resale arrangements for other cloud providers and for some AWS scenarios. These arrangements can be more complex and carry more risk, but they remain a possibility for companies determined to move unused capacity.

The third alternative is to optimize usage. Companies can invest in better utilization monitoring, workload placement, and automation to ensure that reserved capacity is used as efficiently as possible. This does not recover the money already spent, but it reduces future waste.

My client explored all three alternatives and found that each had significant limitations. Modifications were possible, but only within a narrow range. Third-party brokers were interested, but the process was opaque and uncertain. Optimization helped, but it could not eliminate the fundamental overcommitment they had already made.

The broader implications

Cloud commitments are more rigid than many enterprises initially realize because they lack a liquid market and because providers control modifications, transfers, or cancellations. Right now, I see this pattern most often in the AI space. Companies commit to massive amounts of compute for training and inference based on projections that rarely reflect the actual workloads. Then they are surprised to find themselves locked into payments. The AI boom has led to significant overcommitment because enterprises remain unaware that the resale mechanisms that once existed have been largely shut down.

This is why cloud financial management has become such an important discipline. Companies need to be far more thoughtful about how they commit to cloud resources, how they model their future consumption, and how they build flexibility into their cloud strategies. The days of assuming you can always resell your way out of an overcommitment are effectively over, at least with AWS.

For Azure and Google Cloud, the resale landscape is slightly different, but the same general principles apply. These providers have their own capacity transfer policies and, like AWS, those policies can change at any time. Companies should understand their options before making large, committed purchases and build contingency plans in case their actual usage diverges from their projections—or if resale policies change.

The bottom line is that reselling unused reserved cloud instances is far more complicated than it sounds. The market is not as open as it once was, the options are limited, and the providers themselves hold most of the cards. My client got burned, and I doubt they will be the only one. Companies that want to optimize their cloud spending should focus on accurate forecasting, thoughtful commitment sizing, and ongoing optimization rather than relying on resale as a safety valve. That approach worked at one point, but those days are largely gone.

Meta is reportedly planning a cloud business that would sell access to AI computing power and models, extending its internal infrastructure into a commercial service for outside developers and enterprises. Reuters, citing Bloomberg’s reporting, noted that the planned offering would allow customers to access AI models hosted on Meta’s infrastructure and pay based on usage, effectively positioning the company in the infrastructure-as-a-service and AI platform markets. On the surface, this seems like a logical next step. If you are already spending enormous amounts of money to build AI infrastructure, there is a natural temptation to ask whether some of that investment can be monetized beyond your own internal use.

I have seen this pattern before. A company builds sophisticated internal systems, recognizes their value, and then begins to imagine that becoming a cloud provider is simply a matter of exposing those capabilities to external customers. It sounds straightforward, especially given the excitement around AI and the demand for high-performance infrastructure. But cloud computing is not just another distribution model. It is not simply a matter of offering on-demand multitenant services and charging a fee. It is a deeply operational, trust-based business in a market that punishes companies that do not fully understand what enterprise customers require.

A crowded neocloud market

The first problem Meta faces is that this is not an open opportunity. The neocloud space, meaning purpose-built AI infrastructure delivered as a service, is already crowded and increasingly difficult to enter. Amazon, Microsoft, and Google dominate the conversation for obvious reasons. They have years of cloud operating experience, broad service portfolios, global reach, mature ecosystems, and deeply established enterprise relationships. Oracle remains a serious player as well, especially in enterprise applications, data platforms, and performance-sensitive workloads. IBM still matters in hybrid cloud, operations, and industries where governance and regulatory rigor remain central.

That list alone should give Meta pause. These companies are not just infrastructure vendors. They are experienced cloud operators. They have spent years building not only the underlying platforms, but also the native capabilities enterprises now expect by default. Those capabilities include security, governance, identity management, observability, support, compliance, billing controls, resilience planning, and integration with the broader enterprise technology estate. These are not secondary features. They are part of the core value proposition.

This is why late entry into the cloud market is so hard. A new provider is not just competing on price or capacity. It is competing against accumulated trust. Enterprises are not casual buyers. They are selecting long-term operating environments for applications, data, AI models, and business-critical processes. They want confidence that the provider understands how these services will be consumed, governed, and supported over time. Meta is entering a market where the incumbents already have a major head start on all of those fronts.

Harder than it looks

Over the years, I have had many technology companies come to me and say they wanted to reposition their technology in the cloud space, either as software as a service or infrastructure as a service. In the beginning, enthusiasm is always high. The technology is impressive. The market size looks attractive. The revenue models appear compelling. Investors love the story. Then we begin to walk through what it really means to operate as a cloud provider, and the optimism usually fades fast.

The questions become very practical and very uncomfortable. How will tenants be isolated? How will identity and access controls work across different kinds of customers? What governance models will be built in natively? How will workloads be monitored, optimized, and secured? What does support look like 24 hours a day, across regions, across industries, across compliance boundaries? How will outages be handled, communicated, and remediated? How will the platform integrate with existing customer tools for operations, policy management, and security response? How much investment will it take just to become credible before you even begin to differentiate?

Once companies fully understand the complexities, market dynamics, and the capital and execution required to compete even with secondary players, many of them back off. They realize that cloud technology is not a packaging exercise. It is a transformation in how a company designs, operates, supports, sells, and evolves technology. That is why I remain skeptical when any company assumes it can translate internal infrastructure excellence into external cloud success without a very long, disciplined commitment.

Meta’s market readiness

Of course, Meta is not lacking in financial resources. If any company can afford to spend aggressively in this space, it is Meta. The company has the capital to build infrastructure, absorb losses, hire experienced talent, and stay in the market long enough to make a serious attempt. I would never argue that Meta is too small or too poor to try. Quite the opposite. If there is any non-traditional entrant with the financial scale to force itself into the conversation, Meta would be high on the list.

But money does not erase complexity. It only gives you the chance to confront it. The real question is not whether Meta can afford to become a cloud provider. The question is whether Meta has what it takes to become an excellent cloud provider. Those are two very different things. Enterprises are not going to move meaningful workloads to a new platform simply because the company behind it is wealthy or technically famous. They are going to ask whether the provider understands enterprise consumption patterns, enterprise risk, enterprise governance, and enterprise operations.

That is where the challenge becomes much more serious. Meta has extensive experience running infrastructure for itself. That is valuable, but internal operating excellence is not the same thing as external service maturity. Running systems for your own workloads allows a high degree of control over architecture, standards, priorities, and operating assumptions. Running systems for paying customers requires flexibility, consistency, transparency, and support across a wide range of use cases that you do not control. Those are very different disciplines, and companies often underestimate the gap between them.

What exactly is Meta?

Another concern here is strategic clarity. Meta already has a complicated market identity. It is a social media company, an advertising platform company, a hardware company, an AI company, and still, in the minds of many, the company that spent billions pursuing the metaverse. If it now wants to be viewed as a serious cloud infrastructure provider, it will need to explain not only what it is offering, but why customers should believe this is a durable long-term commitment and not just another adjacent experiment.

That uncertainty can be damaging. Customers want stable providers with clear strategic intent. They do not want to architect important systems around a platform if they suspect the provider may lose interest, shift direction, or reframe the business after a few years of uneven results. Cloud computing requires patience, consistency, and deep customer orientation. It is not a market where strategic ambiguity helps.

This could become confusing for Meta internally as well. Building a true cloud business demands focus. It demands years of investment in areas that may not be glamorous but are absolutely necessary, such as governance, operations, controls, support frameworks, partner programs, and enterprise sales alignment. If the company is not willing to make those sacrifices fully and for the long term, the initiative will struggle.

AI coding agents are making it easier than ever to produce software. Ensuring that software is secure before deployment is another matter — one that AWS thinks AI should help with too.

As enterprises adopt agentic development workflows, the volume of first-party code being created and modified is rising rapidly. Yet the process of validating vulnerabilities, determining whether they are exploitable, and fixing them often still depends on developers and security teams working through findings manually.

AWS is aiming to address that imbalance with Continuum, a new service designed to continuously discover, investigate, and remediate vulnerabilities in enterprise environments, whether the code is their own or from third parties.

Rather than simply generating alerts, the service is intended to help enterprises move findings through the entire remediation lifecycle, AWS VP of Security and Observability Chet Kapoor wrote in a blog post.

For first-party applications, Continuum can analyze code, validate whether vulnerabilities are exploitable, generate remediation recommendations, and propose fixes that can be reviewed through existing software development workflows, helping developers address security issues without requiring security teams to manually investigate every finding, Kapoor said.

Once users think Continuum has learned enough about their environment and understands their guardrails, they can put it in what AWS calls “enforce mode” to autonomously fix any code lapses, Kapoor said.

Continuum borrows some of its capabilities, penetration testing and code scanning features, from an existing service, Security Agent.

Other capabilities are all-new, including threat modeling, which is designed to automatically generate threat models from source code or design documents and output them in STRIDE format.

Keeping pace with AI-driven software development

Analysts see Continuum helping enterprise developer teams ship more secure code while keeping pace with AI coding tools.

“The harder problem is no longer just finding issues, it is knowing which ones are real, which ones matter in their environment, and which ones need to be fixed first,” said Akshat Tyagi, associate practice leader at HFS Research. “Traditional workflows built around dashboards and manual triage struggle with that volume. A dashboard can show the backlog, but it does not validate the finding, assess business impact, or help remediate it.”

Continuum’s value, according to Tyagi, “is not just more detection, but using AI to prioritize risk findings, suggest mitigations, and support faster action while keeping humans in control of high-risk decisions.”

Taking faster action is becoming increasingly important as attackers are gaining access to many of the same AI capabilities that enterprises are using to accelerate software development and security testing, according to Amit Chandak, chief analytics officer at IT consulting firm Kanerika. “The gap between a flaw being disclosed and a working exploit is shrinking rapidly from months to hours,” he said.

While Continuum may reduce repetitive work for developers and SREs, it could also create new responsibilities for CISOs around governance, oversight, testing, and maintaining guardrails for automated actions.

“Continuum changes the CISO’s role from managing findings to governing how findings are handled. The focus moves to setting rules: what can be automated, what needs human approval, and what level of risk is acceptable in production,” Tyagi said. “Staffing will shift too. There may be less manual triage, but more need for people who can review AI-generated fixes, set guardrails, and know when not to trust the system.”

Even so, Chandak does not expect the offering to lead to immediate headcount reductions, particularly given that Continuum is only available as a gated preview.

Continuum could change how CISOs measure work, Tyagi said: “Ticket count matters less. Better measures are how quickly real risks are validated and fixed, how many false positives are removed, and whether automation is reducing risk without causing new problems.”

Those same metrics could also become a yardstick for CISOs determining how much autonomy to give tools like Continuum, said Chandak. Most enterprises’ data and governance practices are not yet ready for fully autonomous remediation, said Chandak, adding that, “AWS’ graduated trust design, under which enterprises have the option of choosing the degree of autonomy, from human in the loop to fully automatic remediation, is an admission of that fact.”

Beyond first-party code

Continuum could also help CISOs with third-party code vulnerability analysis, where enterprises often have less visibility and control.

“Most third party vulnerability alerts are noise. A tool may flag a vulnerable library, but the real question is whether that vulnerable code is actually used in production. If Continuum can answer that, it helps teams focus on the few issues that matter,” Tyagi said. “This is especially useful for open-source and software supply chain risk, where enterprises depend on packages and hidden transitive dependencies they may not fully track. It also helps when no patch is available yet.”

However, he warned, Continuum might not offer a direct fix to third-party code: “You usually cannot patch third-party code yourself as you don’t own it, so remediation there means version pinning or compensating controls.”

It is tempting to date cloud computing from the launch of Amazon S3 in 2006 and the rise of infrastructure as a service (IaaS) that followed. That was certainly the moment the market changed in a visible, irreversible way. But the truth is that cloud began earlier, in the 1990s, when software as a service (SaaS), application hosting, managed services providers, and various forms of remote subscription computing started to reshape how enterprises thought about owning and operating technology. Even then, the core value proposition was familiar: Let someone else run the infrastructure, abstract the complexity, deliver capability as a service, and allow the business to consume only what it needs.

What AWS changed was the scale, accessibility, and precision of the execution. Amazon turned infrastructure into a programmable utility. It made compute and storage available in ways that were elastic, self-service, API-driven, and globally reachable. That was the breakthrough. Enterprises had outsourced pieces of technology before, but now they could rent raw infrastructure with unprecedented speed and flexibility. The launch of Amazon S3 was especially important because it provided a durable, scalable storage foundation that became one of the building blocks for modern digital business.

AWS changed everything

Technology markets are rarely transformed by the first company to think of an idea. They are transformed by the first company to make that idea operationally real, economically viable, and broadly consumable. AWS did exactly that. It built a model for infrastructure as a service that allowed enterprises, startups, and eventually governments to rethink the entire life cycle of IT delivery.

Looking back from 2026, it is difficult to remember how radical this concept once seemed. At the time, many enterprise leaders considered public cloud too risky, too immature, too uncontrolled, or simply too foreign for conventional IT governance. There were concerns about security, compliance, vendor dependency, performance, data residency, and reliability. Many of those concerns were valid. Early cloud adoption often ran ahead of cloud maturity, and many organizations discovered that moving quickly did not always mean moving wisely.

Still, the economics of agility overwhelmed the inertia of the old model. Provisioning that once took months could be done in minutes. Capital expenditure gave way, at least in part, to operating expenditure. Experimental workloads became easier to justify. Digital businesses could scale without building data centers first. AWS led that transition, and the rest of the industry followed, including competitors that helped mature the market.

Cloud’s strengths and liabilities

If the first decade of cloud was about acceleration, the second decade was about correction. Enterprises learned that cloud was not automatically cheaper, not automatically simpler, and not automatically better. It was better when used with discipline. It was more cost-effective when architected intelligently. It was more resilient when governance, operations, and security were designed into the system rather than added later.

This is when the industry grew up. We learned about cloud financial management because too many organizations assumed elasticity would control cost, only to discover that unused resources, poor workload placement, and fragmented accountability could drive spending far beyond expectations. We learned that public cloud could provide extraordinary innovation and reach, but also that not every workload belongs there. Latency, sovereignty, compliance constraints, legacy integration challenges, and predictable high-volume workloads all forced a more nuanced view.

We also learned about concentration risk. As enterprises standardized on a small number of hyperscalers, questions emerged around resilience, lock-in, and strategic dependency. The answer was never simplistic multicloud posturing for its own sake. It was architectural realism. Use the public cloud where it creates a clear advantage. Keep options open where business risk requires it. Understand portability, but do not romanticize it. In other words, cloud became less ideological and more practical.

Cloud is now an assumption

Perhaps the most important shift of all is that we no longer debate whether cloud is real or whether enterprises should use it. That argument is over. Cloud is baked into the cake. It is part of enterprise operating reality. The modern enterprise assumes on-demand infrastructure, platform services, automation pipelines, managed databases, identity fabrics, observability stacks, and globally distributed application delivery. Even when workloads remain on-premises or at the edge, they are often built, governed, or operated with cloud-native thinking.

This is maturity. Cloud is not a project or a trend. It is not even a strategy by itself. It is an enabling model that now underpins enterprise strategy. Businesses no longer ask whether to adopt cloud in the abstract. They ask how much cloud, which cloud services, under what governance model, at what cost profile, and in support of which business outcomes.

That may sound less exciting than the early days of disruption, but it is actually the mark of success. The most powerful technologies eventually disappear into standard practice. Electricity, networking, virtualization, and mobile platforms all went through this process. Cloud has done the same.

How cloud supports the AI race

As enterprises move aggressively into AI, cloud has entered another pivotal phase. AI is not replacing cloud. It is intensifying the importance of cloud while also changing how value is measured. Training, tuning, deploying, and governing AI systems require immense computational scale, specialized infrastructure, distributed data access, and operational consistency. Public cloud providers are well positioned to offer those capabilities, particularly with GPUs, AI platforms, managed model services, and data integration tools.

But this is not a repeat of the early cloud era. Enterprises are more sober now. They know the importance of cost, latency, and data gravity. They know that governance and accountability matter more in AI than perhaps anywhere else in modern IT. The role of cloud in the AI race is therefore foundational, but not absolute. Some AI workloads will run in public cloud. Some will be distributed across edge computing environments. Some will remain in private environments for reasons of sovereignty, economics, or control. The key is not to force a universal answer. The key is to create an architecture that aligns AI ambitions with operational reality.

Cloud should play the role it has gradually earned: not as a religion, but as a strategic utility. For AI, the cloud is where many enterprises will source scale, experimentation speed, global reach, and managed innovation. The winning organizations understand where cloud creates leverage and where other operating models make more sense.

Changing how enterprises think

The real story of the past 20 years is not just that AWS launched S3 and helped popularize infrastructure as a service. It is that cloud changed enterprise behavior. It normalized service consumption over asset ownership. It moved architecture toward abstraction, automation, and modularity. It forced IT organizations to broker capability rather than build everything from scratch. It redefined speed as a core competitive requirement.

And now, as AI becomes the next forcing function, cloud stands less as a novelty and more as the platform on which the next era will be built. That is a remarkable outcome for something that, in many ways, started with the old idea that computing could be delivered remotely on a subscription basis. We have been heading here for longer than many people realize. In the past two decades, led in large measure by AWS and the broader hyperscale movement it accelerated, cloud has evolved from a gamble to an indispensable foundation.

Hard to believe? Yes. But also inevitable in retrospect.

We’re seeing an interesting infrastructure tug of war today where GPU clouds are being pulled in two directions. For the economics of AI to work, the enterprise market needs to carve expensive hardware into smaller, shareable units and hand it to customers on demand, similar to how CPUs are doled in public cloud infrastructure. But the more the providers push GPUs to behave like elastic cloud infrastructure, the more they run into the reality that this GPU hardware was never built for safe multitenant use, fast fault recovery, or clean isolation between workloads. That tension is becoming one of the defining operational problems of the AI infrastructure market.

When a gamer launches Steam or the Epic Games Store on their laptop, they don’t have to worry about which GPU is being scheduled, how memory is going to be divided, or really any of the security boundaries or hardware assignment issues on their PC. For consumer PCs, these issues are not just hidden from view, they are irrelevant.

But for today’s IT teams managing GPU-driven AI workloads across distributed systems, those types of allocations and partitions need to be managed manually and carefully. This includes deciding which GPU to assign to each workload, how to divide memory, how to isolate tasks, and how to maximize utilization of this very expensive hardware. That’s why you heard so much about “Day 2” AI infrastructure themes around Nvidia’s GTC event this year.

The legacy hardware bottleneck

GPUs were originally developed to speed up graphic rendering and to perform local compute via shaders in service of graphical rendering. Their design assumes a trusted computing environment in which a single application controls the device. When a user runs an application on a GPU, it accelerates that particular workload.

But while GPUs are optimized for throughput, and therefore have thousands of simple cores designed to execute the same instruction over large datasets, this design paradigm creates several major technical limitations regarding context switching and memory isolation. GPUs were designed to produce pixels, not to run sensitive AI applications from multiple tenants using the same hardware.

As a result, GPU infrastructure today behaves less like elastic cloud infrastructure and more like carefully managed hardware appliances.

The partitioning paradox

Today’s AI infrastructure requires GPUs to behave like shared, elastic cloud resources. As inference workloads begin to outstrip large-scale training runs, the ability to slice and share expensive hardware among multiple tenants in real time while maintaining acceptable fault tolerance is no longer optional.

Hardware vendors have introduced new approaches to dividing GPUs into multiple isolated compute slices. Other frameworks approach partitioning through schedulers and container runtimes. But resource partitioning is just one slice of the overall GPU operations pie.

There is currently no widely adopted, cross-vendor operating model for achieving this safely at scale. Most providers are faced with either dedicating a single customer to a physical machine (thus wasting available capacity), or accepting the multi-tenancy security risks currently without a known solution. The current engineering challenge has moved from beneath the model layer and into the infrastructure layer. Success now relies on the ability to quickly launch new workloads and to rapidly contain hardware faults so that a single GPU failure does not bring down all workloads running on the server.

Untrusted code and tenants

The vast majority of current GPU programming models rely on the idea that the driver has complete control over memory protection and that no user will act maliciously. Unfortunately, that assumption completely falls apart in a cloud environment where one VM or container can leave behind data remnants in memory that another VM or container may access. Especially considering that how GPUs execute code is often completely opaque.

A single faulty workload or a single faulty driver failure in a shared GPU environment can also take down every workload (job) that was being run on the same server, increasing the amount of damage caused by an operational failure.

Currently, there are immature options for runtime inspection or behavioral auditing, limiting both visibility and control for security teams. GPU drivers provide a large attack surface and generally limited telemetry from the hardware. In these shared environments, embeddings, weights, prompts, and tokens are now all sensitive data points, creating significant blind spots for those attempting to protect intellectual property.

The high cost of cold starts

The real constraint in many GPU clouds is not model performance but operational efficiency. Right now GPU operations looks like 30 minute tenant spin-up times, 70% idle rates, and engineers continuously debugging infrastructure stacks. Today’s GPU clouds are stalled not due to inferior models, but because the infrastructure layer underpinning these clouds was never designed to support such a high degree of scale.

A 30 minute cold start is a fundamental limitation on the modern AI business model. Those GPU clouds that can spin up workloads in seconds will ultimately win against those that do not. Multitenancy is the only viable means of producing sufficient unit economics to make this very expensive hardware viable for the long term.

Bridging the orchestration gap

Platform teams are beginning to recognize that GPUs require a specialized operating layer between the hardware and workloads. Operators need a unified operating model that supports multiple hardware vendors and GPU models. Cloud providers need a method to safely slice and share servers among tenants, and to prevent cascading failures, while rapidly launching new workloads.

Enterprises are increasingly seeking to run sensitive AI applications with stronger isolation guarantees, and thus are turning to newer categories of software designed specifically to manage the “dirty work” of hardware orchestration.

The race to make GPUs more operational

Prior to Kubernetes standardizing container orchestration, the industry was constantly debating the efficiency of container scheduling and bin packing across clusters. Those operational concerns were eventually automated and incorporated into infrastructure layers that made the complexities invisible to the end user. 

A similar evolution is occurring around GPUs today. While platform teams continue to argue over placement strategies and memory tuning, these decisions will likely be automated within five to 10 years. As AI infrastructure evolves, the most valuable layer may not be the GPUs themselves but the operating layers that make them secure, elastic, and efficiently sharable. So the winners in the AI race won’t necessarily be just those with the most silicon, but those who have the best operating models for making that silicon secure and elastic.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

Licensing can be complicated, particularly when enterprises are forced to double-pay because the software they already own is only licensed for a specific environment, and moving it requires a whole different licensing model. Without proper portability rights, they need to make additional financial investments to run the same workloads in a different home.

AWS says its new Bring Your Own Media (BYOM) service eliminates this duplication. Customers can now reuse their existing Microsoft SQL Server media and licenses on Amazon Relational Database Service (RDS) with no additional licensing fees.

This means enterprises no longer have to justify workload migrations to the cloud against existing licensing commitments and infrastructure investments, AWS said.

“What used to be a licensing barrier between operational SQL Server data and agentic AI is now gone,” AWS data engineer Srikanth Katakam and product marketing manager Colleen Betik wrote in a blog post announcing the service.

Avoiding the double-licensing problem

To drive true value, agentic AI apps must have access to elastic GPU capacity as well as direct, low-latency access to the data they need to reason and make decisions. But, AWS argued, that can be difficult in self-managed data centers and on premises infrastructure with limited access to agentic AI cloud-native services. But a lot of enterprise data lives in Microsoft SQL Server, and until now customers have had to pay for a second license to use a fully managed cloud service like RDS.

Now, with BYOM on Amazon RDS for SQL Server, enterprises can reuse their existing SQL Server Enterprise Edition or Standard Edition licenses through a “lift-and-shift” model. This brings their data into a fully-managed environment.

According to the licensing distribution terms, enterprises must provide their licensed SQL Server Release to Manufacturing (RTM) media to Amazon RDS. They can then upload that media to Amazon S3 and launch BYOM instances. Enterprises must configure AWS License Manager to perform automatic instance tracking.

From there, RDS automates patching, backups, high availability, and monitoring, while providing direct access to agentic AI and analytics services native to AWS. The platform also reports vCPU usage to provide visibility into SQL Server license usage, Katakam and Betik wrote.

“You do not need to choose between protecting your SQL Server licensing investments and giving your data a path to the AWS analytics and agentic AI services redefining what’s possible in the cloud,” they said. 

It’s important to note that this service is only available to enterprises with Microsoft Software Assurance (SA), an add-on which supports mobility and rehosting of existing licenses. Enterprises must verify that their SQL Server licenses comply with Microsoft’s licensing agreement, and that they have a SQL Server License (Standard or Enterprise) with SA. They also must submit a License Mobility Verification Form to Microsoft; it, in turn, will notify AWS once verification is complete.

AWS emphasized that enterprises remain responsible for compliance; it does not block operations if license limits are exceeded.

More control over workloads, loosening Microsoft’s grip

The advantage of this service, explained Mike Leone, principal analyst at Moor Insights & Strategy, is that enterprises can get a Microsoft SQL Server workload onto a managed AWS service without rewriting it for Aurora or paying for the license twice. This means they can modernize on their own schedule rather than being forced into rewrites before they’re ready.

“For a lot of shops, that control over timing is worth more than the license saving itself,” Leone said.

This also loosens Microsoft’s grip on enterprise workloads, because organizations finally have somewhere else to run SQL Server using licenses they already own, he noted. Realistically, their dependency shifts to AWS rather than disappearing entirely, as their data ends up living next to AWS’s AI services.

“For a lot of teams, that’s a trade they’re glad to make for the managed infrastructure and the AI tooling they get in return,” Leone said.

Yaz Palanichamy, a senior advisory analyst at Info-Tech Research Group, also pointed to significant improvements stemming from the migration. Notably, it allows organizations to transition away from static, linear, or reactive storage capabilities towards more dynamically intelligent environments.

“The key is to balance the unification of transactional data towards managed AI and machine learning pipelines,” he said.

Enterprises take on new responsibilities

The catch? Leone noted that enterprises now own the task of licensing compliance. Staying current on SA, the Microsoft maintenance contract that makes any of this legal, and the license mobility rules, “become your headache instead of something baked into the bill,” he said.

Furthermore, he added, lift-and-shift has a way of turning into “lift-and-forget” when enterprises move SQL Server as-is and never actually modernize, “so you end up carrying all the old baggage onto a shinier platform.”

Palanichamy also pointed to skyrocketing AWS costs, since operationalizing AI agents requires a considerable amount of data ingestion and querying. This can potentially be cost prohibitive on RDS for SQL Server.

“One aspect to consider would be the relative time to production value, so that enterprises can better handle the management of volatile AI workloads,” he said.

Just one challenge in the AI race

Although AWS frames licensing complexity as a roadblock to agentic AI, analysts say it’s really just one piece of the puzzle.

Organizations are not in an AI-ready state due to a number of factors, Palanichamy noted. This could be total cost of ownership (TCO)-related, a lack of appropriate acceptable use policies (AUPs), or concerns around security and compliance.

If and when enterprises are ready to adopt AI, they must perform thorough needs analysis/process optimization benchmarking exercises to determine what workflows could benefit from the technology, he said, and, conversely, flag workflows where AI could prove “an impediment or potential disservice.”

Leone also pointed out that moving SQL Server onto RDS doesn’t “magically make your data agent-ready,” nor does it make it smarter. The data proximity and managed plumbing are what actually support agents. Sitting data next to Bedrock and the rest of AWS’s AI services means builders “stop wasting time shipping the data around or building one-off integrations every time an agent needs it.”

Ultimately, though, licensing isn’t the real issue when it comes to AI; it’s more of a migration problem. The real roadblocks are messy data, questionable governance, and teams that aren’t ready to run it in production, and, Leone said, “no change to a license bill touches a single one of them.”

For the past two years, enterprises have focused on grounding AI systems in internal documents, databases and knowledge repositories. Microsoft now contends that the next challenge is giving those systems reliable access to the outside world as they move into production.

At its ongoing annual Build conference, Microsoft unveiled Web IQ, a new suite of AI-native APIs designed to connect AI agents and applications to real-time information from across the web, including web pages, news, images and videos.

The goal is to help developers build more accurate and context-aware AI systems while reducing the complexity of integrating web search, retrieval and grounding capabilities into enterprise applications, the company wrote in a blog post.

The APIs already underpins grounding for Microsoft Copilot and ChatGPT, and unlike traditional search APIs are designed to retrieve highly relevant information while minimizing token consumption, helping reduce both inference costs and response latency, Microsoft said.

Reducing the cost and complexity of web grounding

That focus on reducing inference costs and response latency to deliver Web IQ’s search capabilities will be valuable for CIOs and developers, said Phil Fersht, chief analyst at HFS Research.

“Developers have typically stitched this together themselves using search APIs, web scraping, retrieval-augmented generation, vector databases, custom ranking logic, crawling tools and separate orchestration layers. That works, but it is messy, brittle and expensive to maintain,” he said.

“The hard part is not just finding a web page. It is retrieving the right evidence, ranking it, selecting useful passages, reducing token waste, respecting publisher controls and doing all this fast enough for multi-step agents,” he added.

An agent running in production will typically run at least five or ten retrieval steps and in such scenarios latency and cost can become “ugly,” said Moor Insights and Strategy principal analyst Mike Leone.

“I’ve watched plenty of teams handle this by gluing a search API onto a model and hoping the model can sort through whatever HTML comes back, which isn’t ideal,” he said.

Simplifying development of applications

Beyond the infrastructure and cost advantages, Web IQ could also simplify the AI application building process developers, said Stephanie Walter, practice lead of the AI stack at HyperFrame Research.

“The value of Web IQ is that Microsoft is trying to make web grounding a reusable, agent-native service rather than a bespoke integration,” Walter said.

That reusability, Fersht said, will matter to CIOs as most enterprises do not want every development team rebuilding its own web grounding stack: “They need a common capability that is governed, scalable, low-latency and economically efficient.”

Still, Microsoft is not entering an untapped market and there are several offerings already available, including OpenAI web search, Google grounding, Perplexity APIs, Bing Search APIs, Azure AI Search, vector databases, and custom RAG pipelines.

However, Microsoft might have an advantage with IQ, despite Google’s strength in search and web-scale knowledge, OpenAI’s tooling push, and AWS’ Bedrock-data-service-partner integrations combo, Fersht said.

“Microsoft’s differentiation is that it can combine Bing’s global web index with Azure AI, Copilot, Foundry, Microsoft 365 and enterprise developer channels,” he said.

The other advantage is the existence of existing IQ offerings, such as Work IQ, Fabric IQ, and Foundry IQ, all of which are targeted towards adding more business context for agentic systems, Fersht added.

But uptake, Walter said, will depend on whether these integrations translate into better production outcomes.

Web IQ is currently available in limited access for select Azure customers. Enterprises can request access to Web IQ via their Microsoft account team or submit a form.

Beyond direct API access, developers can also configure the model-agnostic Web IQ as an MCP tool within Foundry IQ, a Microsoft spokesperson said.