My current model-selection strategy is embarrassingly simple… and wrong: I choose the most capable (expensive) model because I’m worried the cheaper one might get something wrong. That makes me spend too much, of course, which is why I (like you, perhaps?) continue to struggle with one of the most fundamental challenges in AI: How can I know when a cheaper model is sufficient for a task? Or, really, which model should I use at all?

I asked a friend, Leo Zheng, who leads marketing for Fireworks AI, an AI infrastructure company that runs and improves open-weight models. Surely it was his job to know? His answer surprised me. I thought the answer would come down to models, but it doesn’t. Fireworks, he said, wants to “enable every company to own the continual learning loop within their four walls.” The idea is to abstract away model updates while companies feed the system new signals as customer behavior changes.

That’s when it clicked. I was asking how to automate model choice, but the harder problem is building a feedback loop that tells a company what worked. Model choice then becomes simply one important action the system can take, not the be-all and end-all decision a developer must get right in advance. Arguably, the more important component is integrating enterprise data into that continual learning loop that Zheng describes.

I’ve made that argument before, and I was right. Sort of. Proprietary data doesn’t automatically guarantee AI success. Neither does dumping that data into a retrieval-augmented generation (RAG) system. Your competitors can do the same. The more durable advantage comes from outcome data: traces that connect what the system saw, what it did, and whether the business result was acceptable. Those traces become valuable when a company uses them to improve the system and carry the learning to the next model.

Make the data earn its keep

Public benchmarks are useful, but they only tell you how a model performed on someone else’s test. They can’t tell a bank whether a fraud alert was correct, inform a retailer whether an AI agent resolved a return, or tell a software company whether AI-generated code survived review. Enterprises already have pieces of those answers buried in support transcripts, accepted patches, abandoned carts, etc. The trick is turning that mess into something a system can learn from.

Think about a software company. It already collects product analytics to improve its product. AI can extend that flywheel. Application signals can improve the system powering the product. How? By capturing user intent, engagement, and preferences.

This isn’t foolproof, however. A fast learning loop built on a weak proxy is simply a fast way to make the system worse. For example, suppose Toni in customer support accepts a suggested response. Was it good? Maybe. But what if Toni was rushing (as she is prone to do) and the customer reopened the ticket the next day? It’s not enough to act on inputs alone; a company needs to connect the suggestion to the eventual outcome or risk rewarding the wrong behavior.

Acceptable or ideal outcomes must be defined by the enterprise. A platform like Fireworks can optimize through learning loops, but only if desired outcomes are defined. This changes the cost calculation. After all, the cheapest model by token can be the most expensive model by outcome if people must review, retry, and repair its work. A company can be hemorrhaging money on AI failures while congratulating itself for lowering inference costs.

Open weights get useful

None of this requires open weights, of course. A company can own its evaluations, improve prompts and retrieval, and route among closed APIs and open models without changing a single parameter. For many workloads, that will be enough.

This generally won’t start with reinforcement learning, either. It starts by picking one workflow with an observable outcome and logging the request, model, cost, answer, and what happened next. Do that for a quarter and patterns should emerge. A cheaper model will be good enough for some work, while a proprietary frontier model will earn its price elsewhere. However, in a third category, every model may fail in the same repeatable way, and it’s this last category that makes the serious case for changing the weights.

A few years ago, I argued that open source succeeds when it makes developers’ lives easier, not when it wins an ideological purity contest. The same applies to open weights. The importance of open weights isn’t that they’re a cheaper facsimile of whatever OpenAI and Anthropic are shipping. Instead, they matter because they shift leverage away from the models and toward users with proprietary outcome data. They let enterprises turn what they learn into something they can shape, keep, and carry forward as models change.

Open weights turn model renters into model owners. That’s a very big deal.

For Cursor’s Composer 2, the company started with the open-weight Kimi K2.5 model, then specialized it through continued pretraining and reinforcement learning in environments built to resemble Cursor itself. The interesting thing isn’t that Cursor picked the “right” model, but rather that Cursor can use what it learns from developers to keep making that model more “right” for them. In this case, Fireworks ran the reinforcement learning inference, while Cursor supplied what only Cursor could: its product environment and definition of good. Cursor subsequently built Composer 2.5 on the same Kimi K2.5 checkpoint and says 85% of its compute went into additional training and reinforcement learning.

The base model was the starting point, not the finished product.

This is why open weights matter. They turn the model from a finished product into raw material. The foundation labs have broad knowledge, far more than any normal company could hope to collect. What they don’t know is whether your patch shipped, your fraud was stopped, your customer stayed, or Toni’s response made matters worse. You do.

RAG lets a model consult your data. A learning loop lets the system learn from it. One helps answer today’s question, but the other helps the company answer tomorrow’s question better.

Control must be convenient

Raw weights alone won’t deliver that control. Open source didn’t take over the enterprise because CIOs wanted tarballs. It won because Red Hat, and especially cloud providers, turned theoretical control into everyday convenience. Open-weight AI needs the same translation.

Enterprises want the convenience of a closed API without surrendering control over the intelligence their data creates. That’s the real Fireworks opportunity. Not faster tokens, but a shorter distance between learning something about your customers and making the product smarter because of it. Fireworks CEO Lin Qiao calls this “application-specific intelligence,” and that’s exactly right. Enterprises don’t need to outbuild OpenAI or Anthropic. Even if they wanted to, they couldn’t. Instead, they need to turn what they know about their customers and workflows into intelligence those labs can’t supply.

Fireworks isn’t alone. Together AI also spans training, evaluation, and serving, while Baseten now talks about connecting production feedback to model training. This is becoming a category, not simply a Fireworks feature. Fireworks’ particular bet is that the entire learning loop should belong to the customer.

So, back to the question I was asking Zheng: How do I pick the right model in advance? By now it’s clear: I can’t. Neither can you, and that’s fine, because it’s not really the right question.

Once the enterprise owns the test, it no longer has to trust a vendor’s benchmark or brand. New models must prove themselves against the work that matters to that company, then be promoted, specialized, or discarded accordingly. Model choice becomes a consequence of learning, not an act of faith. In effect, the models audition for the enterprise, not the other way around.

No one can predict whether OpenAI, Anthropic, Google, or the open-weight community will lead six months from now. The sensible response is to stop making that prediction central to the strategy. The enterprise’s own data should decide which model to pick. That’s the bet I find compelling in Fireworks: Make open-weight models as easy to improve as APIs are to call, so enterprises can stop treating their data as context for somebody else’s model and start using it to build intelligence they control.

Microsoft is betting that the integration of HorizonDB, the cloud-native PostgreSQL alternative it is developing, with Azure will attract more enterprise AI and agentic workloads to its cloud services.

Enterprises may not be willing to take that bet.

It’s been nine months since Microsoft unveiled HorizonDB, but the service remains in public preview with no announced general availability date. Why put AI projects on hold waiting for HorizonDB to arrive, when AWS, Google, Databricks, Snowflake, and others already have production-ready PostgreSQL services positioned for the same AI workloads that Microsoft says it is building HorizonDB to handle?

AWS has had the longest head start. Aurora PostgreSQL became generally available in 2017 and has since evolved from a cloud-native PostgreSQL database into an AI-ready service with vector search and integrations with Amazon Bedrock. Similarly, Google’s AlloyDB, which followed in 2022, now includes AlloyDB AI with vector search, embeddings and model interaction for generative AI and agentic applications.

Databricks and Snowflake, too, have their own platform-centric services in the form of Lakebase, which became generally available on AWS and Azure this year, and Snowflake Postgres, which was made generally available in February 2026.

As the latecomer, when Microsoft pitched HorizonDB at Ignite in November 2025 it talked up its new architectural approach to cloud-native PostgreSQL, built around disaggregated compute and storage and a database-as-log design. The hyperscaler also positioned native vector search and deep integration with Foundry and Fabric as key differentiators for AI-heavy workloads.

No reason to wait

Those architectural differences may not be compelling enough for CIOs to wait for HorizonDB to become generally available, though.

“Most enterprises with urgent needs will not wait. A long preview window creates uncertainty around SLAs, pricing, operational maturity, and roadmap confidence,” said David Linthicum, an independent cloud consultant.

And, said Stephanie Walter, practice lead of AI stack at Hyperframe Research, enterprises cannot build mission-critical production plans around an undefined GA date, regional footprint or support commitment.

Given the difficulty of unwinding a poor database choice, enterprises will approach unknown quantities with caution.

“Database platforms eventually become sticky control points. Once the database is connected to the rest of the application, analytics, AI, and governance stack, switching becomes a business transformation rather than just an infrastructure swap,” said Michael Ni, principal analyst at Constellation Research.

In the case of a cloud database, there’s also the unwelcome possibility of “huge egress fees” in case of change, said Bradley Shimmin, lead of the data and analytics practice at The Futurum Group.

All that uncertainty is likely to lead enterprises to restrict HorizonDB to experimental use cases for now, Shimmin added.

Performance anxiety

Analysts also questioned whether HorizonDB’s technical differences will show up in performance benchmarks.

Microsoft has said HorizonDB can deliver up to three times the throughput of open-source PostgreSQL, but makes no comparisons with rival offerings such as Aurora or AlloyDB that it will compete with, Walter said.

The bigger question, according to Igor Ikonnikov, advisory fellow at Info-Tech Research Group, is whether those performance advantages, still largely on paper, translate into a meaningful difference in production.

“A database with a better compute benchmark can still be more expensive once resilience and ecosystem costs are included,” Ikonnikov said.

The economics also point to another HorizonDB limitation, particularly for workloads that are not continuously running, said Advait Patel, senior site reliability engineer at Broadcom.

HorizonDB currently uses provisioned compute rather than a serverless, scale-to-zero model, meaning customers continue to incur compute charges while an instance is provisioned, even if its workload is intermittent or idle, Patel said.

There are developer considerations too.

HorizonDB’s PostgreSQL compatibility does not necessarily mean every existing PostgreSQL application will move cleanly as in its current form the database supports only an approved set of PostgreSQL extensions rather than arbitrary ones, Walter said.

Who should wait?

For enterprises already deeply invested in Microsoft’s Azure ecosystem, those limitations may not be enough to rule out waiting for HorizonDB, Patel said: The chance to integrate the database with AI services and the wider Microsoft stack may outweigh immediate availability, he added.

That calculus also reflects how enterprises typically make database decisions in the first place: not by comparing databases in isolation, but by weighing how well they fit into the broader technology stack, including the cloud platform they have standardized on, Ikonnikov said.

For Azure shops, the choice may therefore be less about moving an existing workload away from Aurora or AlloyDB and more about whether a new Azure workload should start on Azure Database for PostgreSQL today or wait for HorizonDB when it becomes available, he said.

That may be an open question for some enterprises, said Devin Pratt, research director at IDC. “Plenty of organizations are still mid-decision, not locked in,” he said.

Microsoft finally offers a timeframe

Microsoft still won’t say exactly when HorizonDB will launch, with Shireesh Thota, corporate vice president for Azure Databases at Microsoft, saying only, “General availability for Azure HorizonDB is currently targeted for the second half of 2026.”

That narrows it down to a period of a little over four months, including Microsoft’s FabCon and Ignite conferences — an eternity in AI.

For years, AI progress was measured by scale: more data, larger models, and more compute. That formula produced real breakthroughs, but autonomous mobility was one of the first industries to discover its limits.

On the road, AI does not fail quietly. A misread scene, an ambiguous gesture from a pedestrian, or a construction zone that does not match prior examples can create immediate and visible risk. That pressure forced autonomous mobility teams to confront a reality the rest of enterprise AI is now beginning to face: the hardest problem is not access to models. It is reliable ground truth.

As AI moves from pilots into production systems, organizations are discovering that performance depends not only on model capability but on the quality, consistency, and defensibility of the data used to train, evaluate, and improve those systems.

The scaling myth is breaking in production

Early autonomous vehicle development followed a familiar playbook: collect more data, train larger models, and improve performance over time. That approach worked to a point. But real-world driving data does not behave like benchmark data. Road environments are messy and unpredictable. Human behavior is inconsistent. Context changes quickly. Many situations are ambiguous even to trained human observers.

As datasets grew, teams often discovered that they were not simply collecting more signal. They were also collecting more contradictions. Different teams interpreted the same scenes differently. Edge cases accumulated, and ambiguity multiplied. Instead of producing clarity, scale sometimes introduced confusion.

The same pattern is now repeating across enterprise AI. Organizations have access to increasingly capable foundation models, but many AI initiatives still struggle when deployed into production environments. Production exposes weaknesses that benchmarks rarely capture: inconsistent data, ambiguous edge cases, shifting context, and gaps between model behavior and human expectations.

The result is a growing gap between model access and operational reliability. More data is still valuable, but only when it is curated, consistently interpreted, and tied to clear operational definitions. In many production settings, low-quality or inconsistently interpreted data introduces noise faster than models can resolve it.

Real-world AI is multimodal by nature

Autonomous mobility also exposed another reality that is now spreading across AI: real-world intelligence is multimodal. A vehicle does not understand the road through images alone. It must reconcile camera feeds, LiDAR, radar, maps, localization signals, motion history, weather conditions, and human behavior into one coherent interpretation of the scene.

The same requirement is emerging across other high-consequence AI domains. In healthcare, systems may need to connect medical imaging, clinical notes, lab results, and patient history. In agriculture, models may combine satellite imagery, drone footage, soil data, weather patterns, and field observations. In manufacturing and robotics, AI systems increasingly need to reason across video, sensor telemetry, 3D spatial data, maintenance logs, and human instructions.

This raises the standard for ground truth. The question is no longer simply whether an object is correctly labeled in an image or whether a text response is accurate. The harder question is whether the system correctly understands a situation across multiple signals, some of which may be incomplete, noisy, contradictory, or changing over time.

Multimodal AI does not just need aligned datasets. It needs aligned interpretation, and that cannot be engineered into the model after the fact.

Edge cases are not the exception

One of the most important lessons from autonomy is that edge cases are not a small part of the problem. In high-stakes AI, they are the problem.

AI systems often perform well on averages, but real-world systems fail on exceptions. In autonomy, those exceptions may include a pedestrian behaving unpredictably, a construction zone that does not match prior examples, a partially occluded object, unusual road geometry, or an interaction where human intent is unclear. These situations represent a small fraction of total driving events, but they dominate risk.

The same principle applies beyond mobility. In healthcare, it may be a rare presentation of disease. In finance, it may be an unusual transaction pattern. In manufacturing, it may be a combination of operating conditions never previously encountered. In robotics, it may be a physical interaction that looks simple in simulation but behaves differently in the real world.

Organizations often discover that the final increments of reliability require dramatically more effort than the first 90% of performance improvement. This final gap is why many teams find themselves stuck in “hill climbing” — expending enormous effort for increasingly marginal gains.

In multimodal systems, this effect compounds. Each additional sensor stream introduces new edge-case permutations, and reconciling conflicting signals requires expert judgment.

Ground truth must be defensible, not just labeled

Autonomous-vehicle programs that got this right learned that ground truth cannot be treated as static input: a label, a bounding box, or a classification. In the scenarios that mattered, the correct interpretation of a scene had to be reasoned about, reconciled across sensor streams, and defended against alternative readings.

The same standard now applies in other critical settings. A medical annotation may require reconciling imaging findings with clinical context. A financial model’s training data may need to account for regulatory intent, not just transactional patterns. A robotics system operating in an unstructured environment may need ground truth that captures not just what happened, but the chain of causation leading to that moment.

In these cases, ground truth is not simply data that has been labeled. It is data that has been structured, reviewed, and validated through expert judgment. It needs to explain not only what the correct answer is, but why that answer is correct.

This distinction matters because AI systems increasingly operate in environments where being approximately right is not enough. The more consequential the use case, the more important it becomes to create ground truth that is consistent, auditable, and operationally meaningful.

The shift from annotation to judgment

Perhaps the most significant transformation happening in AI today is the evolution of human involvement. The data labeling industry is moving away from simple, crowd-sourced annotation toward more specialized work: scenario design, failure analysis, model evaluation, red teaming, reasoning validation, and edge-case identification.

This work requires people who understand context, ambiguity, intent, and risk. In high-stakes environments, the key questions are not simply whether a system reached the correct answer. Organizations also need to understand why the system behaved the way it did, whether that behavior can be explained, and how to prevent similar failures in the future.

Trust cannot be added through marketing, interface design, or post-hoc explanation alone. It needs to be engineered upstream, embedded throughout the life cycle from data creation to training, evaluation, monitoring, and auditing.

The difference between a model that looks correct and a model that behaves reliably often comes down to whether expert judgment has been systematically incorporated into development.

Software provides scale, and expertise provides judgment. Reliable AI needs both.

What autonomous mobility teaches the rest of AI

Autonomous mobility is often treated as a specialized vertical. It should be treated as an early warning system.

It showed what happens when AI enters environments where the cost of error is high, edge cases dominate risk, multimodal evidence must be interpreted correctly, and opaque decisions are unacceptable. That is exactly where enterprise AI is now heading in healthcare, finance, critical infrastructure, robotics, agriculture, and other high-consequence domains.

The lesson from autonomy is not limited to vehicles. It is that real-world AI must handle ambiguity, edge cases, and multimodal evidence at the same time. Models can generate outputs, but defining what is acceptable remains a human and operational decision.

The next major leap in applied AI will not come from model size alone. It will come from the systems that create, validate, and continuously refine defensible ground truth.

Those systems depend on three things: high-quality data, expert human judgment, and disciplined operational processes. The organizations that recognize this early will have a significant advantage as AI moves from impressive demonstrations to dependable real-world systems.

Autonomous mobility learned this first.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.