Microsoft has open-sourced an AI evaluation framework that converts natural-language requirements into executable tests, expanding its push into enterprise AI governance as organizations struggle to validate agent behavior before production deployments systematically.

The framework, called ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), generates evaluation scenarios, datasets, metrics, and scorecards from written specifications, product requirements, and governance documents, Microsoft said in a blog post announcing the release.

“Agents fail in ways that are hard to see,” Microsoft wrote in the blog post. “They drift from policy, produce unsafe outputs in edge cases, and behave differently in production than they did in testing. Generic benchmarks do not catch these failures because they are not built around your policies, your agent, or your use case.”

Rather than requiring developers to manually create evaluation suites, ASSERT translates written intent into reusable tests that can be integrated into AI development pipelines, the company said in the blog post.

With ASSERT, Microsoft is entering an increasingly competitive AI evaluation market that already includes platforms such as LangChain’s LangSmith, Braintrust, Patronus AI, Galileo, Arize AI’s Phoenix, and Promptfoo, which help enterprises benchmark, monitor, and validate large language model applications.

Behavioral testing remains immature

The release comes as enterprises rapidly expand AI agent deployments while formal evaluation practices remain the exception rather than the rule.

“Most organizations, in fact, 99% of them, do not evaluate any AI agents pre-production,” said Anushree Verma, senior director analyst at Gartner.

According to Verma, the industry’s next competitive advantage will depend less on advances in reasoning models than on how effectively organizations simulate and stress-test AI agents before deployment.

“The next competitive moat in agentic AI is not about the sophistication of reasoning models or the underlying architecture,” she said. “It will be about the depth and realism of the training environment through agentic simulation, particularly for mission-critical deployments.”

Gartner estimates that by 2029, more than 75% of domain-specific agents designed without agentic simulation in regulated industries will fail to deliver value.

Forrester sees enterprises moving toward behavioral evaluation but says most organizations have yet to make it a formal production requirement.

“Most enterprises are still in an intermediate stage where behavioral evaluation is inconsistently applied rather than treated as a formal production gate,” said Biswajeet Mahapatra, principal analyst at Forrester.

According to Forrester data, more than 45% of organizations are already using AI agents, and another 25% are piloting them, yet many continue to struggle with scaling because of immature governance and limited operational rigor.

“The net is that behavioral evaluation is becoming important, but for most organizations it is still ad hoc or tool-driven rather than a standardized release gate enforced across the lifecycle,” Mahapatra said.

AI judges still need human oversight

Microsoft said ASSERT uses large language models as judges, with model-generated evaluations agreeing with human reviewers 80% to 90% of the time in the company’s internal validation.

That level of agreement can help automate large portions of AI testing, but should not be treated as a standalone governance mechanism, Mahapatra said.

“An 80% to 90% agreement rate with human reviewers indicates strong alignment but is not sufficient as a standalone control for governance or compliance,” he said.

Instead, enterprises should adopt layered oversight where AI evaluates AI at scale while humans retain supervisory accountability for high-risk, regulated, or ambiguous scenarios. Buyers should also watch for bias, consistency issues, and overreliance on a single model acting as both generator and evaluator, he added.

Open source reduces lock-in, not governance risk

Microsoft released ASSERT under the MIT open-source license, allowing organizations to inspect, modify, and integrate the framework into existing AI development workflows.

But open sourcing a framework does not eliminate questions around evaluation neutrality, Mahapatra said.

“Open sourcing under an MIT license reduces lock-in concerns and enables broad interoperability across model ecosystems,” he said. “However, it does not fully eliminate trust or conflict-of-interest questions because the originating vendor still influences how evaluation criteria, scoring logic, and definitions of acceptable behaviour are encoded.”

Instead of relying on a single evaluation framework, enterprises should validate AI systems against multiple evaluation approaches and retain ownership of internal evaluation policies, he said.

Databricks on Wednesday unveiled OpenSharing, a new open protocol designed to let enterprises share AI models, agent skills, dashboards, and unstructured data across platforms without having to copy or move those assets.

That sharing is made possible by OpenSharing’s zero-copy credential vending model that allows recipients to securely access shared assets directly from a provider’s cloud storage using temporary, scoped credentials rather than requiring the assets themselves to be copied, moved, or replicated, the company wrote on its GitHub page.

Reducing the integration tax of enterprise AI

The ability to share AI assets without creating duplicate copies could help reduce integration complexity, improve governance, and limit the operational overhead associated with operationalizing AI systems across environments for CIOs, said Ashish Chaturvedi, leader of executive research at HFS Research.

“Every organization building AI, such as multi-agentic systems, is hitting the same wall, i.e., the model, the skill, and the consumer reside on three different platforms. The integration tax is enormous, and it grows exponentially with every new partner, customer, or internal team,” Chaturvedi said.

Echoing Chaturvedi, The Futurum Group’s lead of the CIO practice, Dion Hinchcliffe, pointed out that the reduction in operational overhead could help CIOs cut down on the hidden costs of integration around AI deployments: “Today, hidden costs include more than just model development. It is the endless packaging, translation, sync, and governance effort required to operationalize AI assets across organizational boundaries.”

From data sharing to AI asset sharing

That cost reduction is becoming even more important because enterprises are beginning to treat AI assets as business assets that need to be shared, said Stephanie Walter, practice lead of the AI stack at HyperFRAME Research.

“Enterprises are quickly realizing that the value is no longer just in the dataset. It is in the governed context, logic, and intelligence built around the dataset. Existing approaches can share datasets well, but they often do not address the broader AI package,” Walter said.

“OpenSharing is directionally aligned with that shift because it extends the sharing model beyond tables and files toward the artifacts that power AI workflows,” Walter added.

For Hinchcliffe, that alignment should work in CIOs’ favor, trying to operationalize AI across their systems: “CIOs increasingly want AI supply chains, not isolated data lakes like before.”

Additionally, Chaturvedi pointed out that the new protocol can help CIOs accelerate the monetization of AI investments.

“For CIOs, the speed at which you can share AI assets across partners, subsidiaries, and customers determines the speed at which you can monetize your AI investments. If sharing an agent skill takes six weeks of integration work, you’ve lost the window. If it takes a protocol call, you’ve turned AI into a distribution business,” he said.

How OpenSharing could simplify AI development

Achieving those benefits, however, will require developers to move AI assets across disparate platforms more efficiently, and analysts pointed out that OpenSharing’s ability to reduce integration complexity could significantly improve productivity.

“Developer productivity depends on reducing platform translation work. Developers do not want to rebuild the same asset for every consuming environment, and enterprises do not want every partner or customer interaction to become a platform migration conversation,” Walter said.

In fact, Chaturvedi sees the new protocol as unique in the industry, in the specific sense that “no other open protocol covers agent skills and AI models as shareable, governed objects”.

Walter, in contrast, sees the openness of the protocol as novel: “What is more interesting is the combination: an open protocol, cross-platform interoperability, Linux Foundation governance, and a broader asset model that extends beyond datasets into AI models, agent skills, dashboards, applications, and unstructured data.”

“The novelty is not that Databricks invented sharing, zero-copy access, or marketplace-style distribution. Those capabilities already exist in various forms across the market,” Walter said, pointing towards Snowflake’s offerings, such as the Zero-Copy integrations.

The difference, though, the analyst noted, is that Snowflake allows data to be copied only if both provider and receiver are on Snowflake.

With Databricks’ OpenSharing, data can be copied across platforms, Walter added.

OpenSharing, which is an evolution of Databricks’ existing Delta Sharing protocol, is currently a sandbox project under the Linux Foundation AI & Data Foundation and is available via GitHub.

Its current list of generally available connectors includes Python, Apache Spark, Tableau, PowerBI, Snowflake, DuckDB, Clojure, Node.js, Java, Arcurate, Rust, Go, C++, and R.

Other connectors that are expected to be made generally available soon include Google Spreadsheet, Excel, Airflow, and Lakehouse Sharing.