From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence

The state of enterprise AI

A data-driven look at enterprise AI adoption, showing how organizations move from experimentation to real productivity gains and new capabilities.

Beyond RAGs: Building Actually Truthful AI Harnesses

The article "Beyond RAGs: Building Actually Truthful AI Harnesses" discusses the limitations of Retrieval-Augmented Generation (RAG) systems, emphasizing that retrieval alone does not guarantee evidence for AI claims. It explores methods for constructing AI systems that can substantiate their statements, moving beyond simple retrieval to more robust proof mechanisms. The piece highlights the importance of developing AI that can verify its own outputs rather than merely retrieve information.

Generating a Consistent Enterprise: Synthesis and Reference-Free Evaluation of Multi-System Business Data

The paper introduces a synthetic data generator that creates fully consistent, fictional enterprises—complete with workforce, customers, sales, support, and communication records—without relying on any real dataset. It validates realism through a five‑axis scorecard, an adversarial detector, and soundness checks, achieving a mean realism score of 99.1 across 23 generated companies. A second generator produces relational databases from business questions, ensuring qualifying rows and exact labels, and is available as a hosted service and containerized simulators.

Many Companies Use AI. Few Know How to Build an AI-Native Enterprise Data Platform.

A practical enterprise AI architecture with data agents, AI-powered QA, and AI governance. The post Many Companies Use AI.

Ethics and Society Newsletter #6: Building Better AI: The Importance of Data Quality

One year in: How Microsoft Research Asia – Singapore is advancing research, partnership and talent for real-world impact

Microsoft Research Asia – Singapore, launched a year ago, has built a solid foundation and strengthened collaboration with government, academia, and industry. The lab is exploring how frontier AI research can generate real‑world value. It continues to advance research, partnership, and talent for practical impact.

The Measurement Revolution? Credible Measurement and Inference in the Age of AI

The article discusses how artificial intelligence is reshaping measurement in economics by converting unstructured data into structured variables at low cost, enabling large‑scale measurement that was previously infeasible. It outlines three stages—discovery, construct definition, and observation—where AI impacts the measurement pipeline and stresses the importance of rigorous validation to ensure credible inference. The review offers guidance on navigating the shift from a single scalable measure to multiple plausible ones that can lead to differing empirical conclusions.

Microsoft Research Focus – Edition #97

Microsoft Research Focus – Edition #97 features host Sowbhagya Suresh discussing recent developments across Microsoft. Topics include challenges faced by coding agents, insights into how people use AI at work, and advancements in extended‑range weather forecasting. The episode highlights cutting‑edge research and practical applications within the company.

An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

The paper introduces the Systemic Risk Index, an open pipeline and dashboard that aggregates evidence from 19 public AI benchmarks into four systemic‑risk categories defined by the EU GPAI Code of Practice. It evaluates 18 models using harm‑preserving perturbations and simulated deployment contexts, offering users the ability to switch between average and worst‑case aggregation and to trace each risk rating back to its benchmark evidence. The study finds that worst‑case scores can be 14 to 37 points lower than average scores, and that LLM judges agree with human graders at a level comparable to human‑human agreement.

The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents

The Era by Eon Benchmark is a new dataset for evaluating large language model agents that interact with enterprise tools. It constructs a complete fictional company with product simulators, internal databases, and benchmark questions, all generated from a shared entity graph to ensure consistency. Exact answer keys are computed from the generated records, allowing precise grading and validation of realism and adversarial robustness across 23 simulated companies.

When Compliance Data Masquerades as Evaluation: Measurement Validity for Deployed AI Systems

arXiv:2609.13642v1 Announce Type: new Abstract: We argue that a recurring failure in the evaluation of deployed AI systems occurs when data collected for operational monitoring or regulatory complian...

How UK AISI and EvalEval Are Making Benchmark Results Reproducible