arXiv AI By Zehao Xu, Tony Paek, Kevin O'Sullivan, Attila Dobi

Calibrate Globally, Measure Everywhere: Scaling LLM-Based Prevalence Measurement Across A/B Experiments

Read the original on arXiv AI →

arXiv:2602. 16111v2 Announce Type: replace-cross Abstract: Online media platforms track the share of impressions associated with content attributes, or prevalence, to evaluate trade-offs and set guardrails in A/B experiments.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning
Jul 30

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv:2607. 27083v1 Announce Type: new Abstract: As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure.

By Yicheng Feng, Yan Zhang, Yan Cheng, Wei Qi
arXiv AI
Aug 6

ORCA-bench: How Ready Are Language Model Agents for Oncall?

arXiv:2607. 28545v2 Announce Type: replace-cross Abstract: Large language models can write, patch, and search code, but oncall root cause analysis (RCA) demands something different: reasoning over noisy metrics, logs, traces, and source code, starting from ambiguous user-facing reports, often hours after the incident began.

By Albert Gong, Kyuseong Choi, Abhineet Agarwal, Jason Schechner, Ryan Huang, Raj Agrawal, Anish Agarwal, Raaz Dwivedi