arXiv:2609.33153v2 Announce Type: replace-cross
Abstract: Reported improvements from tools and reusable skills in large language model agents refer to different comparisons. This critical narrative r...
By Shuyang Zhang
arXiv:2607. 26587v1 Announce Type: cross Abstract: Automated research systems use experimental scores both to deliver artifacts and to decide which ideas to retain, transfer, and pursue.
By Jingjie Ning, Shanshan Zhong, Xiaochuan Li, Ji Zeng, Chenyan Xiong
The paper presents a layered framework for evaluating conversational AI by aligning offline proxy signals with online A/B experiment outcomes. It introduces a three‑step alignment chain—behavioral label to product outcome, classifier to candidate behavior, and offline signal to experiment effect—alongside an audit protocol that compares confidence intervals and rankings. In a real‑world deployment, the composite proxy achieved 81.1% F1 versus 34.3% for the raw classifier, correctly predicting direction on all 113 contrasts and enabling efficient prioritization of candidate models before costly online testing.
By Xuanyi Li, Vaskar Nath, Hossein Amirkhani, Jay Li, Alex Deng
arXiv:2609.15624v1 Announce Type: cross
Abstract: Researchers assessing competent generative-AI use at work must choose among self-reports, objective tests, and measures of oversight and reliance. We...
By Daniele Veri'
arXiv:2608. 01366v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts.
By B. Sankar, Pawni Yadav, Srinidhi Ranjini Girish, Amogh A. S
The paper re‑evaluates memory‑based self‑improving agents by adding multiple runs to measure variance and by randomizing task order. It finds that agent performance is noisy in complex, multi‑step environments and that improvement depends heavily on the sequence of tasks, revealing a hidden curriculum effect. The authors suggest that underspecification of tasks and environments contributes to this fragility and demonstrate that adding detailed rubrics and feedback can partially mitigate performance drops, though gaps remain.
By Qinyuan Ye, Yu Li, Yada Pruksachatkun, Jiaxin Zhang, Chien-Sheng Wu