← Back to all news
arXiv AI By Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang, Jett Chen, Michael Ryan, Qianou Ma, Luxi He, Zhoujun Cheng, Andre He, Seungone Kim, Jiayi Geng, Mingqian Zheng, Weiwei Sun, Zheyuan Zhang, Xinran Zhao, Yike Wang, Abe Hou, Liwei Jiang, Pang Wei Koh, Diyi Yang, Graham Neubig, Daniel Fried

Efficient Test-Time Adaptation through Human-AI Interaction

Read the original on arXiv AI →

The paper introduces Test-Time Adaptation through Human‑Agent Interaction (TAHI), a method that uses iterative human feedback to adapt AI agents to individual users’ criteria. By integrating cross‑session interaction data into agent context and weights, and building an evolving rubric module, the authors demonstrate that agents can improve task success by 4.5–20.9% after only a few interactions. The evolving rubric also serves as a scalable annotation tool, detecting 16.0–22.3% more failures than language models or humans alone, and personalized agents can even generalize improvements up to 8.8% across users.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
3d ago

Evolving Self-tuning Benchmarks for Long-Horizon Tool Use

The paper introduces TURTLE, an Unsupervised Environment Design framework that evolves tool‑use tasks for AI assistants by using a proposer, solver, and feedback generator powered by large language models. TURTLE automatically generates self‑tuning, long‑horizon benchmarks that increase task difficulty, diversity, and failure modes across multiple applications. Experiments on AppWorld, τ‑Bench, and AutomationBench demonstrate significant gains in hard task percentages and broader reasoning challenges.

By Hyunji Alex Nam, Michael Dennis, Natasha Jaques, Minsuk Chang
llmsagentsbenchmarks
More like this →
arXiv Machine Learning
Oct 16

Neurodata Without Boredom: Benchmarking Agentic AI for Data Reuse

Neuroscience data are fragmented across labs and formats, making reuse labor‑intensive. This study benchmarks coding agents on eight recent mouse neural recording papers, asking them to reformat data for a common decoder task. While agents handle many subtasks, they rarely produce fully error‑free solutions; the authors pair outcome metrics with a process rubric to evaluate errors and propose agent‑assisted data‑sharing practices.

By Ling-Qi Zhang, Kristin Branson
agentsbenchmarks
More like this →
arXiv AI
Jul 21

Agent psychometrics: Task-level performance prediction in agentic coding benchmarks

arXiv:2604. 00594v2 Announce Type: replace Abstract: As the focus in LLM-based coding shifts from static single-step code generation to multi-step agentic interaction with tools and environments, understanding which tasks will challenge agents and why becomes increasingly difficult.

By Chris Ge, Daria Kryvosheieva, Daniel Fried, Uzay Girit, Kaivalya Hariharan
llmsagentsbenchmarks
More like this →