Towards Data Science

Stop Ranking Agent Configs by Average Score

Best-worst comparisons, MaxDiff-style judging, and Plackett-Luce utility scores give agent teams a cleaner way to decide which configs to ship, prune, and route toward next. The post Stop Ranking Agent Configs by Average Score appeared first on Towards Data Science .

arXiv AI
Jun 19

Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents

arXiv:2606. 19704v1 Announce Type: new Abstract: Agent benchmarks are growing fast, but no single benchmark touches more than four or five of the dimensions that deployment exposes.

By Dhaval C. Patel, Kaoutar El Maghraoui, Shuxin Lin, Yusheng Li, Tianjun Feng, Chun-Yi Tsai, Yihan Sun, Wei Alexander Xin, Akshat Bhandari, Tanisha Rathod, Aaron Fan, Sanskruti Vijay Shejwal, Tomas Pasiecznik, Sagar Chethan Kumar, Tanmay Agarwal, Rohith Kanathur, Sam Colman, Amaan Sheikh, Dev Bahl, Ann Li, Krish Veera, Alimurtaza Mustafa Merchant, Shambhawi Baswaraj Bhure, Sajal Kumar Goyla, Chengrui Li, Kirthana Natarajan, Rui Li, Thomas Ajai, Rujing Li, Vivek G. Iyer, Sanjaii Vijayakumar, Yitong Bai, Ayal Yakobe, Darief Maes, Yassine Jebbouri, Tianyang Xu, Thai Quoc On, Vera Mazeeva, Winston Li, Yuval Shemla, Yeshitha Bhuvanesh, Rushin Bhatt, Siddharth Chethan Gowda, Alisha Vinod, Caroline Cahill, Shriya Aishani Rachakonda, Yunfeng Chen, Aryaman Agrawal, Aman Upganlawar, Mao Le Jonathan Ang, Yubin Sally Go, Madhav Rajkondawar, Yang-Jung Chen, Trisha Maturi, Ananya Kapoor, Andrew Li, Shrey Arora, Mana Abbaszadeh, Shen Li, Charles Xu, Byeolah Kwon
Towards Data Science
Sep 28

Guided Merge Sort : An Optimized Sorting that Picks the Best from Ordinary and Multi-Way Merge Sort Algorithms

The article introduces Guided Merge Sort, an optimized sorting technique that combines elements of ordinary merge sort and multi‑way merge sort. It highlights how the use of the "goto" operator becomes essential in this approach. The post explains the algorithm’s design and its potential advantages over traditional methods.

By Tigran Hayrapetyan
Towards Data Science
Oct 30

Your Agent’s Tools Can Make Decisions, Too

The article titled "Your Agent’s Tools Can Make Decisions, Too" discusses decision models as tools and offers a hands‑on guide for their implementation. It highlights how agents can leverage these tools to make informed choices. The post was originally published on Towards Data Science.

By Shuai Guo
Towards Data Science
Aug 30

8 Tips for Writing Effective Agent Instructions

The article "8 Tips for Writing Effective Agent Instructions" offers quick and simple guidance to improve the clarity and effectiveness of instructions given to agents. It presents a concise set of practical tips aimed at helping readers craft better agent instructions. The post is published on Towards Data Science.

By Payal Patel
arXiv Computation and Language
Oct 22

Credit Where Credit Is Due: Contrastive Attribution Topology Search for Multi-Agent Systems

The paper introduces Contrastive Attribution Topology Search (CATS), a method for selecting compact, high‑performing teams of agents within a larger multi‑agent system. CATS uses an LLM attributer to assign credit to agents based on execution traces, then iteratively prunes weak edges and agents to form a small, efficient team. Experiments show CATS improves accuracy on BigCodeBench by 5.3 percentage points over heuristic search while reducing deployment tokens by 54%, matches strong baselines on MBPP with fewer tokens, and successfully removes adversarial agents in a HotpotQA stress test.

By Fabian Tritthart, Tom Zehle