arXiv:2607. 19375v1 Announce Type: cross Abstract: Language models perform economically valuable work, yet they are not currently assessed for how well they perform every economically valuable task.
By Alexander Wan, Stephane Hatgis-Kessell, Tom\'as Aguirre, Percy Liang, Rishi Bommasani
The study examines how the choice of large language model (LLM) used to write job application cover letters influences hiring decisions, independent of the applicant’s qualifications. Using a multi‑model audit with eleven writer agents and nine evaluator agents across ten job descriptions, the authors find a consistent ranking of writer models—locally hosted models are disadvantaged and evaluators often favor their own outputs. The hierarchy has tangible effects, with up to 1.8 of 25 qualified candidates being rejected solely due to model mismatch, and the patterns are explained by shared stylistic cues that both writers and evaluators reward or penalise.
By Nicol\`o Pagan, Michael Andreas Ebling, Stefania Ionescu, Anik\'o Hann\'ak
arXiv:2603. 19005v2 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains.
By An Luo, Jin Du, Xun Xian, Robert Specht, Fangqiao Tian, Ganghua Wang, Xuan Bi, Charles Fleming, Ashish Kundu, Jayanth Srinivasa, Mingyi Hong, Rui Zhang, Tianxi Li, Galin Jones, Jie Ding
IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.
By Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar, Chenyan Xiong
arXiv:2608. 13940v1 Announce Type: new Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled by cost: a candidate solution can be written in minutes, whereas evaluating it can take hours to days of GPU time.
By Thomas Simon Foster, Bassel Al Omari, Tingchen Fu, Thomas Mann, Carl Domond, Lucia Cipolina-Kun, Bhavul Gauri, Muna Aghamelu, Alexander D. Goldie, Eryk Helenowski, Jean-Christophe Gagnon-Audet, Alberto Pepe, Saba Nazir, Daniel Izcovich, Noam Levi, Rishi Hazra, Karen Hambardzumyan, Nicolas Baldwin, Xian Li, Martin Josifoski, Paris Giampouras, Masoud Jalili Sabet, Anya Sims, Hela Momand, Tatiana Shavrina, Despoina Magka, Jason Weston, Yulin Wang, Anirudh Goyal, Jo\~ao Henriques, Yoram Bachrach, Emily McMilin, Jakob Nicolaus Foerster
arXiv:2407.20371v3 Announce Type: replace-cross
Abstract: Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the...
By Kyra Wilson, Aylin Caliskan