arXiv AI

Science Utopia? Closed-Loop LLM Simulation of Academic Research Ecosystems

The paper introduces SciUtopia, a closed‑loop large‑language‑model simulation framework that models the evolving academic research ecosystem, including research direction, collaboration, publication, peer review, funding, and researcher attrition. Running 61 simulation worlds, the system simulates over 40,000 researchers and 1.2 million LLM‑generated peer reviews, revealing that rejection‑driven resubmission increases reviewer burden, cautious exploration balances citation impact with career success, and resource inequality can arise without early‑funding advantage.

arXiv Computation and Language
Sep 22

SocioVerse2: A Longitudinal Dynamic Social Simulation Framework under a Human-AI Co-evolutionary Paradigm

arXiv:2609.24911v1 Announce Type: new Abstract: Social simulation offers the social sciences an experimental instrument that the real world cannot supply, and generative agents have transformed it by...

By Xinnong Zhang, Jiayu Lin, Jia Wang, Yixu Huang, Xinyi Mou, Yingqian Wu, Jingcong Liang, Shijun Lei, Jianing Shi, Guanying Li, Siyuan Wang, Hanjia Lyu, Zhenfei Yin, Yunlu Yin, Siming Chen, Yulan He, Jiebo Luo, Xuanjing Huang, Liyin Jin, Baohua Zhou, Hanqi Yan, Zhongyu Wei
arXiv AI
Jun 8

Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

arXiv:2606. 07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution.

By Jiayu Wang, Weijiang Lv, Bowen Fu, Jing Fu, Jiayi Song, Lingyu Zhang, Lanxuan Xue, Luodi Chen, Zepeng Xin, Kaiyu Li, Xiangyong Cao
arXiv AI
Aug 17

ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond

arXiv:2608. 14354v1 Announce Type: new Abstract: Enabling LLM agents to sustain productive, stable, and goal-aligned research over extended horizons is a central challenge for autonomous machine learning and scientific discovery, as progress hinges on continuously managing evolving state, exploration decisions, and computational resources.

By Mingming Zhao, Jiqian Dong, Kangping Xu, Zadid Hasan, Chengrui Fan, Shan Jiang, Shuai Mao, Ting Lingya, Linyi Zou, Tailin Zhou, Yun Hin Chan, Wenkai Zhang, Zhanhong Zhou, Guowei Huang, Hongliang Li, Wenjing Cun, Zhitang Chen, Mingxuan Yuan, Yanhui Geng
arXiv AI
Jul 14

FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights

arXiv:2602. 02905v2 Announce Type: replace Abstract: Autonomous agents powered by large language models (LLMs) promise to accelerate scientific discovery end-to-end, but rigorously evaluating their capacity for verifiable discovery remains a central challenge.

By Zhen Wang, Fan Bai, Zhongyan Luo, Jinyan Su, Kaiser Sun, Xinle Yu, Jieyuan Liu, Kun Zhou, Claire Cardie, Mark Dredze, Zhiting Hu, Eric P. Xing
arXiv AI
Sep 10

Mapping the Emerging Social Science of Large Language Models

The paper maps the nascent social‑science literature on large language models (LLMs) by analysing 198 curated papers and 47,719 field‑scale papers. It identifies three main domains—LLM as Social Minds, LLM Societies, and LLM‑Human Interactions—each containing 13 subcategories such as reasoning, bias, collective intelligence, and trust. The taxonomy is validated through clustering stability, author classification agreement, and topic mapping, revealing differing prominence across conference and journal venues.

By Yi Yang, Xiao Jia, Zeyun Dong, Chenzhang Wang, Zhanzhan Zhao
Hugging Face Trending Papers
Aug 17

Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies

Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all contemporaneous or future literature, and asks models to propose hypotheses that an independent large language model judge matches against the held-out ground-truth idea.