arXiv AI

A Rigorous Turing Test: a Foundation for Evaluating Artificial General Intelligence

arXiv:2501. 17629v2 Announce Type: replace-cross Abstract: Several studies claim that large language models have passed the Turing Test and hence can "think", yet none follow Turing's original instructions precisely.

arXiv AI
Sep 10

Turing's First Imitation Game: Design Concepts and a Human-Approximates-Machine Reading

The paper analyzes Turing’s 1948 report "Intelligent Machinery" as a foundational source for later imitation games, highlighting key design concepts such as the possibility of machine errors, the exclusion of irrelevant physical traits, the role of a human judge, and Turing’s view that intellectual activity is largely search. It argues that limiting the human contestant to a weak chess player heightens the importance of intellectual search, making human behavior more comparable to machine behavior. This reframes the 1948 game as a human‑approximates‑machine scenario, suggesting that imitation games can probe when human intelligence becomes machine‑like under specific task constraints.

By Sharon Temtsin, Christoph Bartneck
arXiv AI
Sep 2

Can machines think efficiently?

The article proposes an updated Turing Test that incorporates energy consumption as a key metric, arguing that the original test is insufficient for distinguishing human from machine intelligence in the context of modern AI. It suggests that by adding an energy constraint, the test evaluates intelligence through the lens of efficiency, linking abstract thinking to tangible resource limits. The new test also provides a measurable, practical endpoint, encouraging society to balance AI time savings against total resource costs.

By Adam Winchell
arXiv AI
Sep 21

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition

CogGym is a scalable, unified framework that standardizes diverse cognitive experiments into a task‑agnostic Experiment Markup Language (EML) for systematic comparison of human and AI behavior. The initial release curates 258 experiments from 100 papers focused on human commonsense reasoning and evaluates 50 large language models, revealing a scaling trend where larger models better reproduce human judgments but still lag far behind human split‑half reliability. The framework aims to continually incorporate new cognitive science experiments to track where model behavior aligns with or diverges from human cognition as models evolve.

By Lance Ying, Jinzhou Wu, Yingshan Susan Wang, Shivam Aarya, Luca M. Schulze Buschoff, Harry Chen, Katherine M. Collins, Andrea de Varda, Shuhao Fu, Sean Dae Houlihan, Akshay K. Jagadish, Guangyuan Jiang, Samuel Kiegeland, Tetsu Kurumisawa, Rongzhi Liu, Ryan Liu, Ningshan Ma, Kathryn McGregor, Younes Strittmatter, Polina Tsvilodub, Jacob Hoover Vigly, Sarah Wu, Enjie Xu, Yiling Yun, Kelsey Allen, Tyler Brooke-Wilson, Brian Christian, Evelina Fedorenko, Michael C. Frank, Michael Franke, Tao Gao, Samuel J. Gershman, Robert D. Hawkins, Jennifer Hu, Julian Jara-Ettinger, Max Kleiman-Weiner, Sydney Levine, Tal Linzen, Hongjing Lu, Timothy O'Donnell, Desmond C. Ong, Steven T. Piantadosi, Rebecca Saxe, Eric Schulz, Tianmin Shu, Felix A. Sosa, Ilia Sucholutsky, Tan Zhi-Xuan, Tomer Ullman, Fei Xu, Ilker Yildirim, Jian-Qiao Zhu, Thomas L. Griffiths, Tobias Gerstenberg, Kevin Smith, Joshua B. Tenenbaum
arXiv AI
Aug 3

Shall We Play a Game? Language Models for Open-ended Wargames

arXiv:2509. 17192v3 Announce Type: replace Abstract: LLM-based social simulations can make a generated transcript look like a single behavioral signal, but the model behind that transcript may be doing several different jobs: choosing what an actor says or does, deciding what happens after an action, or both.

By Glenn Matlin, Isaac Song, Yixiong Hao, Parv Mahajan, Evan Montoya, Ryan Bard, Stuart R. Topp, Anthony Wen-Ming Zang, Mohammed Rehan Parwani, Soham Shetty, Mark Riedl
arXiv AI
Sep 17

Cultural Competence in Context: A Large Language Model Passes the Turing Test in Finland

The study reports that ChatGPT 5.2 successfully passed a Finnish-language Turing Test conducted in Finland, contrary to expectations that uneven representation of Finnish in training data would hinder performance. The authors attribute failures mainly to participants using colloquial Finnish cues to identify human authorship. They propose reinterpreting the Turing Test as a comparative method to assess an AI’s credible membership in a specific social world, highlighting its utility for probing the human‑machine boundary across domains.

By Otto Segersven, Pentti Henttonen
arXiv AI
Aug 13

On Benchmarking Human-Like Intelligence in Machines

arXiv:2502. 20502v2 Announce Type: replace Abstract: Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as approximate models of human cognition.

By Lance Ying, Katherine M. Collins, Lionel Wong, Ilia Sucholutsky, Ryan Liu, Adrian Weller, Tianmin Shu, Thomas L. Griffiths, Joshua B. Tenenbaum