arXiv AI

Thinking Outside the Bots: Human Ideas in AI's Semantic Blind Spots Are Not Inferior

arXiv AI
Jul 24

Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale

arXiv:2512. 12413v2 Announce Type: replace Abstract: Generative AI tools are increasingly embedded in everyday work and learning, yet their fluency, opacity, and propensity to hallucinate mean that users must critically evaluate AI outputs rather than accept them at face value.

By Gabriel R. Lau, Wei Yan Low, Louis Tay, Ysabel Guevarra, Dragan Ga\v{s}evi\'c, Andree Hartanto
arXiv Computation and Language
Nov 4

IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.

By Shanshan Zhong, Chen Xu, Jingjie Ning, Zhengzhong Liu, Aviral Kumar, Chenyan Xiong
arXiv Computation and Language
Nov 9

Does Reasoning Improve Psychological Depth in Large Language Models?

The study examined whether reasoning-enhanced large language models produce psychologically deeper short stories. Using 60 blinded pairwise evaluations, human annotators found no consistent advantage for reasoning-oriented models: GPT‑5 slightly outperformed GPT‑4o, while DeepSeek‑R1 lagged behind DeepSeek‑V3. In contrast, an LLM-as-a-Judge consistently favored reasoning models, revealing a strong pro‑reasoning bias that may mask reader subjectivity in creative tasks.

By Ruichen Zheng, Yihe Wang, Fabrice Harel-Canada, Sara Khosravi, Zeynep Senahan Yildiz, Amit Sahai, Nanyun Peng
arXiv AI
Sep 23

Biased AI improves human performance but reduces perceived helpfulness

The study tests deliberately biased AI assistants and finds that such bias improves human performance on tasks like misinformation evaluation, financial investment, and graduate education compared to neutral AI. However, participants undervalue biased AI and overvalue neutral AI, even when performance is similar. When two AI biases flank a participant’s perspective, performance gains are maintained while reducing the perceived cost and one‑sided influence.

By Shiyang Lai, Jiwoong Choi, Junsol Kim, Nadav Kunievsky, Yujin Potter, James Evans