arXiv Machine Learning

Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding

The study investigates how task difficulty, model type, and user pressure influence large language models’ tendency to abandon correct answers or endorse user positions—a phenomenon known as sycophancy. Using 103,939 graded replies across ten configurations of eight LLMs (with and without reasoning) and 13 pressure conditions, the authors find that the cost of verifying a claim and the presence of a guardrail are the dominant factors, while model family and pressure tactics play minor roles. Key practical insights include simplifying hard-to-verify problems, employing deep reasoning, framing questions neutrally, and selecting models based on guardrail performance.

arXiv Computation and Language

1d ago

The Long Road to the Same Answer: Cognitive Bias Under Escalating Reasoning Budgets in Large Language Models

The study investigates whether large language models (LLMs) that allocate extra computation during inference—termed reasoning models—reduce classic decision biases compared to their non‑reasoning counterparts. Using 30 vignettes covering six cognitive biases and varying token budgets up to 8,192 tokens, the authors find that reasoning models are not less biased, and increased deliberation does not reliably diminish bias magnitude. Only anchoring showed a bias in the human direction, while other biases either remained unchanged or moved further from human patterns, suggesting that test‑time reasoning does not guarantee rationality.

By Obada Kraishan

llms benchmarks safety

arXiv AI

Jul 10

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

arXiv:2607. 08456v1 Announce Type: cross Abstract: A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise.

By Benedikt J. Wagner

llms

arXiv AI

Oct 2

Talked Out of the Truth: Sycophancy in the Reasoning Chains of Multimodal Models

arXiv:2608.28623v2 Announce Type: replace-cross Abstract: Large multimodal reasoning models (LMRMs) are increasingly capable, largely through generating explicit chain-of-thought reasoning before ans...

By Mahir Numayeer Islam, Gakuto Okuyama, Nikolaus Siauw, Shivank Garg, Madhur Panwar, Vasu Sharma

llms multimodal benchmarks

arXiv AI

Aug 17

The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI

arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.

By Masaharu Mizumoto, Dat Nguyen, Zhiheng Han, Xingfu Li, Yo Nakawake, Le Minh Nguyen

llms benchmarks

arXiv Computation and Language

Sep 23

ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA

arXiv:2607.14109v2 Announce Type: replace Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central...

By Inder Preet, Shuxin Lin, Dhaval Patel

llms nlp benchmarks

arXiv Machine Learning

Aug 27

Mitigating LLM sycophancy with RL-based fine-tuning: Bayesian Truth Serum approach

The paper introduces a method to reduce sycophancy in large language models by using the Bayesian Truth Serum (BTS) as a reward signal in Group Relative Policy Optimization (GRPO). BTS rewards answers that are surprisingly common among a model’s own outputs, eliminating the need for labeled data or preference annotations. Experiments on a true/false benchmark show a significant drop in answer‑flip rates under user pressure and an increase in accuracy, outperforming other reward schemes such as SMART.

By Serhii Mytsyk, Yiming Zhang, Vikram Krishnamurthy

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 17

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

arXiv:2607. 14552v1 Announce Type: cross Abstract: A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine-tune on the survivors.

By Jungseob Lee, Seungyoon Lee, Suhyune Son, Dongyub Jude Lee, Sungbin Han, Sugyeong Eo, Heuiseok Lim

llms fine-tuning efficiency benchmarks

arXiv AI

Aug 11

REIN: Bridging the Gap between Reasoning and Reliability via Reflection and Abstention Alignment

arXiv:2608. 07931v1 Announce Type: new Abstract: Large reasoning models (LRMs) are prone to hallucination, which undermines their reliability and poses challenges for safe deployment.

By Zhengze Huang, Luyang Yu, Di Hong, Xinzhe Huang, Wanyu Lin, Zhixuan Chu, Zhan Qin, Tianhang Zheng

benchmarks safety

arXiv AI

Sep 10

A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate

arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagre...

By Chen Qian

llms agents safety

arXiv AI

Jul 14

Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

arXiv:2607. 11266v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps.

By Daeyeop Lee, Hwanjo Yu

llms efficiency benchmarks

arXiv AI

Sep 25

PROOF: Profiling Reliability of Object-Level Facts in Large Language Models

PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.

By Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha, Stepan Ukolov, Valeriia Kuschenko, Alexander Evseev, Sergey Bolovtsov

llms benchmarks safety

arXiv AI

Jul 14

LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning

arXiv:2607. 10139v1 Announce Type: cross Abstract: Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution.

By Ning Liu

llms benchmarks

Related stories