arXiv AI
6d ago
The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
arXiv:2608. 11694v1 Announce Type: cross Abstract: A benchmark score comes from a single phrasing of each problem.
The Shape of Wisdom: Decision Trajectories in Language Models
arXiv:2606. 01202v1 Announce Type: new Abstract: Language models do not simply choose an answer at the output layer.
Related stories
arXiv AI
Jul 28
What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation
arXiv:2605. 26795v2 Announce Type: replace Abstract: Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear.
arXiv AI
2d ago
AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs
arXiv:2608. 14320v1 Announce Type: new Abstract: The anchoring effect is a cognitive bias in which an initial reference value shifts a later judgment toward itself.
arXiv AI
Jul 14
Length Penalties Make Chain-of-Thought Less Monitorable
arXiv:2607. 09786v1 Announce Type: new Abstract: Length-penalized reinforcement learning can shorten chain-of-thought reasoning while hiding an influence that drives the model's answer.
arXiv AI
Jun 16
Vernier: Probing Representational Misalignment Behind Lexical Gaps in Causal Reasoning
arXiv:2606. 15733v1 Announce Type: cross Abstract: Instruction-tuned language models can answer the same causal-reasoning question differently after its English variable names are replaced by type-preserving placeholders, although the structural causal model and the gold answer are unchanged.
arXiv AI
1d ago
Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization
arXiv:2608. 15445v1 Announce Type: new Abstract: When a reward is correct on every training example yet consistent with more than one goal, a model can acquire an unintended one, a failure known as goal misgeneralization.
arXiv AI
Jul 24
Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers
arXiv:2603. 01437v2 Announce Type: replace Abstract: As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning.
arXiv AI
Jun 24
Pigeonholing: Bad prompts hurt models to collapse and make mistakes
arXiv:2606. 24267v1 Announce Type: cross Abstract: While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing.
arXiv AI
Jul 15
Pigeonholing: how bad prompts hurt models, causing collapse and mistakes
arXiv:2606. 24267v2 Announce Type: replace-cross Abstract: While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing.
arXiv AI
Jun 30
The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling
arXiv:2606. 29278v1 Announce Type: new Abstract: We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows.
arXiv Machine Learning
Jul 27
Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets
arXiv:2607. 21692v1 Announce Type: new Abstract: Sparse attention reduces the cost of long contexts by allowing each query to read only selected parts of the input.
arXiv Machine Learning
22h ago
Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See
arXiv:2608. 17744v1 Announce Type: cross Abstract: Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.