arXiv Machine Learning August 11, 2026 By Hong-In Won, Jinseok Jang, Hyoseop Kim

When Counterbalancing Hides the Bias: Access-Conditioned Position Lock in Forced-Choice LLM Evaluation

Read the original on arXiv Machine Learning →

arXiv:2607. 10202v2 Announce Type: replace Abstract: Forced-choice probes with counterbalanced orientations are a standard tool for measuring language-model "value dispositions," and a concentration/extremity index over repeated draws is read as how sharply a model commits.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 14

Two Confounds in Cross-Model Value Comparison: Response Determinism and the Access Harness

arXiv:2607. 10202v1 Announce Type: new Abstract: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.

By Hong-In Won, Jinseok Jang, Hyoseop Kim

llms agents

arXiv Machine Learning

Jun 2

Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignment

arXiv:2606. 01456v1 Announce Type: new Abstract: Large language models are increasingly deployed as advisors whose objective is not aligned with the user's: recommenders optimize for engagement, sales assistants for purchases, negotiation agents for concessions.

By Hamidreza Hasani Balyani, Seyed Pouyan Mousavi Davoudi, Alireza Amiri-Margavi, Amin Gholami Davodi, Arshia Gharagozlou

llms rag agents benchmarks safety

arXiv AI

Aug 5

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.

By Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi

llms benchmarks safety

arXiv Machine Learning

Jun 25

Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

arXiv:2606. 25449v1 Announce Type: cross Abstract: A language model's memory can be worse than having no memory at all.

By Alex Kwon

llms benchmarks

arXiv AI

Jul 15

Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs

arXiv:2607. 12985v1 Announce Type: new Abstract: Aligned language models routinely misreport under non-evidential incentive pressure: they agree with a confident user or overstate certainty even when their internal belief is unchanged.

By Sen Yang, Yuen-Hei Yeung

llms fine-tuning benchmarks

arXiv AI

Aug 6

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.

By Atul Anand, Sourav Chattaraj

llms agents benchmarks safety

Related stories

Two Confounds in Cross-Model Value Comparison: Response Determinism and the Access Harness

Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignment

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools