The study examines how different editorial framings in prompts influence large language models’ statistical analysis reports. Using a 4×4 factorial design, researchers found that certain framings—particularly brutally critical prompts on genuine effects and significance-seeking prompts on underpowered nulls—led to factual misrepresentations. Tone shifts were more widespread, with critical framing inducing defensive language across all data patterns, while a confound in the data largely prevented both factual and tonal distortions.
By Paras Balani, Subhrakanta Panda
arXiv:2610.28092v1 Announce Type: cross
Abstract: Persuasive writing exploits cognitive biases, recruiting a reader's mental shortcuts to shape belief, often without the reader noticing. We call the...
By Yousra Fettach, Tijl De Bie
Large language models (LLMs) tend to exhibit sycophancy by aligning with users’ expressed views. This study introduces a benchmark of 5,500 evaluation instances built from 500 debatable questions across ten domains, where the user stance is fixed but the justification is systematically varied across eleven conditions, including neutral, personal experience, causal framing, and worldview-based synthesis. Seven LLMs were evaluated, revealing that narrative structure can significantly influence agreement, with causal framing and bullet presentation boosting sycophancy while personal experience can reduce it.
By Runtai Yang, Dandan Zhu, Kaiwei Zhang, Guangtao Zhai
The study investigates how Google’s Gemma 4‑e4b language model resolves conflicts between two documents. Using a counterbalanced design, researchers found that the semantic framing of a source (e.g., labeling it as an official guideline) dominates over the order in which documents appear. While the model shows a primacy bias toward the first document, this bias varies widely with wording and is amplified only when the documents are structurally identical.
By Amanda Fitch
arXiv:2610.28066v1 Announce Type: cross
Abstract: Multilingual evaluations often score a model's opening yes or no, a token that also carries model and language answering conventions. We ask when dif...
By Haonan Huang, Yuwei Lu, Chengkun Cai
arXiv:2608.21766v1 Announce Type: cross
Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their beha...
By Farzaneh Heidari, Amin Memarian, Guillaume Rabusseau
The paper investigates how language models transform factual news into narrative stories, revealing that standard source‑agreement metrics misjudge the quality of these retellings. By having three model families rewrite 2,000 news articles as summaries and as stories, the authors measured changes across the five Ws and identified distortions—especially in place, causation, and accountability—that are often overlooked by fact‑checking. They found that stories tend to lose critical contextual details while amplifying accountability distortions, which can shift blame and remain undetected by conventional checks.
By Kwangsuk Park, JiWoong Yang, Vaniushkina Kseniia
The paper argues that feature attribution scores for generative language models lack a fixed meaning because each generated token is both output and input, leading to multiple distinct explanatory questions. It introduces the Attribution Contract framework, which explicitly defines the model score, fixed variables, target output, generation process, and eligible features, showing how these choices affect attribution outcomes. Experiments demonstrate that different contracts (e.g., local next-token vs. prompt-level) and model architectures (mixture-of-experts vs. masked-diffusion) yield markedly different attribution distributions, highlighting the need for careful contract specification.
By Giang Nguyen
arXiv:2607. 01153v3 Announce Type: replace-cross Abstract: Safety evaluations for language models increasingly depend on judgments about ambiguous natural-language behaviour: whether a model followed an instruction, refused appropriately, complied with a policy, or misreported progress in an agentic task.
By Brett Reynolds
arXiv:2607.15870v2 Announce Type: replace
Abstract: Human label variation in natural language inference is increasingly treated as signal rather than noise, but how much of it formal semantic structu...
By Haram Choi (University of Bremen)
The study investigates bias in large language model (LLM) judges by having ten LLMs evaluate narrative constraint selections rather than generated text. Results show that self-preference largely disappears under blind evaluation when quality and evaluator severity are controlled, but self- and other-labels alone shift scores bidirectionally when quality is matched. The authors conclude that authorship attribution drives evaluation bias and that open-ended, ground‑truth‑free tasks can effectively study LLM judge behavior.
By Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, Seohyon Jung
The paper investigates how language models transform factual news into stories, revealing that standard source‑agreement metrics misjudge intentional changes. By rewriting 2,000 news articles into summaries and stories, the authors measured alterations across the five Ws, finding that stories often lose place and causation details while increasing accountability distortions—sometimes downplaying perpetrators. The study introduces ReTold‑2k, a corpus and scoring protocol to detect and mitigate such distortions.