arXiv Computation and Language August 31, 2026 By Beiduo Chen, Pingjun Hong, Ziyun Zhang, Benjamin Roth, Anna Korhonen, Barbara Plank

Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization

Read the original on arXiv Computation and Language →

The paper investigates whether large language models can learn and reproduce annotator‑specific label‑explanation behavior, using two sentence‑pair tasks with four annotators each. It finds that individual annotator patterns are weak at the single‑annotation level but become detectable after reducing input‑content effects and aggregating across annotators. The authors propose cross‑annotator preference optimization (CAPO), which improves upon prompting and supervised fine‑tuning by better capturing annotator‑specific reasoning while maintaining stable attribution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Aug 18

When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

arXiv:2608. 16627v1 Announce Type: cross Abstract: Natural language explanations (NLEs) are increasingly used as inputs, for example, as few-shot rationales that influence model behavior in in-context learning (ICL).

By Mahdi Dhaini, Adam Dejl, Juraj Vladika, Volkan \"Ozer, Barbara Plank, Gjergji Kasneci

llms benchmarks safety

arXiv AI

Jun 3

Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs

arXiv:2602. 10352v2 Announce Type: replace-cross Abstract: Self-interpretation methods prompt language models to describe their own internal states, but remain unreliable due to hyperparameter sensitivity.

By Keenan Pepper, Alex McKenzie, Florin Pop, Stijn Servaes, Martin Leitgab, Mike Vaiana, Judd Rosenblatt, Michael S. A. Graziano, Diogo de Lucena

llms safety

arXiv AI

Jun 6

Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization

arXiv:2605. 11632v2 Announce Type: replace-cross Abstract: Self-generated counterfactual explanations (SCEs) are minimally modified inputs (minimality) generated by large language models (LLMs) that flip their own predictions (validity), offering a causally grounded approach to unraveling black-box LLM behavior.

By Yilong Wang, Qianli Wang, Bohao Chu, Yihong Liu, Jing Yang, Simon Ostermann

llms reinforcement-learning fine-tuning safety

arXiv AI

Aug 20

The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.

By Emma Yanyang Kong, JJ Tan, Ishan Gupta, Lars Olds, Claire Campbell, David Fagnan, Veli Balin, Rohan Gosain, Louis Garcia, Minsu Jang

llms benchmarks safety

arXiv AI

Jun 24

Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules

arXiv:2508. 10971v2 Announce Type: replace-cross Abstract: Knowledge graphs (KGs) can be enhanced through rule mining; however, the resulting logical rules are often difficult for humans to interpret due to their inherent complexity and the idiosyncratic labeling conventions of individual KGs.

By Nasim Shirvani-Mahdavi, Chengkai Li

llms fine-tuning

arXiv AI

Jul 3

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

arXiv:2607. 02460v1 Announce Type: cross Abstract: Post-training large language models (LLMs) without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain.

By Zhuowei Chen, Xiang Lorraine Li

llms reinforcement-learning fine-tuning efficiency benchmarks

One email a morning, machine-written

Related stories

When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs

Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization

The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation