arXiv Computation and Language By Charles Chang (Duke Kunshan University), Yuen Yuen Ang (Johns Hopkins University), Bolun Sun (Northwestern University)

Dual-Axis Policy Entailment: A Benchmark for Policy-Change Reasoning

Read the original on arXiv Computation and Language →

Dual-Axis Policy Entailment (DAPE) is a benchmark that evaluates ordered pairs of policy directives on two dimensions: whether they govern the same policy domain and whether the later directive continues, broadens, narrows, reverses, or shifts to a different domain. The benchmark includes 1,000 expert-annotated pairs from Chinese government policy and a calibration protocol for annotation consistency. Models tested on DAPE perform poorly zero‑shot, but a fine‑tuned Qwen3‑14B model reaches the human ceiling in agreement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Sep 7

Can Large Language Models Anticipate Behavioral Responses to Social Policies? A Case of Pension Enrollment Prediction among China's Flexible Workers

The paper introduces FlexPension-LLM, a domain‑specialized large language model designed to predict pension enrollment among China’s flexible workers. By injecting policy‑grounded cues and using LoRA/SFT for rationale‑augmented supervision, the model achieves a Composite F1 score of 0.9316 on a blind split, outperforming several baselines and matching top commercial models. External survey tests confirm its robustness, with the narrowest performance range among strong systems.

By Yumiao Li, Peixin Liu, Donglin Di, Chen Li, Runhuan Feng
arXiv Computation and Language
Oct 9

The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection

The paper introduces MARS-Gov, a standpoint-aware multi-agent system designed to detect and mitigate bias in Dutch government documents. It combines legal evidence retrieval, open-set target screening, specialized jurors, conservative routing, and rewrite verification, and adds a dynamic "10th juror" when new bias groups are identified. On the DGDB benchmark, MARS-Gov achieves state‑of‑the‑art performance with an F1 score of 0.880, surpassing zero‑shot LLM detectors and supervised Dutch encoders while reducing unnecessary interventions to 2.5%.

By Yuchen Miao, Zijun Wang, Chang Han, Yurui Shi, Mingtai Zhang, Siyang Xu
arXiv AI
Aug 26

Comparing Explanations is Not Enough, Explain the Change: New Standards are Needed to Explain Behavioral Shifts in Large Language Models

arXiv:2602. 02304v3 Announce Type: replace Abstract: Large-scale foundation models exhibit behavioral shifts when subjected to interventions such as scaling, fine-tuning, reinforcement learning with human feedback, or in-context learning.

By Martino Ciaperoni, Marzio Di Vece, Roberto Pellungrini, Luca Pappalardo, Fosca Giannotti, Francesco Giannini