arXiv:2602.02219v3 Announce Type: replace Abstract: Large language models are widely employed as evaluators, a paradigm commonly referred to as LLM-as-a-judge. Prior research has predominantly examin...
Qistas is a new protocol designed to evaluate how large language models treat the social, confessional, and regional constituencies of a society in a portable, pre‑registered manner. The pilot implementation focuses on Iraq and measures procedural symmetry, differential refusal, stereotype reliance, and terminological framing using paired statistical tests and a power analysis that specifies detectable effect sizes before data collection. The authors release an authoring apparatus—including a taxonomy, checklists, audit logs, and a human review notebook—and report a field test that uncovered several defect patterns common to counterfactual fairness benchmarks.
arXiv:2607. 09262v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to summarize and evaluate policy-relevant information, but it remains unclear whether their judgments are implicitly shaped by geopolitical cues.
arXiv:2607. 02104v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as cheap, scalable judges that compare candidate outputs pairwise -- to rank responses, select models, or triage papers.
arXiv:2512. 15792v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.
arXiv:2608.22432v1 Announce Type: cross Abstract: Multilingual LLM judges produce different evaluator-backbone rankings depending on the prompt language: on an eight-language Agent-as-a-Judge benchma...
MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.
The paper presents a red‑teaming framework to evaluate how open‑source large language models (LLMs) can be used for political influence, focusing on models that can be deployed locally without weight modifications. It measures LLM Overton Windows (OWs) to assess expression fidelity of sampled political opinions and examines how natural‑language jailbreaks alter these scores. The study evaluates over 30 LLMs from 10 families across five countries, finding that left‑leaning content is expressed with higher fidelity, larger models tend to have lower mean OW scores, and jailbreak effectiveness varies significantly between model families.
arXiv:2606. 24391v1 Announce Type: new Abstract: We introduce Age of LLM, a turn-based 1v1 benchmark in which two LLMs face off on a 13x7 grid to destroy the enemy base.
arXiv:2608. 06123v1 Announce Type: new Abstract: Measuring political bias in large language models (LLMs) remains challenging as it can manifest through subtle differences in framing, argumentation, and legal reasoning that are difficult to capture with a single metric.
arXiv:2512. 15792v4 Announce Type: replace-cross Abstract: Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.
The paper introduces MARS-Gov, a standpoint-aware multi-agent system designed to detect and mitigate bias in Dutch government documents. It combines legal evidence retrieval, open-set target screening, specialized jurors, conservative routing, and rewrite verification, and adds a dynamic "10th juror" when new bias groups are identified. On the DGDB benchmark, MARS-Gov achieves state‑of‑the‑art performance with an F1 score of 0.880, surpassing zero‑shot LLM detectors and supervised Dutch encoders while reducing unnecessary interventions to 2.5%.