arXiv AI October 31, 2026 By Tarek Gara

The Model as One Rater Among Several: Measuring Political Positions in Data-Sparse Regions with a Language-Model Panel

The paper introduces a method for measuring political positions in data‑sparse regions by treating a large language model as a single fallible rater within a panel, analogous to expert surveys. It demonstrates that this panel approach yields reliable agreement across multiple models and reveals that disagreements often stem from differing interpretations of the regime rather than factual errors. The authors provide a full rubric, prompt, and data set, using the Middle East and North Africa as a worked example.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

2d ago

How Green Is Your Manifesto? Scoring Environmental Positions in the 2024 EU Elections with an LLM Panel

The study introduces a panel of three large language models that score each sentence of political manifestos on environmental protection and climate change, producing averaged manifesto-level ratings. Applied to 41 2024 European Parliament election manifestos, the model’s rankings align closely with the Chapel Hill Expert Survey but tend to rate parties as greener than experts, suggesting a gap between promises and practice. The approach offers a low‑cost, multilingual, and reproducible alternative to traditional expert surveys and hand‑coding, with inter‑model disagreement serving as a built‑in reliability check.

By Franco Corti, Sajid Siraj

llms benchmarks

arXiv AI

Jul 21

Trusting sovereign language models as scientific instruments: evidence from Portugal's AMALIA

arXiv:2607. 08731v2 Announce Type: replace-cross Abstract: National language models are becoming publicly funded epistemic infrastructure.

By Manuel Pita

llms

arXiv AI

Jul 10

Validity of LLMs as data annotators: AMALIA on authority

arXiv:2607. 08731v1 Announce Type: cross Abstract: A national language model offers a linguistic community its own instrument for measuring what its citizens say and value.

By Manuel Pita

llms benchmarks

Hugging Face Trending Papers

Jul 9

Validity of LLMs as data annotators: AMALIA on authority

A national language model offers a linguistic community its own instrument for measuring what its citizens say and value. Portugal's AMALIA, a publicly funded 9B-parameter model for European Portuguese, appears competitive on agreement alone: asked to code the moral foundation of authority, it agrees with trained human coders to within six F1 points of open models eight to thirteen times its size.

llms benchmarks

arXiv AI

Jul 8

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

arXiv:2607. 05552v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans.

By Haonan Huang

llms safety

arXiv Computation and Language

Sep 22

Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity

arXiv:2609.23039v1 Announce Type: new Abstract: LLM-based AI systems answer political questions for hundreds of millions of people. Current audits measure what they say to an average user, but their...

By Joan C. Timoneda

llms safety

One email a morning, machine-written

Related stories

How Green Is Your Manifesto? Scoring Environmental Positions in the 2024 EU Elections with an LLM Panel

Trusting sovereign language models as scientific instruments: evidence from Portugal's AMALIA

Validity of LLMs as data annotators: AMALIA on authority

Validity of LLMs as data annotators: AMALIA on authority

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity