arXiv Computation and Language By Franco Corti, Sajid Siraj

How Green Is Your Manifesto? Scoring Environmental Positions in the 2024 EU Elections with an LLM Panel

Read the original on arXiv Computation and Language →

The study introduces a panel of three large language models that score each sentence of political manifestos on environmental protection and climate change, producing averaged manifesto-level ratings. Applied to 41 2024 European Parliament election manifestos, the model’s rankings align closely with the Chapel Hill Expert Survey but tend to rate parties as greener than experts, suggesting a gap between promises and practice. The approach offers a low‑cost, multilingual, and reproducible alternative to traditional expert surveys and hand‑coding, with inter‑model disagreement serving as a built‑in reliability check.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
6d ago

The Model as One Rater Among Several: Measuring Political Positions in Data-Sparse Regions with a Language-Model Panel

The paper introduces a method for measuring political positions in data‑sparse regions by treating a large language model as a single fallible rater within a panel, analogous to expert surveys. It demonstrates that this panel approach yields reliable agreement across multiple models and reveals that disagreements often stem from differing interpretations of the regime rather than factual errors. The authors provide a full rubric, prompt, and data set, using the Middle East and North Africa as a worked example.

By Tarek Gara
Hugging Face Trending Papers
Jul 9

Validity of LLMs as data annotators: AMALIA on authority

A national language model offers a linguistic community its own instrument for measuring what its citizens say and value. Portugal's AMALIA, a publicly funded 9B-parameter model for European Portuguese, appears competitive on agreement alone: asked to code the moral foundation of authority, it agrees with trained human coders to within six F1 points of open models eight to thirteen times its size.