arXiv Computation and Language

EJAI: Judgment-Preserving Civic AI for Data Center Environmental Justice

EJAI is a source‑grounded civic AI system that links an interactive map, retrieved evidence, missing‑data statements, and two stakeholder personas to help communities evaluate data‑center environmental justice claims. In a benchmark of 21 queries, the system’s evidence supported 98.4 % of factual claims and identified missing evidence in 32 of 33 cases, while persona alignment sometimes omitted conflicting evidence. A within‑subjects study of 15 residents showed that the system enabled varied approaches to preparing for a simulated town hall, though some users noted simplifications or biases from the personas.

Hugging Face Trending Papers
Nov 10

EJAI: Judgment-Preserving Civic AI for Data Center Environmental Justice

EJAI is a source‑grounded civic AI system designed to help communities evaluate the environmental justice impacts of data centers. It combines an interactive map, retrieved evidence, missing‑data statements, and two stakeholder personas to let users inspect evidence and craft policy recommendations. In tests, the system accurately supported most factual claims, highlighted evidence gaps, and revealed how persona alignment can influence evidence presentation.

arXiv AI
Jul 1

A Technical Typology of AI Systems in Public Administration

arXiv:2606. 31755v1 Announce Type: cross Abstract: Research on artificial intelligence (AI) in the public sector often treats "AI" as a single category, neglecting technical distinctions between different AI systems.

By Jonathan Rystr{\o}m, Chris Schmitz, Nathan Davies, Gerhard Hammerschmid, Albert Meijer, Chris Russell
arXiv AI
Jun 9

Can Data Work be Reparative?

arXiv:2606. 09408v1 Announce Type: cross Abstract: We present an ethnographic study of an alternative approach to data work, developed by a civic-tech initiative that builds datasets for training and benchmarking online safety systems.

By Srravya Chandhiramowuli, Ding Wang, Alex Taylor
arXiv AI
Sep 10

Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?

The paper investigates whether large language model (LLM) chatbots can emulate human legal judgments of reasonableness. By comparing responses from 26 LLMs to those of human participants across 25 legal scenarios, the study finds that chatbots generally track human answers but tend to produce more homogeneous, government‑ and corporation‑friendly responses and align more closely with white, male, older, and more educated respondents. The authors note that these patterns warrant further systematic research.

By Nirav Patel, Emily Wenger, Christopher Buccafusco
arXiv AI
Sep 10

How AI Models Manage Epistemic Authority: A Taxonomy and Comparative Analysis of Responses to User Disagreement

The paper introduces a taxonomy of six user challenge types and a four-layer framework to analyze how large language models respond to user disagreement. Using a dataset of 2,310 challenge scenarios and 32,340 responses from 14 models, the study finds that models often validate users (85%) while still maintaining their original claim (65%). It also reports that models frequently apologize (33%) and transfer authority in advice contexts, with significant variation across model types and task domains.

By Riyadh Alnasser, Yusuf M\"ucahit \c{C}etinkaya, Sumin Zhao, Tu\u{g}rulcan Elmas
arXiv Computation and Language
1d ago

Agent-as-a-Judge

arXiv:2611.18067v1 Announce Type: new Abstract: Artificial intelligence has progressed from task-specific systems toward general models spanning perception, generation, reasoning, and autonomous inte...

By Runyang You, Hongru Cai, Caiqi Zhang, Qiancheng Xu, Meng Liu, Tiezheng Yu, Yongqi Li, Wenjie Li
arXiv AI
Jul 3

AI Assistance for Human Review of Default Judgments

arXiv:2607. 01256v1 Announce Type: cross Abstract: Overwhelmed courts in the United States review millions of default judgments each year.

By Theodora Worledge, Othman Bensouda Koraichi, Daniel Bernal, Aviv Caspi, Tatsunori Hashimoto, Carlos Guestrin, David Freeman Engstrom
arXiv AI
Oct 7

EIO-Agents: The Missing Semantic Layer for AI Agent Evaluation

EIO-Agents introduces an open specification for interoperable AI agent evaluation, consisting of two layers: the Evaluation Intelligence Ontology (EIO) and the Portable Evaluation Record (PER). The EIO layer supplies a semantic framework that defines typed evidence, behavioral predicates, evidence contracts, claims, witness rules, proof status, recurrence, and computable derivations for metrics, findings, controls, and decisions such as PASS, REVIEW, or BLOCK. The PER layer offers a canonical, content‑addressed record of an evaluation that preserves the evidence‑to‑decision chain, enabling re‑derivation, explanation, and verification of the evaluation outcome.

By Fouad Bousetouane
arXiv AI
Oct 27

Distilling Expert Judgment at Scale

The paper proposes a method to capture expert human judgment and reasoning for automated evaluation systems that can scale across high‑stakes domains such as healthcare, defense, finance, governance, and politics. By assembling expert networks and encoding their reasoning into structured editorial standards, the authors demonstrate that automated judges trained on these standards outperform uncalibrated frontier models on source‑quality and neutrality metrics in AI news reporting. The study validates the approach with strong inter‑annotator agreement and high F1 scores, suggesting a viable framework for expert‑calibrated evaluation in any domain where expertise is scarce.

By Robbie Goldfarb, Andrew B. Hall, Jillian Fisher, Kathryn Salam, Matt Wilde