arXiv AI By Nirav Patel, Emily Wenger, Christopher Buccafusco

Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?

Read the original on arXiv AI →

The paper investigates whether large language model (LLM) chatbots can emulate human legal judgments of reasonableness. By comparing responses from 26 LLMs to those of human participants across 25 legal scenarios, the study finds that chatbots generally track human answers but tend to produce more homogeneous, government‑ and corporation‑friendly responses and align more closely with white, male, older, and more educated respondents. The authors note that these patterns warrant further systematic research.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 6

Assessing and Explaining the Persuadability of Large Language Models as Legal Decision Tools

arXiv:2604. 26233v3 Announce Type: replace Abstract: As Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers, across a range of judicial and administrative contexts, it becomes essential to explore how they answer legal questions, and in particular the factors that lead them to decide difficult questions.

By Oisin Suttle, David Lillis
arXiv AI
Oct 21

Humans Correct Their Judgments Through Debate, but Weaker AI Models Do Not

The paper examines whether AI‑driven debate can help humans and weaker language models (LLMs) correct their judgments. Experiments show that debate improves human accuracy over prior beliefs, especially with weighted voting, but offers little benefit—and sometimes harms—accuracy for weaker LLM supervisors. The study highlights debate as a promising tool for scalable human oversight while questioning its effectiveness for fully automated AI control.

By Maria Victoria Carro, Denise Alejandra Mester, Facundo Nieto, Oscar Agustin Stanchi, Guido Ernesto Bergman, Giovanni Franco Gabriel Marrafini, Trinidad Borrell, Guido Freire, Joaquin Salvador Machulsky, Mario Alejandro Leiva, Ana Lopez, Nicolas Spinelli, Luca Nicolas Forziati Gangi, Eitan Sprejer, Federico Barrera Lemarchand, Gerardo Simari, Maria Vanina Martinez