arXiv:2608.21409v1 Announce Type: cross
Abstract: In medicine, claims remain valid when supported by empirical evidence grounded in stable biological reality. In law, by contrast, truth is contingent...
By Lorenzo Molfetta, Alessio Cocchieri, Luca Ragazzi, Ilaria Bartolini, Marco Patella, Gianluca Moro
arXiv:2604. 26233v3 Announce Type: replace Abstract: As Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers, across a range of judicial and administrative contexts, it becomes essential to explore how they answer legal questions, and in particular the factors that lead them to decide difficult questions.
By Oisin Suttle, David Lillis
The study examines how Large Language Models (LLMs) can exhibit an ‘inertia of confidence’, giving incorrect legal verdicts with high certainty, and tests this on Indian Contract Act cases. Phase I audits ChatGPT, Meta AI, and Perplexity AI, introducing the High‑Confidence Error Rate (HCER) to measure dangerous certainty, finding Meta AI most prone to errors. Phase II surveys 380 Indian law students, revealing that exposure to hallucinated citations increases verification efforts but most students lack formal ethical AI training.
By Angel Mary John, Vipin Kumar Singh, Jerrin Thomas Panachakel
The paper argues that hallucinations by legal language models should be judged as failures of legal warrant rather than mere factual or citation errors. It defines claim-authority warrant as a context-sensitive relationship between a legal claim and applicable, current authority, and proposes that evaluating warrant can uncover failures missed by traditional accuracy or citation metrics. The authors outline a pilot study, benchmark specifications, and a research agenda to assess whether legal AI systems’ claims are properly licensed by law.
By Maksym Taranukhin, Vered Shwartz
JusticeAxis is a new benchmark that formalizes legal judgment as a reference‑anchored task, requiring a single decision tied simultaneously to the statute and the surrounding circumstances. It comprises 256 real‑world criminal cases from 18 countries, each with audio, image, and text evidence, and three lawyer‑written judgments per case: the recorded judgment and two failure judgments. The authors also introduce JusticeAgent, a modular system that first establishes facts and then applies the law, with skills distilled from execution trajectories and validated under Bayesian credible bounds. Experiments reveal that open‑weight backbones tend to drift toward unsupported grounds while frontier models default to statutory interpretations, and that JusticeAgent can effectively plug into commercial systems.
By Zhengkai Tu, Mingda Zhang, Zijia Wang, Xiaoying Tang, Jimmy Huang
The paper investigates how a defendant’s courtroom statement influences decisions made by large language model (LLM)–simulated jurors. Using the newly introduced JuryBench benchmark, the authors analyze 432,000 verdicts from 20 frontier LLMs, varying defendant background, emotional appeal, and rebuttal content. Results show that emotional persuasion can backfire, that jurors are harsher toward defendants from different backgrounds and more lenient toward same‑background defendants, and that juror ideology strongly shapes verdict severity.
By Cho-Ying Wu