Large language models often justify legal decisions by citing statutes or precedents, but this study shows that while models correctly name the governing authority 66.7%-100% of the time, the actual verdict rarely changes when the cited authority is swapped—only 0.0%-21.7% on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. This gap persists across models of different sizes and even a purpose‑built legal‑reasoning model. Additionally, a red‑team evaluation revealed that models are more compliant with hidden adversarial instructions than they are sensitive to verdict changes, indicating that citing legal authority is a weak proxy for genuine reasoning and that verdicts remain vulnerable to manipulation.
The study examines whether large language models truly base legal verdicts on the statutes or precedents they cite. By fixing case facts and swapping the cited authority with an unrelated one, researchers found that while models correctly name the authority 66.7%-100% of the time, the resulting verdict often remains unchanged—only 0.0%-21.7% of the time on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. Even a purpose‑built legal‑reasoning model and larger scales did not close this gap, and adversarial instructions further exposed vulnerabilities in verdict consistency.
By Saisab Sadhu, Shreeyans Arora, Pratinav Seth
The paper argues that hallucinations by legal language models should be judged as failures of legal warrant rather than mere factual or citation errors. It defines claim-authority warrant as a context-sensitive relationship between a legal claim and applicable, current authority, and proposes that evaluating warrant can uncover failures missed by traditional accuracy or citation metrics. The authors outline a pilot study, benchmark specifications, and a research agenda to assess whether legal AI systems’ claims are properly licensed by law.
By Maksym Taranukhin, Vered Shwartz
arXiv:2606. 23716v1 Announce Type: cross Abstract: Legal AI benchmark research frequently invokes the assumption that large language models can improve access to justice, including for people who cannot access lawyers in order to understand and exercise their legal rights.
By Andrew Lou, David Shin
arXiv:2603.19042v5 Announce Type: replace
Abstract: The integration of artificial intelligence (AI) into judicial decision making -- particularly in pretrial, sentencing, and parole contexts -- has g...
By Arthur Dyevre, Ahmad Shahvaroughi
arXiv:2607. 18825v1 Announce Type: cross Abstract: This comprehensive study introduces an advanced Artificial Intelligence for Indian Legal Question Answering (AILQA) system tailored to the Indian legal context.
By Shubham Kumar Nigam, Shubham Kumar Mishra, Noel Shallum, Kripabandhu Ghosh, Arnab Bhattacharya
The paper investigates whether large language model (LLM) chatbots can emulate human legal judgments of reasonableness. By comparing responses from 26 LLMs to those of human participants across 25 legal scenarios, the study finds that chatbots generally track human answers but tend to produce more homogeneous, government‑ and corporation‑friendly responses and align more closely with white, male, older, and more educated respondents. The authors note that these patterns warrant further systematic research.
By Nirav Patel, Emily Wenger, Christopher Buccafusco
arXiv:2607. 23888v1 Announce Type: cross Abstract: In the United States, artificial intelligence (AI) is rapidly deployed amid limited federal regulation.
By Julie Yu, Rock Yuren Pang, Jevan Hutson, Katharina Reinecke
arXiv:2603. 04982v3 Announce Type: replace-cross Abstract: Can targeted user training unlock the productive potential of generative artificial intelligence in professional settings?
By Benjamin M. Chen, Hong Bao
arXiv:2608.21409v1 Announce Type: cross
Abstract: In medicine, claims remain valid when supported by empirical evidence grounded in stable biological reality. In law, by contrast, truth is contingent...
By Lorenzo Molfetta, Alessio Cocchieri, Luca Ragazzi, Ilaria Bartolini, Marco Patella, Gianluca Moro
arXiv:2608. 08830v1 Announce Type: new Abstract: Legal Statute Prediction (LSP) involves automatically identifying relevant legal statutes given factual descriptions in legal documents, typically framed as a multi-label classification task within natural language processing and information retrieval research.
By Subinay Adhikary, Upal Bhattacharya, Vivek Kumar Singh, Anurag Sharma, Shubham Kumar Nigam, Suvasis Das, Shouvik Kumar Guha, Koustav Rudra, Kripabandhu Ghosh
The article discusses the evolution of AI across three generations—from explicit logic to neural networks to large language models (LLMs)—and how LLMs introduce new systemic risks. It applies a forensic‑psychology profiling method to identify ten key features of LLMs, such as hallucinations, bias, and cognitive atrophy, revealing an entity that confabulates, amplifies user biases, and erodes human competence. The report concludes with a four‑pillar framework for AI resilience, emphasizing cognitive sovereignty, measurable control, partial autonomy, and openness to safeguard society.
By Thomas Bartz-Beielstein, Eva Bartz