arXiv:2608.21409v1 Announce Type: cross
Abstract: In medicine, claims remain valid when supported by empirical evidence grounded in stable biological reality. In law, by contrast, truth is contingent...
By Lorenzo Molfetta, Alessio Cocchieri, Luca Ragazzi, Ilaria Bartolini, Marco Patella, Gianluca Moro
arXiv:2606. 15646v1 Announce Type: new Abstract: Large Language Models (LLMs) have transformed natural language processing, but their lack of interpretable reasoning and tendency to hallucinate pose significant challenges for legal applications.
By Deepa Tilwani, Yash Saxena, Ankur Padia, Srinivasan Parthasarathy, Manas Gaur
The paper introduces PARCEL, a benchmark designed to evaluate whether legal claims are supported by cited authorities. Using 3,396 claims from New York State Court of Appeals decisions, the authors label each claim as Supported, Refuted, or Not Found and cast the task as a three‑way natural language inference problem. State‑of‑the‑art LLMs achieve up to 0.97 accuracy in zero‑shot settings, yet they still frequently misclassify unsupported claims as supported, especially when fabricated but plausible citations are present.
By M. Mikail Demir, M. Abdullah Canbaz
Legal Research Bench (LRB) is a new benchmark comprising 413 open-ended U.S. legal research questions, each paired with a gold answer, supporting authorities, and a binary grading rubric. The study evaluates thirteen advanced language‑model agents using web search, case‑law search, page parsing, and retrieval tools, scoring responses only when all required criteria are met and cited authorities verify. Results show that even the best model, Claude Opus 4.8, achieves full correctness on only 42.9% of questions, with performance varying by legal area and task complexity, and no clear link between more tool calls or inference cost and higher accuracy.
By Katrina Drozdov, Oliver Chen, Langston Nashold, Rayan Krishnan
The paper introduces a systematic method for comparing different formalizations of the same legal provision by analyzing their inferences on individual cases. It matches formalizations at the node level, derives shared interfaces, and uses a SAT solver to identify edge cases where any two formalizations disagree. The authors apply this approach to ten EU provisions formalized by nine advanced LLMs, finding that behavioral divergence is largely uncorrelated with structural agreement and that the resulting edge cases expose distinct types of disagreement, some reflecting real legal controversies.
By Julius Vernie, Matthias Grabmair
arXiv:2610.19427v1 Announce Type: new
Abstract: Legal reasoning is a fundamental capability for large language models (LLMs) to complete legal AI tasks such as legal judgment prediction. In legal pra...
By Shuyuan Zheng, Yun-seo Lee, Xing Gao, Jialu Wu, Jiabao Hu, Katrina Qiyao Wang, Xinbo Lin, Qianying Liu, Chuan Xiao, Tao Huang