arXiv:2607. 24449v1 Announce Type: cross Abstract: International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks.
International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks. We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage.
ViLegalExpert is a large-scale Vietnamese legal benchmark built from real citizen–lawyer consultations, comprising over 172,000 questions across 34 legal domains with professional answers and expert-verified evidence. It supports legal information retrieval, extractive QA, and abstractive QA. Experiments show that while pretrained language models perform well on QA, hybrid retrieval methods achieve the best evidence retrieval, highlighting significant challenges in grounding legal answers to authoritative sources.
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage o...
arXiv:2605. 21071v4 Announce Type: replace-cross Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs generate responses.
The paper presents a cross‑lingual legal QA system for Vietnamese labour law, introducing a bilingual evaluation suite of 231 Vietnamese–English question–answer pairs, 75 of which are annotated for five complex legal reasoning phenomena. It evaluates a verifier‑guided pipeline that decomposes answers into claims, checks citation reachability and entailment, and corrects citation failures and contradictions, and introduces six automatic diagnostics for faithfulness to retrieved evidence. Experiments show that dense retrieval outperforms sparse and hybrid retrieval, translation placement has no significant effect on diagnostics, and verifier‑guided correction modestly improves citation preservation but not other dimensions, with human evaluation indicating a gap between automatic diagnostics and human judgments.
arXiv:2609.14739v1 Announce Type: cross Abstract: Large language models are increasingly used in high-stakes domains such as law, where systems must ground their reasoning in retrieved evidence and a...
CanLegalRAGBench is a Canadian legal QA benchmark that uses realistic queries and expert‑annotated answers grounded in case law to evaluate retrieval‑augmented generation (RAG) systems. The study finds that retrieval performance varies with design choices and that open‑source embedding models can compete with closed‑source ones. It also highlights limitations of automatic evaluations, noting that RAG systems often produce hallucinations or overly detailed responses, with 8‑29% of claims unsupported by retrieved documents.
arXiv:2603. 22973v2 Announce Type: replace Abstract: Applying computational methods to law at scale requires separating genuine legal reasoning from surface similarity.
Legal Research Bench (LRB) is a new benchmark comprising 413 open-ended U.S. legal research questions, each paired with a gold answer, supporting authorities, and a binary grading rubric. The study evaluates thirteen advanced language‑model agents using web search, case‑law search, page parsing, and retrieval tools, scoring responses only when all required criteria are met and cited authorities verify. Results show that even the best model, Claude Opus 4.8, achieves full correctness on only 42.9% of questions, with performance varying by legal area and task complexity, and no clear link between more tool calls or inference cost and higher accuracy.
GreenLeaf Law Embed Tiny is a 0.6 B parameter embedding model designed for legal domain retrieval. It achieves 75.11 % on the Massive Legal Embedding Benchmark and 64.38 % on MTEB(Law, v1), outperforming other models under 1 B parameters. The model is trained via a two‑stage pipeline that distills knowledge from a larger teacher, fine‑tunes with hard negative mining, and uses a curated dataset of 3.4 million query‑passage pairs, including 150,000 human‑curated samples from diverse legal jurisdictions, while supporting efficient inference with multiple quantization levels.
CoAL‑RAG is a complexity‑aware legal retrieval‑augmented generation method that adapts its retrieval strategy based on a multi‑dimensional evaluation of question essence and retrieval consistency. It quantifies reasoning demand from the logical structure of a question and uses the discrepancy between semantic and keyword retrieval to gauge problem complexity, thereby selecting the most suitable retrieval approach and filtering context dynamically. Experiments show that CoAL‑RAG outperforms baseline models on Chinese legal benchmarks (SocialLawQA, LawBench) with a 42.5% BLEU improvement and 3.6× ROUGE‑L, while also achieving strong cross‑jurisdictional performance on English datasets (LexGLUE, CaseHold).
arXiv:2509.00141v2 Announce Type: replace-cross Abstract: Statutory corpora and judicial decisions are growing faster than legal professionals can read them, while individual judgments often exceed t...