arXiv:2607. 24449v1 Announce Type: cross Abstract: International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks.
By Annia Abtout, Julien Delaunay, Monika Ewa Rakoczy
arXiv:2605. 21071v4 Announce Type: replace-cross Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs generate responses.
By Souvick Das, Sallam Abualhaija, Domenico Bianculli
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage o...
ViLegalExpert is a large-scale Vietnamese legal benchmark built from real citizen–lawyer consultations, comprising over 172,000 questions across 34 legal domains with professional answers and expert-verified evidence. It supports legal information retrieval, extractive QA, and abstractive QA. Experiments show that while pretrained language models perform well on QA, hybrid retrieval methods achieve the best evidence retrieval, highlighting significant challenges in grounding legal answers to authoritative sources.
By Dat Tien Nguyen, Nghia Hieu Nguyen, Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
The paper presents a cross‑lingual legal QA system for Vietnamese labour law, introducing a bilingual evaluation suite of 231 Vietnamese–English question–answer pairs, 75 of which are annotated for five complex legal reasoning phenomena. It evaluates a verifier‑guided pipeline that decomposes answers into claims, checks citation reachability and entailment, and corrects citation failures and contradictions, and introduces six automatic diagnostics for faithfulness to retrieved evidence. Experiments show that dense retrieval outperforms sparse and hybrid retrieval, translation placement has no significant effect on diagnostics, and verifier‑guided correction modestly improves citation preservation but not other dimensions, with human evaluation indicating a gap between automatic diagnostics and human judgments.
By Nguyen Minh Chi, Mo El-Haj, Nguyen Ha Thanh, Dawn Knight, Paul Rayson
CanLegalRAGBench is a Canadian legal QA benchmark that uses realistic queries and expert‑annotated answers grounded in case law to evaluate retrieval‑augmented generation (RAG) systems. The study finds that retrieval performance varies with design choices and that open‑source embedding models can compete with closed‑source ones. It also highlights limitations of automatic evaluations, noting that RAG systems often produce hallucinations or overly detailed responses, with 8‑29% of claims unsupported by retrieved documents.
By Ethan Zhao, Maksym Taranukhin, Wei Cui, Moira Aikenhead, Vered Shwartz