arXiv AI
Jun 18
RedactionBench
arXiv:2606. 18782v1 Announce Type: cross Abstract: Large Language Models are increasingly applied to sensitive domains that require redaction of personally identifiable information (PII).
Domain Agnostic Text Redaction from Natural Language Rules using Instruction Tuning
arXiv:2608. 14693v1 Announce Type: cross Abstract: With the increasing digitization of personal and corporate communication, the automatic sanitization of textual data has become a crucial component of data privacy and compliance frameworks.
Related stories
arXiv AI
Aug 20
Redakto - The Incognito Tab for LLMs
Redakto is a new tool designed to anonymize text before it is processed by large language models (LLMs). It offers state‑of‑the‑art redaction of personally identifiable information (PII) and pseudonymization, accessible via a web interface, REST APIs, and model context protocol hooks. The authors evaluate its performance on legal and medical datasets, showing that anonymized texts retain utility comparable to the originals, enabling LLM tasks without significant loss of effectiveness.
arXiv Computation and Language
Sep 15
PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents
arXiv:2606.17467v3 Announce Type: replace-cross Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and int...
arXiv AI
Jun 6
RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
arXiv:2509. 20324v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is an emerging approach in natural language processing that combines large language models (LLMs) with external document retrieval to produce more accurate and grounded responses.
arXiv AI
Jun 17
Combating Data Laundering in LLM Training
arXiv:2604. 01904v3 Announce Type: replace-cross Abstract: Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.
arXiv Computation and Language
Sep 11
A Short Survey of Viewing Large Language Models in Legal Aspect
The paper surveys how large language models (LLMs) are being applied in legal tasks such as judgement prediction, document analysis, and drafting. It reviews the benefits of automation while highlighting legal challenges like privacy, bias, and explainability. The authors also discuss data resources for legal domain specialization and outline future research directions.
arXiv AI
Aug 20
From Threat Intelligence to Detection: Knowledge-driven Enrichment and Template-based Rule Grounding for Automated Sigma Rule Generation
The paper introduces AUTOSIGMA, an automated system that converts unstructured cyber threat intelligence reports into Sigma detection rules. It enriches input data with a structured knowledge base, matches it against existing Sigma rule repositories, and uses a large language model as a judge to validate the generated rules. Experiments on real-world APT reports and security blogs show that AUTOSIGMA outperforms other methods in rule validity, relevancy, MITRE ATT&CK coverage, and robustness to input quality.
arXiv AI
Aug 18
Bye-bye, Bluebook? Automating Legal Drudgery With AI-Augmented Rule Following
arXiv:2505. 02763v2 Announce Type: replace-cross Abstract: One of the central promises of legal AI is to automate drudgery -- the formal, repetitive tasks of lawyers' work that consume time without calling for much discretion.
arXiv Computation and Language
Sep 11
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that baseline safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe generation.
arXiv Machine Learning
Sep 10
Data Quality Rule Generation with LLMs
The paper introduces LeDQeR, an approach that uses large language models to automatically generate data quality rules for rule‑based enterprise tools. It follows a generate‑filter framework where the LLM proposes candidate rules from dirty data, and four filters ensure the rules are executable, correct, generalizable, and non‑redundant. Experiments show that LeDQeR produces effective, compact rule sets across diverse datasets and error types.
Hugging Face Trending Papers
Sep 10
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that standard safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe outputs.
arXiv Computation and Language
Aug 27
Gavel: Agent Meets Checklist for Evaluating LLMs on Long-Context Legal Summarization
The paper introduces Gavel, a framework for evaluating large language models (LLMs) on long-context legal summarization tasks. Gavel includes a reference-based component (Gavel-Ref) with checklist, residual-fact, and writing-style checks, and a reference-free component (Gavel-Agent) that assesses factual coverage directly from source documents. Experiments on 12 frontier LLMs reveal that models tend to omit key information more than hallucinate, perform well on simple checklist items but struggle with rare, complex items, and their performance degrades with longer cases. Gavel-Agent cuts token usage by at least 36% compared to traditional methods while maintaining competitive accuracy, and it also generalizes effectively to the medical domain.