arXiv Machine Learning

Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languages

arXiv:2608. 05163v1 Announce Type: cross Abstract: A common assumption holds that switching to a non-English language makes a multilingual RAG system easier to attack for personal information.

arXiv Machine Learning

2d ago

Language-Specific Gaps in AI Safety Training Datasets

arXiv:2608. 13695v1 Announce Type: cross Abstract: Large language model providers routinely cite multilingual safety benchmarks spanning a dozen or more languages as evidence that their models are safe for non-English-speaking users.

By Chialuka Prisca-Mary Onuoha, Bright Etornam Sunu, Rashidat Sikiru

llms benchmarks safety

arXiv AI

Jun 30

The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

arXiv:2606. 28843v1 Announce Type: cross Abstract: Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task.

By Will Hawkins, Kaivalya Rawal, Jonathan Rystr{\o}m, Stratis Tsirtsis, Zihao Fu, Greta Warren, Ryan Brown, Eoin Delaney, Sandra Wachter, Brent Mittelstadt, Chris Russell

llms fine-tuning safety

arXiv AI

Aug 5

Evaluating OpenAI's Privacy Filter: Cross-Lingual, Cross-Domain PII Detection Across 42 Benchmarks

arXiv:2608. 02616v1 Announce Type: cross Abstract: We present the first independent, systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.

By Rohith Uppala

llms benchmarks safety

arXiv AI

Aug 7

OpenAI Privacy Filter: A Cross-Lingual, Cross-Domain PII Evaluation Across 32 Benchmarks

arXiv:2608. 02616v2 Announce Type: replace-cross Abstract: We present what is, to our knowledge, the first systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.

By Rohith Uppala

llms fine-tuning benchmarks safety

arXiv AI

Jul 14

Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingual LLM Safety

arXiv:2607. 10112v1 Announce Type: cross Abstract: Safety alignment in large language models remains brittle across languages: prompts reliably refused in English can elicit harmful compliance in non-English and low-resource settings.

By Chigozirim Ifebi, Brent Kong, Ayushi Mehrotra

llms benchmarks safety

arXiv Machine Learning

Jul 30

RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning

arXiv:2607. 26339v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems ground large language models (LLMs) in external corpora, but this reliance exposes them to corpus poisoning: maliciously injected passages that manipulate retrieved evidence.

By Pushkal Kumar, Tucker Nielson, Tanish Kolhe, Shubham Zala, Vincent Li

llms rag safety

arXiv Machine Learning

Jul 3

HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety

arXiv:2607. 02079v1 Announce Type: cross Abstract: We present HaloGuard 1.

By Navaneeth Sangameswaran, Preetham S, Ashmiya Lenin

agents benchmarks safety

arXiv AI

Jul 28

TriShieldRAG: A Three-Ring Defense-in-Depth Framework Against Knowledge Corruption in Retrieval-Augmented Generation

arXiv:2607. 23838v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) lets a large language model answer questions using documents retrieved from an external knowledge base at query time.

By Susil Kumar Mohanty, Rohit Patel, Kosuru Yuvaraj, Jeenal Chaudhary, Disha Singhania

llms rag safety

arXiv Machine Learning

Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y

llms nlp benchmarks safety

arXiv AI

Jun 30

Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.

By Mohammed Abu Baker, Luca Baroni, Dan Wilhelm

llms agents fine-tuning benchmarks

arXiv Machine Learning

Jul 22

On the Limits of Support-Preserving Alignment and Bounded Filtering

arXiv:2607. 18295v1 Announce Type: new Abstract: We study whether alignment schemes that reshape a base model's output distribution, combined with bounded safety filters, can drive the probability of harmful behavior to zero in modern large language models.

By Aryan Dutt, Rui Mao, Anupam Chattopadhyay

llms benchmarks safety

arXiv Machine Learning

Aug 12

Leak It: Per-Document Extraction Beyond Aggregate Membership Inference

arXiv:2608. 00144v2 Announce Type: replace Abstract: Membership inference (MIA) on language models is usually summarised by aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines can separate members from non-members using surface text alone.

By Victor Maricato

llms safety

Related stories