arXiv Machine Learning

Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languages

arXiv:2608. 05163v1 Announce Type: cross Abstract: A common assumption holds that switching to a non-English language makes a multilingual RAG system easier to attack for personal information.

arXiv Machine Learning

Aug 17

Language-Specific Gaps in AI Safety Training Datasets

arXiv:2608. 13695v1 Announce Type: cross Abstract: Large language model providers routinely cite multilingual safety benchmarks spanning a dozen or more languages as evidence that their models are safe for non-English-speaking users.

By Chialuka Prisca-Mary Onuoha, Bright Etornam Sunu, Rashidat Sikiru

llms benchmarks safety

arXiv AI

Jun 30

The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

arXiv:2606. 28843v1 Announce Type: cross Abstract: Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task.

By Will Hawkins, Kaivalya Rawal, Jonathan Rystr{\o}m, Stratis Tsirtsis, Zihao Fu, Greta Warren, Ryan Brown, Eoin Delaney, Sandra Wachter, Brent Mittelstadt, Chris Russell

llms fine-tuning safety

arXiv AI

Aug 5

Evaluating OpenAI's Privacy Filter: Cross-Lingual, Cross-Domain PII Detection Across 42 Benchmarks

arXiv:2608. 02616v1 Announce Type: cross Abstract: We present the first independent, systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.

By Rohith Uppala

llms benchmarks safety

arXiv AI

Aug 7

OpenAI Privacy Filter: A Cross-Lingual, Cross-Domain PII Evaluation Across 32 Benchmarks

arXiv:2608. 02616v2 Announce Type: replace-cross Abstract: We present what is, to our knowledge, the first systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.

By Rohith Uppala

llms fine-tuning benchmarks safety

arXiv AI

Jul 14

Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingual LLM Safety

arXiv:2607. 10112v1 Announce Type: cross Abstract: Safety alignment in large language models remains brittle across languages: prompts reliably refused in English can elicit harmful compliance in non-English and low-resource settings.

By Chigozirim Ifebi, Brent Kong, Ayushi Mehrotra

llms benchmarks safety

Hugging Face Trending Papers

Sep 1

Who Judges the Judges? A Chinese Safety QA Benchmark for Evaluating LLM Responses and Safety Judges

Safety benchmarks for large language models often assess the risk of a user query, although the outcome of question answering depends on whether the response violates a policy. This distinction is cri...

llms nlp benchmarks safety

arXiv Machine Learning

Aug 24

Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Audit

arXiv:2608.20351v1 Announce Type: cross Abstract: We ask whether stereotype-loaded queries about culturally marked people leak more personal information from a retrieval-augmented generation (RAG) sy...

By Yanhang Li, Zhichao Fan, Zexin Zhuang

rag

arXiv Machine Learning

Jul 30

RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning

arXiv:2607. 26339v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems ground large language models (LLMs) in external corpora, but this reliance exposes them to corpus poisoning: maliciously injected passages that manipulate retrieved evidence.

By Pushkal Kumar, Tucker Nielson, Tanish Kolhe, Shubham Zala, Vincent Li

llms rag safety

arXiv AI

Sep 2

Who Judges the Judges? A Chinese Safety QA Benchmark for Evaluating LLM Responses and Safety Judges

arXiv:2609.01210v1 Announce Type: cross Abstract: Safety benchmarks for large language models often assess the risk of a user query, although the outcome of question answering depends on whether the...

By Rui Yang, Shuang Huang, Junhua Liu, Ziqi Zhao, Qingzhong Yan, Yuhang Sun, Cong Liu, Guoping Hu, Rui Mei, Jing Shao

llms nlp benchmarks safety

arXiv Machine Learning

Jul 3

HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety

arXiv:2607. 02079v1 Announce Type: cross Abstract: We present HaloGuard 1.

By Navaneeth Sangameswaran, Preetham S, Ashmiya Lenin

agents benchmarks safety

arXiv AI

Jul 28

TriShieldRAG: A Three-Ring Defense-in-Depth Framework Against Knowledge Corruption in Retrieval-Augmented Generation

arXiv:2607. 23838v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) lets a large language model answer questions using documents retrieved from an external knowledge base at query time.

By Susil Kumar Mohanty, Rohit Patel, Kosuru Yuvaraj, Jeenal Chaudhary, Disha Singhania

llms rag safety

arXiv Machine Learning

Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y

llms nlp benchmarks safety

Related stories