arXiv AI

Prompt Injection in Automated R\'esum\'e Screening with Large Language Models: Single and Multi-Injection Settings

arXiv:2606. 27287v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems.

arXiv AI
Sep 25

Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

The paper investigates prompt injection attacks on Jev, a non‑generative decision model, using 510 reconstructed cases. It finds that malicious prompts can shift Jev’s action probabilities, though rarely cause it to choose the attacker’s target. Techniques such as override markers mitigate influence, while adaptive attacks that use score feedback roughly double the highest attacker‑target probability and increase success rates on new validation calls from 1.8% to 3.5%.

By Tiantong Wu, Wei Yang Bryan Lim
arXiv Computation and Language
Aug 27

Lower-Resource, Higher Scores: Language Bias in LLM Evaluators

The paper demonstrates that large language model (LLM) evaluators, whether reward‑model based or prompted LLM‑as‑a‑Judge, exhibit significant language bias in multilingual settings. Experiments with semantically identical instruction‑response pairs across 23 languages reveal that lower‑resource languages receive higher scores, a bias that persists across eight open‑weight evaluators and is not detectable by standard pairwise accuracy metrics. The authors link the bias to model uncertainty and language identity, showing it cannot be explained by content difficulty alone.

By Ej Zhou, Lucas Resck, Zheng Hui, Anna Korhonen
arXiv Computation and Language
Sep 28

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

The paper introduces a prompt‑injection detection framework for email assistants that models attacks as a chain of stages. It combines a text detector, stage‑specific verifiers, rule‑based risk signals, user intent consistency checks, and a logistic decision policy. Experiments on five benchmarks show the framework outperforms pretrained detectors, achieving a mean F1 of 0.406 versus 0.216, and demonstrate that training on benign emails resembling attacks reduces false alarms.

By Ahmad Hashmi, Dhyey Patel, Yunting Yin
arXiv Machine Learning
Jul 30

ToxScreen: Detecting Whether an LLM Has Been Poisoned

arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.

By Anthony Hughes, Nicole Xing, Collin Francel, Andy Kim, Andrew Draganov
Hugging Face Trending Papers
Jul 29

ToxScreen: Detecting Whether an LLM Has Been Poisoned

As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned.