QuasiLeak-Bench is a new benchmark that evaluates large language model (LLM) based privacy detectors on their ability to recover quasi‑evidence groups—sets of weak quasi‑identifiers that jointly reveal private attributes—while respecting sender‑specified disclosure policies. The benchmark uses a reverse‑construction pipeline to generate documents containing verified quasi‑evidence groups and applies automated quality control to produce reliable ground truth at scale. Experiments show that current privacy detectors perform poorly on this task, but a fine‑tuned 4B LLM can outperform the best prompted models, indicating the task is learnable.
By Yufan Zhu, Yangfan Jiang, Yang Fei, Shizheng Hou, Shubhankar Mohapatra, Xiaokui Xiao
MoCo is a Python library that consolidates 26 model collaboration methods for language models, enabling execution, benchmarking, and comparison at scale. It includes 25 evaluation datasets across reasoning, QA, code, safety, and more, allowing users to add their own data. Experiments show that most collaboration strategies outperform non‑collaborative models in 61.0% of settings, with top methods achieving up to 25.8% improvement.
By Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang, Zhaoxuan Tan, Jiajie Yan, Zhenyu Lei, Wenxuan Ding, Weijia Shi, Haojin Wang, Zhenting Qi, Yuru Jiang, Heng Wang, Chengsong Huang, Yu Fei, Jihan Yao, Yilun Du, Luke Zettlemoyer, Yejin Choi, Yulia Tsvetkov
CarryOn is a dual-loop system that enhances frozen vision‑language‑action (VLA) policies for robotic assistance by adding personalization and reliable execution. The planning loop maintains persistent user and scene context to ground subtask decomposition in user preferences, while the execution loop supervises policy rollouts to prevent hazardous contact and recover from failures using reusable control skills. Experiments on a real robot and in simulation show that CarryOn reduces safety violations from 52.6% to 6.1% and increases recovery success from 13% to 80%.
By Zixuan Wang, Yuqi Liu, Yuxin Chen, Sengqiao Yang, Jinhui Ye, Pengguang Chen, Changsheng Lu, Shu Liu, Bei Yu, Jiaya Jia
The paper investigates when supervised learning can recover the true latent variables that generate data, focusing on two causal settings: generative (labels cause latents) and discriminative (latents cause labels). It shows that if the representation space has at least as many dimensions as the latent space, optimal encoders are linear; otherwise, identifiability fails and encoders become nonlinear. Additional constraints—such as non‑negative activations in the generative case or sparsity and spark conditions in the discriminative case—enable recovery of individual latent variables, linking nonlinear ICA, compressed sensing, and neural network interpretability.
By Junze Zhou, David Klindt
The paper examines whether transcripts of model solutions can reveal capabilities after benchmark scores saturate. By simulating saturation with historical data, the authors test protocols that use only correctly solved transcripts to predict performance on unsaturated items. Pairwise judging of transcripts outperforms other methods, reducing prediction error by 5.3×, though its effectiveness can vary across distribution shifts.
By Morio Hamada, Jehyeok Yeon, Matteo Migliarini, Justin Olive, Magda Dubois
The paper studies how large language models (LLMs) plan under partial observability, comparing aggressive planners that commit to long plans based on unobserved assumptions with conservative planners that only use observed information. Both strategies, whether the LLM outputs a plan directly or generates a PDDL formulation, achieve similar accuracy across three LLMs on two domains. Conservative planners reduce failures but increase API calls, while aggressive planners fail more often but replan efficiently when priors are informative. The study also finds that LLMs treat priors almost as facts, risking failure if priors are unreliable, and proposes hybrid strategies with tunable hyperparameters to balance safety and efficiency.
By Cassie Huang, Zhanwei Cao, Yeawon Lee, Nathalia Gomez, Zhuoyi Yang, Ian G. Harris, Tiffany Do, Jiayuan Mao, Li Zhang
CleanRM is a reward‑modeling approach that mitigates the bias caused by noisy preference data in RLHF. It calibrates sample‑wise noise ratios using a small set of clean feedback via optimal transport, aligning the joint distribution of noisy and clean preferences. The method then trains a reward model with a noise‑corrected surrogate loss, proven to be unbiased with respect to the clean risk, and outperforms existing denoising baselines in experiments.
By Licheng Pan, Haocheng Yang, Yuzhou Chen, Yichen Sun, Yuan Lu, Yunfei Teng, Jiajun Fan, Guangyi Liu, Zhixuan Chu, Hao Wang
The paper introduces MINE-SAMPLE-RENDER (MSR), a test-time method that enhances diversity in generative models by externalizing it into a corpus-derived sampling process. MSR first mines an interpretable categorical schema from a corpus of real artifacts and uses the resulting statistics to sample attribute configurations. These configurations are then fed into a generative model, which acts purely as a conditional renderer, producing outputs that better align with the corpus distribution and exhibit greater diversity across several application domains.
By Kostas Triaridis, Morris E. Sharp, Nebojsa Jojic
The paper introduces Localized Context Distillation (LoCD), a method that improves learning from interaction experience for large language models. LoCD identifies critical spans in a student’s trajectory that need revision, then has a teacher regenerate only those spans from the student’s own prefix, reducing exposure bias while focusing on informative states. Experiments across math reasoning, code generation, and agent tasks show that LoCD outperforms strong baselines and yields more stable training with smaller, sparser updates.
By Zhezheng Hao, Yongkang Yang, Xilin Xia, Jiarui Yu, Yi Liu, Fanjunduo Wei, Peijie Dong, Tianfu Wang, Qiang Lin, Can Wang, Hande Dong, Jiawei Chen
The paper introduces a tool‑augmented large language model framework that improves the reliability of medical calculations derived from free‑form patient notes. It separates clinical language understanding from rule execution by retrieving calculator specifications, generating executable tools, and using a constrained extractor to map notes to calculator inputs. On the MedCalc‑Bench dataset, the method achieves 86.73% final‑answer accuracy, outperforming one‑shot prompting and fine‑tuned baselines by 24.0 and 17.27 percentage points.
By Yang Zhong, Ling Luo, Xuanlyu Zhu, Yanhua Wang, Yuanyuan Sun, Jian Wang, Hongfei Lin
The paper presents a privacy‑preserving text inference framework using CKKS homomorphic encryption for health‑misinformation classification and semantic retrieval. It combines BioBERT, MPNet, and TF‑IDF features for classification, achieving an accuracy of 0.971 and macro‑F1 of 0.950, while a late‑fusion similarity model reaches a Pearson correlation of 0.8691. Optimized ciphertext retrieval reduces latency from 222.028 s to 5.168 s and communication overhead from 6.11 GB to 2.13 MB over 7,128 items.
By Jasleen Kaur, Chlo\'e Cachet, Koray Karabina, Jean Belo Klamti, Philippe Lamontagne
The paper investigates how encouraging action predictability in latent world models can preserve intrinsic dynamics, which are crucial for planning. By deriving a theoretical lower bound and conducting empirical tests, the authors show that action predictability prevents the encoder from merging distinct action outcomes, even when visual appearance varies. Using a method called Concat IDM, they achieve a substantial increase in planning success on the Push‑T benchmark, from 9.1% to 77.2%, demonstrating the practical benefit of this inductive bias.
By Jiamin Shi, Yuanhong Zhang, Zhiyang Liu, Jiawei Sun, Jingwen Fu
The paper introduces SAFE-PRO, a method for enhancing safety in multimodal large language models by proactively asking clarifying questions when cross‑modal ambiguity poses intent‑level risks. SAFE‑PRO employs a Risk‑Conditioned Uncertainty signal that blends predictive entropy with a learned cross‑modal risk score to guide both question‑asking and path optimization, prioritizing safety while reducing over‑refusal. The authors also present the SSUI‑Dialog dataset and demonstrate that SAFE‑PRO outperforms existing approaches, including GPT‑4o and Claude‑3.5‑Sonnet, across multiple benchmarks.
By Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li
The study examines how groups of large language model (LLM) agents handle misinformation when a few adversarial agents inject false claims into a discussion. In experiments with ten agents across six models and three datasets, a single adversarial agent caused 44.8% of benign agents to adopt the false answer, while three adversaries raised this to 60.5%. Interestingly, many benign agents privately held the false answer yet publicly disagreed, indicating hidden belief collapse that is not visible in the discussion itself.
By Liangjie Huang, Henry Peng Zou, Wei-Chieh Huang, Wooseong Yang, Yuwei Han, Philip S. Yu
The paper introduces a lightweight privacy-preserving technique for Retrieval-Augmented Generation (RAG) systems that uses random projection to transform both datastore and query embeddings. This approach maintains semantic similarity for effective retrieval while significantly reducing the risk of data extraction attacks. Experiments across various RAG architectures and datasets demonstrate that the method outperforms prior differential privacy-based techniques in both retrieval and generation performance while offering comparable empirical privacy.
By Dixi Yao, Tahseen Rabbani, Tian Li
The paper introduces adVersarial Parameter Decomposition (VPD), a method that splits a language model’s parameters into simple subcomponents, each responsible for a small part of the model’s algorithm. VPD optimizes these decompositions so that the model’s input‑output behavior remains intact even when many subcomponents are removed, including adversarially chosen ones. This approach enables a new form of circuit analysis by tracing how sequences of parameter subcomponents interact to produce outputs for specific inputs.
By Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin, Bart Bussmann, Nathan Hu, Michael Ivanitskiy, Linda Linsefors, Lee Sharkey
INFUSE is a new benchmark that tests indirect prompt injection (IPI) in large‑language‑model agents performing long‑horizon workflows. It contains 40 realistic tasks and 56 attack scenarios across five threat categories, with attacks injected through realistic interaction surfaces rather than just initial prompts. The study evaluated 19 LLMs, generating over 41,000 adversarial trajectories and found mean attack success rates of 19.6% (LLM evaluation) and 15.4% (heuristic verification), while prompt‑level hardening reduced success from 58.2% to 14.3%.
By Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson, Kevin Leach
The paper investigates how vision‑language models (VLMs) handle negation by comparing their attention patterns to human gaze when processing affirmative versus negated statements. Using synthetic and realistic datasets, the authors find that most VLMs do not mimic human-like attention and rely on mechanisms beyond attention, with PaliGemma2‑3B shifting focus to alternatives under negation and LLaVA‑1.5‑7B showing a left‑side spatial bias. The study highlights a performance gap between affirmative and negated queries across downstream tasks, underscoring the need for better negation processing in VLMs.
By Beth Pearson, Michael Wray, Martha Lewis
Screwtape is a pipeline that transforms real agent transcripts into adversarial edits designed to bypass action‑only monitors. Using an LLM attacker to modify transcripts and an LLM verifier to ensure consistency, the system generated 35 successful attacks on 94 Claude Code and Codex transcripts, with six confirmed by humans to also evade an external monitor. The study shows that a red‑teamer can discover monitor failures without direct access to a developer’s infrastructure, suggesting a method for improving safety oversight.
By Gosha Kozhevnikov, Tyler Tracy
The paper investigates when it is worthwhile to pay for higher‑fidelity large language models (LLMs) in stochastic optimization tasks. It introduces two cost‑aware algorithmic frameworks that adaptively switch between mixed‑fidelity first‑order oracles, one for known bias and one for unknown bias, and demonstrates that these methods can reduce cumulative cost while maintaining accuracy. Experiments with synthetic data and LLM completions confirm that the adaptive policies skip expensive oracles and achieve comparable performance to fixed high‑fidelity approaches.
By Swati Gupta, Tianjiao Li