UXBench is the first user‑centric benchmark that uses real user feedback to evaluate AI assistants. It comprises three tasks—UX Judge, UX Eval, and UX Recovery—built from 7,400 test instances drawn from over 70,000 interaction logs of a mainstream Chinese AI assistant, covering 8 scenarios and 83 domains. Experiments on 26 state‑of‑the‑art language models reveal how model capability affects user experience and highlight six key findings about user feedback prediction and engagement.
By Mengze Hong, Xia Zeng, Zeyang Lei, Sheng Wang, Chen Jason Zhang, Di Jiang, Taiming Fu, Jinfeng Huang, Mengqiao Liu, Qinghe Chang, Haosheng Zou, Qiongyi Zhou, Sijun He, Simonjmdeng, Minlong Peng, Di Liang
HaloGuard 2.0 is an open‑weight, streaming output guard that assigns a safe/unsafe probability to each response token in a single forward pass, allowing deployments to halt a stream once harm is detected. It builds on HaloGuard 1.0’s input‑guard constitution, training on a 478,532‑response corpus and a second corrective phase with 106,043 counterfactual responses to address a shortcut that previously caused the model to judge prompts instead of responses. The resulting 0.8 B‑parameter model achieves an 83.6 macro F1 on six English safety benchmarks—up 10.8 points over the prior version—while using 5–10× fewer parameters than the strongest full‑response guards, and its multilingual variant reaches 83.1 F1 across 17 languages.
By Ashmiya Lenin, Navaneeth Sangameswaran, Preetham S
The paper presents a multilingual audit of stereotype bias in large language models (LLMs) using 12,000 locally written Reddit posts in English, Spanish, French, and German. It evaluates gender, racial, and religious bias across six models (three open-weight and three proprietary APIs) with a two-blank design to reduce grammatical cues, finding that religion shows the highest imbalance, Spanish leads in gender bias, and German in race and religion. The study highlights that aggregate scores can mask category- and context-specific biases, underscoring the need for detailed inspection before deploying LLMs in consequential settings.
By Zihao Sun
The paper investigates whether large language model (LLM) agents can acquire new knowledge from tutorial videos without updating their parameters. It introduces Tutorial Agentic Learning (TAL), a non‑parametric continual‑learning framework that builds a structured tutorial‑book memory via a two‑stage adversarial process, and retrieves information through a Table‑of‑Contents tree during inference. On the VidLearn multimodal QA benchmark—comprising 72 chess tutorial videos—TAL outperforms baseline models, achieving up to 113% relative gains on training data and 91% on test data, and improves in real‑world chess openings in 69% of head‑to‑head games, with additional evidence of generality in a second domain.
By Haoyu Zhao, Zafeirios Fountas, Jun Wang
arXiv:2601.16724v3 Announce Type: replace
Abstract: Automated Essay Scoring systems disproportionately penalize high-proficiency English as a Second Language (ESL) learners. We propose Contrastive Le...
By Kevin Fan, Eric Yun
arXiv:2611.03374v1 Announce Type: cross
Abstract: Recent language model (LM) safety failures have largely been studied as disparate phenomena. In agentic misalignment, a model blackmails an executive...
By Brandon Jaipersaud, Elliot Creager
arXiv:2604.24693v2 Announce Type: replace
Abstract: Linear activation steering is a powerful approach for controlling and specializing large language models by adding linear representations of concep...
By Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan, Mikhail Belkin
arXiv:2611.02332v1 Announce Type: cross
Abstract: Tool-using language-model agents often execute actions that their domain policies forbid. We present LeanGuard, a runtime that checks every proposed...
By Sreehari Sankar, Debargha Ganguly, Vikash Singh, Mohsen Hariri, Weicong Chen, Yanyan Zhang, Shouren Wang, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Idan Davidovich, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Joe Stubbs, Erman Ayday, Jing Ma, Vipin Chaudhary
arXiv:2611.00690v1 Announce Type: cross
Abstract: Speech-to-speech translation (S2ST) increasingly preserves what is said and how it sounds, but not necessarily when speech and silence occur. For dub...
By Zhisheng Zheng, David Harwath
arXiv:2611.02940v1 Announce Type: new
Abstract: PlagiaCorpus is a bilingual Arabic-English dataset for benchmarking plagiarism and text-reuse detection, including reuse produced by large language mod...
By Adel Ammar, Serry Sibaee, Wadii Boulila
The paper surveys the state of embedding models across different modalities, noting that despite shared mathematical foundations, research remains fragmented. It argues that recent advances—contrastive learning, Transformer architectures, and a shift toward finer granularity—are driving a convergence toward common principles, while modality-specific challenges are still underexplored. The authors propose a framework of three capabilities—coverage, comparability, and compositionality—and outline three unification paths: transformation, alignment, and joint embedding, with the latter advocated as the ultimate goal.
By Ziyin Zhang, Yuyu Guo, Ying Zhang, Siqiao Xue, Zihan Liao, Jin Qin, Wenjie Yang, Siyuan Yang, Shijie Li, Cheng Chen, Hao Wu, Zhaoyang Zhu, Ruipeng Li, Zhaokun Jiang, Lei Lei, Rui Wang, Hang Yu, Rui Wang
Fairness evaluation of clinical prediction models for binary outcomes often involves fairness metrics that compare the area under the receiver operating characteristic curve (AUC, or c-statistic) acro...
Robustness verification is essential for ensuring the safety and security of neural networks, but current methods focus heavily on
norm-ball attacks, which do not adequately model typical real-world...
Reinforcement learning objectives that maximize expected return can concentrate probability mass on a small subset of high-reward outcomes, limiting diversity when multiple distinct solutions are desi...
Multimodal perception is central to many computer vision applications, including robotics and autonomous driving, where different sensing modalities provide complementary cues. Self-supervised learnin...
Vision-Language-Action (VLA) models generalize across tasks, but their autoregressive decoders need sequential forward passes in proportion to the action horizon, each bounded below on embedded aerial...
The article describes how an individual discovered that a subtle background sound in a YouTube video can reveal the viewer’s exact location. By extracting faint plane noises from the soundtrack, analyzing their Doppler shifts, and matching these patterns with ADS‑B flight data, the author can pinpoint the video’s geographic origin based on aircraft speed, distance, and flight gaps. This method demonstrates a novel privacy vulnerability in seemingly innocuous media.
The paper introduces ANGLER, an adaptive framework that uses state-based verifiers from the REEF benchmark to guide the evolution of web‑agent attacks. REEF provides 9 verifier‑grounded websites and 5 attack types, enabling realistic red‑teaming of VLM‑based agents. ANGLER improves attack success rates to 20.36% versus 15.24% for non‑adaptive methods and lowers benign task success from 61.9% to 44.4%.
By Yini Huang, Aiwen LU, Zhen Sun, Yi Han, Yuzhi Zhao, Jiaheng Wei
Speech-DRAME is a framework for evaluating spoken role‑play that considers linguistic content, prosody, emotional expression, character embodiment, and contextual fit. It includes a bilingual, human‑annotated benchmark (Speech‑DRAME‑EvalBench) with contrastive mismatch conditions and a fine‑tuned audio‑language evaluator (DRAME‑Eval). While DRAME‑Eval performs well on synthesized data, it shows a significant performance drop on authentic human acting, highlighting a synthetic‑to‑authentic transfer gap.
By Jiatong Shi, Jionghao Han, Yichen Lu, Santiago Pascual, Pengfei Wu, Chenye Cui, Shinji Watanabe, Chao Weng, Cong Zhou
The study examines how the choice of large language model (LLM) used to write job application cover letters influences hiring decisions, independent of the applicant’s qualifications. Using a multi‑model audit with eleven writer agents and nine evaluator agents across ten job descriptions, the authors find a consistent ranking of writer models—locally hosted models are disadvantaged and evaluators often favor their own outputs. The hierarchy has tangible effects, with up to 1.8 of 25 qualified candidates being rejected solely due to model mismatch, and the patterns are explained by shared stylistic cues that both writers and evaluators reward or penalise.
By Nicol\`o Pagan, Michael Andreas Ebling, Stefania Ionescu, Anik\'o Hann\'ak