CHARM is a multicultural benchmark that tests large language models’ ability to adopt a character’s style while respecting knowledge boundaries. It includes 40 real and fictional characters from five cultural-linguistic regions and evaluates two boundary types—Temporal and Cross-Universe—using abstention-enabled multiple-choice questions. The study finds that hallucinations mainly stem from compliance failures: models often recognize a query is out of scope yet still provide out-of-character answers, revealing systematic cultural variations in these errors.
By Sunkyung Han, Nahyeon Park, Gaeun Seo, Seunghyun Yoon, JinYeong Bak
TempCloze is a video cloze benchmark designed to evaluate visual temporal reasoning in Video-LLMs. The task presents a video’s beginning and ending clips and asks models to select the correct missing middle from four candidates, focusing on semantic, alignment, and progression aspects while minimizing appearance cues. Evaluation of 31 models shows that temporal alignment is the main challenge, with models performing better on semantic content and event progression but struggling to place events correctly in time.
By Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du
The paper investigates how the strategic placement of Byzantine nodes in decentralized federated learning (DFL) affects the propagation of malicious influence across the communication graph. It introduces Byzantine Placement Influence (BPI), a measure that captures cumulative exposure of honest nodes to Byzantine sources over time, and develops algorithms to optimize BPI across various network structures and attack types. Experiments demonstrate that BPI-guided placements consistently yield highly damaging configurations, highlighting the importance of considering node placement in DFL threat models.
By Edoardo Gabrielli, Gabriele Tolomei
arXiv:2609.01595v1 Announce Type: cross
Abstract: We develop a framework for mechanism design with AI agents whose alignment (preferences) and capabilities (feasible actions and information) are unkn...
By Dirk Bergemann, Andrew Koh, Stephen Morris
arXiv:2510.01030v2 Announce Type: replace
Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust repres...
By Zach Studdiford, Timothy T. Rogers, Kushin Mukherjee, Siddharth Suresh
arXiv:2510.15144v4 Announce Type: replace
Abstract: Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now appr...
By Chance Jiajie Li, Zhenze Mo, Yuhan Tang, Ao Qu, Jiayi Wu, Kaiya Ivy Zhao, Yulu Gan, Jie Fan, Jiangbo Yu, Hang Jiang, Paul Pu Liang, Jinhua Zhao, Luis Alberto Alonso Pastor, Kent Larson
arXiv:2602.17826v2 Announce Type: replace
Abstract: Language models exhibit fundamental limitations -- hallucination, brittleness, and lack of formal grounding -- that are particularly problematic in...
By Marcelo Labre
The paper introduces Progressive Risk Vesting (PRV), a framework for controlling the authority of recursive large language model agents that spawn specialist branches. PRV maintains a risk budget in escrow, debiting it as branches are activated, and provides theoretical guarantees on harm bounds for adaptively generated trees. The authors analyze how risk behaves relative to an authority reproduction number, derive risk and compute shadow prices, and suggest a design rule to search broadly in a sandbox while granting recursive authority sparingly.
By Molly Wang (Imperial Business School)
The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.
By Frederic Sadrieh, Michal \v{S}tef\'anik
The paper introduces TPGC, a dual-prior prompt initialization method for multi-task graph pre-training. It first uses a Task-Prior Injection Module to pre-train prompts on an auxiliary graph, then a Structure-Prior Injection Module to embed global structural context into layer-wise prompt vectors. Experiments on six node and graph classification benchmarks show that TPGC outperforms state‑of‑the‑art baselines in few‑shot settings while requiring fewer tunable parameters and less runtime.
By Zhiyang Qiu, Yangtao Wang, Xiaocui Li, Yanzhao Xie, Siyuan Chen, Wensheng Zhang
ValueGraph is a graph pre‑training framework that incorporates automatically inferred moral‑value signals as soft constraints to learn contextualized user representations. By constructing post‑reply graphs, it captures both semantic and structural information and aligns users through contrastive and clustering objectives based on relative value similarity. Experiments on stance detection and Twitter bot detection demonstrate consistent improvements over strong text‑based, graph‑based, and LLM baselines, underscoring the benefit of value‑signal guidance for socially informed user modeling.
By Yitong Han, Wei Gao, Yi Zhao, Prasanta Bhattacharya, Fengzhu Zeng, Mohammad Amanlou
The study evaluates large language models for assessing suicide risk in Arabic crisis helpline calls, comparing Arabic and English models. Using de‑identified transcripts from Lebanon’s National Lifeline, the researchers fine‑tuned instruction‑tuned LLMs and transformer encoders, achieving a macro‑F1 of 81.19 and ROC‑AUC of 90.61 for high‑risk calls in Arabic, and 85.00/92.59 in English. The results show that high‑risk calls are more distinguishable than at‑risk calls, and translating to English does not degrade performance, indicating potential for operator‑facing tools.
By Linhai Ma, Rita El Hachem, Mahatab El Hajj, Lilian Ghandour, Samah Fodeh
The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.
By Bhuvan Koduru, Dareen Safar B Alharthi, Rita Singh, Bhiksha Raj
arXiv:2609.01231v1 Announce Type: new
Abstract: We prove that a multi-head scaled dot product attention can be viewed as a parameter identification strategy. The ratio of unidentified parameters to t...
By W. Ross Morrow
arXiv:2609.01273v1 Announce Type: new
Abstract: Synthetic data has become a common component of machine learning research. While widely adopted, its use in privacy-sensitive contexts has quietly shif...
By Jiachen Zhao, Antonia Januszewicz, Taeho Jung
arXiv:2506.12454v2 Announce Type: replace-cross
Abstract: What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this wor...
By Matteo Vilucchio, Lenka Zdeborov\'a, Bruno Loureiro
arXiv:2609.00222v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...
By Daniela Occhipinti, Andrea Piergentili, Marco Guerini
arXiv:2609.00549v1 Announce Type: new
Abstract: Large Language Model (LLM) agents increasingly rely on external skills, yet standard evaluations obscure whether retrieving these skills actually helps...
By Seonghyeon Cho, Chanjun Park
arXiv:2609.00760v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly trained to decline queries that fall outside their knowledge (knowledge-based refusal, KR) or violate saf...
By Yuri Son, Seunghee Kim, Hyuhng Joon Kim, Taeuk Kim
arXiv:2609.01246v1 Announce Type: new
Abstract: Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet p...
By Thibaut Thonet, Jos Rozen, Laurent Besacier