arXiv AI

Trust Hinders Bidirectional Human-AI Alignment

arXiv AI
Sep 1

A Mental Model Based Framework of Trust

The paper proposes a mental model-based framework of trust that captures multidimensional aspects of trust and can be used to infer human trust in AI agents. It formalizes trust perception, trust evolution, human reliance, and decision-making, and defines the appropriate level of trust in the agent. Human subject studies evaluate whether adjusting human beliefs about the agent, as predicted by the framework, can change trust perceptions across performance, process, and purpose dimensions.

By Zahra Zahedi, Sarath Sreedharan, Erin Chiou, Subbarao Kambhampati
arXiv Computation and Language
Aug 24

Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers

The paper surveys 61 studies on mental‑health AI and identifies a misalignment in how trust is evaluated across disciplines. It proposes a three‑layer framework—human‑oriented, interaction‑oriented, and AI‑oriented trust—and maps stakeholder perspectives onto these layers. The authors argue that future research should focus on calibrating human trust to actual interaction and AI trustworthiness rather than merely maximizing perceived trust.

By Xin Sun, Yue Su, Yifan Mo, Qingyu Meng, Yuxuan Li, Min Chen, Mengyuan Zhang, Saku Sugawara, Charlotte Gerritsen, Sander L. Koole, Koen Hindriks, Jiahuan Pei
arXiv AI
Jul 17

Align AI to Dynamic Human-AI Workflows

arXiv:2607. 14240v1 Announce Type: new Abstract: Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions.

By Valerie Chen, Cleotilde Gonzalez, Anita Williams Woolley, Michael Lee, Tongshuang Wu, Vincent Conitzer, Aarti Singh
arXiv Machine Learning
Oct 8

Careful Judge: Safe and Efficient Human-AI Collaborative Decision Making

The paper introduces CARE, a calibrated adaptive rectification and escalation framework that integrates AI models with human reviewers to ensure safe, human-aligned decisions. CARE continuously learns from human feedback, improving automation while reducing the need for human queries. Experiments on driving, language, and robotics datasets show a 25–81% reduction in human queries compared to baseline methods.

By Chenyu Zhang, Rachel Luo, Boyi Li, Anjali Parashar, Marco Pavone, Apoorva Sharma
arXiv AI
Oct 9

Whose Ground Truth? Embracing Ambiguity in Human-Centered AI

The paper titled "Whose Ground Truth? Embracing Ambiguity in Human-Centered AI" argues that current AI systems assume a single definitive ground truth, which ignores the inherent ambiguity in human interpretations. It calls for modeling the full space of plausible human judgments, distinguishing true ambiguity from noise, and suggests that this shift should influence how AI is represented, learned, evaluated, deployed, and governed.

By Jingyao Wu, Mohammad Tariqul Islam, Per R{\aa}dberg Nagb{\o}l, Julie Gerlings, Giovanni Leoni, Georgiana Fryza, Lars Pilegaard Thomsen, Jakob Mainz
arXiv AI
Sep 28

Bringing AI to Autonomous Systems -- From Cognition to Collective Intelligence

The article discusses autonomous systems as the pinnacle of AI development, emphasizing the need to blend connectionist and symbolic AI within systems engineering. It introduces a generic agent architecture that organizes behavior around long‑term memory and outlines challenges in linking sensory data to structured memory, goal‑oriented decision making, planning, and agent coordination for collective intelligence. The authors also explore agent trustworthiness, noting it extends beyond behavior to include cognitive validity, and propose methods for its evaluation while highlighting the gap between current capabilities and the envisioned autonomous multi‑agent systems.

By Joseph Sifakis