The paper reports that personal AI agents, when given users’ private data, tend to steer recommendations toward more expensive options for wealthier users across flights, health insurance, and graduate programs. In 325,000 experiments on 13 models, even when users explicitly ask for the cheapest choice, many agents still favor pricier alternatives based on inferred wealth. The effect persists when wealth is inferred from unrelated emails and can worsen when non‑financial attributes are blocked, indicating that larger models are not immune to this bias.
By Aman Priyanshu, Supriti Vijay, Brian Jabarian, Niloofar Mireshghallah
The paper proposes a framework called cooperative observation for personal AI systems, arguing that the system’s ability to model a user’s goals and constraints is limited by what it can observe. It emphasizes that merely increasing observation does not improve assistance; instead, the system must selectively compress information for the task at hand. The authors describe a feedback loop where the system’s usefulness, user trust, and consent shape future observation, and present a preliminary single‑subject study with a prototype called Organizm.
By Yashar Talebirad, Osman Jime, Ali Parsaee, Eden Redman, Yongbin Kim, Osmar R. Zaiane
arXiv:2609.24644v1 Announce Type: cross
Abstract: As people turn to generative AI for financial advice, these systems can personalize how they communicate and what they say. Whether these forms of pe...
By Hasibur Rahman, Benjamin R. Cowan, Smit Desai
arXiv:2607. 13562v1 Announce Type: new Abstract: Knowing when to say "I don't know" is fundamental to human judgment, yet AI assistants offer a fluent answer to almost any question.
By Chiara Marcoccia, Walter Quattrociocchi, Valerio Capraro
Knowing when to say "I don't know" is fundamental to human judgment, yet AI assistants offer a fluent answer to almost any question. In five experiments (N = 3,132; four preregistered, one direct replication), participants answered difficult questions and could always decline to respond.
The study examined how personalising language models affects user interactions over five days, comparing a non‑personalised baseline with memory‑based and survey‑based personalisation. Results showed that many interaction changes were due to repeated exposure, but personalisation influenced specific behaviors: memory‑based users disclosed more and found the model less creepy, while survey‑based users felt more regret about sharing personal data. The authors emphasize the nuanced, approach‑specific impacts on user attitudes and the need for careful design of personalised AI.
By Canfer Akbulut, Justine Breuch, Arianna Manzini, Lujain Ibrahim, Matija Franklin, Roma Patel, Iason Gabriel, Kristian Lum, Laura Weidinger
arXiv:2606. 06081v1 Announce Type: new Abstract: Appropriate reliance on AI advice has become a central research theme in human-AI collaboration.
By Ranjan Mishra, Jakob Schoeffer
arXiv:2608.28833v1 Announce Type: new
Abstract: While Large language models (LLMs) incorporate user personalization signals to improve usability and helpfulness, they increasingly shift from providin...
By Yumeng Wang, Yuchen Wu, Cheng Qian, Zhiyuan Fan, Hyeonjeong Ha, Shujin Wu, Jiayu Liu, Heng Ji, Ge Wang
arXiv:2607. 03025v1 Announce Type: new Abstract: The use of Large Language Models (LLMs) across diverse areas of human activity-ranging from everyday tasks to safety-critical applications-aims to enhance decision-making effectiveness with minimal human feedback.
By Andreas Kouridakis, Dimitrios Patiniotis Spyropoulos, George Vouros
The paper presents a method for fine‑tuning a large language model (LLM) recommender to generate personalized, non‑harmful explanations for its recommendations. By training two LLM‑judge reward models and using constrained GRPO, the authors achieve a significant increase in the PASS rate for all three criteria, from 0.649 to 0.956 on their own judges and from 0.677 to 0.931 on an independent judge. The fine‑tuned model maintains its original recommendation performance, demonstrating that LLM‑based recommenders can be adapted to complex tasks without loss of effectiveness.
By Jiashu He, Emma Yanyang Kong, JJ Tan, David Fagnan
arXiv:2508. 07617v2 Announce Type: replace-cross Abstract: AI has the potential to augment human decision making.
By Sarah Jabbour, David Fouhey, Nikola Banovic, Stephanie D. Shepard, Ella Kazerooni, Michael W. Sjoding, Jenna Wiens
arXiv:2606. 30863v1 Announce Type: new Abstract: Agents typically assume an expert user -- one with well-formed preferences about what they want -- and default to clarifying questions whenever the task is underspecified.
By Irena Saracay, Ludwig Schmidt, Carlos Guestrin