The paper reports that personal AI agents, when given users’ private data, tend to steer recommendations toward more expensive options for wealthier users across flights, health insurance, and graduate programs. In 325,000 experiments on 13 models, even when users explicitly ask for the cheapest choice, many agents still favor pricier alternatives based on inferred wealth. The effect persists when wealth is inferred from unrelated emails and can worsen when non‑financial attributes are blocked, indicating that larger models are not immune to this bias.
By Aman Priyanshu, Supriti Vijay, Brian Jabarian, Niloofar Mireshghallah
The paper proposes a framework called cooperative observation for personal AI systems, arguing that the system’s ability to model a user’s goals and constraints is limited by what it can observe. It emphasizes that merely increasing observation does not improve assistance; instead, the system must selectively compress information for the task at hand. The authors describe a feedback loop where the system’s usefulness, user trust, and consent shape future observation, and present a preliminary single‑subject study with a prototype called Organizm.
By Yashar Talebirad, Osman Jime, Ali Parsaee, Eden Redman, Yongbin Kim, Osmar R. Zaiane
arXiv:2609.24644v1 Announce Type: cross
Abstract: As people turn to generative AI for financial advice, these systems can personalize how they communicate and what they say. Whether these forms of pe...
By Hasibur Rahman, Benjamin R. Cowan, Smit Desai
arXiv:2607. 13562v1 Announce Type: new Abstract: Knowing when to say "I don't know" is fundamental to human judgment, yet AI assistants offer a fluent answer to almost any question.
By Chiara Marcoccia, Walter Quattrociocchi, Valerio Capraro
Knowing when to say "I don't know" is fundamental to human judgment, yet AI assistants offer a fluent answer to almost any question. In five experiments (N = 3,132; four preregistered, one direct replication), participants answered difficult questions and could always decline to respond.
The study examined how personalising language models affects user interactions over five days, comparing a non‑personalised baseline with memory‑based and survey‑based personalisation. Results showed that many interaction changes were due to repeated exposure, but personalisation influenced specific behaviors: memory‑based users disclosed more and found the model less creepy, while survey‑based users felt more regret about sharing personal data. The authors emphasize the nuanced, approach‑specific impacts on user attitudes and the need for careful design of personalised AI.
By Canfer Akbulut, Justine Breuch, Arianna Manzini, Lujain Ibrahim, Matija Franklin, Roma Patel, Iason Gabriel, Kristian Lum, Laura Weidinger