The study explores how children design AI chatbots and what they consider trustworthy. Using a custom chatbot-building environment, 115 learners aged 8‑18 created 119 chatbots and adjusted traits such as confidence, transparency, and formality. Findings show younger children equate trust with purpose‑fulfillment, while older children focus on transparent, calibrated design, and that students calibrate academic chatbots to be more formal and transparent than hobby ones.
By Deniz Ozturk, Jiayu Li, Daksh Pratap Singh, Yasitha Rajapaksha, Fasika Melese, Bahare Riahi, Shiyan Jiang, Qiao Jin, Joey Huang, Veronica Catet\'e, Tiffany Barnes, Xiaoyi Tian
The paper introduces Safety Nudges, a browser-based tool that displays lightweight, in situ flags when a conversational AI exhibits risky behavior such as hallucination or overconfidence. In a two‑week field study with 45 frequent chatbot users, participants reported that the nudges were useful, clear, and minimally disruptive, and most felt more aware of potential AI harms. However, increased awareness did not automatically translate into measurable changes in user behavior, underscoring the need for relevance, calibration, and user control in nudge design.
By Varshini Elangovan, James Wedgwood, Chhavi Yadav, William Agnew, Sauvik Das, Virginia Smith
arXiv:2607. 25057v1 Announce Type: new Abstract: As conversational AI systems become increasingly integrated into daily life, their potential effects on user well-being require ongoing attention.
By Jina Suh, Mihaela Vorvoreanu, Forough Poursabzi-Sangdeh, Emily Tseng, Eugenia Kim, Luke Nicholls, James W. Pennebaker, Eric Horvitz
The paper evaluates the safety of conversational AI therapy bots for Generation Alpha, revealing that while these models understand 76‑82% of youth‑specific vocabulary, they correctly assess clinical risk only 64‑72% of the time, creating a significant vocabulary‑comprehension gap. Six failure patterns—such as sarcasm masking, minimization acceptance, and semantic drift—were identified, with compounded errors leading to a 94% miss rate when three or more patterns co‑occur. The authors estimate 146,880 missed crises annually and recommend mandatory human‑in‑the‑loop systems, quarterly youth‑specific validation, transparent performance disclosure, and regulatory oversight for youth‑facing mental health AI.
By Manisha Mehta, Virendra Mehta
arXiv:2606. 29685v1 Announce Type: new Abstract: How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm?
By Kaavya Krishna-Kumar, Elaine Lau, Vaughn Robinson, Jay Caldwell, Sheriff Issaka, Skyler Wang, Francisco Guzm\'an, Steven Kelling, Jonas Mueller
How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse material, yet many child-safety failures begin earlier: in model assistance that helps adults manipulate, impersonate, profile, or isolate minors, and in model responses that deepen children's emotional dependence on AI systems rather than redirecting them toward human support.
arXiv:2601.09721v2 Announce Type: replace-cross
Abstract: Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evalua...
By Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi, Farzaneh Sedaghatkar, MohammadReza Kargozari
arXiv:2607. 05407v1 Announce Type: cross Abstract: Modern artificial intelligence (AI) systems present profound new risks to child safety.
By Neil Kale, Rebecca Portnoff, Pratiksha Thaker, Michael Simpson, Robertson Wang, Kevin Kuo, Chhavi Yadav, Virginia Smith
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.
By Kate H. Bentley, Luca Belli, Adam M. Chekroud, Emily J. Ward, Emily R. Dworkin, Emily Van Ark, Kelly M. Johnston, Will Alexander, Millard Brown, Matt Hawrilenko
arXiv:2604.25415v2 Announce Type: replace-cross
Abstract: People increasingly turn to general-purpose AI chatbots for advice about emotional and mental health problems, but the ability of these syste...
By Veith Weilnhammer, Lennart Luettgau, Christopher Summerfield, Raymond Dolan, Elise Wilkinson, Virginia Corno, Viknesh Sounderajah, Matthew M Nour
Learn how OpenAI is making ChatGPT safer for teens with age-appropriate protections, learning tools, parental controls, and expert partnerships.
arXiv:2607. 22692v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for emotional support despite lacking mechanisms to safely govern evolving mental health risk.
By Anabela C. Areias, Catarina Botelho, Ant\'onio Farinhas, Areti Vassilopoulos, Dora Janela, Xin Tong, Nuno M. Guerreiro, Maya D'Eon, Fab\'iola Costa, Ricardo Rei