arXiv AI By Robin Welsch

When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving

Read the original on arXiv AI →

The study investigates how large language model (LLM) advisors affect complex problem‑solving in a simulated clothing‑factory setting. Two preregistered experiments (N=200 and N=198) found that participants with AI support reported higher confidence and understanding, expended less effort, and in some cases achieved better performance or avoided bankruptcy. Within the AI‑supported group, more frequent changes to the AI’s recommendations were linked to improved unaided performance and knowledge.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 16

Beyond "ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work

The paper investigates how to help users monitor their own and an AI system’s competence when using AI assistance. It identifies 30 interventions from experts and organizes them into a design space based on timing, target competence, and source of cue. A large experiment shows that reliability cards and contrasting replies reduce estimation error and overconfidence, though they do not improve task performance.

By Manuel A. D. Santos, Paul Thiesse, Steeven Villa, Daniela Fernandes, Albrecht Schmidt, Verena Distler, Robin Welsch