arXiv AI
Aug 7
Unified Agent: Managing Interactions across Devices
arXiv:2608. 05729v1 Announce Type: new Abstract: As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time.
One Is Not Enough: How People Use Multiple AI Models in Everyday Life
The article reports that people increasingly use multiple multimodal large language models (MLLMs) at the same time, choosing each model for its perceived strengths. This practice creates coordination challenges such as adapting prompts, calibrating trust, and managing separate conversation histories. A diary study and interviews with ten participants show that users build primary and secondary hierarchies among models, develop switching patterns based on task aggregation, and adjust effort, latency, and output credibility to manage multi‑MLLM workflows.
Related stories
arXiv AI
Sep 10
Large-Scale User Behavior Analysis in Multimodal AI-Assisted Manual Task Execution
arXiv:2609.07594v1 Announce Type: cross Abstract: Conversational Task Assistants (CTAs) are multimodal dialogue systems that support users in complex real-world tasks such as cooking and DIY through...
arXiv AI
Sep 17
MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents
MIRAGE is a controlled study that examines how multimodal personal agents use historical evidence when conversation state changes. The study keeps evidence, questions, and scoring constant while varying only the conversation state, then checks if agents can determine answerability, recover the correct source, and answer from it. Results across seven multimodal backbones show distinct failure regimes before and after compaction, heavy reliance on context continuity by open-weight models, and mixed effects of retrieval pressure on source attribution.
arXiv AI
Jun 10
Human-AI Coordination Zones: A Framework for Designing Human-in-the-Loop Experiences with Agentic AI
arXiv:2606. 09848v1 Announce Type: cross Abstract: As generative and agentic AI becomes embedded in everyday products, practitioners face a persistent challenge: how to design human-AI coordination -- the ongoing mutual adjustment between users and AI systems as mediate through interfaces-that supports usability, trust, and safety.
arXiv AI
Aug 19
Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges
The article surveys multi‑turn conversational AI, highlighting its shift from isolated text prompts to sustained, multimodal interactions that involve clarifying goals, revising requests, and switching topics. It reviews literature across text‑only dialogue, AudioLLMs, multimodal and omni‑modal systems, and tool‑augmented agents, organizing findings around datasets, models, training, evaluation, and cross‑cutting challenges. The analysis reveals that while multimodal perception and action have progressed rapidly, systems still struggle with persistent memory, cross‑turn grounding, full‑duplex interaction, robust evaluation, and cultural alignment.
arXiv AI
Sep 25
DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents
DocuTeam is a mixed‑initiative multi‑agent discussion system that allows both users and agents to start and steer conversations around evolving documents. Agents monitor changes to the document and proactively initiate or redirect discussions, while users can shape the dialogue or adopt agent suggestions. In a within‑subjects study with 20 participants, DocuTeam produced outcomes that were rated as more novel, relevant, and specific compared to a baseline, without increasing cognitive load.
arXiv AI
Jun 6
The Virtual Roundtable: Multi-Agent Personas Simulating the Dynamics of Human Brainstorming
arXiv:2606. 05178v1 Announce Type: cross Abstract: As AI-driven product development accelerates, the bottleneck is shifting from how we build to what we build.
arXiv AI
Jun 9
Scaling Participation in Modular AI Systems
arXiv:2606. 07812v1 Announce Type: new Abstract: Humanity is a mosaic of multifaceted talents and needs, and any truly intelligent AI must reflect that richness.
arXiv Computation and Language
Oct 16
MoCo: A One-Stop Shop for Model Collaboration Research
MoCo is a Python library that consolidates 26 model collaboration methods for language models, enabling execution, benchmarking, and comparison at scale. It includes 25 evaluation datasets across reasoning, QA, code, safety, and more, allowing users to add their own data. Experiments show that most collaboration strategies outperform non‑collaborative models in 61.0% of settings, with top methods achieving up to 25.8% improvement.
arXiv AI
Jun 11
Designing AI-Supported Focus Groups: A Role x Modality Playbook
arXiv:2606. 11835v1 Announce Type: cross Abstract: Collecting participants' lived experiences is central to design research.
arXiv AI
Oct 16
Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows
Kolab is a new dataset comprising over 115 hours of screen recordings from 50 professionals across 40 occupations who collaborate with AI agents on realistic, long‑horizon tasks. The dataset includes participants’ judgments of collaboration quality and output, revealing varied coordination patterns and the need for agents to anticipate how experts will use or modify their contributions. Analysis shows current state‑of‑the‑art models overestimate collaboration success and struggle to differentiate high‑ from low‑quality interactions, indicating a gap in understanding long‑horizon human‑agent teamwork.
arXiv AI
Jun 17
LVLMs and Humans Ground Differently in Referential Communication
arXiv:2601. 19792v4 Announce Type: replace-cross Abstract: For generative AI agents to partner effectively with human users, the ability to accurately predict human intent is critical.