Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through w...
CogGym is a scalable, unified framework that standardizes diverse cognitive experiments into a task‑agnostic Experiment Markup Language (EML) for systematic comparison of human and AI behavior. The initial release curates 258 experiments from 100 papers focused on human commonsense reasoning and evaluates 50 large language models, revealing a scaling trend where larger models better reproduce human judgments but still lag far behind human split‑half reliability. The framework aims to continually incorporate new cognitive science experiments to track where model behavior aligns with or diverges from human cognition as models evolve.
Co-Design Bench is a new, reference‑free benchmark comprising 273 multimodal coding tasks derived from 26,314 real user sessions. It features two tracks—Multimodal Editing and Repository Editing—that test agents on localized edits with optional image, audio, video, or document inputs and coordinated repository changes across three capability dimensions: Grounded Understanding, UI Transformation, and Functional Implementation. The benchmark uses a judge‑then‑refute protocol with hidden expert‑authored acceptance criteria, and the best of ten model endpoints achieves an average of 69.9% criterion completion but fully satisfies only 26.4% of tasks, especially struggling with repository edits involving multiple change types.
arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to sp...
arXiv:2608. 05729v1 Announce Type: new Abstract: As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time.
arXiv:2606. 31966v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) have strong potential as embodied agents, but their ability to collaborate in visually grounded environments remains underexplored.
DREAM (Dynamic Resolution Assignment For Multimodal Multi-Agent Debate) enhances multimodal multi-agent debate by dynamically assigning each agent the optimal visual resolution through a zero-shot probe round that measures uncertainty with Average Normalized Log-Likelihood. It also introduces Uncertainty-Guided Rollback Aggregation to mitigate groupthink by tracking and restoring low-uncertainty answers that are overridden by group pressure. Across six multimodal datasets, DREAM improves the accuracy-token trade-off by 1.5–3.2% over existing baselines without requiring dataset-specific tuning.
arXiv:2601. 02854v2 Announce Type: replace Abstract: As an agent-level reasoning and coordination paradigm, Multi-Agent Debate (MAD) orchestrates multiple agents through structured debate to improve answer quality and support complex reasoning.
arXiv:2606. 26348v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can process diverse inputs, e.
We present SimpleSearch-VL, an efficient, reliable, and practical framework for multimodal agentic search. Its core idea is to improve the agent's own search-and-verification process rather than scaling data, tools, or auxiliary model components.
arXiv:2602.00685v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as simulated participants in social science experiments, but their behavior is often unstable an...
arXiv:2606. 02449v1 Announce Type: new Abstract: Multimodal agents are increasingly expected to operate interfaces on behalf of users, raising a central deployment question: can they truly substitute for humans in workflows that services deliberately protect against automation?