Hugging Face Trending Papers
CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
4d ago
CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
arXiv:2610.36484v1 Announce Type: new Abstract: Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing...
arXiv AI
Sep 21
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
CogGym is a scalable, unified framework that standardizes diverse cognitive experiments into a task‑agnostic Experiment Markup Language (EML) for systematic comparison of human and AI behavior. The initial release curates 258 experiments from 100 papers focused on human commonsense reasoning and evaluates 50 large language models, revealing a scaling trend where larger models better reproduce human judgments but still lag far behind human split‑half reliability. The framework aims to continually incorporate new cognitive science experiments to track where model behavior aligns with or diverges from human cognition as models evolve.
arXiv AI
5d ago
Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models
Co-Design Bench is a new, reference‑free benchmark comprising 273 multimodal coding tasks derived from 26,314 real user sessions. It features two tracks—Multimodal Editing and Repository Editing—that test agents on localized edits with optional image, audio, video, or document inputs and coordinated repository changes across three capability dimensions: Grounded Understanding, UI Transformation, and Functional Implementation. The benchmark uses a judge‑then‑refute protocol with hidden expert‑authored acceptance criteria, and the best of ten model endpoints achieves an average of 69.9% criterion completion but fully satisfies only 26.4% of tasks, especially struggling with repository edits involving multiple change types.
arXiv AI
Oct 15
GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?
arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to sp...
arXiv AI
Jun 26
What We are Missing in Multimodal LLM Evaluation?
arXiv:2606. 26348v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can process diverse inputs, e.
arXiv AI
Sep 1
MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However...