arXiv AI
FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflows
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
6d ago
Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models
The paper introduces Auto-Comp, a fully automated, concept-driven pipeline that generates photorealistic compositional benchmarks for vision‑language models. Auto‑Comp creates paired Minimal and Contextual samples for each concept, enabling isolation of core binding abilities from visio‑linguistic complexity. Evaluations across 25 models reveal consistent failures in attribute and relational binding, with context helping relational tasks but hindering attribute tasks due to visual clutter.
arXiv AI
4d ago
MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows
MatToolBench is a new benchmark that evaluates multimodal GUI agents on professional materials science software. It contains 204 tasks across 10 tools in three modalities—GUI operation, OriginPro scripting, and code-based database queries—executed inside a Windows 11 VM. The benchmark offers fine-grained, expert-decomposed scoring and a high-performing multimodal judge for aesthetic assessment, revealing that strong general benchmark performance does not transfer to scientific workflows.
Hugging Face Trending Papers
5d ago
Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut
Timeline-Bench is a benchmark comprising 56 real video‑editing tasks that require AI agents to transform raw production material into finished videos. Each task includes a brief, source assets, a container, and a set of tests that assess format, content, brief compliance, and quality based on 2,582 blind judgments by 43 video editors. In evaluations, the best agent resolved only 15 of the 56 tasks, and most failures were due to quality tests rather than technical errors.
arXiv Machine Learning
Jun 30
DataComp-VLM: Improved Open Datasets for Vision-Language Models
arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.
arXiv AI
Jun 19
CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation
arXiv:2605. 10873v2 Announce Type: replace-cross Abstract: Recovering editable CAD programs from images or 3D observations is central to AI-assisted design, but progress is difficult to measure because existing evaluations are fragmented across datasets, modalities, and metrics.
arXiv Machine Learning
Jun 2
PaintBench: Deterministic Evaluation of Precise Visual Editing
arXiv:2606. 00188v1 Announce Type: cross Abstract: While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle.