← Back to all news
arXiv Computation and Language By Sajjad Ghiasvand, Maryam Amirizaniani, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani

Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models

Read the original on arXiv Computation and Language →

The study evaluates whether multimodal large language models (MLLMs) can produce open‑ended aesthetic critiques comparable to humans. Eight open‑weight MLLMs (7 B–397 B) and GPT‑5.5 were tested on 1,227 r/photocritique posts under various prompts, revealing that reference‑based similarity metrics often misrepresent model performance, while shorter critiques and image‑omission had limited impact. Human judges and annotators found the models’ critiques largely different from human ones, noting that models tend to be overly comprehensive and repetitive rather than selective and specific.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Aug 28

AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability

AesCanvas is a new dataset and benchmark that evaluates image aesthetic models on two fronts: CritiqueCanvas, which contains 519,136 instruction–response pairs for long‑form, multi‑dimensional critique across photography, painting, and virtual imagery, and ContextCanvas, which offers 301 expert‑reviewed use scenarios to assess contextual aesthetic suitability. The benchmark tests closed‑source, open‑weight general, and aesthetic‑specific multimodal large language models, revealing that models excel at critique generation but lag in context‑sensitive judgment. The study shows that aesthetic specialization does not reliably transfer to contextual suitability and highlights the need for culturally situated, evidence‑grounded suitability as a distinct objective for aesthetic modeling.

By Xuanwei Hu, Haoyu Dong, Kejun Wu, Tianyi Liu, Jianjun Gao
llmsmultimodalbenchmarks
More like this →
arXiv AI
Jun 2

Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

arXiv:2606. 02578v1 Announce Type: cross Abstract: Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a critical weakness: when visual evidence conflicts with textual cues, MLLM judges tend to reward plausible narratives over perceptually correct answers.

By Seojeong Park, Jiho Choi, Junyong Kang, Seonho Lee, Jaeyo Shin, Hyunjung Shim
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv AI
Jul 21

From Weights to Words: Expressing and Editing Preference Model Inferences in Natural Language

arXiv:2607. 16232v1 Announce Type: cross Abstract: The growing use of statistical learning algorithms to infer human preferences from high-dimensional choice data runs up against a fundamental challenge: choice alternatives typically differ in many ways simultaneously, so it is generally unclear which factors actually drove an observed decision and should be credited as preferences.

By Zachary Wojtowicz, Ayush Nayak, Jacob Andreas
llms
More like this →