arXiv Machine Learning
Undertone: Personalizing Frozen VLM Aesthetic Judges from Few Ratings
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
2d ago
Parsed Text Understates Vision-Language Models: From Below to Above Expert Agreement on Fine-Grained Facial Emotion
The paper introduces EmoNet‑Face‑HQ, a benchmark of 2,500 AI‑generated portraits each evaluated by four psychology experts across 40 emotion categories. It shows that most vision‑language models (VLMs) perform no better than random guessing unless fine‑tuned, but that changing the prompt to a simple yes/no per category can raise a VLM’s agreement with experts from chance to near expert levels. When evaluated using probability‑of‑yes scores on the five most‑agreed‑upon categories, eleven open‑weight VLMs achieve a quadratic‑weighted κ of 0.425, below the 0.468 two‑expert agreement, yet their average probability‑of‑yes score (0.553) matches the benchmark’s fine‑tuned model and outperforms it in three cases without any training.
arXiv Computation and Language
Oct 7
The Failure Is in the Readout: Fine-Grained Emotion Recognition Benchmarks Measure Elicitation, Not Perception
The paper introduces EmoNet‑Face‑HQ, a fine‑grained emotion recognition benchmark that uses generated portraits and a 40‑category taxonomy to evaluate vision‑language models (VLMs). It finds that VLMs perform poorly when asked to generate responses but can match or surpass a fine‑tuned model (Empathic‑Insight‑Face) when their logits are read directly as binary queries. The study shows that the benchmark’s difficulty lies in the readout process rather than in perception, and that graded probability outputs yield better performance than simple yes/no questions.
arXiv Machine Learning
Sep 24
Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judges
The paper investigates whether panels of vision‑language models (VLMs) can reliably judge image aesthetics. It shows that a panel of holistic judges rarely outperforms its best member, but when each model scores images on five rubric‑defined dimensions and these dimension scores are fused across model families, the panel consistently beats the best single VLM on two datasets (EVA and PARA). The study demonstrates that the value of a panel depends on the type of input it receives, and that dimension‑based fusion yields measurable gains at the cost of additional labeling and API usage.
arXiv AI
Oct 12
Unsupervised Multimodal Reward Modeling
The paper presents an unsupervised multimodal reward model built on a frozen Qwen3‑VL backbone. By adding a linear head to an intermediate layer and training it to rank a web caption higher than a perturbed version of itself, the authors avoid human preference labels and achieve competitive performance—outperforming GPT‑4o on several benchmarks with far fewer parameters and compute. The approach demonstrates that a carefully chosen intermediate layer, simple perturbation-based training, and verifier‑driven fine‑tuning can yield effective vision‑language rewards without costly annotation.
arXiv Computer Vision
21h ago
ScoreDial: Precise Control of Visual Attributes in Image Generation via Regressor Guidance from Limited Human Ratings
arXiv:2611.05004v2 Announce Type: replace Abstract: Requested-value control is stricter than continuous steering: given $c$ on a human-rating scale, the generated image should attain that value from...
arXiv Machine Learning
Oct 2
The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization
The paper introduces Bayesian Pairwise Merging (BPM), a method that personalizes a shared expert library by inferring reward trade‑off weights from users’ pairwise choices. BPM treats these weights as latent variables, learns a posterior over them, and uses the posterior mean as merge coefficients, thereby converting natural pairwise feedback into actionable model personalization. Experiments on radiology summarization, image captioning, and story generation show BPM outperforming uniform merging and achieving high win rates with limited feedback, while also quantifying uncertainty in the inferred trade‑offs.