arXiv Computer Vision By Daniel Chong, Haomiao Chen, Keith W Jamison, Raja Marjieh, Ilia Sucholutsky, Nori Jacoby, Amy Kuceyeski

Alignment of visual brain encoding models with individual perceptual similarity judgments

Read the original on arXiv Computer Vision →

The study explores how to fine‑tune visual brain encoding models so that their internal representations better match individual human perceptual similarity judgments. Using similarity data from 100 images collected from eight participants in the Natural Scenes Dataset, the authors adapted a generic similarity model (DreamSim) to create personalized perceptual similarity models for each subject. They then fine‑tuned each participant’s baseline encoder with either generic or personalized similarity information, finding that models trained with personalized judgments were most aligned with human perceptual similarity and preserved inter‑subject correlations of those judgments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Aug 31

Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision

The study investigates whether transferring relational structure from human mental representations to deep neural networks (DNNs) can improve fine‑grained alignment between the two. Using unsupervised Gromov‑Wasserstein optimal transport, the authors show that fine‑tuning pre‑trained DNNs with Relational Knowledge Distillation (RKD) brings the networks close enough to human representations to align at the individual‑object level on a test set of concepts not seen during training. The improvement is driven mainly by a more human‑like global structure of category distances, while local nearest‑neighbor overlap remains largely unchanged.

By Yuria Shimizu, Soh Takahashi, Takato Horii, Masafumi Oizumi