← Back to all news
arXiv Computer Vision By Yongsheng Lian

Single-Image Food Mass Estimators Do Not Transfer Their Gram Scale Across Datasets

Read the original on arXiv Computer Vision →

The study fine‑tuned DINOv2‑base on 2,253 weighed Nutrition5k dishes to estimate food mass from a single RGB photo and compared it with the zero‑shot vision‑language model Qwen3‑VL‑32B across Nutrition5k and three external datasets. While the fine‑tuned model achieved a mean absolute error of 24.8 g on its own domain—better than Qwen—it failed to outperform Qwen on external datasets and its gram scale varied with scene coverage and differed per food type. Adding a single labeled photo of the same food reduced external error significantly, indicating that single‑image mass estimators require labeled photographs from the target setting.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Aug 5

OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet

arXiv:2608. 03428v1 Announce Type: cross Abstract: Image based dietary assessment offers a scalable alternative to self reported food diaries, yet fine-grained food recognition remains challenging due to high intra-class variability and visually similar dishes.

By Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas, Daphni Plati, Eugenia Mylona, Eleni Georga, Nikos Tsiknakis, Antonis Sakellarios, Dimitrios I. Fotiadis
llmsfine-tuningmultimodal
More like this →
arXiv AI
Oct 9

Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery

The paper presents a framework that enhances single-image nutrition estimation by using multimodal large language models to inventory visible foods and verify both food identity and the suitability of proposed 2D regions for portion estimation. A whole-image review identifies gaps and triggers a single targeted recovery pass, after which recovered regions are re-verified and reconciled into a final item set. The approach, which requires no task-specific fine-tuning, improves item-level grounding, mass accuracy, energy accuracy, and item-identity precision and recall compared to a retrieval baseline.

By Jingbo Yue, Bruce Coburn, Jinge Ma, Jui-Feng Chi, Fengqing Zhu
llmsfine-tuningmultimodal
More like this →
arXiv AI
Oct 19

NutriVision: Ingredient-Conditioned Fusion and Prediction for Single-Image Food Nutrition Estimation

NutriVision is an end‑to‑end framework that estimates calories, mass, fat, carbohydrates, and protein from a single RGB image and optional ingredient list. It generates depth information with DepthAnything‑V3, encodes ingredient descriptions via CLIP, and fuses visual geometry and ingredient semantics through an Ingredient‑Conditioned Frequency‑Aligned Fusion Module, an Ingredient‑Aware Mask‑based Prediction Head, and modality‑specific Internal Semantic Modeling blocks. On the Nutrition5k dataset, NutriVision achieves a mean PMAE of 13.60 ± 0.10 %, outperforming prior methods by significant margins.

By Avinash Anand, Aman Kumar, Chaitanya Lakhchaura, Ashutosh Kumar, Akshita Abrol, Timothy Liu, Zhengkui Wang
multimodal
More like this →