← Back to all news
arXiv AI By Avinash Anand, Aman Kumar, Chaitanya Lakhchaura, Ashutosh Kumar, Akshita Abrol, Timothy Liu, Zhengkui Wang

NutriVision: Ingredient-Conditioned Fusion and Prediction for Single-Image Food Nutrition Estimation

Read the original on arXiv AI →

NutriVision is an end‑to‑end framework that estimates calories, mass, fat, carbohydrates, and protein from a single RGB image and optional ingredient list. It generates depth information with DepthAnything‑V3, encodes ingredient descriptions via CLIP, and fuses visual geometry and ingredient semantics through an Ingredient‑Conditioned Frequency‑Aligned Fusion Module, an Ingredient‑Aware Mask‑based Prediction Head, and modality‑specific Internal Semantic Modeling blocks. On the Nutrition5k dataset, NutriVision achieves a mean PMAE of 13.60 ± 0.10 %, outperforming prior methods by significant margins.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jun 9

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

arXiv:2606. 08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large multimodal datasets linking diverse foods to complete nutrient profiles.

By Runze Yan, Minxiao Wang, Jiaying Lu, Darren Liu, Xiao Hu, Hanqi Luo
llmsdiffusionfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 9

Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery

The paper presents a framework that enhances single-image nutrition estimation by using multimodal large language models to inventory visible foods and verify both food identity and the suitability of proposed 2D regions for portion estimation. A whole-image review identifies gaps and triggers a single targeted recovery pass, after which recovered regions are re-verified and reconciled into a final item set. The approach, which requires no task-specific fine-tuning, improves item-level grounding, mass accuracy, energy accuracy, and item-identity precision and recall compared to a retrieval baseline.

By Jingbo Yue, Bruce Coburn, Jinge Ma, Jui-Feng Chi, Fengqing Zhu
llmsfine-tuningmultimodal
More like this →
arXiv AI
Sep 23

Predicting Postprandial Glycemic Response from Meal Images, Clinical Variables, and Gut Microbiome Information

The study introduces a multimodal framework that predicts postprandial glycemic response (PPGR) by combining image-derived macronutrient estimates with clinical variables and gut microbiome data. It jointly performs macronutrient estimation from meal images and glucose prediction, using an attention-based module to model interactions between dietary and host-specific information. Evaluated on a real-world dataset, the model outperforms existing PPGR baselines that use image-derived inputs and nearly matches methods relying on manually reported macronutrients.

By Varvara Kondratyeva, Kamilia Zaripova, Nassir Navab, Azade Farshad
multimodal
More like this →
arXiv AI
Aug 5

OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet

arXiv:2608. 03428v1 Announce Type: cross Abstract: Image based dietary assessment offers a scalable alternative to self reported food diaries, yet fine-grained food recognition remains challenging due to high intra-class variability and visually similar dishes.

By Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas, Daphni Plati, Eugenia Mylona, Eleni Georga, Nikos Tsiknakis, Antonis Sakellarios, Dimitrios I. Fotiadis
llmsfine-tuningmultimodal
More like this →