arXiv AI

Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery

The paper presents a framework that enhances single-image nutrition estimation by using multimodal large language models to inventory visible foods and verify both food identity and the suitability of proposed 2D regions for portion estimation. A whole-image review identifies gaps and triggers a single targeted recovery pass, after which recovered regions are re-verified and reconciled into a final item set. The approach, which requires no task-specific fine-tuning, improves item-level grounding, mass accuracy, energy accuracy, and item-identity precision and recall compared to a retrieval baseline.

arXiv AI
Jun 9

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

arXiv:2606. 08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large multimodal datasets linking diverse foods to complete nutrient profiles.

By Runze Yan, Minxiao Wang, Jiaying Lu, Darren Liu, Xiao Hu, Hanqi Luo
arXiv AI
Aug 5

OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet

arXiv:2608. 03428v1 Announce Type: cross Abstract: Image based dietary assessment offers a scalable alternative to self reported food diaries, yet fine-grained food recognition remains challenging due to high intra-class variability and visually similar dishes.

By Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas, Daphni Plati, Eugenia Mylona, Eleni Georga, Nikos Tsiknakis, Antonis Sakellarios, Dimitrios I. Fotiadis
arXiv AI
Sep 23

Predicting Postprandial Glycemic Response from Meal Images, Clinical Variables, and Gut Microbiome Information

The study introduces a multimodal framework that predicts postprandial glycemic response (PPGR) by combining image-derived macronutrient estimates with clinical variables and gut microbiome data. It jointly performs macronutrient estimation from meal images and glucose prediction, using an attention-based module to model interactions between dietary and host-specific information. Evaluated on a real-world dataset, the model outperforms existing PPGR baselines that use image-derived inputs and nearly matches methods relying on manually reported macronutrients.

By Varvara Kondratyeva, Kamilia Zaripova, Nassir Navab, Azade Farshad
Hugging Face Trending Papers
Jul 7

RFHNet: Relational and Frequency-Aware Hashing Network for Large-Scale Fine-Grained Food Image Retrieval

Fine-grained food image retrieval is a key task in computational gastronomy, with applications in food traceability, dietary monitoring, and smart catering systems. Although hashing-based retrieval is attractive for large-scale search due to its storage efficiency and fast Hamming-distance computation, existing methods often perform poorly in fine-grained food scenarios, where subtle local semantics and frequency-sensitive visual cues are essential.

arXiv AI
Jun 10

MetaPlate: Counterfactual-Guided RAG-LLM Tool for Personalized Food Recommendation and Hyperglycemia Prevention

arXiv:2606. 10120v1 Announce Type: cross Abstract: Postprandial hyperglycemia is a key risk factor for metabolic disorders; however, existing dietary guidance is often static, impractical, and insufficiently personalized, providing recommendations that are difficult to follow or not impactful.

By Asiful Arefeen, Carol Johnston, Hassan Ghasemzadeh
arXiv AI
Sep 4

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

CulturalMenuBench is a new benchmark comprising 4,870 culinary items in 10 languages across 18 regions, designed to test multimodal language models on tasks that combine dish recognition, step-by-step cooking images, ingredients, procedural text, and regional labels. The benchmark reveals a large knowledge‑application gap: models that score over 94% on standard multiple‑choice questions fall to at most 56% when attributing dishes to Chinese regional cuisines, indicating that cultural knowledge is present but not activated by visual input. Diagnostic analyses show that accuracy is driven by visual distinctiveness rather than cultural structure, and that removing sequential cooking images selectively harms process‑grounded tasks, confirming the need for procedural evidence.

By Bo Zeng, Linfeng Gao, Peiqin Lin, Yu Zhao, Mingyan Zeng, Yu Tong, Xintong Wang, Linlong Xu, Longyue Wang, Weihua Luo, Qinggang Zhang, Jinsong Su