arXiv Machine Learning

VisKG-LM: Compiling Knowledge Graphs into Visual Memory for Multiple-Choice Question Answering

VisKG‑LM proposes compiling retrieved knowledge graph subgraphs into static visual memories rather than re‑encoding them during each inference step. The method serializes each subgraph as Relation‑Labeled Paths, renders them as images that preserve the graph’s branching structure, and caches these images for reuse. At inference, a language model processes the question and candidate text first, then consults the cached visual memory only at its final layer, yielding improved performance on CommonsenseQA, OpenBookQA, and MedQA‑USMLE compared to both text‑only baselines and a large vision‑language model.

arXiv Machine Learning

Aug 27

A Storage-Retrieval Gap in Parametric Knowledge Graph Memory

The paper investigates a parametric approach to knowledge graph memory by compiling each entity into a LoRA adapter, enabling zero‑cost query-time retrieval via weight injection. On the MetaQA dataset, these adapters encode context‑free factual knowledge, improving exact‑match scores by up to +0.243 over a base model and achieving an oracle gap of +0.283. However, the stored knowledge is not recoverable through similarity or embedding‑based methods, indicating that knowledge is stored locally and does not transfer across semantically neighboring entities.

By Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp

llms rag fine-tuning

arXiv Computer Vision

Sep 22

Visual Graph Reasoning via Knowledge Compilation

arXiv:2609.22327v1 Announce Type: cross Abstract: Visual graph reasoning requires answering graph-theoretic questions directly from graph images, where graph topology and state are conveyed visually...

By Rongzheng Wang, Zhe Wang, Ke Qin, Rongwei Wang, Muquan Li, Yizhuo Ma, Yihong Huang, Jielei Wang, Shuang Liang

llms reinforcement-learning multimodal benchmarks

arXiv Computation and Language

4d ago

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

arXiv:2609.35942v1 Announce Type: new Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs i...

By Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

llms nlp reinforcement-learning multimodal benchmarks

arXiv AI

Jun 3

Visual Graph Scaffolds for Structural Reasoning in Large Language Models

arXiv:2606. 02673v1 Announce Type: new Abstract: Graphs have been used to enhance large language models (LLMs) for structured reasoning, mostly as external knowledge sources are provided to models at test time.

By Runlin Lei, Xiaokui Xiao, Zhewei Wei

llms nlp fine-tuning efficiency

arXiv AI

Aug 7

ViSR-KGC: Visual Subgraph Reasoning with Vision-Language Models for Multimodal Knowledge Graph Completion

arXiv:2608. 05833v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to infer missing entities or relations from incomplete graph structures, and has evolved into multimodal knowledge graph completion (MMKGC), where entities are associated with multiple modalities such as text and images.

By Jiafan Li, Mengxue Yang, Jiaqi Zhu, Liang Chang, Ying Li, Hongan Wang

llms rag multimodal

arXiv AI

Sep 1

Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

arXiv:2608.28978v1 Announce Type: new Abstract: Knowledge graphs have been proposed as a structured alternative to flat retrieval-augmented generation for long-term agent memory, on the assumption th...

By Theo Rusu, Sourena Khanzadeh, Manar Alalfi

llms rag agents benchmarks

arXiv AI

Jun 2

PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models

arXiv:2602. 00415v2 Announce Type: replace Abstract: Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief.

By Zhisheng Chen, Tingyu Wu, Zijie Zhou, Zhengwei Xie, Jinhan Li, Ziyan Weng, Liang Lin, Jingwei Song, Zikai Xiao, Yingwei Zhang

llms multimodal benchmarks

arXiv AI

Jul 21

Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering

arXiv:2510. 01483v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query.

By Mohamad Al Mdfaa, Svetlana Lukina, Timur Akhtyamov, Arthur Nigmatzyanov, Dmitrii Nalberskii, Sergey Zagoruyko, Gonzalo Ferrer

llms agents nlp robotics multimodal benchmarks

arXiv Machine Learning

Jun 2

KG-Guard: Graph-Based Hallucination Detection for Knowledge Base Question Answering

arXiv:2606. 00328v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for knowledge base question answering (KBQA), where answering requires selecting entities from a question-specific knowledge-graph subgraph.

By Albert Sawczyn, Piotr Bielak, Tomasz Kajdanowicz

llms rag nlp benchmarks safety

arXiv AI

Sep 15

FedV-KGQA in Practice: Design Lessons and an Interactive Prototype

FedV-KGQA addresses multi‑hop question answering over vertically partitioned knowledge graphs where each silo holds disjoint relation types. The system trains local embeddings, concatenates silo‑specific entity views, anchors questions at a topic entity, and ranks candidates without sharing raw triples. Experiments show federated fusion nearly matches centralized accuracy, that anchoring and enrichment are more critical than embedding choice, and that the cheapest encoder depends on target accuracy.

By Md Saikat Islam Khan Bappy, Oshani Seneviratne

rag nlp

arXiv AI

3d ago

Concept-Grounded Attention: A Controlled Evaluation of Graph-Injected Attention, Temporal Versioning, and Epistemic Status

arXiv:2609.38684v1 Announce Type: new Abstract: Knowledge-intensive language-model systems typically represent external knowledge as text chunks or static graphs, with limited support for concept evo...

By Sachin Dev Duggal, Pradyumna Swarnalatha Ramanna, Alexandros Vassiliades

llms fine-tuning safety

arXiv AI

Sep 3

Seed-Anchored Budget-Bounded Graph Rendering for Question Answering on Industry-Standard Power-Grid Information and Exchange Models

The paper introduces seed‑anchored graph rendering, a deterministic technique for generating context‑bounded graph representations that prioritize query‑local evidence without adding extra parameters beyond a shared hop bound and context budget. Evaluations on Common Information Model (CIM) network models and the SmallGrid topology show that this method retains all single‑ and multi‑hop evidence, improving accuracy from 0.450 to 0.970 under an 8,000‑character budget, and outperforms or matches other graph‑RAG approaches while avoiding LLM graph‑construction tokens.

By Jayakumar Manoharan, Yamini Sehgal

llms nlp

Related stories