arXiv Machine Learning

Visual-Aware Representation of Web Pages for Machine Learning Applications

The paper introduces a platform that renders web pages using FitLayout and captures their visual and structural properties in an RDF-based representation. The system offers a REST API for pipeline control, SPARQL queries for data retrieval, and a Python client to integrate with machine learning workflows. It demonstrates how rendered pages can be converted into graph representations to train graph neural networks for key content recognition, highlighting reproducibility and dataset sharing.

arXiv Computer Vision
Oct 15

Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation

arXiv:2610.20335v1 Announce Type: new Abstract: Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction,...

By Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
arXiv AI
2d ago

AmbiHTMLBench: A Benchmark for HTML-based Visually Rich Document parsing

AmbiHTMLBench is a new benchmark that evaluates how well models can convert real-world documents into executable HTML while preserving content, organization, and visual appearance. It contains 1,115 pages with structured annotations and uses both objective metrics (table fidelity, text error, layout consistency) and a reference-anchored subjective protocol (LLM judges aligned with human preferences). Results show that different systems excel in objective versus subjective measures, and fine‑tuning a small expert model improves performance, highlighting the benchmark’s role as a diagnostic tool for advanced document parsing.

By Boyu Niu, Yingjie Xiong, Shurui Li, Linhan Pang, Yuntong Zhang, Han Wang, Hao Wu, Kai Zuo, Xu Tang, Xuanhe Zhou, Fan Wu
arXiv AI
Jun 30

PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

arXiv:2606. 28344v1 Announce Type: cross Abstract: Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems depend on complex parsing pipelines that linearize HTML and discard layout, visual structure, and formatting.

By Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, Sewon Min
arXiv Machine Learning
Jul 1

FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

arXiv:2606. 32016v1 Announce Type: new Abstract: Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks.

By Zekai Chen, Kairui Yang, Xuaner Chen, Xunkai Li, Xun Wu, Rong-Hua Li, Guoren Wang
arXiv AI
Aug 18

Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement

arXiv:2608. 16628v1 Announce Type: new Abstract: Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities, such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data.

By Shenao Chen, Yidan Xu, Xiangmin Han, Rundong Xue, Duanpo Wu, Yuhan Gao, Chenggang Yan, Yue Gao
arXiv Machine Learning
Jul 22

One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models

arXiv:2607. 19128v1 Announce Type: new Abstract: Vision-language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored.

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan
arXiv Machine Learning
Jul 2

Generated Contents Enrichment

arXiv:2405. 03650v4 Announce Type: replace-cross Abstract: We study Generated Contents Enrichment (GCE), a conditional image-generation task in which a sparse scene description is first enriched through an explicit scene representation and then rendered into semantically richer visual content.

By Mahdi Naseri, Jiayan Qiu, Zhou Wang
arXiv Machine Learning
Oct 8

KGATE : a Knowledge Graph Embedding Training Environment

KGATE is a modular Python library for training knowledge graph embedding (KGE) models, built on PyTorch Geometric and TorchKGE. It allows users to assemble or plug in components such as initializers, encoders, decoders, losses, negative samplers, and evaluation metrics, and includes a preprocessing step to prevent data leakage. Benchmarks against six existing KGE libraries show that KGATE’s training time is comparable to the fastest while offering a broader set of features.

By Benjamin Loire, Galadriel Bri\`ere, C\'elia Brahimi, Antoine Toffano, Ana\"is Baudot