← Back to all news
arXiv Computation and Language By Karthik Sridhar, Atharva Gupta, Nishant Pradhan, Murari Mandal, Dhruv Kumar, Saurabh Deshpande

Semantics or Structure? Auditing Text Sensitivity in Multimodal Time-Series Forecasting

Read the original on arXiv Computation and Language →

The paper investigates whether multimodal time‑series forecasting models actually use the semantic content of accompanying text. By systematically perturbing the text—replacing it with empty, constant, shuffled, or cross‑domain sentences—the authors find that mean squared error changes by less than 0.5 % across several architectures, indicating that text does not drive performance gains. They also show that removing a co‑shipped numeric column restores the reported improvements, suggesting that the models rely on other signals rather than textual semantics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 12

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

The paper introduces a synthetic benchmark for multimodal time‑series forecasting that evaluates how well text annotations contribute to predictions. By generating controlled signals with semantically correct, incorrect, and irrelevant annotations, the authors can precisely measure the true information content. Six mutual‑information estimators (KSG, MINE, InfoNCE, CCA, PID, and V‑information) are tested, all correctly ranking useful annotations and enabling annotation auditing without model training. The benchmark also highlights each estimator’s limitations and validates findings on seven real datasets, providing practical guidelines for metric implementation.

By Emma Andrews, Gianmarco Mengaldo
multimodalbenchmarks
More like this →
arXiv Computation and Language
Oct 28

DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding

DocXEval is the first comprehensive framework that adapts existing explainable AI methods and evaluation metrics to multimodal document understanding, covering text, layout, and image modalities. The authors systematically evaluate 13 XAI methods across 3 tasks, 8 datasets, 4 models, and 17 metrics, finding that Occlusion consistently performs best in faithfulness and interpretability. The study also provides meta‑evaluation insights, diagnostic analyses of modality contributions, spatial attribution patterns, and potential dataset biases, offering a benchmark and practical guide for future research.

By Saifullah Saifullah, David Dembinsky, Stefan Agne, Andreas Dengel, Sheraz Ahmed
llmsmultimodalbenchmarks
More like this →