← Back to all news
arXiv Machine Learning By Aadi Dash, Lennon J. Shikhman, Michael Galarnyk

Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIP

Read the original on arXiv Machine Learning →

The study investigates whether text conditioning improves neural partial differential equation (PDE) surrogates by using OperatorCLIP. Experiments compare an unconditioned Fourier Neural Operator (FNO), a constant-sentence FiLM control, and a task‑specific text description trained with contrastive alignment across Darcy2D, ShallowWater2D, and 3D compressible Navier‑Stokes. Results show that constant conditioning yields lower mean test error on 2D tasks, while task text plus alignment performs similarly or worse, with significant seed uncertainty and no clear semantic ordering from prompt interventions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 17

From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation

The paper introduces a 3D-CLIP encoder trained with structured hard negatives to improve vision‑language alignment for text‑to‑CT generation. This encoder drives a latent diffusion model that operates directly in 3D latent space, eliminating spatial artifacts from super‑resolution pipelines. Experiments on the CT‑RATE dataset show state‑of‑the‑art image fidelity and factual correctness across 18 pathological conditions, with lower inference time and GPU memory usage than competing methods.

By Daniele Molino, Camillo Maria Caruso, Filippo Ruffini, Paolo Soda, Valerio Guarrasi
diffusionmultimodalbenchmarkssafety
More like this →
arXiv AI
Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu
llmsragagentsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 15

Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP

The study evaluates whether signals from a frozen vision‑language model, InstructBLIP, can reliably indicate answerability in visual question answering. Using a 10K subset of UNK‑VQA, the authors test predictive uncertainty, cross‑modal alignment, visual grounding, visual dependence, internal representations, and learned probes, finding that all signals yield weak discrimination (AUROC 0.50‑0.54). Even interventions that remove question‑relevant visual evidence or increase probe capacity do not substantially improve answerability detection, underscoring that visual dependence and related metrics are not robust proxies for answerability.

By Minjun Kim, Hyunmin Song, Changwon Lim
llmsnlpmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 12

Teaching neural decoders with language

The paper introduces Language as Supervision (LaS), a method that aligns neural activity with natural‑language class descriptions via learned projections and uses candidate text prototypes for classification. Evaluated across seven encoder architectures on visual, motor, and speech decoding tasks from scalp and intracranial recordings, LaS boosts speech‑decoding accuracy by up to 17.15 percentage points compared to traditional class‑label supervision. The approach also enables reusable candidate sets, subject‑ and session‑adaptation, and conditional unseen‑frequency decoding in steady‑state visual evoked potentials, with dual‑sided attribution visualizations highlighting both neural and textual contributions to class matching.

By Xuan Wen, Yonghao Song, Yuhong Zhang, Guoliang Li, Tzyy-Ping Jung, Xiaorong Gao
More like this →