arXiv AI
Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processing
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 4
MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers
arXiv:2606. 04366v1 Announce Type: new Abstract: Conventional patchified Transformers operate on uniform spatial partitions, distributing computational effort evenly across the domain irrespective of local features.
arXiv AI
Oct 1
Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations
The paper introduces the Cluster Attention Neural Operator (CANO), a neural operator that uses a novel cross‑attention mechanism to dynamically cluster queries while keeping full‑resolution keys and values. This design eliminates the slice compression and weight‑sharing limitations of previous Transformer‑based operators, maintaining fast computation and global interactions. Experiments on a range of fluid and solid dynamics benchmarks—including Navier‑Stokes, Airfoil, Plasticity, Pipe Turbulence, and Composites—show that CANO achieves lower errors than existing baselines and demonstrates strong geometric adaptability and temporal consistency.
arXiv AI
4d ago
Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention
arXiv:2610.42147v1 Announce Type: new Abstract: Agentic workflows are increasingly shifting towards long-horizon tasks, such as open-ended scientific work, where models benefit from long context wind...
arXiv Machine Learning
Sep 7
Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures
The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
arXiv Computation and Language
Oct 12
Contextual Correction of Parallel Draftsfor Speculative Decoding
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
arXiv AI
Jun 26
Error-Conditioned Neural Solvers
arXiv:2606. 27354v1 Announce Type: cross Abstract: Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution.