Executing Causal Structure Learning with Linear-Attention Transformers

The paper demonstrates that a fixed‑weight transformer can exactly execute one update step of a standard continuous causal discovery algorithm, and by stacking such blocks it reproduces the algorithm’s optimization trajectory. The transformer maintains both the current graph and the algorithm’s multiplier, which is shown to be essential for accurate updates. Experiments confirm that the constructed block matches the reference solver to floating‑point precision and that arithmetic replay on synthetic and benchmark networks inherits the solver’s performance, while ordinary attention models fail to reliably execute the update or scale to larger graphs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• benchmarks