The paper investigates the macroscopic dynamics of large language model (LLM)-driven agents by applying a least‑action principle to estimate the generative directionality of LLMs embedded within agents. Through experimental measurement of transition probabilities between LLM‑generated states, the authors find evidence of a tendency toward detailed balance, suggesting that LLM generation may be governed by underlying potential functions rather than explicit learning rules. This approach offers a model‑agnostic, effective description of LLM generative dynamics that could serve as a foundational theory for complex AI systems.
By Zhuo-Yang Song, Qing-Hong Cao, Ming-xing Luo, Hua Xing Zhu
arXiv:2310.04363v3 Announce Type: replace
Abstract: Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits t...
By Edward J. Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, Esmeralda S. Whitammer
The paper introduces Particle GFlowNets, showing that Generative Marginalization Models (MaMs) are equivalent to Generative Flow Networks. It extends MaMs to non‑autoregressive sampling and proposes an automatic full‑state rejuvenation criterion based on the Gelman‑Rubin statistic to accelerate learning. Experiments demonstrate significant training speedups in large combinatorial spaces.
By Tiago da Silva, Diego Mesquita, Salem Lahlou
arXiv:2502. 19049v3 Announce Type: replace Abstract: Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations, dictated by a diffusion function.
By Patrick Seifner, Kostadin Cvejoski, David Berghaus, Cesar Ojeda, Ramses J. Sanchez
arXiv:2607.00588v2 Announce Type: replace
Abstract: Continuous diffusion language models offer an alternative to autoregressive generation, but their generations may suffer from repetition. We find t...
By Shuai Zhang, Zijie Chen, Huachuan Qiu, Hongliang He, Lun Du, Zhenzhong Lan
arXiv:2602. 14885v2 Announce Type: replace-cross Abstract: Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that restricts network dynamics to gradient-like flows.
By Ram\'on Nartallo-Kaluarachchi, Renaud Lambiotte, Alain Goriely
arXiv:2606. 10530v1 Announce Type: cross Abstract: Recent developments in brain recording are driving a demand for machine learning tools capable of decoding the latent structure of large populations of neurons.
By Shufeng Kong, Fumei Deng, Xinyi Dong, Caihua Liu, Weiwei Chen, Yingheng Wang, Daniel Cao, Azahara Oliva, Antonio Fernandez-Ruiz, Carla Gomes
arXiv:2506. 20771v2 Announce Type: replace Abstract: We propose a latent score-based generative AI framework for learning stochastic, non-local closure models and constitutive laws in nonlinear dynamical systems of computational mechanics.
By Xinghao Dong, Huchen Yang, Jin-Long Wu
The paper investigates when language diffusion models, specifically Uniform-based Discrete Diffusion Models (UDDMs), shift from memorizing training data to generalizing to new data. It shows that UDDMs act as associative memories, forming basins of attraction around stored examples without requiring an explicit energy function. By measuring token recovery and conditional entropy, the authors identify a sharp transition governed by training set size, where memorization (vanishing entropy) gives way to generalization (finite entropy).
By Bao Pham, Mohammed J. Zaki, Luca Ambrogioni, Dmitry Krotov, Matteo Negri
arXiv:2606. 31576v1 Announce Type: new Abstract: The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomolecule generation.
By Ole Winther, Paul Jeha, Sander Dieleman, Andriy Mnih, Manfred Opper, Andrea Dittadi
arXiv:2602. 03564v2 Announce Type: replace Abstract: Time series forecasting can be viewed as a generative problem that requires both semantic understanding over contextual conditions and stochastic modeling of continuous temporal dynamics.
By Mingyue Cheng, Yaguo Liu, Daoyu Wang, Xiaoyu Tao, Qi Liu
arXiv:2606. 02133v1 Announce Type: cross Abstract: Non-monotonic sequence generation methods, such as masked diffusion models, provide a flexible alternative to left-to-right autoregressive modeling by allowing tokens to be generated in non-fixed and prescribed orders.
By Yangtian Zhang, Zhe Wang, Arthur Gretton, Rex Ying, David van Dijk, Michalis K. Titsias, Jiaxin Shi