The paper introduces a certified continuation framework for computing and training deep equilibrium networks (DEQs). It uses compact input homotopy and a rounded Newton tracker for inference, and augments local-plus-low-rank recurrence with programmable dormant bilinear rank‑one channels for training. The approach guarantees polynomial‑time bit complexity, with certified bounds on inference and training error budgets.
arXiv:2608. 12655v1 Announce Type: new Abstract: A flat training curve does not reveal whether a neural network has reached a global optimum, is locally trapped, is representation-limited, or is mismatched to its trainer.
Deep equilibrium models promise input-adaptive implicit computation: harder problems should demand more solver iterations, and the solved equilibrium should encode the result of genuine iterative inference. We report a cautionary study of a port-Hamiltonian DEQ with a learned initialization on two reasoning tasks -- ProofWriter entailment over frozen DeBERTa embeddings and a BFS-verified graph-reachability benchmark -- in which the implicit computation is a silent no-op.
arXiv:2607. 11116v1 Announce Type: cross Abstract: Deep equilibrium models promise input-adaptive implicit computation: harder problems should demand more solver iterations, and the solved equilibrium should encode the result of genuine iterative inference.
arXiv:2610. 31386v1 Announce Type: cross Abstract: Equilibrium neural networks define their hidden state as the fixed point of a contraction, so their effective depth is set at evaluation time rather than by the architecture.
arXiv:2606. 00716v1 Announce Type: new Abstract: Inference and control in engineered physical systems pay a heavy physics cost at deployment: state estimators, inverse-problem solvers, model-predictive controllers, schedulers, and observers are often not closed-form and must re-solve a numerical optimization per instance, with the operator re-supplied each time.
arXiv:2602. 03024v2 Announce Type: replace-cross Abstract: Deep Equilibrium Models (DEQs) have emerged as a powerful paradigm in deep learning, offering the ability to model infinite-depth networks with constant memory usage.
arXiv:2601. 16884v3 Announce Type: replace Abstract: We study multigrade deep learning (MGDL) as a principled framework for structured error refinement in deep neural networks.
arXiv:2512. 23075v5 Announce Type: replace-cross Abstract: Policy gradient methods for Large Language Models optimize a policy $\pi_\theta$ via a surrogate objective computed from samples of a rollout policy $\pi_{\text{roll}}$.
arXiv:2607.14576v2 Announce Type: replace-cross Abstract: Whether a deep residual architecture trains stably is usually determined by training it, which is expensive and answers the question only for...
The paper presents a convergence framework for deep $V$‑learning over a finite horizon $H$, deriving explicit bounds on policy loss by decomposing the Bellman update error into six residuals. It shows how $L^s$ concentrability controls expected $L^1$ loss, quantifies the impact of shared sampling across horizon levels, and provides optimal and near‑optimal sample allocations for statistical error rates. The work also establishes sharp action‑gap bounds under a margin condition, transfers optimal‑gap results to frozen‑iterate gaps, and offers consistency guarantees for generative‑reset approximate‑ERM procedures with exact action scores.
arXiv:2607. 20594v1 Announce Type: cross Abstract: When does a weight-tied looped transformer -- one block applied T times -- implement an actual algorithm?
The paper investigates weight perturbation (WP) as a derivative-free alternative to backpropagation for training fully connected MNIST networks. In shallow architectures, large-batch WP attains about 97% test accuracy, close to backpropagation, but under a ten‑epoch budget WP lags behind (90.2% vs. 97.4%). The study also shows diminishing returns beyond a batch size of 4,000 and that learning‑rate reduction and averaging of perturbation updates can stabilize deeper networks, though they do not yet match backpropagation accuracy.