The paper introduces PotentialEP, a backpropagation‑free method for training generative models at CIFAR scale using Equilibrium Propagation. It shows that a predictive coding network with a small linear injection can relax to a state that inherently carries the gradient needed for sampling, allowing an Energy Matching model trained with backpropagation to be sampled by relaxation alone with negligible loss in FID. Moreover, the authors demonstrate that training end‑to‑end with Equilibrium Propagation yields FID scores comparable to backpropagation, marking the first competitive CIFAR‑10 generative model trained without backpropagation.
arXiv:2606. 03584v1 Announce Type: new Abstract: Equilibrium Propagation (EP) is a physics-based training framework that has primarily been employed in energy-based models, including continuous Hopfield networks, nonlinear resistive networks and coupled phase oscillators.
FORGE is a forward‑only test‑time adaptation technique designed for integer‑only vision models running on microcontrollers. It restores batch‑normalization statistics after BN folding by re‑normalizing each convolution’s per‑channel output using only forward‑pass estimates, enabling adaptation on deployed, folded integer models. The method achieves accuracy gains comparable to gradient‑based TENT, requires adapting only a few layers, works with single‑sample streaming, and has been validated on an ESP32‑S3 with minimal energy and latency overhead.
arXiv:2506.11030v2 Announce Type: replace-cross Abstract: Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that, despite its widespread success, s...
Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly.
arXiv:2608. 19540v1 Announce Type: new Abstract: Training fast generators on new domains with limited data remains challenging for two reasons.
The paper introduces SCION, a single-model framework that jointly learns generative and self‑supervised representation tasks without relying on labels or pretrained encoders. SCION uses a pixel‑space encoder conditioned on a flow timestep and a shared embedding, training contrastive and flow‑matching objectives simultaneously with gradient‑norm balancing and stop‑gradient techniques. On ImageNet 256×256, SCION achieves state‑of‑the‑art FID scores (8.92 with JiT‑B and 5.89/3.47 with JiT‑L) surpassing prior class‑unconditional and pretrained‑conditioned methods.
arXiv:2511.01061v2 Announce Type: replace-cross Abstract: Backpropagation (BP) powers modern deep learning, but it must hold every activation in memory until a global backward sweep consumes it, and...
CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.
The paper introduces Conditional Flow Matching (CFM) for engineering inverse design, comparing it to conditional diffusion models and cGANs on EngiBench structural and thermal benchmarks. CFM outperforms the baselines in cumulative and final optimality gaps, mean volume‑fraction deviation, and throughput, achieving up to 66× faster sample generation with fewer network evaluations. The study demonstrates CFM’s effectiveness as a warm‑start generator for gradient‑based refinement in PDE‑constrained design problems.
arXiv:2607. 13380v1 Announce Type: new Abstract: Predictive Coding (PC) offers a biologically motivated alternative to backpropagation via local weight updates, yet routing error between layers still relies on an autograd Jacobian-transpose ($J^\top$) product - the last non-local operation in PC.
Training fast generators on new domains with limited data remains challenging for two reasons. First, adapting a pretrained diffusion or flow model to a new domain leaves its costly multi-step sampling unaddressed, and existing acceleration methods are tied to the source parameterization--$ε$, $x$, $v$, or $u$--leaving heterogeneous pretrained models with no common acceleration target.
arXiv:2607. 27372v1 Announce Type: new Abstract: The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages.