arXiv:2607. 15412v2 Announce Type: replace Abstract: Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously.
arXiv:2607. 15412v1 Announce Type: new Abstract: Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously.
arXiv:2605.05660v3 Announce Type: replace Abstract: Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, most exis...
arXiv:2610. 13773v1 Announce Type: cross Abstract: Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms.
arXiv:2607. 22906v1 Announce Type: new Abstract: We study adaptive gradient descent for continuously differentiable, possibly nonconvex objectives under one-sided H\"older regularity.
arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
The paper introduces Federated Adaptive Gradient Correction (FA‑GC), a method that corrects each gradient before it is incorporated into the running averages used by adaptive optimizers like Adam in federated learning. Compared to a naive correction approach (FA‑NT) that only adjusts the final update direction, FA‑GC ensures that the corrected direction only vanishes at global stationary points and provides convergence guarantees without client‑heterogeneity terms. Experiments on seven image and language tasks show that FA‑GC with Adam achieves target accuracy in fewer rounds and, in some cases, reduces total communication compared to uncorrected local Adam and FA‑NT.
The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.
arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.
arXiv:2610.07290v1 Announce Type: cross Abstract: Recent advances in penalty-based methods for stochastic bilevel optimization (SBO) have eliminated the need for second-order derivative oracles. Howe...
arXiv:2606. 17526v1 Announce Type: new Abstract: Efficient optimization is essential for training large language models.
arXiv:2609.08133v1 Announce Type: cross Abstract: In nonconvex optimization problems arising in geometric machine learning, data augmentation is commonly used to promote invariance by averaging empir...