The Flow has not summarised this story yet — read it at arXiv Computation and Language.
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the one-hidden-layer ReLU model of Awasthi et al.
arXiv:2609.09130v1 Announce Type: new Abstract: An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input...
The paper establishes the optimal incremental first‑order oracle (IFO) complexity for nonconvex finite‑sum optimization under individual smoothness, proving a matching lower bound that closes a previously missing √{n} factor. It also refines the analysis of the PAGE algorithm under the global Polyak‑Lojasiewicz condition, providing tighter guarantees for different ranges of the condition number. The authors introduce a novel dense weak hiding construction that yields these lower bounds and demonstrates the limits of existing methods.
The paper investigates restricted eigenvalue (RE) bounds for norm‑regularized estimators under heavy‑tailed designs. It shows that the previously conjectured sample‑size law based on Gaussian width fails for heavy‑tailed measurements, due to a phenomenon called simultaneous threshold occupancy. The authors provide explicit counterexamples, derive worst‑case sample‑complexity bounds, and compare the behavior of heavy‑tailed versus Gaussian designs on constant‑width polyhedral descent cones.
arXiv:2608. 28150v1 Announce Type: new Abstract: Which geometry controls the rank complexity of normalized softmax attention?
arXiv:2608. 16878v1 Announce Type: cross Abstract: For any convex body $\mathcal{K}\subset\mathbb{R}^{n}$ containing a unit ball, the spectral gap of Hit-and-Run is $\Omega(1/(n^2 C_{\mathsf{PI}}))$, where $C_{\mathsf{PI}}$ is the Poincar\'e constant of the uniform distribution $\pi$ over $\mathcal{K}$.