← Back to all news
arXiv AI

One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: A Controlled Comparison of an Iterative Neural Minimizer, Set Transformers and Classical Estimators

The paper introduces the Neural Function Minimizer (NFM), a one-shot method that iteratively moves a position across a one‑dimensional domain to locate the global minimum from twenty noisy samples. In a controlled benchmark, the NFM matches Set Transformers on location error but achieves lower regret, especially with mixture heads, and outperforms classical estimators on unseen functions. The authors plan to release the benchmark, models, and harness for further research.

Related stories

arXiv AI
Oct 8

One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: An Iterative Neural Minimizer Compared with Set Transformers and Classical Estimators

The paper introduces the Neural Function Minimizer (NFM), an iterative neural model that predicts the global minimum of a noisy one‑dimensional function from only twenty samples. It compares the NFM to two Set Transformers and classical zero‑query estimators, finding that all learned models outperform classical methods on held‑out data, with the NFM achieving the lowest error relative to the Gaussian‑process posterior argmin. The study also examines how different estimator outputs affect tie‑breaking between equally deep valleys and plans to release a benchmark and self‑checking harness.

By Qusay Muzaffar, David Levin, Michael Werman
llmsbenchmarks
More like this →
arXiv Machine Learning
Aug 27

Why and When Neural Networks Improve Local Approximation in Optimization

The paper investigates why neural network surrogates sometimes improve and sometimes worsen derivative‑free optimisation performance. It identifies three key factors—role (whether the surrogate proposes candidates or replaces gradients), radius (the neighbourhood within which a local model is reliable), and room (whether the base method can still progress)—that determine when a learned local model is beneficial. Experiments on 117 benchmark instances show that providing surrogate‑approved candidates boosts success rates, while replacing gradients or ignoring the radius can reduce them.

By Chengkuo Bian, Pengcheng Xie
benchmarks
More like this →
arXiv AI
Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko
llmsfine-tuning
More like this →
Hugging Face Trending Papers
Jun 28

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks

A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation. Adam's per-coordinate preconditioner drifts along each symmetry orbit, which pulls the trajectory off the symmetry quotient where the optimization lives and blurs the singular-learning rate the quotient makes readable.

llmscomputer-vision
More like this →
arXiv Machine Learning
Jul 13

Training, Reading, and Editing Legible Transformers

arXiv:2607. 08946v1 Announce Type: new Abstract: A transformer can be built from operators that are legible by construction -- bounded, named units that read as fuzzy set operations rather than dense activations -- but legibility must be pressed for during training, and the pressure has a failure mode.

By Mark Oskin
llms
More like this →