arXiv:2606. 17649v1 Announce Type: cross Abstract: The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense.
arXiv:2510. 07750v3 Announce Type: replace-cross Abstract: Robust optimization safeguards decisions against uncertainty by optimizing against worst-case scenarios, yet their effectiveness hinges on a prespecified robustness level that is often chosen ad hoc, leading to either insufficient protection or overly conservative and costly solutions.
arXiv:2602. 24207v2 Announce Type: replace Abstract: The use of algorithmic predictions in decision-making leads to a feedback loop where the models we deploy actively influence the data distributions we see, and later use to retrain on.
arXiv:2601. 07094v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) iteratively fits a Gaussian process (GP) surrogate to accumulated evaluations and selects new queries via an acquisition function.
arXiv:2607. 21773v1 Announce Type: new Abstract: In this paper, we propose and study a robust variant of the smart predict-then-optimize approach that accounts for prediction shifts due to disturbance in the covariate feature space.
The paper introduces SCROLL, a method for forecasting multiple observables in stochastic dynamical systems by composing each observable’s likelihood into per‑task free‑routed last‑layer beliefs on a shared backbone. This approach learns unit‑dependent loss scaling directly from data, enabling accurate predictive variance estimation without separate tuning. Experiments on the Ornstein–Uhlenbeck process, stochastic Lorenz‑63, and real air‑quality data show that SCROLL recovers analytic kernels, achieves superior negative log‑likelihood on state and regime tasks, and maintains calibration while reducing hyper‑parameter search costs.
arXiv:2608. 25551v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory.
arXiv:2606. 27711v1 Announce Type: cross Abstract: We introduce a neural network-based framework for learning time series estimators through a process we term decision-theoretic pretraining.
arXiv:2606. 17660v1 Announce Type: cross Abstract: Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and na\"ive runs can even degrade model performance.
arXiv:2606. 04342v1 Announce Type: cross Abstract: Multi-step time series forecasting (MSF) is commonly evaluated using point-wise error metrics such as mean squared error (MSE), implicitly treating the conditional mean as a sufficient target.
The paper investigates whether a looped estimator—one that repeatedly applies a single fitted operator with shared parameters—can enhance statistical accuracy while staying within a fixed parameter budget. It establishes upper and lower bounds on squared Hellinger risk for looped sieve maximum likelihood and compares them to the untied counterpart, revealing a tradeoff between parameter sharing, iteration count, and accuracy. For models with known H"older smoothness, looped residual feedforward networks and a post‑layer‑normalized Transformer achieve minimax polynomial rates with a fixed number of bounded real parameters, and under certain conditions the looped estimator’s worst‑case risk vanishes as sample size grows, outperforming the untied approach.
arXiv:2609.14065v1 Announce Type: new Abstract: When algorithmic predictions inform people's decisions, the models we deploy are performative and actively shape the data we see. This feedback loop be...
arXiv:2607. 10694v1 Announce Type: cross Abstract: We study the problem of optimal continual fine-tuning for a pre-trained Foundation Model deployed at a resource-limited device.