← Back to all news
arXiv Machine Learning By Congwei Song

The General Theory of Localization Methods

Read the original on arXiv Machine Learning →

The paper introduces a general machine learning framework called the localization method, built on localization kernels and local means that underpin self‑attention. It formally defines the framework through a local‑ized model and a localization trick, and systematically connects it to a wide array of existing models such as kernel methods, MeanShift, Hopfield networks, LLE, fuzzy inference, and denoising autoencoders. The authors also extend the framework with adaptive kernels, hierarchical local models, and non‑local models, showing that the Transformer can be constructed using hierarchical local models, thereby unifying and generalizing state‑of‑the‑art architectures.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 30

AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

AdaKerNet is a task‑adaptive neural kernel decoder that operates on frozen multimodal representations from large foundation models, without requiring access to the models’ parameters. It learns Lipschitz‑controlled multimodal features, a reference kernel providing a soft structural prior, and a lightweight nonlinear predictor that deforms this structure. Experiments on four multimodal large language models and diverse input modalities show consistent improvements over baseline decoders, achieving up to 41% error reduction in scarce‑label settings.

By Konstantinos D. Polyzos, Eleni Oikonomou, Tara Javidi
llmsragfine-tuningmultimodal
More like this →
arXiv Machine Learning
Sep 21

Diagonalized Attention for Individualized Regression: Latent-Row Localization and Prediction

The paper introduces an individualized sparse regression framework for matrix‑valued covariates, where each observation has its own relevant rows while regression effects are shared across the population. It proposes a diagonalized attention mechanism that uses query–key scores to localize sample‑specific signal rows and a value matrix for downstream regression, achieving a parameter dimension independent of sample size. The authors provide existence theorems guaranteeing recovery of latent rows under score‑separation and concentration conditions, and demonstrate strong prediction, localization, and classification performance in simulations and real sentiment analysis.

By Borui Peng, Liwei Lin, Feifei Wang, Long Feng
More like this →