arXiv Machine Learning

Covariance Last-Layer Ensembles: Function-Space Diversity for Efficient Uncertainty Quantification

arXiv:2607. 23856v1 Announce Type: new Abstract: A Last-Layer Ensemble (LLE), $K$ linear units on one shared frozen feature map, is an efficient single-pass approach to the disagreement-based epistemic uncertainty for out-of-distribution (OOD) detection.

arXiv AI
1d ago

Don't Replace, Complement: A Detector-Agnostic Subspace Signal for Near-OOD Detection

The paper introduces PLUS, a detector‑agnostic subspace signal that enhances near‑out‑of‑distribution (near‑OOD) detection. PLUS selects the most class‑active neurons via a sparse autoencoder, orthonormalizes their decoder directions to form a low‑dimensional subspace, and scores inputs using a per‑class whitened squared Mahalanobis distance within that subspace. The resulting signal is fused additively with any calibrated base score using a single global weight, improving near‑OOD AUROC across 126 detector–backbone combinations while largely preserving far‑OOD performance.

By Wonseok Shin, Songkuk Kim
arXiv Machine Learning
Jun 18

Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions

arXiv:2602. 21160v3 Announce Type: replace-cross Abstract: In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class.

By Mame Diarra Toure, David A. Stephens
arXiv Machine Learning
Sep 24

CORE-STACK+: Meta-Learning for Deep Stacked Generalization

CORE-STACK+ is a new meta‑learning framework for deep stacked generalization that tackles two key problems in heterogeneous vision ensembles: prediction‑space multicollinearity and calibration collapse. It introduces a four‑step preconditioning pipeline—kernelized redundancy filtering, a lightweight differentiable meta‑feature gate, a spectrum‑adaptive ridge penalty, and a Laplace‑approximate Bayesian blender—to jointly improve conditioning and calibration. Across six vision benchmarks, CORE‑STACK+ boosts accuracy, reduces model count and inference cost, and significantly lowers expected calibration error compared to existing methods.

By Noor Islam S. Mohammad
arXiv Computation and Language
Aug 31

Layered LLM Defenses as an Ensemble: Access Tiers, Inference Cost, and the Measured Failure Correlation Between Defense Layers

The paper investigates whether stacking multiple defenses around large language models (LLMs) truly compounds security. Using the Adversary Access‑Tier Model (AATM) and a cost‑tiering system, the authors analyze a seven‑layer defense stack and find that failure correlations between layers are consistently positive, meaning the residual attack success is higher than the multiplicative prediction. Despite high coverage and low false refusals, the stack’s performance is largely driven by common architectural causes rather than diverse, independent defenses.

By Abrar Alotaibi, Muhammad Shahid Jabbar, Sadam Al-Azani, Moataz Ahmed
Hugging Face Trending Papers
Jun 28

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks

A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation. Adam's per-coordinate preconditioner drifts along each symmetry orbit, which pulls the trajectory off the symmetry quotient where the optimization lives and blurs the singular-learning rate the quotient makes readable.

arXiv Machine Learning
Jul 14

Vertical Fusion: Condensing Internal Representations for Robust ViT Classification

arXiv:2607. 10391v1 Announce Type: cross Abstract: Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks.

By Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig