arXiv:2608. 14227v1 Announce Type: new Abstract: Preprocessing invariance is an appealing goal for spectral foundation models: a frozen model should remain useful when laboratories preprocess spectra differently.
arXiv:2608. 02157v1 Announce Type: new Abstract: Raman spectroscopy enables non-destructive, label-free molecular characterization across materials science, biomedicine and process monitoring.
The study evaluates whether multimodal models produce consistent decisions when presented with equivalent optical spectra in different representations—tables in nanometers, micrometers, scientific notation, or photon energy, and plots in wavelength or energy. Using datasets of silicon metasurfaces and gold nanoparticles, the authors test several models (d3-nano, Jev 1.13, GPT‑6 Luna variants, Qwen3.5‑2B) and find that representation changes can shift accuracy by up to 41 percentage points, with some models benefiting from photon‑energy inputs while others suffer. Fine‑tuning with LoRA improves performance on trained formats but often relies on a shortcut of abstaining when data are truncated; training with answerable truncations mitigates this shortcut, revealing significant gaps in models’ handling of missing evidence.
arXiv:2610.09709v1 Announce Type: new Abstract: Can we determine whether a foundation model will generalize out-of-distribution (OOD) before any target data is available? Existing diagnostics require...
The paper investigates the use of the squared norm of a whitened foundation‑model embedding as a training‑free likelihood surrogate. It shows that the apparent Gaussianity of whitened coordinates stems from the projection central limit theorem, not from a true joint Gaussian distribution, and that the norm is systematically over‑dispersed compared to a Gaussian reference. The authors explain that whitening reverses the encoder’s spectral hierarchy, concentrating norm contributions in near‑degenerate directions dominated by noise, and propose interpreting the squared norm as a Mahalanobis measure of semantic atypicality rather than a log‑likelihood.
arXiv:2607. 13006v1 Announce Type: new Abstract: A growing family of indices scores how predictable a series is from its spectrum.
Raman spectroscopy enables non-destructive, label-free molecular characterization across materials science, biomedicine and process monitoring. Predictive Raman datasets often contain few labelled spectra and thousands of ordered wavenumbers, with informative variation within bands and across distant spectral regions.
The paper presents a physics‑aware, multi‑domain self‑supervised learning framework for Raman spectroscopy that uses a masked autoencoder with a sparse Voigt‑profile decoder and a denoising objective. It learns transferable spectral representations from diverse biological, chemical, forensic, and materials datasets, and demonstrates improved performance on substance classification, component identification, concentration estimation, and spectral denoising. The study shows that self‑supervised pretraining is especially beneficial for tasks with limited labeled data, that attention‑based probing reveals richer information than linear probing, and that larger encoders benefit more from pretraining.
The paper investigates why the orthogonal optimiser Muon outperforms Adam in large language model pretraining by analysing the spectral properties of Transformer loss landscapes. It finds that Muon’s momentum buffers exhibit an anisotropic spectral profile with a volatile head and a tolerant bulk, enabling larger effective step sizes. Building on this insight, the authors propose Spectral‑Aware Muon (SAMuon) and a lightweight variant, which adjust the bulk scaling while keeping the head unchanged, achieving 13–24 % fewer training tokens than Muon without extra FLOPs.
arXiv:2606. 31591v1 Announce Type: cross Abstract: Emergent misalignment (EM) is a recently discovered phenomenon in LLMs where fine-tuning on a narrow misaligned task, such as writing insecure code, leads to broadly misaligned behaviour on unrelated prompts.
arXiv:2607. 23711v1 Announce Type: new Abstract: LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting.
The paper investigates whether different parameter‑efficient fine‑tuning (PEFT) methods truly differ in their approach and impact. By comparing six methods on language and diffusion models, the authors find that many LoRA‑family techniques approximately preserve pretrained geometry, and that restoring slightly drifted singular‑value spectra largely maintains task performance, challenging the need for explicit geometric preservation. They also observe distinct adaptation–retention trade‑offs: LoRA limits forgetting best, DoRA scores higher on average, and PiSSA incurs more retention costs, with performance gains linked to modifications in specific spectral components.
arXiv:2608. 05025v1 Announce Type: new Abstract: Joint Energy-Based Models (JEM) unify classification and generation within a single network and support out-of-distribution (OOD) detection.