arXiv Machine Learning
Jul 10
Structural Bottlenecks on Frequency Representation in End-to-End Audio Models
arXiv:2607. 08545v1 Announce Type: cross Abstract: End-to-end neural audio models achieve high-fidelity compression and generation.
Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints
arXiv:2607. 16027v1 Announce Type: new Abstract: Introduction: Biological systems face anatomical and metabolic constraints, including costly synaptic maintenance and limited connectivity.
Related stories
arXiv AI
Aug 11
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
arXiv:2608. 09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive.
arXiv AI
4d ago
AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models
AdaKerNet is a task‑adaptive neural kernel decoder that operates on frozen multimodal representations from large foundation models, without requiring access to the models’ parameters. It learns Lipschitz‑controlled multimodal features, a reference kernel providing a soft structural prior, and a lightweight nonlinear predictor that deforms this structure. Experiments on four multimodal large language models and diverse input modalities show consistent improvements over baseline decoders, achieving up to 41% error reduction in scarce‑label settings.
arXiv AI
Aug 11
Scaling Audio Models Efficiently: A Joint Study of Compute Constraints and Optimization Behavior
arXiv:2606. 22790v2 Announce Type: replace-cross Abstract: In this paper, we investigate the tradeoffs between compute allocation and model performance for two speech processing tasks: Automatic Speech Recognition (ASR) and Speech Emotion Recognition (SER).
arXiv Machine Learning
Aug 4
Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval
arXiv:2608. 01481v1 Announce Type: new Abstract: Short segments of perceived speech can be retrieved from non-invasive magnetoencephalographic (MEG) recordings by deep networks trained with a CLIP-style objective against wav2vec 2.
Hugging Face Trending Papers
Aug 10
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive. While recent token compression methods attempt to alleviate this burden, compressing modalities in isolation often destroys the temporal cross-modal anchors necessary for coherent reasoning.
arXiv Machine Learning
Jun 16
MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation
arXiv:2606. 16408v1 Announce Type: new Abstract: We introduce MUNI, an end-to-end multimodal latent diffusion framework for any-to-any generation that unifies subset-conditioned cross-modal generation and unconditional joint sampling through a shared stochastic latent.
arXiv Machine Learning
Sep 11
Building Supervision into Hebbian Plasticity through Spike Agreement
The paper introduces Supervised Spike Agreement-Dependent Plasticity (Supervised SADP), a gradient‑free Hebbian learning rule that embeds class labels directly into spike‑driven plasticity. SADP trains output neurons with a supervised Hebbian rule and hidden neurons by measuring Cohen’s kappa agreement with the correct‑class output spike train, optionally aggregating over temporal offsets (K‑shift). Across six benchmark and medical imaging datasets, SADP consistently outperforms reward‑modulated STDP, achieving higher accuracy (e.g., 86.46 % on MNIST) and faster training (up to 2.86× speedup).
arXiv Machine Learning
Jun 10
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
arXiv:2606. 09853v1 Announce Type: new Abstract: A central objective in multimodal learning is to capture synergy: task-relevant information that arises only from the joint use of multiple modalities, and is not available from any single modality alone.
arXiv AI
Aug 11
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.
arXiv AI
Aug 11
Topographic Constraints Shape Brain-Like Component Structure in Auditory Models
arXiv:2509. 24039v2 Announce Type: replace-cross Abstract: If topography is a fundamental feature of the brain, it should influence both how neurons are arranged in space (i.
arXiv Computation and Language
Sep 23
Modality-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservation
The paper introduces modality‑gated deep adapters, a parameter‑efficient method for adding new modalities to a frozen multimodal embedding language model without altering its existing outputs. These adapters are bottleneck modules attached to each decoder layer, grouped into modality‑specific packs that activate only during encoding of their own modality, ensuring exact preservation of the base model’s computation graph. Experiments on a 2B base model show significant gains in audio‑to‑text and thermal‑to‑text retrieval metrics, and the authors release the audio and thermal packs along with training and evaluation code.