arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.
arXiv:2607.01630v2 Announce Type: replace Abstract: Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our...
arXiv:2608. 06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they rely on abundant, diverse data that industrial and scientific domains often lack.
Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our analyses suggest that classification supervision alone does not sufficiently preserve task-agnostic shared backbone representations over long incremental sequences. We identify two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate.
arXiv:2410. 07299v3 Announce Type: replace-cross Abstract: We introduce OTIS, an open time series encoder that yields high-quality time series features for downstream deployment on any system, including resource-constrained wearables and industrial sensors.
The paper introduces a new training strategy called Staged Depth Training (SDT) that implements a representation curriculum for physics-informed neural networks (PINNs). SDT trains a shallow prefix with a temporary physics-informed head, freezes it, and then adds depth, improving performance across many benchmark problems without changing the final architecture. Experiments show significant error reductions and deeper fitted depth-scaling exponents, indicating that explicit representation learning can enhance PINN performance.
arXiv:2606. 14373v1 Announce Type: cross Abstract: The workflow from particle collision to physics analysis passes through a series of reconstruction steps that are traditionally modular and disconnected, with no shared representation linking low-level detector data to high-level analysis tasks.
arXiv:2610.28555v1 Announce Type: cross Abstract: Statistical learning in high dimensions suffers from the curse of dimensionality. Fortunately, many functions of practical interest are compositional...
arXiv:2608. 12090v1 Announce Type: new Abstract: Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology.
The paper introduces the Latent Interconnect Protocol (LIP), a framework that allows large language models (LLMs) to directly exchange rich, high‑dimensional representations with specialist models (e.g., protein and small‑molecule experts) via learned connectors. By training only these connectors, the approach enables zero‑shot transfer to cross‑domain tasks and compositional reasoning without joint training of the LLMs or specialists. Experiments with Gemma 3 and Kimi K2.6 show improved task performance and compositionality, especially in zero‑shot settings, while highlighting the need for consistent specialist interfaces and LLM training to handle such inputs.
arXiv:2608.31096v1 Announce Type: cross Abstract: Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while...
The paper presents a spiking neural network (SNN) approach that uses time-to-first-spike (TTFS) coding to limit each neuron to at most one spike per time window, enabling energy-efficient large language models (LLMs). A reference-based strategy is introduced to encode the four core LLM components—embedding layers, layer normalization, attention-related operations, and dropout—allowing the construction of a fully TTFS-based SNN architecture trained end-to-end. Experiments on BERT and GPT-2 show performance comparable to artificial neural network (ANN) counterparts on natural language understanding and common-sense reasoning, while achieving a 1.5‑billion‑parameter spiking LLM and providing an estimate of spike-related energy consumption.