arXiv:2609.18697v1 Announce Type: new Abstract: Historians have reconstructed the twentieth-century transformation of general relativity and gravitation (GRG) at the field level and through individua...
The paper presents a SciBERT-based method for automatically classifying scientific papers into four telescope-related categories—science, instrumentation, mention, and not telescope—within strict 512-token limits. Despite truncation challenges, the approach achieved a macro F1 score of 0.89, topping the WASP-2025 leaderboard. The authors analyze truncation effects, compare chunking and long-context models, and offer insights into efficient scientific text curation.
arXiv:2607. 04525v1 Announce Type: cross Abstract: How concepts are represented in neural networks is a fundamental question in machine learning.
The study investigates whether contextual embeddings can detect meaning changes in scientific terminology beyond traditional frequency counts. Using Astrophysics and NLP corpora from 2010 to 2024, the authors extract candidate terms with KeyBERT, filter for significant frequency rises, and then evaluate semantic drift via multiple embedding‑based metrics. Results show that frequency methods slightly outperform embedding metrics in aligning with expert judgments, yet embedding‑only detections (e.g., "primordial black holes") reveal critical conceptual shifts missed by frequency alone, suggesting complementary value.
How concepts are represented in neural networks is a fundamental question in machine learning. The dominant view treats concept representations as stationary geometric objects.
arXiv:2605. 24782v2 Announce Type: replace Abstract: While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invariants, making even perception-based out-of-distribution accuracy a poor proxy for scientific utility.
The paper introduces a time‑aligned evolving concept graph framework that jointly models semantic and structural changes in scientific literature. By treating dated papers as shared update events, it reconstructs both semantic and structural states from the same publication history for each prediction time, and fuses these states at the pair level to forecast co‑occurrence, relation formation, and conditional relation type. Experiments on a large graph of 187,848 papers and 270,687 concepts show that refreshing context with graph updates boosts mean relation AUPRC by 16.6% and raises mean relation AUROC from 0.9290 to 0.9722.
The paper introduces the task of Scientific Claim Unlearning and presents a new benchmark, SciUnlearn, to evaluate it. It highlights that language models trained on static scientific corpora risk disseminating outdated or retracted claims as scientific knowledge evolves. Current machine unlearning methods fail to effectively remove claim-level knowledge, often only suppressing it superficially, underscoring the need for specialized techniques for structured knowledge removal.
The paper investigates whether embedding spaces capture objective physical measurements such as mass, distance, time, and volume. It finds that these embeddings only weakly model such measurements and instead exhibit peculiar patterns. Further analysis shows that superficial string similarity heavily influences the representation of physical measurements, and recalibrating similarity does not significantly improve alignment.
The paper studies how transformer representations evolve across layers by examining the intrinsic dimensionality (ID) of token embeddings and their neighborhood structures. It finds that closed‑class tokens expand and collapse earlier than open‑class tokens, and that these changes are linked to shifts in local geometry. The authors compare encoder and decoder models, showing distinct layer‑wise behaviors, and demonstrate that geometric features alone can predict a token’s part‑of‑speech and reveal how semantic content changes across layers.
The study analyzes 207,111 astronomy papers from 2015 to mid‑2026 to quantify how many contain language‑model‑generated vocabulary. Using a hierarchical Bayesian model calibrated on pre‑2020 unassisted papers and 392 papers that disclose model use, the authors estimate that in 2025 roughly 54% (±8% statistical, ±26% systematic) of papers show a language‑model trace, with the estimate remaining above 36% under various assumptions. Despite only 0.81% of 2025 papers explicitly declaring model assistance, the trace is pervasive, and the detectable signal is fading as authors adapt to the characteristic words. whyItMatters":"The findings reveal that language‑model assistance has become widespread in recent astronomy research, yet most authors do not disclose its use, highlighting a growing gap between actual practice and transparency in scholarly writing."
arXiv:2602. 15029v3 Announce Type: replace Abstract: The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe.