The paper investigates whether large language models (LLMs) can reinvent foundational computer‑science algorithms by suppressing direct recall of target algorithms through unlearning techniques. Experiments show that LLMs fail to recover nearly half of the algorithms, and even without unlearning, recovery rates drop dramatically, indicating an overreliance on memorized knowledge. The study introduces a generative verifier to maintain reasoning strength and concludes that current LLMs have limited capacity for foundational algorithm innovation.
arXiv:2603. 11784v2 Announce Type: replace Abstract: As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed.
The paper introduces the task of Scientific Claim Unlearning and presents a new benchmark, SciUnlearn, to evaluate it. It highlights that language models trained on static scientific corpora risk disseminating outdated or retracted claims as scientific knowledge evolves. Current machine unlearning methods fail to effectively remove claim-level knowledge, often only suppressing it superficially, underscoring the need for specialized techniques for structured knowledge removal.
The paper argues that Large Language Models (LLMs) do not function as Solomonoff induction estimators because their training objectives—cross‑entropy, negative log‑likelihood, and next‑token prediction—optimize fit to a supplied conditional distribution rather than a program‑weighted universal mixture. It further contends that additional computation alone does not transform these models into optimal predictors without external hyper‑parameter or architectural changes. The authors suggest that neurosymbolic machine learning, exemplified by models such as Fable and Astra, represents a shift toward symbolic model synthesis, moving beyond purely statistical LLMs.
arXiv:2604. 27540v2 Announce Type: replace Abstract: Scientific reasoning rarely stops at what is directly observable; it often requires uncovering hidden structure from data.
arXiv:2601. 05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at the intersection of Algorithmic Information Theory (AIT) and Machine Learning (ML) of great interest.
arXiv:2609.37891v1 Announce Type: cross Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...
arXiv:2607. 28684v1 Announce Type: new Abstract: Existing benchmarks for scientific equation discovery are largely composed of well-known equations available in the public domain, making it difficult to determine whether a model is discovering laws from data or merely recalling answers from its training corpus.
The paper introduces GUARD, a method for natural forgetting in large reasoning models that transforms unsafe disclosures into safe-exit trajectories using guided answer‑reasoning distillation. It aligns a frozen model with guidance tokens and distills this behavior into the parameters, aiming for a coherent, non‑disclosing chain of thought followed by a refusal‑style answer. The authors also propose the Natural Forgetting Reasoning Score (NFRS) to evaluate structural stability, fluency, and unsupported substitutes, and demonstrate GUARD’s effectiveness on R‑TOFU and a STAR‑1‑derived harmful‑intent setting.
Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic ta...
arXiv:2603. 14147v2 Announce Type: replace Abstract: The generative artificial intelligence (AI) ecosystem is undergoing rapid transformations that threaten its sustainability.
Prospector is a compact model that learns to traverse a domain knowledge graph step by step, calling a small set of tools to build context and produce a chain of evidence that can be verified against the graph. Trained on a semiconductor‑manufacturing knowledge graph, it matches or surpasses off‑the‑shelf models that use the same graph and tools, and it also transfers to unseen graphs in bio‑inspired materials, PFAS, and materials properties. Running on a single GPU, Prospector is faster and cheaper than hosted services while keeping proprietary data on the owner’s hardware.
arXiv:2607. 11696v1 Announce Type: new Abstract: Self-refinement often fails to strengthen few-shot inductive reasoning in large language models.