The paper introduces SAE‑IDF, a data‑driven approach that uses sparse autoencoders to encode symbols into interpretable sparse codes, re‑weights dimensions by rarity, and matches units across heterogeneous symbolic systems. On 12 tasks spanning six symbolic system types, SAE‑IDF outperforms dense LLM baselines (Gemma‑2‑9B and Llama‑3.1‑8B) by up to 45 points, consistently across layers and configurations. Each sparse dimension carries a human‑readable meaning, allowing matches to decompose into named concepts, thereby enabling a scalable, universal symbolic backbone.
arXiv:2610.01519v1 Announce Type: cross Abstract: Neuro-Symbolic (NeSy) predictors incorporate prior knowledge into the prediction process of neural networks, ensuring that outputs satisfy specified...
arXiv:2603. 03031v2 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved strong complex reasoning capabilities through Chain-of-Thought (CoT) reasoning.
arXiv:2607. 14149v1 Announce Type: new Abstract: Although large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment remains cost-prohibitive and environmentally taxing.
arXiv:2601. 07372v2 Announce Type: replace-cross Abstract: While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.
The paper introduces Reward‑Informed Sparse Autoencoders (RI‑SAEs), which use reinforcement‑learning rewards to curate data for training sparse autoencoders on language‑model activations. On Llama‑3.1‑8B, a sparse subset of features separates high‑reward from low‑reward reasoning continuations, but control experiments show this separation largely reflects solution completeness rather than true reasoning quality. The authors conclude that reward filtering can cheaply reuse RL signals for interpretability, though most of the discovered features capture completion form rather than deep reasoning.
arXiv:2608. 15360v1 Announce Type: cross Abstract: While Parameter-Efficient Fine-Tuning (PEFT) has substantially reduced the hardware cost of adapting Large Language Models (LLMs) by decreasing the number of trainable parameters, recent studies have sought to further improve PEFT through parameter sharing.
arXiv:2609.25438v1 Announce Type: new Abstract: Diverse pretraining has been shown to be an effective method for learning reusable, domain-aware representations that provide a starting point for fine...
arXiv:2606. 29604v1 Announce Type: cross Abstract: We aim to discover diverse, generalizable perturbations of LLM internals that can surface hidden behavioral modes.
Large Reasoning Models often overthink, producing redundant reasoning steps that consume many tokens. The authors observe that efficient reasoning steps cluster in a concentrated region of latent space, while verbose steps lie outside this region. By projecting deviating hidden states back into the region via a quadratic program, they propose a training‑free framework that cuts token generation by up to 52.8% while improving accuracy by up to 12.1% across several benchmarks.
arXiv:2510. 14538v3 Announce Type: replace Abstract: Neuro-symbolic (NeSy) AI aims to develop deep neural networks whose predictions comply with prior knowledge encoding, e.
arXiv:2610.31894v1 Announce Type: new Abstract: Recent Sparse Autoencoders (SAEs) decompose the representations of large language models (LLMs) into interpretable features, enabling precise readout a...
Euclid-Omni is a unified neuro‑symbolic framework that integrates a formal geometry system with Large Language Models and Vision‑Language Models to solve both calculation and proving problems in Euclidean geometry up to Olympiad level. Its core component, Euclidea, automatically generates deductive reasoning steps and algebraic computations, while a data‑generation pipeline creates synthetic symbolic problems, diagrams, and natural‑language translations for training. Experiments show that VLMs trained on this synthetic data outperform on calculation tasks, and LLMs paired with Euclidea match state‑of‑the‑art proving systems using far less compute and data.