arXiv:2608. 18242v1 Announce Type: new Abstract: We introduce ClosureBench, a constructive benchmark for compositional graph-relational reasoning with programmatically verified ground truth.
arXiv:2606. 03705v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are widely used to mitigate the limitations of Large Language Models (LLMs), such as outdated knowledge and hallucinations.
arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
arXiv:2608. 12282v1 Announce Type: new Abstract: Agents deployed in enterprise settings must reason across structured APIs and document collections, yet existing benchmarks evaluate these capabilities in isolation.
arXiv:2609.00759v1 Announce Type: new Abstract: Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output s...
arXiv:2608. 07885v1 Announce Type: new Abstract: Reasoning modes of language models outperform their non-reasoning counterparts on multi-step agentic tasks, but pay a 3-6x premium in output tokens on every episode -- much of it spent re-deriving procedures that are shared across episodes of the same domain.
arXiv:2604. 12176v2 Announce Type: replace Abstract: Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables.
The paper investigates how different components of a graph retrieval‑augmented generation pipeline affect large language model performance on knowledge‑graph question answering. It examines four variables—whether the answer path is included, the syntax of triples, the order of triples, and the subgraph size—across six LLMs and two benchmarks. The study finds that including the answer path is crucial, while the grounding instruction dramatically reduces accuracy when no facts are provided, and that syntax, order, and subgraph size have negligible measurable impact at multi‑hop depth.
The paper introduces Reasoning DAG Probing, a method that treats premises, intermediate conclusions, and final answers as nodes in a directed acyclic graph (DAG). By training lightweight probes on LLM hidden states, the authors show that DAG structure is encoded in the models, peaking in intermediate layers and becoming stronger with larger models. During autoregressive generation, the latent DAG structure emerges progressively, anticipating answer correctness and enabling correction of some incorrect outputs by steering activations toward the ground‑truth DAG.
arXiv:2608. 12585v1 Announce Type: new Abstract: Improving reasoning LLMs requires the ability to judge the quality of long reasoning traces for effective reasoning data curation, strong training signals during reinforcement learning, and an in-depth understanding of reasoning behaviors during model performance evaluation.
The paper introduces PolyHuman, a dataset of human-written programs in C++, Java, and Python, to test whether large language models can judge functional equivalence across languages. Using this dataset, the authors evaluate several open-weight and proprietary LLMs, finding that models struggle more with harder problems, show language-specific biases, and rely partly on superficial similarity cues. They also observe run‑to‑run instability in GPT‑o4‑mini, concluding that current LLMs do not reliably capture functional equivalence within or across programming languages.
arXiv:2606. 03144v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as self-study assistants in technical disciplines, yet their reliability as mathematical reasoning assistants remains poorly understood.