Scientific studies of language agents need behavioral variables that support hypotheses across tasks and models. We formulate this research problem as learning and testing a hierarchy of trajectory ab...
The paper proposes a general framework for analysing the behaviour of large language model-driven agents. It connects data-driven behavioural discovery with analysis and experimental investigation, using categories derived from recorded activity to measure behaviours and preserve their positions in the process. The framework includes statistical assessment in held-out contexts and experiments such as prompt steering to test how behaviours and outcomes change under varied conditions, illustrated by a study of iterative self-reflection.
arXiv:2602. 02304v3 Announce Type: replace Abstract: Large-scale foundation models exhibit behavioral shifts when subjected to interventions such as scaling, fine-tuning, reinforcement learning with human feedback, or in-context learning.
The paper evaluates the planning abilities of large language model (LLM) agents by comparing their performance against formal semantics defined in hidden PDDL models. Across five domains, four LLMs, six prompting strategies, and three agent architectures, the best setup solved 78% of solvable tasks offline but only 40% online, with limited recognition of unsolvable tasks and no clear benefit from more elaborate prompting. The study concludes that claims of LLM planning competence require validated action traces rather than end‑to‑end success rates.
arXiv:2604. 19791v3 Announce Type: replace Abstract: Attitude change - the process by which individuals revise their evaluative stances - has been explained by a set of influential but competing verbal theories.
The paper introduces domain-specific languages (DSLs) as an explicit representation layer to bridge the gap between high-level natural-language abstractions and concrete actions. Experiments on the visually-grounded “Hexagons” task and the “Text2CAD” benchmark show that code generation using a human-curated DSL yields dramatic performance gains, especially at higher abstraction levels, while automatically induced DSLs only partially recover these gains. The study highlights that the right representation can close much of the performance gap caused by abstraction, but that automatically learning such representations remains a challenge.
arXiv:2603. 14465v2 Announce Type: replace Abstract: While Large Language Models (LLMs) have evolved into tool-using agents, they remain brittle in long-horizon interactions.
The paper critiques a recent NLI benchmark that tests the imperfective paradox, arguing that the benchmark suffers from conceptual and evaluation mis-specifications, notably Aspectual Reduction and a lack of strict NLI standards. The authors re-evaluate the benchmark, identify mis-specifications, and construct lexically matched minimal pairs to control for lexical variation. Their experiments reveal that models often exhibit a Sufficiency Bias, accept simple‑past hypotheses without affirming culmination, and that prompting interventions shift label decisions without improving true semantic understanding, highlighting additional failure modes such as compositional aspectual classification errors and surface‑form attraction.
The paper introduces INTENT-AS-A-TOOL, a method that equips large language models with intent-targeted tools to provide a fine-grained, judge‑free signal of their commitment to specific behaviors during reasoning. By monitoring the probability of calling these intent tools, the authors can track how intent evolves throughout generation, complementing chain‑of‑thought monitoring and expanding post‑hoc labels into dense trajectories. The approach identifies critical steps for online intervention, demonstrating that action preferences are useful for detecting agentic misalignment in autonomous agents.
The study examines how three context‑engineering techniques—semantic layers, orchestration, and compaction—affect data agents that answer questions over heterogeneous data. Using seven large language models on 1,514 long‑horizon tasks, the authors identify three agent behavior groups (investigators, premature answerers, early stoppers) and show that each technique benefits only the group it aligns with, while sometimes harming others. Combining the techniques yields smaller net gains than the sum of individual effects, leading to guidance on selecting techniques based on agent behavior, task demands, and accuracy‑cost trade‑offs.
The paper introduces HypoAgent, an agentic framework for refining abductive hypotheses over knowledge graphs. It treats user-provided conditions as steering operators, using a small generator for initial proposals, root-cause analysis to identify problematic branches, and a refiner agent that revises hypotheses based on diagnosis and prior proposals. Experiments on BioKG, PharmKG8k, and DBpedia50 show HypoAgent surpasses one-shot generation in both single-turn and multi-turn scenarios, including unconditional settings.
arXiv:2606. 08682v1 Announce Type: cross Abstract: Activation steering has emerged as a popular inference-time technique for modulating the behavior of large language models (LLMs).