The paper proposes a general framework for analysing the behaviour of large language model-driven agents. It connects data-driven behavioural discovery with analysis and experimental investigation, using categories derived from recorded activity to measure behaviours and preserve their positions in the process. The framework includes statistical assessment in held-out contexts and experiments such as prompt steering to test how behaviours and outcomes change under varied conditions, illustrated by a study of iterative self-reflection.
arXiv:2610.09237v1 Announce Type: cross Abstract: Scientific studies of language agents need behavioral variables that support hypotheses across tasks and models. We formulate this research problem a...
Scientific studies of language agents need behavioral variables that support hypotheses across tasks and models. We formulate this research problem as learning and testing a hierarchy of trajectory ab...
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties supervision to what a transition happens to reveal, which may omit the dynamics most relevant to the agent's current decision.
The study investigates how language models acquire in-context learning (ICL) by distinguishing between copying a nearby demonstration and integrating information across a prompt. Across 17 models, roughly one-third default to copied answers, a few are undecided, and the remainder favor rule-consistent answers that yield higher accuracy and are guided by the requested value rather than the nearest conflicting example. Analyses show that models prioritize task-relevant prompt tokens before accuracy surpasses chance, and that synthetic or fine-tuned training can shift models toward compositional responses or improve accuracy when prompts reassign label meanings.
arXiv:2608. 14352v1 Announce Type: cross Abstract: Large Language Model (LLM)-based agents are increasingly used for complex tasks such as software testing and cybersecurity assessment.
arXiv:2610.14574v1 Announce Type: new Abstract: In-context learning (ICL) allows large language models to use demonstrations without updating their parameters. For interactive agents, demonstrations...
arXiv:2610.08364v1 Announce Type: new Abstract: Frontier AI evaluations increasingly use open-ended, agentic, long-horizon tasks whose transcripts can span hundreds of pages of outputs and actions fr...
arXiv:2608.30391v1 Announce Type: cross Abstract: Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where...
The study examines how three context‑engineering techniques—semantic layers, orchestration, and compaction—affect data agents that answer questions over heterogeneous data. Using seven large language models on 1,514 long‑horizon tasks, the authors identify three agent behavior groups (investigators, premature answerers, early stoppers) and show that each technique benefits only the group it aligns with, while sometimes harming others. Combining the techniques yields smaller net gains than the sum of individual effects, leading to guidance on selecting techniques based on agent behavior, task demands, and accuracy‑cost trade‑offs.
Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bri...
arXiv:2608. 16747v1 Announce Type: cross Abstract: Many areas of AI research, such as language model interpretability and chain of thought faithfulness, seek to explain model behaviors.
arXiv:2606.02994v2 Announce Type: replace-cross Abstract: Existing methods induce various forms of knowledge from agent traces, from sample workflows to programmatic skills. We propose ARISE (Agent R...