← Back to all news
arXiv AI By Tianqiang Yan, Lizhen Qu

Behaviour analysis for large language model-driven agents

Read the original on arXiv AI →

The paper proposes a general framework for analysing the behaviour of large language model-driven agents. It connects data-driven behavioural discovery with analysis and experimental investigation, using categories derived from recorded activity to measure behaviours and preserve their positions in the process. The framework includes statistical assessment in held-out contexts and experiments such as prompt steering to test how behaviours and outcomes change under varied conditions, illustrated by a study of iterative self-reflection.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 15

From Copying to Composition: How Language Models Develop In-Context Learning

The study investigates how language models acquire in-context learning (ICL) by distinguishing between copying a nearby demonstration and integrating information across a prompt. Across 17 models, roughly one-third default to copied answers, a few are undecided, and the remainder favor rule-consistent answers that yield higher accuracy and are guided by the requested value rather than the nearest conflicting example. Analyses show that models prioritize task-relevant prompt tokens before accuracy surpasses chance, and that synthetic or fine-tuned training can shift models toward compositional responses or improve accuracy when prompts reassign label meanings.

By Minglai Yang, Xinyu Guo, Renxiong Wang, Sunjie Hou, Daniel Yue Zhang, Yunzhong He
llmsfine-tuning
More like this →