← Back to all news
arXiv Machine Learning By William Dai, Zhenghao Herbert Zhou, Maya Viswanathan, Robert Frank

Where's the Indirect Evidence? Investigating Grammatical Generalization with Influence Functions

Read the original on arXiv Machine Learning →

The paper investigates indirect evidence in language models by applying influence functions to identify training sentences that affect predictions on 15 grammatical phenomena from BLiMP. After filtering out direct evidence, the authors select the 60,000 most influential remaining sentences and show that removing them significantly degrades held‑out accuracy, more so than random removal. They further analyze the characteristics of this indirect evidence and demonstrate that clustering phenomena based on shared indirect evidence recovers known linguistic categories.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Sep 3

Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralization

The paper investigates how language models avoid overgeneralizations by distinguishing between two types of indirect negative evidence: preemption and entrenchment. Through controlled rearing experiments on models trained on child‑caregiver conversations, the authors find that models do not exhibit verb‑specific preemption but show weak abstract preemption. Analysis of training dynamics suggests that competing structures act as indirect positive evidence rather than negative in the verb‑specific condition.

By Yixuan Wang, Freda Shi, Kanishka Misra
llms
More like this →
arXiv Computation and Language
5d ago

CLUE: Uncovering Social Context for Pragmatic Understanding

CLUE is a two‑stage training pipeline that teaches models to discover and use social context cues within a conversation for pragmatic understanding. It combines supervised evidence reasoning with a GRPO reward that credits new, evidence‑supported social inferences while penalizing excessive reasoning effort. Experiments show that CLUE improves weighted F1 scores by 17.3% in‑domain and 9.3% out‑of‑domain, reduces reasoning tokens by over 80%, and aligns its explanations more closely with human‑selected conversational evidence.

By Chengfei Wu, Zhaoqing Wu, Dan Goldwasser
safety
More like this →
arXiv Machine Learning
Jul 9

Towards Understanding Steering Strength

arXiv:2602. 02712v2 Announce Type: replace Abstract: A popular approach to post-training control of large language models (LLMs) is the steering of intermediate latent representations.

By Magamed Taimeskhanov, Samuel Vaiter, Damien Garreau
llms
More like this →