← Back to all news
arXiv AI By Man Liang, Xinzhao Cheng, Faizan Wajid

Decodable but Not Actionable: Auditing the Representation-Behavior Gap in Frozen LLMs for Geometric Constraints

Read the original on arXiv AI →

The paper introduces a controlled auditing framework using parametric CAD to separate failures in encoding, expression, and activation-level control in frozen large language models (LLMs). Applied to eight pretrained models and random controls, it finds that pretraining mainly improves linear decoding of local relations, while global constraint status is already highly decodable from random representations. Despite strong local decodability, the models’ zero‑shot predictions lag far behind supervised probe performance, and simple activation steering fails to provide reliable control; however, low‑rank fine‑tuning with only 400 labeled relation pairs can substantially improve generation on three models, indicating that the representation–behavior gap can be narrowed through adaptation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 19

Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraints

The paper audits frozen decoder‑only large language models (LLMs) on geometric reasoning tasks using parametric CAD constraints. It probes hidden states for linear decodability, forced‑choice generation, activation‑level influence, and behavioral steerability, finding that pretraining improves decoding of local geometric relations but not sketch‑level DOF status. The study shows that decodable information is not always actionable: generation often fails to express it, and steering interventions do not reliably control outputs, revealing divergences among decodability, generation, activation influence, and steerability.

By Man Liang, Xinzhao Cheng, Faizan Wajid
llms
More like this →
arXiv AI
5d ago

Constrained Decoding: Harmful or Beneficial? Revisiting Its Impact on LLM Performance

The paper investigates whether constrained decoding harms or helps large language model (LLM) performance. It introduces a taxonomy of three constraint types—format, validity, and reasoning—and shows that suitable constraints can preserve or even improve performance by eliminating invalid paths. Experiments across multiple model families reveal that perceived performance drops are mainly due to non-equivalent prompting and output formats, while validity constraints boost reliability and reasoning constraints can enhance accuracy when they guide beneficial reasoning trajectories.

By Ngoc Trinh Hung Nguyen, Alonso Silva, Armen Aghasaryan, Mehwish Alam
llmsbenchmarks
More like this →