arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga
arXiv AI
Sep 17

Transcribe, Then Reason: Two-Pass Decomposition for Multimodal Review

The paper demonstrates that a single-pass multimodal model struggles to produce faithful, comprehensive reviews of long recordings or documents, often omitting a third of the content and embellishing the rest. By splitting the task into two passes—first transcribing the source and then reviewing the transcript—the authors show improved faithfulness and coverage across a diverse set of 21 sources. The benefit is most pronounced for longer or weaker baseline cases, while the approach introduces new failure modes such as space constraints and memory confabulation.

By Bojie Li, Noah Shi