arXiv AI By Ivan Viakhirev, Kirill Borodin, Grach Mkrtchian

From Dispersion to Attraction: Spectral Dynamics of Hallucination Across Whisper Model Scales

Read the original on arXiv AI →

arXiv:2604. 08591v2 Announce Type: replace-cross Abstract: Hallucinations in large ASR models present a critical safety risk.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning
Jul 31

Critical attention scaling in long-context transformers

arXiv:2510. 05554v2 Announce Type: replace Abstract: As large language models scale to longer contexts, attention layers suffer from a fundamental pathology: attention scores collapse toward uniformity as context length $n$ increases, causing tokens to cluster excessively, a phenomenon known as rank-collapse.

By Shi Chen, Zhengjiang Lin, Yury Polyanskiy, Philippe Rigollet