Hugging Face Trending Papers July 5, 2026

Geometry of Ordinal Representations in Language Models

Read the original on Hugging Face Trending Papers →

Recent work showed that language models represent character counts on curved 1D manifolds, with attention heads performing geometric transformations to enable computation. We test whether this generalizes across four ordinal tasks (bracket depth, indentation, table position, numeric magnitude) in Gemma-2-2B, Gemma-2-9B, and Qwen3-4B.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jul 7

Geometry of Ordinal Representations in Language Models

arXiv:2607. 04167v1 Announce Type: new Abstract: Recent work showed that language models represent character counts on curved 1D manifolds, with attention heads performing geometric transformations to enable computation.

By Saksham Bassi, Sharvi Tomar

llms benchmarks

arXiv Computation and Language

Sep 7

Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs

The paper investigates how large language models (LLMs) organize reasoning operations—such as problem formulation, goal decomposition, and deduction—within their hidden representation spaces. It shows that these operations are separable in held‑out representations, with peak separability in middle layers, and that token‑wise alignment of operations becomes more distributed across spans as layers deepen. Attention‑masking experiments reveal that representations aligned to operations at chunk onsets depend on prior reasoning context, indicating a geometric correspondence between linguistic reasoning expressions and internal model structure.

By Seogyeong Jeong, Jaehui Hwang, Dongyoon Han, Geonmo Gu, Alice Oh, Taekyung Kim

llms safety

arXiv Computer Vision

Aug 24

Latent Ordinal Evidence, Misaligned Outputs: Inference-Time Ordinal Lens Alignment for Multimodal LLMs

arXiv:2608.20999v1 Announce Type: new Abstract: Multimodal LLMs apply the language model interface to visual inputs, where ordinal regression tasks such as age estimation, image quality assessment, a...

By Haiming Li, Yingsheng Liu, Jingmin Zhu, Siyuan Yan, Xieji Li, Jiajun Sun, Zhen Yu, Zongyuan Ge

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 9

Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings

arXiv:2509. 10534v3 Announce Type: replace-cross Abstract: The attention mechanism in a Transformer architecture matches key to query based on both content -- the what -- and position in a sequence -- the where.

By Anand Gopalakrishnan, Robert Csord\'as, J\"urgen Schmidhuber, Michael C. Mozer

llms rag fine-tuning

arXiv Machine Learning

Jul 20

Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization

arXiv:2601. 22402v2 Announce Type: replace-cross Abstract: Rotary Positional Embeddings (RoPE) have become the standard for Large Language Models (LLMs) due to their ability to encode relative positions through geometric rotation.

By Kanishk Awadhiya

llms rag robotics

arXiv Machine Learning

Jun 11

Visualizing LLM Latent Space Geometry Through Dimensionality Reduction

arXiv:2511. 21594v3 Announce Type: replace Abstract: Large language models (LLMs) achieve state-of-the-art results across many natural language tasks, but their internal mechanisms remain difficult to interpret.

By Alex Ning, Vainateya Rangaraju, Yen-Ling Kuo

llms rag diffusion benchmarks

One email a morning, machine-written

Related stories

Geometry of Ordinal Representations in Language Models

Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs

Latent Ordinal Evidence, Misaligned Outputs: Inference-Time Ordinal Lens Alignment for Multimodal LLMs

Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings

Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization

Visualizing LLM Latent Space Geometry Through Dimensionality Reduction