arXiv AI By Fabrizia Giulia Garavaglia, Marco Giunti

Pyramidal Columns Acting as Attention Heads

Read the original on arXiv AI →

The paper explores whether a group of pyramidal columns in posterior temporal language regions could serve as a biological counterpart to a Transformer attention head. By mapping the functional organization of attention heads onto neurophysiological mechanisms within and across these columns, the authors propose that such a neuromorphic construction is possible, though empirical confirmation remains open. The study does not insist on reproducing the standard Transformer mathematics, allowing for alternative mechanical and mathematical instantiations consistent with biology.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jul 23

Geometric Attention: A Regime-Explicit Operator Semantics for Transformer Attention

arXiv:2601. 11618v2 Announce Type: replace-cross Abstract: Geometric Attention (GA) specifies an attention layer by four independent inputs: a finite carrier (what indices are addressable), an evidence-kernel rule (how masked proto-scores and a link induce nonnegative weights), a probe family (which observables are treated as admissible), and an anchor/update rule (which representative kernel is selected and how it is applied).

By Luis Rosario Freytes