Parabolic Positional Encoding: A Vision-Centric, Principled, Extrapolatable, and General Approach

The paper introduces Parabolic Positional Encoding (PaPE), a parabola‑based method for encoding positions of vision tokens in attention‑based models. PaPE is designed to respect vision‑specific properties such as translation and rotation invariance, distance decay, directionality, and context awareness. Experiments on ImageNet‑1K and eight other datasets across four modalities show that PaPE extrapolates well—improving performance by up to 10.5 points over the next best encoding—and generally matches or surpasses baseline encodings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.