← Back to all news
arXiv Computer Vision By Zheng Liu, Xin Gao, Jinchao Zhu, Gao Huang

Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers

Read the original on arXiv Computer Vision →

Position Anchor Tuning (PAT) is a parameter‑efficient fine‑tuning method for pre‑trained point cloud transformers that improves inference efficiency. PAT reduces the computational cost of multi‑head attention and feed‑forward networks by using token aggregation‑expansion pairs: a token aggregation module (TAM) selects representative tokens based on 3D position anchors, and a token expansion module (TEM) propagates the learned representations back to the full token set. Combined with base‑sharing low‑rank adaptation (BSLoRA) for the TAMs, PAT achieves performance comparable to state‑of‑the‑art methods while requiring fewer trainable parameters and lower computational overhead.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computation and Language
Aug 25

Do Value Vectors in Deep Layers Need Context from the Residual Stream?

The paper investigates whether deep transformer layers require context from the residual stream to compute value vectors. It finds that allowing deeper layers to use a context‑free value vector—preserving original token information—significantly improves performance, and adding context afterward yields little extra benefit. The authors introduce Bank of Values (BoV), a lookup table of token‑specific value vectors for the last third of layers, which reduces compute and memory while matching or surpassing prior methods on large models.

By Muyu He, Yuchen Liu, Qingya Huang, Li Zhang
llmsbenchmarks
More like this →