The paper introduces Hyper^2, a dual‑space consistency framework that applies hyperbolic geometry consistently to both the loss and the encoder in point‑cloud completion tasks. By reusing the same arcosh(1+αd²) function as a positional bias in refinement attention and as the Chamfer loss, Hyper^2 achieves significant Chamfer error reductions—up to 22.9% on ShapeNet‑55 and 37.5% on unseen ShapeNet‑34—while adding only ~1.6% FLOPs. The authors demonstrate that geometric consistency across encoder and loss, rather than either component alone, is key to effective hyperbolic supervision, supported by two model‑agnostic indicators that peak only when both are hyperbolic.
By Guantian Zheng, Haiyang Xu, Tianyu Gao
arXiv:2608. 14691v1 Announce Type: new Abstract: Sequence models are conventionally distinguished by their backbone, the mechanism that routes information across positions, such as attention or recurrence.
By Ahmed Nebli, Hadi Saadatdoorabi, Christopher Keibel, Kevin Yam
arXiv:2609.10305v1 Announce Type: new
Abstract: Language models under one million parameters matter for edge deployment, domain adaptation, and reproducible research, yet a two-layer LSTM or Transfor...
By Fang Li
arXiv:2606. 09059v1 Announce Type: cross Abstract: Two-stage post-training -- a Stage-1 warm-start (supervised fine-tuning, SFT, or on-policy distillation, OPD) followed by Stage-2 reinforcement learning (RL) -- is increasingly used for vision-language models (VLMs).
By Jianxiong Shen
arXiv:2607. 20171v1 Announce Type: cross Abstract: Learned solvers for compressible flow are usually compared to classical methods at equal mesh resolution rather than at equal computational cost, and they typically offer no guarantee that their solutions remain physically admissible.
By Denis Gueyffier (ONERA -- Institut Polytechnique de Paris)
Attention Manifolds introduce learned 2D B-spline surfaces that modulate each value dimension in transformer attention based on query-key interactions. Applied to LLaMA models, they reduce WikiText-2 perplexity by 2–2.5 points with minimal parameter overhead and alter greedy-decoded outputs in 69–83% of diverse prompts, especially on ambiguous inputs. The surfaces can be edited mechanically to steer or block value flow, offering a geometric tool for improving output quality and potentially enhancing safety.
By Naveen Mysore
The paper introduces Stiefel Attention, which constrains the query and key projection matrices of transformers to the Stiefel manifold and optimizes them with a Riemannian Adam variant. It demonstrates that this approach yields steepest‑descent updates, is well‑conditioned, and preserves learned attention geometry during weight decay. Empirical results show significant accuracy gains on modular arithmetic grokking and CIFAR‑10 patches, with the improvement attributed to a step‑scale‑free update rule rather than equivariance or projector changes.
By Rub\'en Dar\'io Guerrero
arXiv:2608.29924v1 Announce Type: cross
Abstract: Large Vision-Language Models (LVLMs) are prone to hallucinations: they fluently describe objects, attributes, and scenes that are not in the image. W...
By Aditi Sarker, Rafi Ibn Sultan, Hui Zhu, Dongxiao Zhu, Prashant Khanduri
arXiv:2607. 10517v1 Announce Type: cross Abstract: Activation steering offers a lightweight alternative to fine-tuning for controlling large language models at inference time.
By Seyed Arshan Dalili, Ajay Narayanan Sridhar, Vijaykrishnan Narayanan, Mehrdad Mahdavi
FishBack introduces a pullback Fisher geometry approach for activation steering in transformers, challenging the common Euclidean assumption of intermediate activation spaces. By deriving a closed‑form steering direction based on the Fisher information metric of the softmax layer, the method achieves target concept changes with minimal off‑target distortion, especially in early and middle layers. Experiments on GPT‑2 Small, Llama‑3‑8B, and Qwen3‑8B demonstrate significant reductions in off‑target KL divergence compared to existing steering baselines.
By Sihan Wang, Jiayi Zhao, Qingyan Cao, Hongbo Yao, Lin Shu
arXiv:2609.37066v1 Announce Type: cross
Abstract: Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed...
By Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry
arXiv:2606. 16620v1 Announce Type: cross Abstract: Inference-time scaling has become the dominant lever for improving language-model reasoning, but existing methods derive rollout diversity from a single source: stochastic token-level sampling.
By Soham Bhattacharjee, Dushyant Singh Chauhan, Salem Lahlou, Martin Takac, Nils Lukas