Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

23,195 stories · RSS feed

arXiv Machine Learning
Jul 8

RayRoPE: Projective Ray Positional Encoding for Multi-view Attention

arXiv:2601. 15275v3 Announce Type: replace-cross Abstract: We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-invariant attention with multi-frequency similarity, and can adapt to the geometry of the underlying 3D scene.

By Yu Wu, Minsik Jeon, Jen-Hao Rick Chang, Oncel Tuzel, Shubham Tulsiani
arXiv Machine Learning
Jul 8

Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices

arXiv:2607. 05933v1 Announce Type: cross Abstract: Dynamic Voltage Frequency Scaling (DVFS) on resource-constrained embedded GPU platforms is essential for energy-efficient small language model (SLM) fine-tuning, as privacy- and personalization-driven adaptation increasingly requires local execution and involves repeated forward-backward optimization over many mini-batches, making it substantially more time- and energy-intensive than single-pass inference.

By Jurn-Gyu Park, Sanzhar Zholdybayev, Aidar Amangeldi, Ademi Zhanuzakova
arXiv Machine Learning
Jul 8

PGOT: A Physics-Geometry Operator Transformer for Complex PDEs

arXiv:2512. 23192v4 Announce Type: replace Abstract: While Transformers have demonstrated remarkable potential in modeling Partial Differential Equations (PDEs), modeling large-scale unstructured meshes with complex geometries remains a significant challenge.

By Zhuo Zhang, Xi Yang, Ying Miao, Xiaobin Hu, Yifu Gao, Yong Yang, Canqun Yang, Boocheong Khoo
arXiv AI
Jul 8

Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers

arXiv:2605. 13974v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) and related flow-based architectures are now among the strongest text-to-image generators, yet the internal mechanisms through which prompts shape image semantics remain poorly understood.

By Evelyn Turri, Davide Bucciarelli, Sara Sarto, Lorenzo Baraldi, Marcella Cornia
arXiv AI
Jul 8

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

arXiv:2607. 06326v1 Announce Type: new Abstract: Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-latency runtime moderation.

By He Liu, Changtao Miao, Xinjie Yang, Tianle Song, Yin Wu, Junchi Chen, Bintao He, Xinyuan Zhang, Bo Zhang, Shi Yan, Wei Lu, Wei Wang, Danyang Xu, Jiansheng Cai, Zhe Li
arXiv AI
Jul 8

FirstResearch: Auditable Question Formation for LLM Scientific Discovery Agents

arXiv:2607. 05682v1 Announce Type: new Abstract: LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect.

By Yufeng Wang
arXiv AI
Jul 8

Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving

arXiv:2607. 05399v1 Announce Type: cross Abstract: Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are difficult to compare because they were evaluated on different models, tasks, budgets, and serving stacks.

By Nikita Agrawal, Ruben Mayer