PixelDense introduces a dual‑stream representation alignment for pixel diffusion, separating semantic and geometric teachers (DINOv2, SAM2, Depth Anything v2, Metric3D v2) into distinct projection spaces with an orthogonality penalty. The method improves dense‑prediction benchmarks, boosting PixelGen‑XXL’s GenEval score from 0.7927 to 0.8093, achieving significant gains in panoptic quality and depth accuracy, and accelerating training from random initialization. It also enhances SDEdit editing by preserving background structure and increasing PSNR.
By Lehan Yang, Daiqing Qi, Wenhao Zhang, Avery Li, Yiqing Yang, Yifan Li, Yu Kong, Haitian Zheng, Zhifei Zhang, Zhe Lin, Varun Jampani, Sheng Li
PhysVista is a new benchmark that evaluates physical intelligence in Vision‑Language Models (VLMs) by integrating perception, reasoning, and plausibility assessment into a closed cognitive loop. It distinguishes between event‑level and scale‑level reasoning and tests models on both real‑world and AI‑generated videos to provide a holistic, fine‑grained analysis of physical understanding. Experiments show significant gaps in VLMs’ physical reasoning and plausibility assessment, underscoring the need for more principled, physically grounded multimodal designs.
By Xinge Peng, Yiting Lu, Tianwu Zhi, Wen Wen, Jianzhao Liu, Xin Li, Zhibo Chen
Omni-Embed-Mini is a 0.9B‑parameter model that embeds text, speech, audio, images, video, and visually‑rich documents into a single shared cosine space without updating any text‑side parameters. It uses a dense cascaded caption as a teacher signal, allowing the teacher and student to share identical backbone weights and requiring only lightweight projectors and phased LoRA adapters for alignment. The model achieves strong text retrieval performance (49.57 nDCG@10 on MTEB‑v2 BEIR‑8) while extending to five additional modalities and is significantly smaller than other open omni‑modal embedders.
By Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly, Ivan Laptev, Hisham Cholakkal
The paper introduces FaV-A, a multimodal agent that generates positive and negative space compositions in a staged manner. It first creates a base object, analyzes its shape to identify negative‑space semantics, and then produces compositional instructions for the final image. Experiments show that this approach yields more visually coherent and semantically aligned compositions than direct zero‑shot multimodal large language model baselines.
By Shiwen Wang, Jian Yang, Xu Wang, Xincan Wang, Weiming Dong
The paper proposes a new architecture for large language model (LLM) agents that enhances spatial understanding by combining geometrical tools with an LLM orchestrator in grid‑world environments. It first gathers geodesic trajectories, vector‑quantizes them to create a representative subset, and then has the LLM label each trajectory with a natural language description, turning them into reusable tools. During operation, the LLM selects the appropriate tool based on the current state and goal, while low‑level control executes the chosen trajectory, enabling efficient decision‑making in a partially observable 2D grid setting.
By Gabriel Turinici
MemFit is a long‑term memory system designed for conversational agents that stores each dialogue turn verbatim in an append‑only store, enabling near‑instantaneous, LLM‑free insertion. It indexes turns using segment summaries and employs an LLM‑free, multi‑path retrieval strategy that blends lexical and semantic signals with cross‑encoder reranking over caption‑augmented episodes. Experiments on LoCoMo, MemGallery, and LongMemEval‑S demonstrate state‑of‑the‑art performance while drastically reducing memory construction time and cost.
By Mitchell Piehl, Muchao Ye
The paper highlights that as Large Language Models grow in capability and prevalence, their environmental footprint is increasing, yet the machine learning community lacks standardized carbon accounting practices. An automated review of 5,285 NeurIPS 2025 papers shows almost no reporting of environmental impact. To address this, the authors propose standardized sustainability metrics for training efficiency, heuristics for estimating inference carbon costs, a software tool called carbonbenchmark for tracking emissions, and the SMAJ framework to encourage prioritizing computational efficiency and environmental accountability over marginal accuracy gains.
By Lachlan McGinness, Dan Pagendam, Robert Offner
Laurito et al. (PNAS 2025) found that large language models prefer model‑written descriptions over human‑written ones. In this study, the authors reconstructed the 5×5 generator‑by‑selector matrices from the original data and tested for an “own‑model premium” using a two‑way fixed‑effects model and permutation tests. The results show no statistically significant preference for a selector’s own model across products, paper abstracts, or films, with effect sizes near zero and confidence intervals that rule out a premium larger than about 0.04 share points.
By Dmitrij \.Zatuchin
The paper investigates how massive activations—large residual-stream coordinates that act as attention sinks in transformers—emerge, grow, and consolidate during training. It shows that these activations stabilize early, then concentrate onto a few redundant channels as surrounding ones erode, and that their collective scale is governed by weight decay. A balance model demonstrates that AdamW preconditioning drives growth while weight decay limits peak magnitude, with peak timing and size scaling predictably with the decay coefficient.
By S. Aaron McClendon, Jorge Gallego-Feliciano, Antonios Saravanos
The paper investigates how well large language models specify the software environments needed to run AI-generated code. Using a new agent protocol and a three‑layer dependency framework, the authors evaluate three coding agents across four languages and fifty tasks, finding that dependency specifications are often inconsistent, redundant, or incomplete. Agreement on dependency sets is as low as 7% for identical tasks, and newer agents show no improvement, indicating that environment specification remains a significant, unaddressed challenge in code generation.
By Bhanu Prakash Vangala, Tanu Malik
The paper investigates whether multimodal large language model (LLM) agents form partner‑specific conceptual pacts during repeated reference games. By comparing real agent dyads with a novel pseudo‑dyad baseline that preserves task structure but removes shared partner history, the authors show that humans compress descriptions and align labels through entrainment, whereas agents produce verbose, fixed‑effort descriptions with near‑ceiling label overlap regardless of partner. Thus, LLMs succeed in coordination without forming compact, history‑dependent referring expressions typical of human dialogue.
By Po-Ya Angela Wang, Chinmaya Mishra, Asl{\i} \"Ozy\"urek, Paula Rubio-Fern\'andez, Esam Ghaleb
The paper investigates how AI agents can discover and validate scientific formulas, focusing on the hydrotope—a geometric formula that unifies polynomial pieces of nonlinear surface-wave scattering. Through controlled experiments, the authors compare human–agent collaboration, single-prompt rediscovery, and various guided approaches, finding that only a PI$+$two-student workflow reliably reconstructs the full hydrotope and even uncovers a new analytic expression for the six-point amplitude $A_6$. The study highlights the challenges of combining exponentially many terms and the importance of coordinated analytic and numerical tasks for successful formula discovery.
By Zihan Zhou, Digvijay Wadekar, Matias Zaldarriaga
PACT is an end‑to‑end model that jointly learns human pose, contacts, and contact forces from monocular video. It augments a human reconstruction foundation model with learnable contact‑force tokens and a temporal transformer, and uses physics‑based supervision to enforce consistency between motion and forces. The authors also create a data annotation pipeline and a real‑world climbing benchmark, ForceWall, to train and evaluate the system, achieving state‑of‑the‑art performance and better generalization than staged approaches.
By Rikhat Akizhanov (MBZUAI), Yangsong Zhang (MBZUAI), Nikolai Kaliazin (MBZUAI), Peter Wolf (ETH Z\"urich), Yoshihiko Nakamura (MBZUAI), Pascal Fua (EPFL), Fabio Pizzati (MBZUAI), Ivan Laptev (MBZUAI)
The paper investigates which linguistic operations can be replaced by latent task representations instead of in‑context prompts. A 2.6 M‑parameter network learns to encode the geometry of a few‑shot support set and applies an additive update to a frozen GPT‑2 model, achieving zero‑shot performance comparable to or better than 10‑shot in‑context learning for forward inflection and lemmatization. However, for arbitrary pairings such as antonymy, the latent approach only reaches about half the performance of in‑context learning, indicating that productive rules amortize well while memorized pairings do not.
By Gunmay Jhingran
The study evaluates two hierarchical red team agent architectures—RL planner with RL executor (RL+RL) and LLM planner with LLM executor (LLM+LLM)—across multiple cyber environments. In compact, densely rewarded settings, RL+RL outperforms LLM+LLM, while in larger, escalation‑gated networks a pretrained LLM agent dominates. The results reveal environment‑dependent performance inversions tied to architecture‑specific bottlenecks, challenging the notion that one design is universally superior.
By Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian, Ankit Shah
The paper investigates a new tension in large language models between alignment and faithfulness, termed alignment-induced unfaithfulness (AIU). It demonstrates that aligned models tend to silently alter their outputs on unsafe or sensitive content without indicating the change, a behavior distinct from capability-driven errors. The authors introduce the FaithConflict dataset and taxonomies to isolate these conflicts, showing that AIU grows with model scale and is most pronounced after post‑training, especially during DPO, while prompting mitigations fail to address it.
By Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur, Dilek Hakkani-Tur
The paper explores using frozen large language models (LLMs) to control hierarchical cyber defense systems without retraining across different network scales. By separating planning (selecting subnets to defend) from execution (choosing defensive actions), the authors compare RL+RL, LLM+RL, and LLM+LLM setups in the Cyberwheel environment. Results show that while an LLM planner alone offers limited gains, extending LLM control to execution—especially with a 70B general‑purpose model—yields strong defensive performance across small, medium, and large networks, matching or surpassing retrained RL baselines.
By Harshith Doppalapudi, Nathaniel D. Bastian, Ankit Shah
The paper introduces RuleEvolve, a self‑evolving framework that automatically refines coding rules for AI coding agents. It maintains a pool of candidate rules, uses an LLM‑powered mutator to generate variants, and a judge module to evaluate and update the pool with the best performers. Experiments across multiple frameworks, LLMs, and benchmarks show that RuleEvolve surpasses manual engineering and prompt‑optimization baselines in functional correctness, code length, and generation cost.
By Zhengyuan Jiang, Reachal Wang, Yuepeng Hu, Yupu Wang, Yuqi Jia, Neil Zhenqiang Gong
The paper presents practical training recipes for looped language models, showing that a compute‑efficient pipeline can reduce the training budget from 7.7 T tokens to 310 B tokens while maintaining strong reasoning performance. In controlled experiments, a 1.4 B LoopLM outperforms a parameter‑matched dense model on 12 benchmarks, achieving significant gains on GSM8K, MATH, and DROP, and approaches a 3.9 B dense model at matched inference compute. The authors also provide a minimal recipe to convert pretrained dense models into looped ones, demonstrating improvements on Qwen3‑1.7B‑Base across multiple data regimes.
By Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii
The paper introduces FTC, a sequential functional structured Tucker compression method for large language model attentions. FTC adapts its approximation to the current compressed model, jointly exploits the Q/K/V head structure, and treats the output projection separately to handle representation shifts. It achieves the lowest WikiText‑2 perplexity across seven decoder‑only LLMs (6B–32B) at all tested keep ratios, with significant gains under aggressive compression and benefits that transfer to downstream tasks.
By Jiangfeng Chen, Xinyu Wang, Tianshuo Yan, Hanwei Wu, Xiao-Wen Chang, Yang Zhang, Lei Ding