The paper introduces MECo, a large language model–driven multi‑task evolutionary framework that enables zero‑shot cross‑problem generalization for combinatorial optimization. MECo maintains task‑conditioned heuristic populations, uses a transfer gap based on cross‑task performance to guide interactions, and selects a compact heuristic set that covers source combinations. Experiments on 32 variants of vehicle routing and flexible job‑shop scheduling demonstrate that MECo achieves lower mean costs than eight automated heuristic design baselines and improves both in‑domain and out‑of‑domain performance when integrated with other methods.
By Zhouliang Xie, Changliang Zhou, Genghui Li, Zhenkun Wang
The study examines whether annual reports can reveal how companies disclose AI-related risks and responses. Using a two-stage classification pipeline on 9,821 reports from 1,362 UK listed firms (2020‑2026), the authors find that mentions of AI risk rose from 2.8% to 41.2% and AI adoption disclosures from 13.8% to 45.2%, with most risk mentions clustering around major vendors like Microsoft. Disclosure varies by sector and market segment, with Critical National Infrastructure and AIM reports lagging, and substantive risk disclosures remain rare—only 4.3% in 2025.
"Why It Matters": The findings show that while AI risk is increasingly referenced in corporate reports, substantive disclosures are scarce, highlighting a gap in transparency that could affect societal resilience.
By Bart Jaworski
Research: Qwen3.8 27B addition in words
Colin Frasier posted on Bluesky about an experiment he ran over two years ago using GPT-4o to see how well it could "compute the sum but return the answ...
The article explains that a language model can remember a fact only in the direction it was trained, leading it to fail when asked to reverse that fact. It uses a toy example to illustrate how this limitation arises. The piece highlights the asymmetry in how models learn and recall relationships.
By Utkarsh Mangal
Simon Willison has released the September edition of his sponsors‑only monthly newsletter. The issue covers new Fable class models, a pricing war, 3D graphics with Blender and pixel art, LLMs applied to mathematics, accidental cyberattacks, the vulnapocalypse affecting Datasette, his current tools, and software releases for 2026. Sponsors can access the full newsletter for $10/month, with an August preview available for free.
The article explores whether large language model (LLM) agents can discover new ideas by examining their creative potential. It frames this inquiry through the lens of creativity, aiming to assess how LLM agents might generate novel insights or solutions. The discussion is presented as a post on Towards Data Science.
By Shitanshu Bhushan
The paper introduces format‑aware fusion, a method that co‑designs quantization producers with their scale domains and consumer layouts to fully exploit four‑bit floating‑point (FP4) Tensor Cores. Using this approach, the authors pretrain the Llama‑3‑family 8B model on 160 billion tokens, achieving up to 37.9 K tokens/s/GPU—significantly higher than standard bfloat16 or Transformer Engine FP4 baselines. The study demonstrates that FP4 performance depends on the interplay of scaling, operand packing, layout, and execution path, with downstream task rankings diverging from training‑loss rankings.
By Robert Hu
arXiv:2610. 00009v1 Announce Type: new Abstract: Frequency-collapse attention [Zeris, 2026e] achieves large gains over standard dot-product attention by replacing the Q/K dot product with a bandpass-filtered inner product at a learned frequency.
By Athanasios Zeris
The paper introduces a small DeltaNet variant that uses a fixed-size recurrent memory to store 32 key‑value pairings per sequence. With only 32 KiB of matrix state, the model attains 99.95 % mean accuracy across three training seeds when selecting the correct value, and this performance persists even when the pre‑query context is extended to 1,798 tokens. Zeroing the first memory block eliminates recall, and exploratory tests with 48 pairs show near‑chance performance after a quarter of the training budget, indicating no clear capacity limit. Baseline models matched in parameters, including a Transformer, remain near chance, preventing a direct memory‑efficiency comparison.
By Samuel Larson
The paper introduces MOVE, a framework for multimodal open‑world verification and expansion in graph learning. MOVE jointly uses visual tokens, textual attributes, and graph context to identify nodes that cannot be assigned to existing classes, then employs a multimodal LLM to generate candidate class descriptions. It selectively expands the class space only when multimodal evidence consistently supports the new classes, avoiding redundancy, and reports an average 11.87% improvement across unknown recognition, open‑domain annotation, and downstream graph learning tasks.
By Zekai Chen, Jiayang Xing, Xun Wu, Miao Zhang, Xunkai Li, Kairui Yang, Zhengyu Wu, Xu Wang, Rong-Hua Li, Guoren Wang
The paper investigates the reliability of attention‑head ablation as a causal inference tool in language models. Using GPT‑2 small, the authors find that a natural post‑projection zeroing method is almost uncorrelated with a corrected pre‑projection ablation and yields a completely different set of top‑5 important heads. They also show that binary accuracy can mask effects near performance floors or ceilings, whereas gold‑token log‑probability provides a graded signal. By employing a discovery/held‑out split and 1,000 matched random‑head and layer‑matched‑head controls, the corrected per‑head effect ranking remains highly stable (Spearman ρ = 0.974) and the top‑5 heads significantly outperform both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is weak on GPT‑2, and replication on DistilGPT‑2 confirms the intervention‑semantic and matched‑control findings.
By Juli Huang
The paper introduces Density-Aware Reward Aggregation (DARA), a method that adjusts reward weighting in multi-reward reinforcement learning based on the density of active rewards within rollout batches. By applying an inverse-square-root density correction, DARA gives more weight to less frequently active rewards, enabling faster learning of targeted behaviors. Experiments on tool calling and mathematical reasoning demonstrate that DARA achieves comparable final performance while reducing training steps by up to 26% and 65% respectively.
By Tong Zheng, Skylar Zhai, Zhan Cheng, TianMing Sha, Youling Huang, Shuo Zhou, Shaotong Qi, Jingcheng Liang, Xuwei Ding, Pengcheng Xu
The study evaluates whether biological reasoning models actually use their biological inputs by testing six models on DNA, protein, and single‑cell tasks. By perturbing one biological input while keeping others fixed, the authors find that many models (e.g., Evo2, ESM3, BioReason, BioReason‑Pro) rely primarily on textual information, with minimal impact from the biological representations. In contrast, models like ChatNT, Prot2Text‑V2, CellWhisperer, and Cell2Sentence‑Scale show greater dependence on their biological inputs, yet overall accuracy gains do not consistently reflect increased biological input contribution.
By Ada Fang, Nikitha Thoduguli, Lukas Fesser, Hanlin Zhang, Sham M. Kakade, Marinka Zitnik
HADRec is a Hierarchy-Aware Drug Recommendation framework that fuses molecular knowledge and electronic health records to improve medication recommendation. It uses LLaMA-7B to encode clinical notes, ChemBERTa to encode drug SMILES strings, and a cross‑attention mechanism for multimodal fusion, while a hierarchical predictor and consistency constraint loss enforce adherence to the ATC classification system. Experiments on MIMIC‑III and MIMIC‑IV show state‑of‑the‑art performance, strong generalization, and well‑calibrated predictions, with counterfactual evaluation indicating clinically aligned reasoning.
By Junke Wang, Hongshun Ling, Li Zhang, Jinjing Wu, Tong Shao, Fang Wang, Yuan Gao
The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.
By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu
Range-GRPO introduces a semi‑supervised post‑training framework that uses conformally calibrated reward ranges instead of single point scores for large language models. By comparing reward ranges pairwise within rollout groups, the method incorporates reward uncertainty into both the magnitude and direction of learning signals. Experiments show that Range‑GRPO outperforms other semi‑supervised approaches on both in‑distribution and out‑of‑distribution tasks while using fewer training resources.
By Ryunyi Lee, Kangjun Noh, Somin Kim, Heedong Kim, Kyungwoo Song
SkillEvoLean introduces a mutation‑enhanced skill evolution framework for Lean theorem provers, jointly refining a high‑level solving policy and its reference knowledge. The method combines progressive updates from successful and failed proof trajectories with mutation‑based exploration when no complete proof is found, sampling mathematical concepts to generate new skill candidates. Evaluations on MiniF2F, PutnamBench, IMO 2025, and USAMO 2026 show significant proof success improvements over baseline approaches.
By Kuo Zhou, ZiXion Yang, Lu Zhang
The paper investigates the components of group attention used in the Chronos-2 time‑series forecasting model, which attends over variates of a group for both multivariate (MV) and in‑context learning (ICL) forecasting. By editing the attention matrix at inference, the authors separate the head into a V/O pathway that projects a weighted summary of the group and a Q/K pathway that determines the weights. They find that uniform pooling (V/O without Q/K weighting) improves performance in most sensor‑network configurations, while learned weighting helps MV but degrades ICL in many cases; applying uniform weighting in the first block alone benefits all tested ICL settings.
By Michael Fore, James Mason Inder, Mrishika Nair, Praneetha Vaddamanu, Sharlina Keshava
arXiv:2610. 01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)?
By Yohan Chatelain (Krembil Centre for Neuroinformatics, CAMH, Toronto, Canada), Pablo de Oliveira Castro (Universite Paris-Saclay, UVSQ, LI-PaRAD, Versailles, France)
The paper presents a systematic comparison of recurrent neural networks and Transformer models for iterative diffusion MRI tractography, focusing on training strategies, input representations, and hyperparameter tuning. It introduces a generation‑validation phase that provides streamline‑level supervision, enabling the models to achieve the best performance reported on the ISMRM2015 challenge dataset. The study also evaluates the effects of missing bundles, noisy training data, and invalid fibers, and demonstrates applicability to in‑vivo data from the Tractoinferno database.
By Emmanuelle Renauld, Philippe Poulin, Hugo Larochelle, Antoine Th\'eberge, Maxime Descoteaux