SWE-Opt is a new benchmark that evaluates large language models (LLMs) on repository‑level software performance optimization. It is built from 102 expert‑written optimizations from open‑source projects and measures runtime, peak memory, and Time‑Weighted Memory Usage (TWMU) under varied input data, execution conditions, and noise‑aware measurements. The study finds that current LLMs achieve few runtime gains and almost no memory improvements, whereas expert implementations deliver a 15.5× speedup and 171.3× memory reduction.
By Ezgi Sar{\i}kayak, Wenchao Gu, Hesham Ghonim, Chunyang Chen
Switch‑SWE is a new benchmark that tests whether coding agents can resume an unfinished task after working on other tasks from the same repository. It provides 200 focus tasks from 19 Python repositories, each paired with matched uninterrupted and switched sessions, and evaluates agents on the final patch only. The benchmark reveals that interleaved work creates an average 11.5‑percentage‑point gap in task resolution, and a context‑management plug‑in can improve performance by 5.5 to 15.5 percentage points.
By Shuochen Chang, Qianli Ma, Yibo Miao, Qingyang Liu, Bingjie Gao, Haonan Zhao, Siyu Sun, Yulin Sun, Han Zhao, Jiangtong Li
TAC-CoT is a framework that replaces explicit chain-of-thought reasoning with compact latent reasoning segments, keeping only textual tokens for tool calls, observations, and final answers. This interleaved design allows large language model agents to use tools without generating full verbal reasoning steps. Experiments on math and QA benchmarks show up to 8.8‑point accuracy gains and a 6.8× reduction in model forward passes, while also improving tool‑use quality in single- and multi‑tool settings.
By Juseon Do, Sangmin Bae, Jeesu Jung, Hwanjun Song
The paper introduces Adaptive Index Reuse (AIR), a method that dynamically determines how many layers of a sparse attention indexer should refresh during a language agent’s reasoning process. AIR replaces the static layer‑refresh schedule of IndexCache with a small controller that selects from a nested library of patterns based on recent computation signals and tool events, optimizing for task reward under a compute budget. Experiments on Qwen3‑4B‑Instruct‑2507 with a 128K context show AIR improving success rates on SWE‑bench Verified, Terminal‑Bench 2.0, and three τ²‑bench domains while reducing time per successful task, and it matches or surpasses the static schedule across 50 varied settings.
By Guangyuan Wang, Yifan Zhang, Shilong Liu
The paper investigates whether large language model (LLM) agents can acquire new knowledge from tutorial videos without updating their parameters. It introduces Tutorial Agentic Learning (TAL), a non‑parametric continual‑learning framework that builds a structured tutorial‑book memory via a two‑stage adversarial process, and retrieves information through a Table‑of‑Contents tree during inference. On the VidLearn multimodal QA benchmark—comprising 72 chess tutorial videos—TAL outperforms baseline models, achieving up to 113% relative gains on training data and 91% on test data, and improves in real‑world chess openings in 69% of head‑to‑head games, with additional evidence of generality in a second domain.
By Haoyu Zhao, Zafeirios Fountas, Jun Wang
The study investigates how often web agents incorrectly treat an HTTP 200 response as a successful fetch of useful content. By sampling 10,307 URLs—including random paths and expert‑verified claim citations—the authors found that roughly 20–25% of 200 responses lack the page’s main content, leading to inflated estimates of claim verifiability and citation drift. They propose simple heuristics and classifiers to detect unusable responses, achieving high recall with low false‑alarm rates, and recommend that agents report fetch outcomes explicitly.
By Shuyang Zhang (The Hong Kong Polytechnic University)
Decentralized Multi-Agent Coordination and Communication in Shared Codebases introduces ParallACT, a benchmark for studying how coding agents with independent, human‑driven requests interact in shared repositories. The benchmark includes complementary tasks that require agents to cooperate via a shared interface, and contradictory tasks where agents’ edits conflict, leading to poor merged outcomes. Experiments show that a single‑agent oracle outperforms parallel agents, especially on contradictory tasks, and that merely notifying agents of each other or providing a communication channel only marginally improves performance at a token cost.
By Rishabh Mediratta, Lingyun Xiao, Peter Stone, Joydeep Biswas, Elias Stengel-Eskin
NitiBench+ is a Thai legal question‑answering benchmark that spans 17 domains and contains 1,670 items derived from real court decisions. Each item includes both the law text that was in force at the decision date and the current text, along with a retrieval corpus of every dated version of every Thai statute section. The benchmark evaluates models using the Thai Bar Council rubric, Bar‑standard score, and citation overlap, and it reveals that reading outdated or missing law significantly degrades performance.
By Pawitsapak Akarajaradwong, Sarana Nutanong, Chompakorn Chaksangchaichot
The paper introduces CREDO, a method that redistributes credit to intermediate evidence steps in Temporal Knowledge Graph Question Answering agents. By comparing rollouts of the same question and using answer overlap as a proxy, CREDO assigns stage-level credit while preserving overall trajectory rewards. Experiments show significant accuracy gains on MultiTQ and TimelineKGQA datasets compared to prior baselines.
By Yue Leng, Shengtao Wen, Xiang Chen, Xinyi Zeng, Yu Tian, Sheng-Jun Huang
arXiv:2602.05842v3 Announce Type: replace
Abstract: Large language models (LLMs) have achieved strong performance in language-centric tasks. However, in agentic settings, LLMs often struggle to antic...
By Xiao Yu, Baolin Peng, Ruize Xu, Yelong Shen, Pengcheng He, Suman Nath, Nikhil Singh, Jiangfeng Gao, Zhou Yu
arXiv:2611.03374v1 Announce Type: cross
Abstract: Recent language model (LM) safety failures have largely been studied as disparate phenomena. In agentic misalignment, a model blackmails an executive...
By Brandon Jaipersaud, Elliot Creager
arXiv:2611.03308v1 Announce Type: cross
Abstract: A frontier language model was recently reported to solve 1D NMR structural elucidation without tools. We ask whether this capability transfers to tan...
By Gabriel Asher, Cailum M. K. Stienstra, Tomo Oga, Devesh Shah Antonio H. de O. Fonseca, Niall O'Connor, Michael Widrich
arXiv:2611.01728v1 Announce Type: cross
Abstract: Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contribution...
By Zerui Chen, Qinggang Zhang, Qichuan Liu, Zhishang Xiang, Zhimin Wei, Yunbo Tang, Zhuo Li, Jinsong Su, Zhihong Zhang
arXiv:2611.02332v1 Announce Type: cross
Abstract: Tool-using language-model agents often execute actions that their domain policies forbid. We present LeanGuard, a runtime that checks every proposed...
By Sreehari Sankar, Debargha Ganguly, Vikash Singh, Mohsen Hariri, Weicong Chen, Yanyan Zhang, Shouren Wang, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Idan Davidovich, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Joe Stubbs, Erman Ayday, Jing Ma, Vipin Chaudhary
arXiv:2608.02372v2 Announce Type: replace
Abstract: Large language model (LLM) agents are increasingly used to support decisions by calling external tools. Existing work on LLM tool-calling assumes t...
By Abdulrahman AlRabah, Xiaocheng Yang, Dilek Hakkani-T\"ur, Abdussalam Alawini
arXiv:2611. 02079v1 Announce Type: cross Abstract: Harness optimization can substantially improve LLM agents without modifying model weights, but existing methods primarily optimize response quality.
By Xingzhi Qian, Sungho Park, Rongyuan Tan, Jianbo Yu, Jue Zhang, Lorenzo Cavallaro, Menglin Xia, Yanzhi Chen, Xinnuo Xu, Yu Kang, Xinran Zheng, Gagan Somashekar, Xuchao Zhang, Chetan Bansal, Qingwei Lin, Victor R\"uhle
Release: datasette-apps 0.3a0
Datasette Apps adds support for hosting and editing custom HTML applications inside of Datasette.
These can be created by hand, but it's easier to create them usi...
DevRev uses GPT-6.1 Sol to power Computer and Operator, balancing reasoning, speed, and cost for enterprise service.
Release: datasette-agent 0.5a0
This is a significant release of Datasette Agent. The most notable new feature is called the panel - it lets the agent chat open and then sit next to pages from...
For a decade, progress in artificial intelligence has been measured by how much a machine can do on its own. Large foundation models now write code and run multistep tasks, and personal agents act on...