TSFLab is a self‑improving, agent‑agnostic workbench designed to streamline time‑series forecasting research. It equips coding agents with shared resources and tools, enabling low‑cost experimentation, reproducibility, and continuous maintenance of research artifacts. Experiments show that TSFLab boosts task completion rates by 20‑163% and reduces token usage and development time significantly compared to baseline approaches.
arXiv:2409. 11363v2 Announce Type: replace-cross Abstract: AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research.
Time-series foundation models provide strong zero-shot forecast, but deployment for real-world applications still requires task-specific designs for model inputs and outputs. Direct inference can leav...
The paper introduces FABench, a scalable backtesting environment designed to evaluate forecasting agents. FABench automatically discovers events, generates questions, and resolves outcomes for any past forecast date, allowing continuous renewal beyond model training cutoffs. In initial tests, retrieval improves Brier scores across eight models, with top agents matching or surpassing market consensus on 54 Polymarket-linked questions, yet analysis shows persistent evidence‑weighting failures that highlight room for improvement.
TimeRSI is a self‑improving deployment system for time‑series foundation models that automates the creation and refinement of forecasting pipelines. A coding agent translates task descriptions into executable changes, tests each revision on historical data, and uses error feedback to guide further revisions. During live deployment, an adaptive selector chooses the best pipeline based on completed forecasts under similar conditions, achieving superior performance on 198 tasks compared to all other submissions.
arXiv:2610.19725v1 Announce Type: new Abstract: Time-series foundation models provide strong zero-shot forecast, but deployment for real- world applications still requires task-specific designs for m...
The paper introduces AIDE^2, an AI research agent that recursively improves its own code by proposing, benchmarking, and selecting modifications. Over an eight‑day autonomous run, it achieved seven successive improvements—including new search policies and memory mechanisms—that transferred to four held‑out benchmarks in machine learning, algorithm engineering, and weather forecasting. The agent’s best version matched or outperformed a top human‑engineered production research agent and also reduced reward‑hacking rates, despite never optimizing for that metric.
MetaCaster is a meta-harness-optimized multi-agent framework that enables few-shot learning for lightweight time series forecasters. It uses agentic data generation to automatically train specialized forecasters from only a few examples and textual contexts, positioning agents as intermediary engineers rather than direct forecasters. Experiments on 18 datasets and 23 lightweight forecasters show that MetaCaster achieves data and computational efficiency while maintaining high forecasting quality.
RSIGym is a new agent-native research environment built on Everything as a Service (EaaS) that exposes training, inference, rollout, evaluation, and sandbox execution through reusable services. It supports shared budget and permission controls for Data, Harness, and Joint improvement tracks, allowing agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment. Using RSIGym, the Opus 5 model achieved the highest RSI-Index of 0.4809 across five benchmarks, improving performance on software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks.
arXiv:2608. 13940v1 Announce Type: new Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled by cost: a candidate solution can be written in minutes, whereas evaluating it can take hours to days of GPU time.
arXiv:2606. 18733v1 Announce Type: cross Abstract: Realistic coding-agent benchmarks often replay public GitHub issues and pull requests, making them vulnerable to overlap with model pretraining, fine-tuning, synthetic-data generation, or benchmark-driven model selection.
The paper investigates whether a language‑model agent can autonomously study an unfamiliar environment without prior task instructions or examples, and decide how to prepare for future tasks. It formalizes task‑agnostic environment preprocessing, where a studying system explores under a budget to produce reusable artifacts for a later solver. Experiments on six diverse benchmarks show that a meta‑agent variant often outperforms fixed methods, though larger budgets do not consistently boost downstream reward, yet the artifacts still reduce test‑time sampling needed to achieve a target score.
The paper introduces AutoMat, a benchmark designed to test large language model (LLM) coding agents on their ability to reproduce claims from computational materials science. AutoMat presents three challenges: reconstructing underspecified procedures, navigating specialized toolchains, and assessing whether the evidence supports a claim. Experiments show that current LLM agents achieve low success rates, with the best setting reaching only 53%, and failures stem mainly from incomplete procedures, methodological deviations, and execution fragility.