The paper introduces BiPA, a method for evaluating skill optimizers that rewrite natural‑language procedures for agents. By replaying the optimizer’s acceptance rule on identical candidates and recording episodes as bipartite graphs of turns and read text, BiPA isolates the effect of selection from the cost of added input tokens. Experiments on three optimizers, three benchmarks, and two models show that learned skills perform only marginally better than hand‑written seeds, and that apparent dev‑set gains often disappear on test, revealing that optimizers tend to add redundant rules that increase token usage.
By Jiawei Zhou, Bo Peng, Yuxuan Huang, Ming Zhang, Youbang Sun, Xiang Wang, Chaochao Lu
The paper introduces BiPA, a method to evaluate skill optimizers that rewrite natural‑language procedures for agents performing long‑horizon tasks. BiPA replays the optimizer’s acceptance rule on identical candidates to isolate selection effects and records each episode as a bipartite graph of turns and read text, revealing that learned skills only marginally outperform hand‑written seeds and that many gains vanish on test data. The study finds optimizers add extra input tokens and turns, and proposes four design principles for more reliable skill editing.
The study compares an equal‑budget Monte Carlo Tree Search (MCTS) scheduler with a randomized‑restart list scheduler for directed‑acyclic‑graph (DAG) workflows, ensuring all methods receive the same number of candidate schedules. Across 768 validated schedules, MCTS performed 2.88% worse than the restarts at a budget of 2048 evaluations, with the deficit persisting even when the selection rule was changed to textbook UCT. The results, limited to synthetic domains with 8–32 tasks and up to 2048 evaluations, suggest that simpler restart strategies can outperform MCTS in certain workflow structures.
The study investigates how inference‑time interventions and weight consolidation affect open‑ended generation in an online bin‑packing task. By iteratively generating, verifying, selecting, and consolidating with LoRA, the model’s outputs shift toward higher value, reducing excess by 1.7 points and outperforming random consolidation by 3.1 points. Across three independent runs, the mean performance remained consistent, and the best candidates converged to the classic heuristic’s level without exceeding it, while consolidation also lowered the proportion of better‑than‑classic candidates but increased their absolute number.
By Roberto I. Ono Filho
The paper investigates how to allocate test‑time computation in recursive reasoning models, specifically the Tiny Recursive Model (TRM). Experiments show that resampling the answer state (y) is more effective than perturbing the reasoning state (z) for recovering failures, especially as compute budgets increase. The proposed Gated Answer Resampling (GAR) method, which restarts only rejected puzzles, boosts TRM performance from 85.9% to 99.4% on Sudoku‑Extreme and from 68.8% to 85.6% on the Pencil Puzzle Bench, outperforming the probabilistic variant that uses far more compute.
By Maxime Illouz
The study compares an MCTS scheduling method with a randomized‑restart list scheduler on DAG workflows under a strictly controlled equal‑budget protocol. With each arm given the same number of candidate schedules (2048), the MCTS method performed 2.88% worse than the restarts, a deficit that persists across various workflow densities and processor counts. Switching to a textbook UCT selection rule largely eliminates this gap, suggesting the original selection strategy was a key factor.
By Yuyu Zhang