arXiv AI By Weixian Xu, Yixiu Liu, Yang Nan, Lyumanshan Ye, Xiangkun Hu, Zhen Qin, Pengfei Liu

Neural Architecture Discovery via Autonomous Evolution

Read the original on arXiv AI →

The paper introduces ASI-Arch, an AI-driven system that autonomously conducts neural architecture research through a closed-loop cycle of research, experiment, analyze, and update. In experiments focused on linear attention, ASI-Arch performed 1,773 iterations and identified 105 state‑of‑the‑art architectures, with its top design surpassing DeltaNet by nearly three times the improvement achieved by Mamba2. The study also dissects how various components of the framework contribute to success in this demanding research context.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Sep 25

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

EvoTreeNAD is a genealogy‑guided evolutionary algorithm that autonomously discovers neural architectures without a predefined seed or search space. Starting from an empty root, it builds a persistent genealogy where each node represents a complete architecture; top‑percentile values from nodes and descendants steer lineage selection. The method combines an Idea Agent that proposes variants and a Code Agent that implements them, with theoretical analysis showing stationary variation regimes and empirical results demonstrating superior performance on CIFAR‑10/100 and MedMNIST‑v2 tasks.

By Lishan Yu, Derek Jiu, Qizhen Lan, Xiaoqian Jiang
arXiv Computation and Language
Oct 22

Building AI Research Teams for Autonomous Discovery of Linear Attention Architectures

The paper proposes a framework for building autonomous AI research teams that can discover linear attention architectures. By assembling agents with diverse disciplinary expertise, private research histories, shared experimental evidence, and adaptive allocation of research opportunities, the team can balance exploration and exploitation under limited experimental budgets. Experiments show that this design outperforms GDN2 at 1.3B scale on language modeling, common‑sense reasoning, and retrieval tasks, and that removing any single component weakens discovery.

By Yujian Liu, Li An, Zelin He, Jacob Helwig, Jiabao Ji, Rohit Jain, Siyu Zhu, Shiyu Chang
Hugging Face Trending Papers
Aug 3

Long-Horizon Autonomous Architecture Research with a Language-Model Agent: A Behavioural Case Study

We study what happens when a single general-purpose large language model acts as the sole researcher on a long-horizon neural architecture design problem. The agent receives a scientific question, an initial hypothesis and motivation, a compute budget, and research affordances (source and experiment management, experiment tracking, literature access, and persistent memory), then autonomously proposes, implements, evaluates, and records experiments over an extended period.