XStack: A Benchmark for ML Repo Migration
ML code should run where researchers choose, not where it was first written. Today, machine learning code mainly lives in two ecosystems, PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repos...
ML code should run where researchers choose, not where it was first written. Today, machine learning code mainly lives in two ecosystems, PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repos...
The study investigates whether coding agents can reliably migrate PyTorch machine learning projects from GPUs to TPUs. Using Claude Code, ten projects were migrated, with eight meeting the project‑goal criterion but 76% of execution attempts failing, often due to compilation issues or silent changes in computation. The research highlights significant human effort (average 2.4 interventions) and variable costs, noting that while some TPUs can be faster and cheaper, failures and accuracy drops remain common.
arXiv:2605.29054v3 Announce Type: replace-cross Abstract: Migrating a machine-learning codebase across frameworks--for example from PyTorch to JAX--is a routine but costly engineering task that is fu...
arXiv:2607. 02469v1 Announce Type: cross Abstract: Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior.
arXiv:2608. 08189v1 Announce Type: new Abstract: LLM-driven program discovery relies on rapid evaluator feedback, but many scientific and engineering tasks require high-fidelity simulations, hardware execution, or physical experiments, making each evaluation expensive.
arXiv:2603. 16011v3 Announce Type: replace-cross Abstract: Large language model (LLM) coding agents increasingly operate at the repository level, motivating benchmarks that evaluate their ability to optimize entire codebases under realistic constraints.
arXiv:2603. 05026v2 Announce Type: replace-cross Abstract: Language model (LM) agents have driven substantial progress in automated software engineering (SWE), yet building and testing software repositories at scale remains a largely manual and labor-intensive bottleneck.
arXiv:2610.09079v1 Announce Type: cross Abstract: Large language models equipped with development environments have moved code generation toward repository-scale construction, yet building complete r...
arXiv:2606. 04023v1 Announce Type: cross Abstract: While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.
MaxKernel is a multi‑agent system designed to generate high‑performance custom kernels for TPUs. It offers three paradigms: a Human‑in‑the‑Loop agent for collaborative design, an Autonomous agent that runs a fully automated optimization loop, and a Graph‑Based Autonomous Search for global exploration. All paradigms share specialized sub‑agents for planning, implementation, debugging, testing, and profiling, and the system consistently matches expert hand‑tuned baselines on the JaxBench suite and real‑world workloads.
arXiv:2511. 06090v3 Announce Type: replace-cross Abstract: Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correctness.
arXiv:2610.34633v1 Announce Type: cross Abstract: Repository-level code migration rewrites an entire project in a new programming language while preserving its observable behavior. It is challenging...