arXiv AI
XStack: A Benchmark for ML Repo Migration
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
1d ago
XStack: A Benchmark for ML Repo Migration
ML code should run where researchers choose, not where it was first written. Today, machine learning code mainly lives in two ecosystems, PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repos...
arXiv Machine Learning
Oct 28
From GPU to TPU with Coding Agents: What Works, What Fails, and What It Costs?
The study investigates whether coding agents can reliably migrate PyTorch machine learning projects from GPUs to TPUs. Using Claude Code, ten projects were migrated, with eight meeting the project‑goal criterion but 76% of execution attempts failing, often due to compilation issues or silent changes in computation. The research highlights significant human effort (average 2.4 interventions) and variable costs, noting that while some TPUs can be faster and cheaper, failures and accuracy drops remain common.
arXiv Computation and Language
Oct 15
T2J-BENCH: Benchmarking Coding Agents on Execution-Grounded PyTorch-to-JAX Repository Migration
arXiv:2605.29054v3 Announce Type: replace-cross Abstract: Migrating a machine-learning codebase across frameworks--for example from PyTorch to JAX--is a routine but costly engineering task that is fu...
arXiv AI
Jul 3
TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution
arXiv:2607. 02469v1 Announce Type: cross Abstract: Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior.
arXiv AI
Aug 11
Janus: An Algorithm-Evaluator Co-Evolution Framework for LLM-Driven Discovery under Expensive Evaluation Budgets
arXiv:2608. 08189v1 Announce Type: new Abstract: LLM-driven program discovery relies on rapid evaluator feedback, but many scientific and engineering tasks require high-fidelity simulations, hardware execution, or physical experiments, making each evaluation expensive.
arXiv AI
Jul 21
FormulaCode: Evaluating Agentic Optimization on Large Codebases
arXiv:2603. 16011v3 Announce Type: replace-cross Abstract: Large language model (LLM) coding agents increasingly operate at the repository level, motivating benchmarks that evaluate their ability to optimize entire codebases under realistic constraints.