arXiv AI
Evaluating Neural Decompilation of Dart AOT Binaries: Fine-Tuning, Metric Validity, Specification Leakage, and Reliability
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 8
Evaluating Fine-Tuning and Metrics for Neural Decompilation of Dart AOT Binaries
arXiv:2607. 06125v1 Announce Type: cross Abstract: Neural decompilation is increasingly studied as a code-generation problem, yet its evaluation methodology remains underdeveloped for modern languages.
arXiv AI
Jul 7
JavaVulBench: A Java Vulnerability Benchmark with Realistic Splits, a Unified Multi-Backend Harness, and a Leakage-Aware Evaluation Mode
arXiv:2607. 02825v1 Announce Type: cross Abstract: We release \textsc{JavaVulBench}, a benchmark dataset and evaluation harness for Java vulnerability detection.
arXiv AI
Sep 10
Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models
The paper compares two output regimes for code-editing language models: direct generation, where the model outputs the entire modified file, and iterative diff-based generation, where the model emits a sequence of localized edits. Experiments on Flutter/Dart tasks show that direct generation consistently outperforms diff-based generation across metrics such as compilation success, token efficiency, and quality judgments. However, diff-based generation can be competitive for short, spatially localized edits, particularly in refactoring and error-handling tasks with few edit steps.
arXiv AI
Sep 7
When LLM Decompilers Recompile More and Preserve Less
The paper examines how large‑language‑model (LLM) decompilers, which produce clean, idiomatic C code, are currently evaluated mainly on recompilability and passing shipped tests. It shows that these metrics can mask significant behavioral differences: a decompiled function may recompile and pass all tests yet diverge on other inputs or lose disclosed vulnerabilities. To address this, the authors propose Decompile‑Diverge, a behavioral oracle that synthesizes drivers, fuzzes inputs, and compares the decompiled code’s behavior to the original, revealing divergences in up to 13% of cases and exposing gaps in current evaluation suites.
arXiv AI
Sep 1
APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows
arXiv:2608.29128v1 Announce Type: new Abstract: Tool-using agents are commonly evaluated by a single bit: whether an end-to-end workflow completed. This metric fails to distinguish failures that matt...
arXiv AI
Aug 11
KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
arXiv:2608. 09412v1 Announce Type: new Abstract: KV-cache compression reduces long-context memory, but aggregate task scores reveal neither which correct executions fail nor why.