arXiv AI

BUSTA: Teaching Small Language Models to Reason About Binary Execution

The paper introduces BUSTA, a method for training small language models to reason about binary execution. By fine‑tuning Qwen3‑1.7B and Nemotron‑3‑Nano with Group Relative Policy Optimization, the authors improve the models’ ability to predict input register values for a given x86‑64 basic block, raising accuracy from 73.2% to 85.3% and from 63.3% to 95.2% respectively. The fine‑tuned Nemotron‑3‑Nano achieves performance within four points of frontier models while using far fewer parameters, and the authors release both the fine‑tuned models and the training dataset.

arXiv Machine Learning
Sep 24

ChipMEM: Verification-Grounded Memory for EDA Agents

ChipMEM introduces a verification‑grounded memory layer for electronic design automation agents that combines cross‑task procedural memory with within‑trajectory statistical guidance. The procedural component stores a skill only after it passes synthesis, simulation, or formal checks, while a Bayesian component ranks recovery strategies based on tool‑call outcomes. Experiments on RTLRewriter‑Bench and CVDP tasks show that ChipMEM improves equivalence‑passing outputs and area metrics compared to agents without memory.

By Abdulrahman AlRabah, Joshua Mabry, Dilek Hakkani-T\"ur, Abdussalam Alawini, Hamid Shojaei, Kartik Hegde, Sandesh Adhikary
arXiv AI
Oct 1

EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses

EngramBench is a new benchmark designed to evaluate skill evolution in autonomous agents by focusing on genuine capability abstraction rather than solution copying. It includes 30 learning tasks and 13 unseen transfer tasks that require agents to manage complex, multi-hour development cycles with LLM‑simulated users. The study shows that while static skill banks cannot eliminate the need for precise code implementation, they effectively reduce redundant context and cut overall coding time by more than 55%.

By Zhixuan Tan, Pengjie Gu, Zhao Li, Yihan Hu, Xu He, Dong Li, Jianye Hao
arXiv Computation and Language
Oct 23

SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs

SCOPE compiles natural‑language policies into neurosymbolic predicate programs that combine small neural models, deterministic extractors, and logical composition. The system is jointly optimized with adversarially generated counterexamples, achieving higher accuracy than prompt‑optimization, task‑decomposition, and workflow‑generation baselines on real‑world policy benchmarks. Programs trained with the weakest executor transfer to other models, enabling deployment on inexpensive models with stable, interpretable predictions.

By Arnon Mazza, Elad Levi
arXiv Machine Learning
Jul 28

Benchmarking LLMs for Verilog Design Flows

arXiv:2607. 22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked.

By Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma, Vinay Chamola
Hugging Face Trending Papers
Jul 30

IFHierBench: Hierarchical Instruction Following for Large Language Models

Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the output satisfying specific constraints. Modern deployments increasingly handle the full task in a single LLM call, with one prompt specifying a layered output whose overall artifact, structural sections, and nested fields must each satisfy concrete constraints.

arXiv AI
Sep 2

Predicting Program Exit Code with LLMs and Programming Language Semantics

The paper introduces Program Executability Prediction (PrEx), a task that asks large language models (LLMs) to determine whether a program is semantically valid or invalid and, if invalid, to identify the violated formal rule. To evaluate this, the authors create a dataset of systematically generated invalid programs derived from valid ones and test open‑source coding LLMs across different semantic formalisms, semantic shifts, and program splits (human‑written, LLM‑translated, fuzzer‑generated). Results show that LLMs rely more on pre‑training priors than on the provided semantics, performing poorly on modified semantics and with increasing program complexity.

By Lara Marinov, Aditya Thimmaiah, Jayanth Srinivasa, Junyi Jessy Li, Milos Gligoric