The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2609.05881v2 Announce Type: replace-cross Abstract: Developers download quantized large language models (LLMs) from public registries to run them locally. None of the registries we examined doc...
The paper "Poison Valleys: Diagnosing Path-Dependent Failure in LLM Code Repair" investigates why large language model (LLM) code‑repair agents sometimes produce incorrect patches. By analyzing 1,844 model requests on two synthetic Python tasks—PersistQueue and Dijkstra—the authors trace how misconceptions in a model’s hypothesis or implementation plan lead to specific software obligation violations. They find that shifting a misconception from hypothesis to plan increases failure rates by roughly 30% for both tasks, revealing distinct, model‑dependent failure patterns linked to upstream misunderstandings.
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
The paper investigates hallucination in large language model–based automated program repair (APR). It defines hallucination as producing patches or intermediate artifacts that are not grounded in available repair evidence, and analyzes it across final patches and intermediate tasks such as triggering test case identification, line coverage prediction, and additional test case generation. Experiments on 832 Defects4J bugs show that only 21.0%–55.9% of patches pass the developer test suite, with 72.7% of sampled repairs exhibiting hallucinations, often due to incorrect causal localization or repair strategies.
arXiv:2610.33949v1 Announce Type: cross Abstract: Pipelines that require structured outputs from hosted language models often specify a Pydantic schema in the prompt, validate the response, and retry...
arXiv:2609.14758v1 Announce Type: cross Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply...