arXiv AI

Can large language models assist choice modelling? Insights into prompting strategies and current models' capabilities

arXiv AI
Sep 10

Beliefs and Behavior in Language Models

arXiv:2609.07943v1 Announce Type: new Abstract: There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In...

By Alex Smolin, Bryan Wilder
arXiv Machine Learning
Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad
arXiv AI
6d ago

Testing the Planning Claims of LLM-Based Agents against Formal Semantics

The paper evaluates the planning abilities of large language model (LLM) agents by comparing their performance against formal semantics defined in hidden PDDL models. Across five domains, four LLMs, six prompting strategies, and three agent architectures, the best setup solved 78% of solvable tasks offline but only 40% online, with limited recognition of unsolvable tasks and no clear benefit from more elaborate prompting. The study concludes that claims of LLM planning competence require validated action traces rather than end‑to‑end success rates.

By Oriol Miro-Lopez-Feliu, Oscar Mir\'o L\'opez-Feliu, Victor Gimenez-Abalos, Javier Vazquez-Salceda, Victor Carballo Araruna, Sergio Alvarez-Napagao