Typed decision models are built for settings where model outputs are consumed directly by software. Instead of generating free-form text, they return a decision over a predefined set of options. By co...
The paper demonstrates that typed decision models can produce type‑correct outputs that nevertheless do not adhere to the explicit definitions of their options. By swapping only the names of options while keeping the definitions, question, and state unchanged, the authors show that decision flips can increase dramatically—up to 70.4 percentage points—when using yes/no names versus neutral controls. This effect is consistent across multiple models and decision rules, and even random strings as names yield flip rates similar to neutral controls, indicating that option‑name polarity contributes to decision instability beyond mere reassignment.
By Yu Sun, Junhao Xu, Jiajia Shi, Zijin Yang
arXiv:2610.02586v1 Announce Type: new
Abstract: A typed decision model answers a fixed question about an input by returning a probability for each of several caller-defined options. Each option carri...
By Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram
arXiv:2610.16655v1 Announce Type: new
Abstract: Decision APIs that return a probability for each option are routinely used as if those probabilities were properties of the options themselves. That re...
By Dushyant Rajput (AltSlate Labs LLP), Nirdesh Chauhan (AltSlate Labs LLP), Siddharth Kosaraju (AltSlate Labs LLP)
arXiv:2602.13576v2 Announce Type: replace-cross
Abstract: Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-langu...
By Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng
arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.
By Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi