arXiv:2508. 11847v4 Announce Type: replace-cross Abstract: We propose a method for evaluating the robustness of widely used LLM ranking systems -- variants of a Bradley--Terry model -- to dropping a worst-case very small fraction of preference data.
By Jenny Y. Huang, Yunyi Shen, Dennis Wei, Tamara Broderick
MiniRep is a reputation‑based aggregation system designed for multi‑agent debate (MAD) that remains robust even when malicious agents are present. It evaluates agents on both their current task performance and historical reputation, while preventing groups of agents with highly similar responses from dominating the final decision. Experiments on the MATH benchmark show that MiniRep consistently outperforms conventional MAD aggregation and other reputation‑based approaches across a wide range of attack scenarios.
By Jiaming Zhang, Yuwan Liu, Yue Huang, Sisi Duan
arXiv:2604. 17805v2 Announce Type: replace-cross Abstract: Pairwise ranking systems based on Maximum Likelihood Estimation (MLE), such as the Bradley-Terry model, are widely used to aggregate preferences from pairwise comparisons.
By Junyi Yao, Zihao Zheng, Jiayu Long
arXiv:2605. 29107v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly rank products, documents, and recommendations for user queries, which makes manipulating these rankings a growing concern for fairness and information integrity.
By Ojas Nimase, Zhe Chen, Gengpei Qi, Yue Zhao, Xiyang Hu
arXiv:2603.15220v2 Announce Type: replace
Abstract: Strict anonymity of model responses is a key for the reliability of voting-based leaderboards, such as LM Arena. While prior studies have attempted...
By Minsung Cho, Jaehyung Kim
arXiv:2501. 00745v3 Announce Type: replace-cross Abstract: The increasing integration of Large Language Model (LLM) based search engines has transformed the landscape of information retrieval.
By Xiyang Hu
arXiv:2510. 06732v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as rerankers in information retrieval, yet their ranking behavior can be steered by small, natural-sounding prompts.
By Tiancheng Xing, Jerry Li, Yixuan Du, Xiyang Hu
arXiv:2607. 24780v1 Announce Type: new Abstract: Evaluating frontier LLMs is challenging: static benchmarks suffer from contamination and saturation -- leaving users unable to distinguish top models and developers blind to specific failure modes -- while human preference is subjective.
By Xingyu Chen, Rui Wang, Zhaopeng Tu, Liefeng Bo
arXiv:2609.23201v1 Announce Type: new
Abstract: Benchmark scores increasingly influence the development, marketing, and selection of large language models (LLMs). Yet an overall score is interpretabl...
By Danial Amin
arXiv:2606. 24381v1 Announce Type: cross Abstract: Prompt-based interaction has become a dominant paradigm for using large language models (LLMs), where multiple candidate prompts are evaluated and the top-ranked one is selected for downstream use.
By Shaoshuai Du, Penghao Liang, Yixian Shen, Chuanqi Shi, Hang Zhang, Lun Wang
arXiv:2606. 25476v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness.
By Abrar Alotaibi, Raed Mughus, Moataz Ahmed
arXiv:2512. 05518v2 Announce Type: replace-cross Abstract: Open-source Large Language Models (LLMs) play a critical role in the democratization of AI, yet their "open" nature introduces more avenues for malicious actors to misuse them for harmful purposes.
By Jason Vega, Gagandeep Singh