← Back to all news
arXiv Machine Learning

Circuits of LLM Classifiers

The paper introduces circuits that organize large language model (LLM) classifiers into an adaptive sequence of queries and decisions to improve classification reliability. By treating LLM predictions as noisy measurements of an underlying label, the authors derive the optimal circuit structure, establish fundamental accuracy limits, and offer practical guidance for implementation. Experiments across seven diverse classification tasks—ranging from safety and health to political science and human preference—show that this method outperforms traditional routing and cascading baselines on the cost‑accuracy frontier.

Related stories

arXiv Computation and Language
Aug 28

The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions

The paper investigates the "score granularity gap" in black-box large language model (LLM) classifiers, asking how finely a confidence score can be thresholded for deployment. By comparing seven confidence construction methods across 25 model-dataset pairs, the authors find that single-shot verbalized confidence, when properly converted to a probability, ranks well but offers only a few distinct threshold values, limiting operational flexibility. The study also shows that multi-query aggregation can improve weak models but may harm strong ones, and provides concrete guidance for deployment trade-offs.

By Ao Sun, Tian Sun, Jiaxing Geng
llmsbenchmarks
More like this →
arXiv Computation and Language
Oct 5

To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

The paper evaluates six structured decision models for hate‑speech moderation across four datasets, comparing them to specialized, zero‑shot, commercial, and supervised baselines. It investigates whether providing dataset definitions or decomposing them into multiple questions improves classification, and measures latency and cost. Results show that commercial LLMs outperform decision models on only one dataset, definitions alter up to 28% of predictions without consistent gains, and decomposition helps in only 20% of cases, yet the best hosted decision model approaches the best commercial LLM’s performance with roughly 97% lower inference cost.

By Demetris Paschalides, George Pallis, Marios D. Dikaiakos
llms
More like this →
arXiv AI
Sep 15

Towards Optimizing SQL Generation via LLM Routing

The paper "Towards Optimizing SQL Generation via LLM Routing" proposes a routing approach for Text-to-SQL tasks that dynamically selects the most cost‑effective large language model (LLM) for each query. Two routing strategies—score‑based and classification‑based—are introduced, achieving accuracy comparable to the best LLM while reducing latency and monetary cost. The authors design the routers for easy training and efficient inference, and demonstrate a practical accuracy‑cost trade‑off on the BIRD dataset.

By Mohammadhossein Malekpour, Nour Shaheen, Foutse Khomh, Amine Mhedhbi
llmsdata-engineering
More like this →
arXiv Machine Learning
Oct 8

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

The paper introduces a three‑stage framework that uses Large Language Models (LLMs) to generate rules and assemble them into decision trees for few‑shot tabular classification. By distilling LLM knowledge into interpretable trees, the method avoids the high inference costs and limited interpretability of direct LLM application while outperforming traditional decision trees in low‑data settings. Experiments on real‑world datasets show superior accuracy and interpretability with lower prompting overhead compared to existing baselines.

By Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li
llmssafety
More like this →