← Back to all news
arXiv Machine Learning By Hadi Khalaf, Sajani Vithana, Kunal Talreja, Haewon Jeong, Viveck R. Cadambe, Flavio Calmon

Circuits of LLM Classifiers

Read the original on arXiv Machine Learning →

The paper introduces circuits that organize large language model (LLM) classifiers into an adaptive sequence of queries and decisions to improve classification reliability. By treating LLM predictions as noisy measurements of an underlying label, the authors derive the optimal circuit structure, establish fundamental accuracy limits, and offer practical guidance for implementation. Experiments across seven diverse classification tasks—ranging from safety and health to political science and human preference—show that this method outperforms traditional routing and cascading baselines on the cost‑accuracy frontier.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Aug 28

The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions

The paper investigates the "score granularity gap" in black-box large language model (LLM) classifiers, asking how finely a confidence score can be thresholded for deployment. By comparing seven confidence construction methods across 25 model-dataset pairs, the authors find that single-shot verbalized confidence, when properly converted to a probability, ranks well but offers only a few distinct threshold values, limiting operational flexibility. The study also shows that multi-query aggregation can improve weak models but may harm strong ones, and provides concrete guidance for deployment trade-offs.

By Ao Sun, Tian Sun, Jiaxing Geng
llmsbenchmarks
More like this →
arXiv Computation and Language
Oct 5

To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

The paper evaluates six structured decision models for hate‑speech moderation across four datasets, comparing them to specialized, zero‑shot, commercial, and supervised baselines. It investigates whether providing dataset definitions or decomposing them into multiple questions improves classification, and measures latency and cost. Results show that commercial LLMs outperform decision models on only one dataset, definitions alter up to 28% of predictions without consistent gains, and decomposition helps in only 20% of cases, yet the best hosted decision model approaches the best commercial LLM’s performance with roughly 97% lower inference cost.

By Demetris Paschalides, George Pallis, Marios D. Dikaiakos
llms
More like this →