arXiv AI

ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions

ThinkJev introduces a hybrid approach that lets a language model decide whether to answer directly or to think first, using the same underlying model for both fast and slow responses. By reading the model’s probability distribution over option letters after any number of thought tokens, the system can provide a calibrated decision under any thinking budget without retraining. Experiments with Qwen3.5-27B show that the method achieves high accuracy while reducing the number of thought tokens needed, and it performs competitively on the JevBench benchmark.

arXiv Computation and Language
2d ago

Concise-9B: Removing the Slack from a Reasoning Model's Thinking

The paper introduces Concise-9B, a version of the Qwen3.5-9B reasoning model that reduces unnecessary thinking tokens through concise self‑training. By applying two rounds of on‑policy rejection‑sampling fine‑tuning and a small share of answer‑synthesis targets, Concise-9B cuts token usage to roughly two‑thirds of the original model while maintaining comparable accuracy on a wide range of hard mathematics and reasoning benchmarks. The study also shows that most of the token savings occur after the answer has already formed, and that the approach’s effectiveness depends on the verbosity of the base model.

By Alexander Boesgaard Lorup
arXiv AI
Oct 16

Verdict: Encode Once, Decide Many

Verdict-MQ is an open encoder that processes a single piece of state once and then answers multiple typed questions about that state, avoiding separate passes for each question. By using a structured attention mask, it ensures the state encoding is independent of question order and prevents questions from influencing one another. On the typed‑decisions benchmark, Verdict‑MQ outperforms a larger specialist model, achieving higher accuracy and better calibration, while on other benchmarks it shows mixed results compared to fine‑tuned baselines.

By Avdhesh Singh Chouhan, Jack Perry
arXiv Machine Learning
Oct 22

Our Own Jev: Typed Decisions in One Forward Pass, from an In-House Judge to Open LoRA Models

arXiv:2610. 29052v1 Announce Type: cross Abstract: A typed decision maps evidence, a question and a fixed set of options to a probability for every option, computed in one forward pass, by reading a language model's logits or a small head over its hidden states, instead of generating and parsing text.

By Yuchuan Weng, Chi Liu, Xin Chen, Srinivasan Manoharan
arXiv AI
Oct 16

JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs

JevAgentHarness is a new framework that benchmarks, deploys, and trains decision models to share control with large language models (LLMs) in agentic tasks. It introduces AgentDecisionBench to evaluate decision models from isolated questions to full agent control, revealing that accurate isolated decisions still miss many tasks and that naïve delegation can reduce success. By fitting confidence thresholds per choice and training a 4B decision model on states after agent mistakes, JevAgentHarness achieves high task success while cutting cost and speeding up execution, with results and resources made publicly available.

By Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji
arXiv AI
Sep 28

LAVOIR: Teaching a Single-Pass Decision Encoder When and What to Ask with Amortized Value of Information

LAVOIR is a single‑pass decision encoder that not only predicts answers to typed questions but also identifies which missing pieces of information (slots) would most improve its confidence. By placing candidate slots next to answer options, one forward pass yields both the decision distribution and the expected value of asking each slot, without requiring human labels. In controlled experiments, LAVOIR’s question policy matches a greedy oracle and improves accuracy by up to 14.1 points over never asking, while on real conversations it raises accuracy by 8.3 points with minimal questioning.

By Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay
arXiv AI
Oct 16

System1-Arena: Which Decisions Can Be Left to System 1?

System1-Arena is a benchmark that evaluates which decisions can be handled by System‑1 models—those that output probabilities in a single forward pass—versus System‑2 models that generate reasoning steps. It contains 92,000 questions across 105 tasks from 268 public sources, grouped into text classification, applied decisions, evaluation and control, and agent/action, and also includes 138 closed‑loop environments such as web shopping, coding control, robotics, games, and driving. The study tests 30 models, showing that large System‑1 models can match or exceed fast closed LLMs in non‑agentic accuracy with better‑calibrated probabilities at lower latency, while reasoning LLMs are most accurate but far more expensive, and large open LLMs perform best in closed‑loop settings.

By Tianyu Chen, Xinyang Liu, Ruyi An, Zhendong Wang, Mingyuan Zhou
arXiv AI
Sep 30

Mnemon: Raw Records, Fast Judgments, Slow Thoughts

Mnemon is a memory agent that stores conversations as raw, dated records and uses a fast System 1 decision model (Jev) to quickly judge the relevance of records, while a slow System 2 LLM plans searches and composes answers. The agent consolidates records into topic timelines and value histories in the background, enabling efficient retrieval without rewriting conversations into structured formats. Experiments show Mnemon achieving high scores on LoCoMo and LongMemEval‑S with low context length and cost, and Jev outperforming LLMs in evidence separation and speed.

By Guangren Wang
arXiv Machine Learning
Oct 8

Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation

The study investigates whether adaptive layer-skipping and repetition programs truly improve language‑model inference or merely exploit input‑blind controls. Using 32 such programs on Qwen3‑4B‑Base and Llama‑3.1‑8B across 4,413 multiple‑choice items, the authors compare real program gains to those from input‑blind perturbations and fixed letter offsets. Results show that input‑blind controls can produce comparable or even larger headroom, and that real programs only outperform controls in specific menu configurations, indicating that the apparent benefits may not be due to the actual layer computation choices.

By Yibei Guo, Rui Liu