arXiv Computation and Language
EDU-Jev Training System-One Decision Models for Education
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 18
QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training
QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.
arXiv AI
Sep 30
Evaluating and Benchmarking the System One Model Jev
arXiv:2609.37647v1 Announce Type: cross Abstract: Jev is a commercial System One model from TypeSafe AI that does not generate text: given a state and typed questions, it returns a choice from fixed...
arXiv AI
Oct 15
Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers
arXiv:2610.15501v1 Announce Type: new Abstract: Large language models make decisions through parallel prefill and sequential decoding. Although effective, this approach incurs decoding costs and does...
arXiv AI
Oct 15
Releasing the Inner Jev: A Simple Recipe for Open Decision Models
arXiv:2610.17205v1 Announce Type: new Abstract: LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a ty...
arXiv AI
Oct 9
TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
TypedBench is a new benchmark designed to evaluate system‑one decision models that output calibrated probabilities for typed answers—categorical, ordinal, or binary—using a non‑generative interface. The benchmark assesses models on policy adherence, wording robustness, probability quality (via selective prediction, ordinal proper scoring rules, and calibration error), and the cost of decisions under asymmetric cost matrices, using seven policy‑labelled generators and nine evaluation suites. Experiments on a hosted model, an open encoder, and a range of open decoders (0.8B–9B parameters) reveal that the hosted model is wording‑sensitive and underconfident, while decoders are slow and less accurate on policy questions, highlighting the need for joint evaluation across multiple dimensions.
arXiv Computation and Language
Oct 9
Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks
The paper introduces Jev, a specialized decision model that selects among given options rather than generating text, and benchmarks it against 19 large language models (LLMs) across 13 multiple-choice tasks in knowledge, reasoning, and multilingual domains. Jev performs competitively with frontier LLMs on knowledge and commonsense benchmarks, achieving the top score on MMLU-Redux and ARC-Challenge, and surpasses most representative and all small LLMs except in mathematical word problems where it lags behind. The study demonstrates that a specialized decision model can match general-purpose LLMs on knowledge-based decisions but struggles with multi-step calculations.