The paper introduces CREAMs, a flexible framework for Concept Bottleneck Models that allows explicit encoding of concept‑concept and concept‑task relationships such as mutual exclusivity, hierarchy, and correlations. CREAMs can optionally use a regularized side‑channel to compensate for incomplete concept sets, maintaining competitive performance while encouraging concept‑grounded predictions. The authors also propose a new metric to assess interpretability when predictions partially rely on the side‑channel, and demonstrate that CREAMs support efficient interventions, avoid concept leakage, and match black‑box performance even with missing concepts.
By Nektarios Kalampalikis, Kavya Gupta, Georgi Vitanov, Isabel Valera
arXiv:2606. 16535v1 Announce Type: new Abstract: Concept Bottleneck Models (CBMs) are a relevant tool for explainable Artificial Intelligence because they make their predictions through human-interpretable symbols.
By Javier Fumanal-Idocin, Javier Andreu-Perez
arXiv:2610.16182v1 Announce Type: new
Abstract: Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit t...
By Daoping Wu, Dong Wang, Siyu Liu, Chen Liang, Jian Geng, Ziqi Li, Shuguang Xiong, Ying Zhang
arXiv:2512. 20638v2 Announce Type: replace-cross Abstract: The evaluation of large language models relies heavily on standardized benchmarks.
By Maty Bohacek, Nino Scherrer, Nicholas Dufour, Thomas Leung, Christoph Bregler, Stephanie C. Y. Chan
NCP-ArchPreview is a latent‑space language model that extends standard next‑token prediction (NTP) with a Next Concept Prediction (NCP) objective, allowing the model to predict discrete concepts spanning multiple tokens. The architecture builds a product‑quantized concept vocabulary from hidden states, uses a dedicated Concept Module to forecast future concepts, and feeds these predictions back to guide token‑level generation, all trained jointly end‑to‑end. Trained on 5.73 T tokens with 8.9 B parameters, it achieves the final pretraining loss of OLMo‑3‑7B using only 51.3 % of the tokens, outperforms OLMo‑3‑7B on downstream tasks (including a 5.99‑point GSM8K gain), and demonstrates that the learned latent space enables lightweight domain adaptation and improved drafting performance.
By NCP Team, Jiaqi Cao, Chiyu Chen, Shuang Cheng, Xu Cheng, Beiya Dai, Yufan Feng, Kewen Ge, Ruijun Ge, Jiayi Huang, Yang Jiao, Dahua Lin, Zhouhan Lin, Yifan Liu, Yuliang Liu, Biqing Qi, Mowen Ruan, Junzhe Shen, Yunchong Song, Hao Sun, Zhongbo Tian, Yixuan Wang, Rubin Wei, Jiaxin Xiong, Kangyu Yang, Qian Yao, Qi Zhang, Bowen Zhou
arXiv:2512. 07355v2 Announce Type: replace Abstract: Two traditions of interpretability have evolved side by side but seldom spoken to each other: Concept Bottleneck Models (CBMs), which prescribe what a concept should be, and Sparse Autoencoders (SAEs), which discover what concepts emerge.
By Alexandre Rocchi, Thomas Fel, Gianni Franchi
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.
By Nour Shaheen, Junwei Ma, Alex Labach, Frank Hutter, Valentin Thomas, Anthony L. Caterini
arXiv:2510.13935v3 Announce Type: replace-cross
Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which ne...
By Kenan Alkiek, David Jurgens, Vinod Vydiswaran
arXiv:2608.15404v2 Announce Type: replace
Abstract: Concept Bottleneck Models (CBMs) are designed to make visual classification interpretable by expressing predictions through human-understandable co...
By Yusuf Meric Karadag, Gulay Oklan, Seref Baris Cagliyan, Umut Ozdemir, Emre Akbas
arXiv:2607. 18358v1 Announce Type: cross Abstract: Document classification is a solved problem in the laboratory and an unsolved one in the enterprise.
By Bogdan Raduta, Horia Velicu, Alexandru Preda, Serban Chiricescu
The paper introduces Graph of Concept Predictors (GCP), a reasoning-aware active distillation framework that captures a large language model’s intermediate reasoning as a directed acyclic graph of concepts and mirrors this structure in a smaller student model. GCP improves sample efficiency by using a graph-aware acquisition strategy that weighs concept uncertainty, gradient diversity, and node centrality, and enhances training stability through targeted sub‑module retraining that updates only the most influential concept predictors. Experiments on eight NLP classification benchmarks show that GCP achieves better performance under limited annotation budgets while providing more interpretable and controllable training dynamics.
By Ziyang Yu, Liang Zhao
arXiv:2512.02004v4 Announce Type: replace
Abstract: Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoen...
By Minglai Yang, Xinyu Guo, Zhengliang Shi, Jinhe Bi, Steven Bethard, Mihai Surdeanu, Liangming Pan