Hugging Face Trending Papers
CIRCE Divines LLM Task Complexity from Constraint Structure
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
5d ago
CIRCE Divines LLM Task Complexity from Constraint Structure
CIRCE is a method that estimates the difficulty of a task for a large language model (LLM) by extracting the problem’s constraint structure into a Constraint Intermediate Representation, which is then compiled into a constraint graph. Unlike prior approaches that rely on the problem text or model outputs, CIRCE uses a lightweight LLM to build this representation and feeds it to a graph neural network to predict whether the LLM will solve the problem correctly. Experiments on seven reasoning benchmarks and four LLMs show that even a randomly initialized GNN improves prediction accuracy, lowering negative log‑likelihood from 0.465 to 0.423 and raising AUROC from 0.691 to 0.727 compared to the best baseline.
arXiv AI
Sep 4
RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data
RECAST is a new framework that generates datasets with far more constraints per example than existing benchmarks, aiming to push large language models (LLMs) to better follow complex instructions. The authors built RECAST-30K, a 30,000‑instance dataset covering 19 constraint types extracted from real prompt‑response pairs, and showed that fine‑tuning on it improves LLMs’ ability to handle complex tasks without harming general performance. RECAST also provides rule‑based and LLM‑based validators for automatic constraint verification, enabling reward‑based reinforcement learning to further enhance model performance on challenging tasks.
arXiv AI
Jun 3
Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models
arXiv:2606. 03624v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) have demonstrated impressive capabilities in many tasks, yet they struggle with reliably following multiple instructions, either by failing to satisfy individual constraints or by struggling to balance competing constraints simultaneously.
arXiv AI
Oct 27
Constrained Decoding: Harmful or Beneficial? Revisiting Its Impact on LLM Performance
The paper investigates whether constrained decoding harms or helps large language model (LLM) performance. It introduces a taxonomy of three constraint types—format, validity, and reasoning—and shows that suitable constraints can preserve or even improve performance by eliminating invalid paths. Experiments across multiple model families reveal that perceived performance drops are mainly due to non-equivalent prompting and output formats, while validity constraints boost reliability and reasoning constraints can enhance accuracy when they guide beneficial reasoning trajectories.
arXiv AI
Jul 21
Constraint-Anchored Reasoning Traces
arXiv:2607. 16727v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) suffer from error snowballing: a single incorrect inference early in a chainof-thought (CoT) trace corrupts all downstream reasoning.
arXiv AI
Aug 14
Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction
arXiv:2608. 12426v1 Announce Type: new Abstract: Large language models are increasingly deployed in settings that require simultaneous adherence to multiple explicit constraints - reasoning structure, safety boundaries, output schemas.