The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provide limited evidence about whether models can satisfy explicit output constraints. We introduce VCIFBench, a benchmark for evaluating complex instruction following in video understanding.
arXiv:2608.25529v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance in video understanding. However, their ability to follow instructions in this do...
arXiv:2503. 06573v3 Announce Type: replace-cross Abstract: Recent LLMs have shown remarkable success in following user instructions, yet handling instructions with multiple constraints remains a significant challenge.
Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the output satisfying specific constraints. Modern deployments increasingly handle the full task in a single LLM call, with one prompt specifying a layered output whose overall artifact, structural sections, and nested fields must each satisfy concrete constraints.
VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.
arXiv:2608. 19207v1 Announce Type: new Abstract: Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior.