EventMemAgent is an active online video agent that uses a hierarchical memory module to handle continuous perception and long‑range reasoning in streaming video. The framework employs a short‑term memory layer to detect event boundaries and sample frames within a fixed buffer, while a long‑term memory layer archives observations event‑by‑event. It also incorporates a multi‑granular perception toolkit and Agentic Reinforcement Learning to internalize reasoning and tool‑use strategies, achieving competitive results on online video benchmarks.
By Siwei Wen, Zhangcheng Wang, Xingjian Zhang, Lei Huang, Wenjun Wu
The paper introduces a post‑training approach that enables a single inference process to transition from text reasoning to image synthesis, eliminating the need for explicit modality switching. Using the 14B BAGEL model, the authors demonstrate that targeted post‑training data and reward‑weighted training improve multimodal image generation across four independent T2I benchmarks. The study highlights the benefits of joint text‑image generation and strategic data selection for enhancing T2I performance.
By Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad
The paper introduces ASCIL, a post‑ASR correction framework that re‑evaluates wake‑up intent by combining acoustic embeddings, linguistic cues, device context, and past misclassifications. ASCIL interprets both implicit (hesitation, disengagement, silence) and explicit (cancellation, repetition) signals as noisy indicators of misclassification, enabling online pattern updates without manual annotation. On a proprietary dataset of 3,667 interactions, ASCIL reduces errors by up to 54.27% relative on a session‑disjoint subset and 24.39% at a 0.90 threshold, while adding less than 60 ms of latency and improving intentional acceptance rates.
By Preeti Saraswat, Divya Neelagiri, Anil Yadav
The paper presents a human‑annotated dataset of 100 Quran recitation recordings, identifying 348 scored units and 162 localized events across ten combined labels. An evaluator that scores both labels and word positions achieves a label‑aware F1 of 0.525 and a localization F1 of 0.826, with adapted production cleaner/alignment components yielding similar scores. A pilot experiment with eight 20‑minute runs across three coders and eight models shows wide variance in F1 (0.143–0.892) and highlights that most gold events are detected, leaving only span extent and label conventions as the remaining challenges.
By Mohamad Al Mdfaa, Nursultan Askarbekuly, Ahmed Helaly, Ubai Sandouk, Manuel Mazzara
ProactiveBench evaluates streaming video models on their ability to interact proactively, rather than reactively. It tests models at one‑second intervals without explicit cues, using six subtasks that vary trigger ambiguity, timing tolerance, and response patterns. The benchmark measures both response and silence rates, distinguishing early, in‑window, and missed responses, and penalizes omissions and repetitions.
By Kaixuan Du, Xin Wan, YuKun Wang, Hang Zhang, Meng Cao, Dai Guan, Ming Chen, Ni Li
The paper proposes defining space as an interventional invariant—a minimal relational structure that maintains local compatibility and future observation laws under admissible actions. It introduces a cross‑modal predictive geometry that combines local state spaces, modality‑specific observation maps, an action groupoid, and a predictive‑state quotient, with causal conditions to distinguish interventional from observational structure. The authors prove that, given joint point separation, equivariance, and interventional faithfulness, the latent space is identifiable up to the centraliser of the intervention group, reducing ambiguity to residual coordinate freedom, and extend the framework to stratified urban systems using sheaf‑valued representations.
By Tao Yang, Xuhui Lin, Kunyao Li, Haijiang Li
HoliBench is a modular benchmarking and deployment toolkit that jointly measures accuracy, latency, and energy for foundation models across a wide range of devices, from single-board computers to GPU servers. It provides a platform abstraction layer that calibrates cross-device measurements and supports multiple model modalities, inference engines, and quantization levels. Using HoliBench, the authors evaluated 20 models on 7 device types, revealing tradeoffs such as limited latency gains from quantization on low‑precision hardware and diminishing accuracy returns relative to energy consumption, while also showing that single-model profiles can predict multi-model pipeline performance within a few percent.
By Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava
SaFeR-Steer is a progressive multi‑turn alignment framework that uses staged synthetic bootstrapping and tutor‑in‑the‑loop GRPO to train a single student model against adaptive, on‑policy attacks. It introduces Trajectory‑Consistent Summative Reward (TCSR) to ensure that low‑quality turns impact the overall trajectory reward. The authors release the STEER dataset and demonstrate that SaFeR‑Steer significantly improves safety and helpfulness on both single‑turn and multi‑turn benchmarks for Qwen2.5‑VL models.
By Haolong Hu, Hanyu Li, Tiancheng He, Huahui Yi, An Zhang, Qiankun Li, Kun Wang, Yang Liu, Zhigang Zeng
arXiv:2609.12397v1 Announce Type: new
Abstract: Multi-modal image generation, particularly subject-driven customization, has garnered growing attention in recent years. Despite the rapid advancement...
By Danning Zhang, Yijing Lin, Shuhan Zhuang, Mengqi Huang, Shaojin Wu, Shancheng Fang, Zhendong Mao
FINESSE is an agent‑based simulation framework that generates synthetic, structured datasets of multiple interdependent financial event streams, such as transactions, payments, account status changes, and policy interventions. Each stream has its own action space, schema, and variable types, and the streams are coupled through agents’ evolving latent states, allowing temporally rich interactions. The accompanying FINESSE‑Bench dataset supports four tasks—balance forecasting, transaction fraud detection, missed payment prediction, and next event prediction—and baseline results are provided using various time‑series and event‑sequence methods.
By Tyler Farnan, Benjamin Eng, Adam Abate, Xirui Hou, Rizal Fathony, Nam H. Nguyen, Senthil Kumar
The study introduces a scalable acoustic‑masking method to quantify how much each consonant contributes to word intelligibility. By silencing individual consonants in isolated words and measuring misrecognition rates with three ASR models, the authors define a mask‑induced misrecognition rate (MMR). Across English, Spanish, German, and Czech, MMR negatively correlates with phoneme frequency and positively with functional load, revealing that consonant importance varies by language.
By Eunjung Yeo, Kwanghee Choi, Krupaben Kothadia, Visar Berisha, Julie M. Liss, David R. Mortensen, David Harwath
UniLayDiff is a Unified Diffusion Transformer that tackles content‑aware layout generation across a wide range of tasks—unconditional, element‑type, size, and relationship‑conditioned generation—using a single end‑to‑end trainable model. It treats layout constraints as a distinct modality within a Multi‑Modal Diffusion Transformer framework, capturing interactions among background images, layout elements, and constraints. The model is further refined with LoRA fine‑tuning to incorporate relation constraints, achieving state‑of‑the‑art performance and, according to the authors, the first unified solution for all content‑aware layout generation tasks.
By Zeyang Liu, Le Wang, Sanping Zhou, Yuxuan Wu, Xiaolong Sun, Gang Hua, Haoxiang Li
arXiv:2606.10468v2 Announce Type: replace
Abstract: Coastline detection in remotely sensed imagery is commonly formulated as pixel-wise segmentation, even though coastlines used in coastal monitoring...
By Rafia Malik, Bernhard Pfahringer, Karin Bryan, Mark Dickson, Eibe Frank
ExpertHTR is a unified vision‑language framework for handwritten text recognition that tackles the challenge of small, heterogeneous datasets by organizing structural annotations into a common Page‑Region‑Line representation. It defines four related training tasks—complete transcription, physical‑line coverage, text localization, and localized recognition—without extra manual labels. The model combines a jointly trained dense backbone with a sparse Mixture‑of‑Experts architecture, using Sparsegen routing and regularization to adaptively activate experts, achieving state‑of‑the‑art results on the IAM benchmark and outperforming general‑purpose OCR systems on most datasets.
By Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu, Vo Nguyen Le Duy
QuPAINT is a physics‑aware multimodal framework designed to characterize two‑dimensional quantum materials using optical microscopy. It employs the Synthetic Materials Framework (Synthia) to generate diverse synthetic images that preserve layer‑dependent optical behavior, and builds the QMat‑Instruct dataset with image‑specific reasoning traces. The framework integrates these signals through Physics‑Informed Attention (PIA) and is evaluated on the newly introduced QF‑Bench benchmark, achieving state‑of‑the‑art performance for flake detection and demonstrating improved spatial grounding and confidence calibration.
By Sankalp Pandey, Xuan-Bac Nguyen, Hoang-Quan Nguyen, Tim Faltermeier, Nicholas Borys, Hugh Churchill, Khoa Luu
The paper presents a cross‑modal learning framework that predicts three‑dimensional dislocation density fields directly from X‑ray diffraction data. By pairing simulated dislocation density fields with virtual diffraction patterns and embedding them into a shared 2‑D latent space via contrastive learning, the authors achieve strong alignment between structural and diffraction representations. Experiments show that model performance improves rapidly with dataset size, reaching near‑saturation with about 500 representative observations out of 10,000, and the predicted structures capture the dominant spatial features of the underlying microstructures.
By Benjamin Udofia, Nicolas Bertin, Markus Stricker
The paper introduces STAG, a post‑hoc framework that provides token‑level spectro‑temporal grounding for captions produced by audio‑based multimodal large language models (MLLMs). STAG estimates temporal support for each token via vocabulary projections of encoded audio, measures frequency‑band relevance through controlled spectral occlusion, and fuses these signals into a spectro‑temporal relevance map. Evaluations across ten explanation methods and four grounding benchmarks show that STAG achieves superior event‑localization performance on every dataset, and counterfactual deletion experiments confirm that removing the identified evidence selectively reduces model confidence and often eliminates the corresponding event from regenerated captions.
By Lucia Cascone, Valeria Fraenza, Michele Nappi, Fabio Narducci, Benedetto Simone
arXiv:2609.12575v1 Announce Type: new
Abstract: Ambiguity is often treated as a bug for AI systems to resolve---but in human communication and culture, ambiguity can also be a generative resource. Fr...
By Cody Kommers, Mingrui Ye, Evelyn Gius, Daniela Mihai, Hoyt Long, Zheng Yuan, Drew Hemment
DementiaCare-Bench is a new video benchmark designed to evaluate video-language models (VLMs) on understanding behavioral and psychological symptoms of dementia (BPSD). It contains 56 caregiver training videos, 94 clips across nine BPSD categories, and 2023 questions that are grounded in transcript spans and labeled by their visual demand. The benchmark reveals that most VLMs perform well on questions answerable by language alone but struggle with those requiring ordered frames, and a lightweight LoRA fine‑tune (DemCare‑VLM) can improve video dependence.
By Afrouz Sheikholeslami, Yuankai Qi, Xuyun Zhang, Luping Zhou, Amin Beheshti, Quan Z. Sheng, Ming-Hsuan Yang
The paper introduces FRIST, a two‑stage EEG decoding framework that uses fMRI data to learn spectral projections and class geometry, then refines EEG predictions. In experiments with 12 participants, FRIST improves finger‑level motor decoding accuracy across both movement execution and motor imagery tasks, outperforming EEG‑only baselines and generalizing across different EEG backbones. The method demonstrates that fMRI’s spatial resolution can enhance noninvasive EEG‑based BCI performance.
By Jintao Zhang, Yidan Ding, Joshua Kosnoff, Maxim Karrenbach, Hanwen Wang, Bin He