The paper introduces Affect-Prototype Guided Fusion (APCF), a framework for open‑vocabulary multimodal emotion recognition that handles incomplete and unsynchronized modal data. APCF builds an affect‑prototype library to model how different emotions contribute across modalities, enabling dynamic fusion of available features. The fused representations are then decoded by an LLM to generate open‑vocabulary emotion labels, achieving superior performance on OV‑MERD+ and MER‑FG datasets compared to existing methods.
By Yichi Zhang, Shenyue Wang, Jing Luo, Chunyang Yu, Xinyu Yang
SyncVoice is a new automatic video dubbing framework that adds a lightweight Text‑Visual Fusion Module to a pretrained TTS system, aligning visual features with linguistic representations to produce temporally synchronized speech. The approach avoids complex architectural changes and achieves state‑of‑the‑art performance on the LRS3 dataset in zero‑shot dubbing. When further trained on a large bilingual audio‑visual corpus, SyncVoice improves vocal fidelity while maintaining synchronization, enabling a single model to dub both Chinese and English videos.
By Kaidi Wang, Yi He, Wenhao Guan, Weijie Wu, Peijie Chen, Hongwu Ding, Xiong Zhang, Di Wu, Meng Meng, Jian Luan, Lin Li, Qingyang Hong
The paper introduces little m, an AI agent that helps formulate industrial process control models by combining a domain-specific knowledge repository with LLM-driven interaction. It tackles the challenge of converting messy real-world specifications, including natural language and spatial diagrams, into rigorous mathematical optimization models. The authors also present IPC-Bench, a multimodal dataset of 50 canonical scenarios, and show through automated and human evaluations that little m outperforms state‑of‑the‑art LLMs in generating semantically correct models.
By Yongchao Ye, Xinyu He, Dutliff Boshoff, Way Kuo, Lishuai Li
arXiv:2609.16070v1 Announce Type: cross
Abstract: Generative recommendation reformulates item prediction as semantic identifier generation, yet episodic content introduces a fundamentally different s...
By Chenxing Wang, Nantao Zheng, Hao Miao, Juyuan Wang, Xinke Jiang, Yuchen Fang, Aolin Li, Haijun Wu
The paper explores whether CLIP embeddings can detect AI-generated images by using a frozen CLIP model to extract visual embeddings and training lightweight classifiers on top. On the CIFAKE benchmark, the approach achieves 95% accuracy without language reasoning, and 85% accuracy after few-shot adaptation with 20% of the data. Certain image types, such as wide-angle photographs and oil paintings, remain challenging, highlighting unexplored difficulties in AI-generated image classification.
By Ziyang Ou
The paper presents a multimodal machine learning framework that classifies Emirati residential architectural styles by combining visual features from images and textual descriptions using OpenAI's CLIP model. The unified 512‑dimensional embeddings are reduced with UMAP, clustered with K‑Means, and then used to train an SVM classifier, achieving a 98% accuracy across eight style clusters. This approach outperforms previous studies and demonstrates the value of integrating visual and textual data for cultural heritage analysis.
By Ahmed Ammar Kubba, Manar Abu Talib, Iman Ibrahim, Qassim Nasir
DGSG-Mind introduces a hybrid instance-aware 3D Gaussian dynamic scene graph system that integrates open‑vocabulary semantic information into dynamic 3D scene representations. By coupling a probabilistic voxel grid with explicit 3D Gaussians, it achieves robust cross‑modal instance fusion, incremental semantic mapping, and dynamic change handling through Gaussian‑based relocalization and masked refinement. The system builds a hierarchical scene graph and a 3D Gaussian Mind for multimodal reasoning, achieving state‑of‑the‑art zero‑shot 3D visual grounding and strong performance in open‑vocabulary semantic segmentation and scene reconstruction, and is demonstrated on real‑world robots.
By Luzhou Ge, Xiangyu Zhu, Jinyan Liu, Xuesong Li
arXiv:2601.09050v2 Announce Type: replace
Abstract: Tonal low-resource languages are widely spoken but remain underserved by modern speech technologies. A central challenge is learning speech represe...
By Tianyi Xu, Xuan Ouyang, Binwei Yao, Shoua Xiong, Sara Misurelli, Maichou Lor, Junjie Hu
arXiv:2609.16464v1 Announce Type: cross
Abstract: The clinical management of autism spectrum disorder (ASD) faces a bottleneck in early screening, mainly because trained specialists are scarce and co...
By Jun Chen, Qi Zhao, Yunliang Jiang, Shuqin Cao, Yunqiang Lin, Chenglong Jia, Qiang Guo, Guang Dai, Xiongtao Zhang, Mengmeng Wang, Xiaoyue Ma
The paper reports a submission to the SHROOM-Visions shared task, aiming to detect and classify hallucinated character spans in vision‑language model outputs across four languages. The authors use multiple fine‑tuned vision‑language models as independent annotators, combine their predictions via character‑level majority voting, and also investigate activation probes. Their method achieved first place in three of the four languages and consistently ranked on the podium for all languages and metrics, with analysis showing that model disagreement mirrors human annotator disagreement.
By Toqeer Ehsan, Nico Penttil\"a, Richard Schmidt, Arash Hajikhani, Victoria Palacin
EviDep is a multimodal evidential regression framework for estimating depression severity from audio–visual recordings, incorporating multi‑scale temporal modeling and shared–private representation learning. It uses frequency‑aware feature extraction to decompose behavioral sequences into multiple frequency bands, refined by scale‑specific experts, and applies disentangled evidential learning to separate cross‑modal shared and modality‑specific information. The model outputs Normal‑Inverse‑Gamma distributions via multi‑branch evidential regression, enabling estimation of depression severity along with aleatoric and epistemic uncertainty, and demonstrates competitive accuracy on several benchmark datasets.
By Fangyuan Liu, Sirui Zhao, Yangsong Zhang, Jinyang Huang, Feng-Qi Cui, Bin Luo, Tong Xu, Enhong Chen
arXiv:2609.17269v1 Announce Type: new
Abstract: Multimodal large language models generate natural-language responses from visual inputs, yet may mention objects absent from an image. In medication as...
By Ziheng Ren, Qian Gao, Jun Fan, Guohui Ding, Zhenyu Yang, Yuteng Xiao
arXiv:2609.16841v1 Announce Type: cross
Abstract: Increasing image resolution produces ever-longer visual-token sequences in vision-language models (VLMs), substantially raising their inference cost....
By Zhenbin Wang, Lei Zhang, Lituan Wang, Wei Huang, Yan Wang, Zhenwei Zhang
The paper introduces PosterVisor, a framework that replaces transient prompts with persistent control for scientific poster generation. It uses an Orchestrator to compile rubrics into Semantic‑Geometric Contracts that bind claims, sources, visuals, budgets, and spatial commitments, and employs Recursive Contract Enforcement to dynamically trigger checks and prevent silent regressions. Implementations in HTML/CSS and PPTX show improved QA accuracy and higher human preference on benchmark datasets.
By Runze Li, Yukun Zhao, Can Xu, Yucheng Shen, Shuaiqiang Wang, Jianmin Wu, Lingyong Yan, Dawei Yin
The paper introduces Relation‑Orbit, a symmetry‑aware method for aggregating likelihoods from frozen vision‑language models to verify fine‑grained left‑right claims. It uses a closed‑form contrast that assigns eight normalized likelihoods based on reflection, inverse relation, and entity exchange, and asserts a claim only when the signed contrast exceeds a threshold chosen via Clopper‑Pearson bounds. Experiments on VSR, GQA, and LLaVA‑1.5/COCO show that Relation‑Orbit achieves higher mean test coverage at a 10% selective‑risk calibration target compared to an all‑eight Orbit‑Max baseline across multiple dataset‑backbone settings.
By Zhouzhi Xiong, Chuxi Zhang, Weizhen He, Yi Chen, Qi Li, Donglian Qi
OmniHarness is a framework that enables generalizable visual generation by learning symbolic policies from verified executions. It abstracts shared procedures and applicability conditions, allowing these policies to be instantiated, adapted, and composed for new tasks while keeping model parameters fixed. The system uses intermediate verification for refinement, self-directed inquiry to generate practice tasks, and continuous feedback to expand capabilities, achieving strong results on multiple benchmarks and outperforming baselines on Creative tasks.
By Xu Xu (Beihang University), Jinxiu Liu (The Chinese University of Hong Kong), Zhangbo Qiao (Beihang University), Jiaxing Lu (Beihang University), Xiangyu Zhang (Beihang University), Yubin Gu (National University of Singapore), Fangwei Ning (Beihang University), Yan Shi (Beihang University)
MUUNRiver-Bench is a diagnostic benchmark for music retrieval that uses natural‑language instructions to define relevance for reference‑audio queries. It contains 3,440 tracks across 13 genres and 116 sub‑genres and covers seven tasks such as similar‑music, style‑preserving lyric‑rewriting, cover, and segment retrieval. Experiments with six models in eight configurations show that acoustic encoders favor local identity while text‑aligned encoders favor semantic relations, and that instruction‑aware and audio‑text fusion systems do not consistently outperform their backbones.
By Zhancheng Guo, Congren Dai, Shangda Wu, Jianhuai Hu, Danni Zhao, Xiaobing Li, Maosong Sun
arXiv:2602.06652v2 Announce Type: replace
Abstract: The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions r...
By Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema, Wai-Chung Kwan, Fazl Barez, Maria Sofia Bucarelli, Fabrizio Silvestri, Pasquale Minervini
arXiv:2609.17509v1 Announce Type: cross
Abstract: Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing comput...
By Thanapat Trachu, Samuele Cornell, William Chen, Shinji Watanabe
FLAT (Flexible‑Length Aligned Transmodal representations) is a joint multimodal pre‑training framework that learns a shared encoder for images and text, producing 1‑D continuous embeddings that can be directly used by downstream generative decoders. By combining contrastive alignment with bidirectional cross‑modal generative objectives, FLAT yields representations that are both discriminative and generative, enabling cross‑modal retrieval and generation with a single pre‑training stage. The model achieves strong performance on T2I generation (GenEval 71.1), image captioning (BLEU‑4 40.5, CIDEr 138.6), and retrieval tasks (Recall@5 86.8/75.8 on MS‑COCO, 98.3/93.6 on Flickr30K), and supports linear interpolation, latent space arithmetic, and zero‑shot composed retrieval.
By Guangyu Sun, Shlok Kumar Mishra, Wentao Bao, Robert Zhenheng Yang, Xiao Wang, Xiyuan Wang, Yujunrong Ma, Chen Yuan, Max Xiangjun Fan, Jun Xiao, Jianpeng Cheng